Patchwork local.conf.sample: Have comments ready to go for quad-core (4 -> 8).

login
register
mail settings
Submitter Robert P. J. Day
Date April 18, 2013, 4:23 p.m.
Message ID <alpine.DEB.2.02.1304181219550.8010@oneiric>
Download mbox | patch
Permalink /patch/48525/
State New
Headers show

Comments

Robert P. J. Day - April 18, 2013, 4:23 p.m.
Have default, commented values for parallelism reflect quad-core
hosts.

Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca>

---

  i'll admit it, i'm just lazy as i do all my work on a quad-core
system and i'm constantly changing 4 to 8. and the proper values for a
quad-core system *should* be 4, not 8, anyway, isn't that right?
Enrico Scholz - April 18, 2013, 4:48 p.m.
"Robert P. J. Day" <rpjday-L09J2beyid0N/H6P543EQg@public.gmane.org>
writes:

> Have default, commented values for parallelism reflect quad-core
> hosts.

fwiw, I use

PARALLEL_MAKE = "\
  -j ${@int(os.sysconf(os.sysconf_names['SC_NPROCESSORS_ONLN'])) * 2} \
  -l ${@int(os.sysconf(os.sysconf_names['SC_NPROCESSORS_ONLN'])) * 150/100} \
"

BB_NUMBER_THREADS ?= "\
  ${@int(os.sysconf(os.sysconf_names['SC_NPROCESSORS_ONLN'])) * 150/100}"


which adjusts parallelism automatically.



Enrico
Robert P. J. Day - April 18, 2013, 4:51 p.m.
On Thu, 18 Apr 2013, Enrico Scholz wrote:

> "Robert P. J. Day" <rpjday-L09J2beyid0N/H6P543EQg@public.gmane.org>
> writes:
>
> > Have default, commented values for parallelism reflect quad-core
> > hosts.
>
> fwiw, I use
>
> PARALLEL_MAKE = "\
>   -j ${@int(os.sysconf(os.sysconf_names['SC_NPROCESSORS_ONLN'])) * 2} \
>   -l ${@int(os.sysconf(os.sysconf_names['SC_NPROCESSORS_ONLN'])) * 150/100} \
> "
>
> BB_NUMBER_THREADS ?= "\
>   ${@int(os.sysconf(os.sysconf_names['SC_NPROCESSORS_ONLN'])) * 150/100}"
>
>
> which adjusts parallelism automatically.

  i'd be perfectly fine with that, too, if someone wants to put that
in instead.

rday
Elvis Dowson - April 18, 2013, 5:57 p.m.
On Apr 18, 2013, at 8:51 PM, "Robert P. J. Day" <rpjday@crashcourse.ca> wrote:

> On Thu, 18 Apr 2013, Enrico Scholz wrote:
> 
>> "Robert P. J. Day" <rpjday-L09J2beyid0N/H6P543EQg@public.gmane.org>
>> writes:
>> 
>>> Have default, commented values for parallelism reflect quad-core
>>> hosts.
>> 
>> fwiw, I use
>> 
>> PARALLEL_MAKE = "\
>>  -j ${@int(os.sysconf(os.sysconf_names['SC_NPROCESSORS_ONLN'])) * 2} \
>>  -l ${@int(os.sysconf(os.sysconf_names['SC_NPROCESSORS_ONLN'])) * 150/100} \
>> "
>> 
>> BB_NUMBER_THREADS ?= "\
>>  ${@int(os.sysconf(os.sysconf_names['SC_NPROCESSORS_ONLN'])) * 150/100}"
>> 
>> 
>> which adjusts parallelism automatically.
> 
>  i'd be perfectly fine with that, too, if someone wants to put that
> in instead.

I've noticed that on a quad-core machine (i7) the best setting is to leave it at 6 for 
BB_NUMBER_THREADS and PARALLEL_MAKE.

8, 16, etc don't have any discernable impact, perhaps save 30 to 40 seconds off
a build. The 3.5Ghz standard CPU frequency can be over-clocked to as high
as 4.5GHz, but I tested it out at 4.2Ghz and 3.8Ghz, and the difference was
only 2 to 3 minutes between 3.8GHz and 4.2GHz over-clocked, so I currently
leave it at 3.8GHz over-clocked.

I'm using two SSDs in a RAID 0 striped configuration, and build times is 
around 22 minutes for a Zynq-7-ZC702 build for core-image-minimal.

Elvis Dowson
Yi Qingliang - April 19, 2013, 12:49 a.m.
On Thursday, April 18, 2013 09:57:16 PM Elvis Dowson wrote:
> On Apr 18, 2013, at 8:51 PM, "Robert P. J. Day" <rpjday@crashcourse.ca> 
wrote:
> > On Thu, 18 Apr 2013, Enrico Scholz wrote:
> >> "Robert P. J. Day" <rpjday-L09J2beyid0N/H6P543EQg@public.gmane.org>
> >> 
> >> writes:
> >>> Have default, commented values for parallelism reflect quad-core
> >>> hosts.
> >> 
> >> fwiw, I use
> >> 
> >> PARALLEL_MAKE = "\
> >> 
> >>  -j ${@int(os.sysconf(os.sysconf_names['SC_NPROCESSORS_ONLN'])) * 2} \
> >>  -l ${@int(os.sysconf(os.sysconf_names['SC_NPROCESSORS_ONLN'])) *
> >>  150/100} \
> >> 
> >> "
> >> 
> >> BB_NUMBER_THREADS ?= "\
> >> 
> >>  ${@int(os.sysconf(os.sysconf_names['SC_NPROCESSORS_ONLN'])) * 150/100}"
> >> 
> >> which adjusts parallelism automatically.
> >> 
> >  i'd be perfectly fine with that, too, if someone wants to put that
> > 
> > in instead.
> 
> I've noticed that on a quad-core machine (i7) the best setting is to leave
> it at 6 for BB_NUMBER_THREADS and PARALLEL_MAKE.
maybe we need consider the hyper-threading.

	CPU_PROC_NUM="`grep "^processor" /proc/cpuinfo | sort -u | wc -l`"
	CPU_CORE_NUM="`grep "^core id" /proc/cpuinfo | sort -u | wc -l`"

> 8, 16, etc don't have any discernable impact, perhaps save 30 to 40 seconds
> off a build. The 3.5Ghz standard CPU frequency can be over-clocked to as
> high as 4.5GHz, but I tested it out at 4.2Ghz and 3.8Ghz, and the
> difference was only 2 to 3 minutes between 3.8GHz and 4.2GHz over-clocked,
> so I currently leave it at 3.8GHz over-clocked.
> 
> I'm using two SSDs in a RAID 0 striped configuration, and build times is
> around 22 minutes for a Zynq-7-ZC702 build for core-image-minimal.
> 
> Elvis Dowson
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Trevor Woerner - April 19, 2013, 8:08 p.m.
All of these "solutions" assume the user wants to use all of their
entire computing resources to do nothing other than their Yocto build.
This isn't _always_ the case, is it? :-)

Personally, I know this build is going to take time. So I start it,
then go off and do other things with my computer. In my case, I don't
like it when the other things I'm trying to do on the computer are
bogged down due to this all-consuming, background build task. I would
much rather that whatever else I'm doing remain responsive to me.
Whether the build finishes in 20 minutes or 25 is irrelevant because
I'll only notice it's done 30 minutes from now :-) As such I never set
my settings to "full power"... knowingly and purposefully.

This being said, I think what is currently in the sample config file
(which are roughly half of full power) are good defaults. If you're
new to this kind of stuff, these settings will suit just fine. If
you're a power user, you'll know how to set them to 8 or 16 or any
other setting which suits your needs.

Patch

diff --git a/meta/conf/local.conf.sample b/meta/conf/local.conf.sample
index 85205c5..85164a0 100644
--- a/meta/conf/local.conf.sample
+++ b/meta/conf/local.conf.sample
@@ -17,14 +17,14 @@ 
 # These two options control how much parallelism BitBake should use. The first
 # option determines how many tasks bitbake should run in parallel:
 #
-#BB_NUMBER_THREADS = "4"
+#BB_NUMBER_THREADS = "8"
 #
 # The second option controls how many processes make should run in parallel when
 # running compile tasks:
 #
-#PARALLEL_MAKE = "-j 4"
+#PARALLEL_MAKE = "-j 8"
 #
-# For a quad-core machine, BB_NUMBER_THREADS = "4", PARALLEL_MAKE = "-j 4" would
+# For a quad-core machine, BB_NUMBER_THREADS = "8", PARALLEL_MAKE = "-j 8" would
 # be appropriate for example.

 #