Patchwork [1/1] libc-locale: split locale handling from libc recipe.

login
register
mail settings
Submitter Dongxiao Xu
Date June 8, 2011, 9:08 a.m.
Message ID <3cb0d7f134013f8dcd664429b7efda396d12790e.1307523829.git.dongxiao.xu@intel.com>
Download mbox | patch
Permalink /patch/5555/
State New, archived
Headers show

Comments

Dongxiao Xu - June 8, 2011, 9:08 a.m.
*libc's do_package will cost a lot of time due to the locale handing,
which may delay the other recipe's do_package task and affect the build
performance.

This commit moves locale handling into a separate recipe *libc-locale.

Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
---
 meta/classes/libc-common.bbclass                   |   23 ++++++++
 meta/classes/libc-package.bbclass                  |   24 --------
 meta/conf/distro/include/tclibc-eglibc.inc         |   12 ++--
 meta/conf/distro/include/tclibc-glibc.inc          |   12 ++--
 meta/recipes-core/eglibc/eglibc-initial_2.13.bb    |    4 ++
 meta/recipes-core/eglibc/eglibc-locale_2.13.bb     |   56 ++++++++++++++++++++
 meta/recipes-core/eglibc/eglibc-package.inc        |   32 +----------
 meta/recipes-core/eglibc/eglibc.inc                |    5 +-
 meta/recipes-core/eglibc/eglibc_2.13.bb            |   12 ++++
 meta/recipes-core/glibc/glibc-initial_2.10.1.bb    |    4 ++
 meta/recipes-core/glibc/glibc-locale_2.10.1.bb     |   48 +++++++++++++++++
 meta/recipes-core/glibc/glibc-package.inc          |   31 ++---------
 meta/recipes-core/glibc/glibc.inc                  |    2 +-
 meta/recipes-core/glibc/glibc_2.10.1.bb            |   12 ++++
 .../meta/external-csl-toolchain_2008q3-72.bb       |    3 +-
 meta/recipes-core/meta/external-poky-toolchain.bb  |    2 +-
 meta/recipes-gnome/gtk+/gtk+.inc                   |    2 +-
 17 files changed, 187 insertions(+), 97 deletions(-)
 create mode 100644 meta/classes/libc-common.bbclass
 create mode 100644 meta/recipes-core/eglibc/eglibc-locale_2.13.bb
 create mode 100644 meta/recipes-core/glibc/glibc-locale_2.10.1.bb
Phil Blundell - June 8, 2011, 9:36 a.m.
On Wed, 2011-06-08 at 17:08 +0800, Dongxiao Xu wrote:
> *libc's do_package will cost a lot of time due to the locale handing,
> which may delay the other recipe's do_package task and affect the build
> performance.
> 
> This commit moves locale handling into a separate recipe *libc-locale.

Can you quantify the effect on build performance a bit?  If I understand
correctly, you're basically saying that the goal is to increase
parallelism.  Does that cause reduced performance for people running
with few threads?

For example, it would be interesting to know what the before and after
times are for some representative image build using (say)
BB_NUMBER_THREADS=1 and BB_NUMBER_THREADS=16.

Also, see below.

> diff --git a/meta/conf/distro/include/tclibc-eglibc.inc b/meta/conf/distro/include/tclibc-eglibc.inc
> index a4c648e..8997d9e 100644
> --- a/meta/conf/distro/include/tclibc-eglibc.inc
> +++ b/meta/conf/distro/include/tclibc-eglibc.inc
> @@ -10,7 +10,7 @@ TARGET_OS_powerpc = "linux${@['','-gnuspe'][bb.data.getVar('BASE_PACKAGE_ARCH',d
>  # Add glibc overrides to the overrides for eglibc.
>  OVERRIDES .= ":libc-glibc"
>  
> -PREFERRED_PROVIDER_virtual/libiconv ?= "eglibc"
> +PREFERRED_PROVIDER_virtual/libiconv ?= "eglibc-locale"

That looks a bit strange.  Is this really correct?

> -    		     eglibc-localedata-i18n \
> -    		     eglibc-gconv-ibm850 \
> -    		     eglibc-gconv-cp1252 \
> -    		     eglibc-gconv-iso8859-1 \
> -    		     eglibc-gconv-iso8859-15 \
> +    		     eglibc-locale-localedata-i18n \
> +    		     eglibc-locale-gconv-ibm850 \
> +    		     eglibc-locale-gconv-cp1252 \
> +    		     eglibc-locale-gconv-iso8859-1 \
> +    		     eglibc-locale-gconv-iso8859-15 \

This kind of thing is definitely not very desirable.  Can you find a way
to keep the output package names the same?

p.
Richard Purdie - June 8, 2011, 3:35 p.m.
On Wed, 2011-06-08 at 10:36 +0100, Phil Blundell wrote:
> On Wed, 2011-06-08 at 17:08 +0800, Dongxiao Xu wrote:
> > *libc's do_package will cost a lot of time due to the locale handing,
> > which may delay the other recipe's do_package task and affect the build
> > performance.
> > 
> > This commit moves locale handling into a separate recipe *libc-locale.
> 
> Can you quantify the effect on build performance a bit?  If I understand
> correctly, you're basically saying that the goal is to increase
> parallelism.  Does that cause reduced performance for people running
> with few threads?

One of the side effects of debian.bbclass is that it requires all
dependencies to do_package before any package itself can do_package so
any renaming of dependencies can be accounted for.

There is an issue if do_package for libc takes an age as it holds up any
other tasks from writing out packages. Pretty much most things depend on
libc.

This patch therefore splits it into two stages and means that packaging
of things depending on libc can happen sooner thereby increasing the
potential parallelism of the packaging stages of builds.

There is a very clear step on the "bootchart" graphs of builds I made
showing this.

I'm not sure how this would reduce performance of builds of a few
threads, it should just make better use of any available "spare"
processing capacity throughout the build.

Cheers,

Richard
Phil Blundell - June 9, 2011, 11:14 a.m.
On Wed, 2011-06-08 at 16:35 +0100, Richard Purdie wrote:
> I'm not sure how this would reduce performance of builds of a few
> threads, it should just make better use of any available "spare"
> processing capacity throughout the build.

If I'm reading the patch right, it does involve a certain amount of
extra copying around of the locale-related files since they need to be
stashed away in some place where libc-locale.bb can find them later.  I
don't think these files are especially big, but there are quite a few of
them (which is the whole reason that libc's do_package was slow in the
first place). 

I must admit that I'm slightly surprised that libc's packaging stage is
becoming a bottleneck for anything other than the very smallest builds.
If there's any substantial amount of other material being compiled then
I would expect that there would be enough do_compile() work to keep the
other threads busy until libc was done packaging.  That's not to say
that I think this change is a bad idea, but I would be interested to
know what the actual impact is for real-world workloads.

And, just on a point of principle, any time we're making a chance for
performance-related reasons I think we should always have measurements
to back it up. 

p.
Richard Purdie - June 9, 2011, 11:29 a.m.
On Thu, 2011-06-09 at 12:14 +0100, Phil Blundell wrote:
> On Wed, 2011-06-08 at 16:35 +0100, Richard Purdie wrote:
> > I'm not sure how this would reduce performance of builds of a few
> > threads, it should just make better use of any available "spare"
> > processing capacity throughout the build.
> 
> If I'm reading the patch right, it does involve a certain amount of
> extra copying around of the locale-related files since they need to be
> stashed away in some place where libc-locale.bb can find them later.  I
> don't think these files are especially big, but there are quite a few of
> them (which is the whole reason that libc's do_package was slow in the
> first place). 
> 
> I must admit that I'm slightly surprised that libc's packaging stage is
> becoming a bottleneck for anything other than the very smallest builds.
> If there's any substantial amount of other material being compiled then
> I would expect that there would be enough do_compile() work to keep the
> other threads busy until libc was done packaging.  That's not to say
> that I think this change is a bad idea, but I would be interested to
> know what the actual impact is for real-world workloads.
> 
> And, just on a point of principle, any time we're making a chance for
> performance-related reasons I think we should always have measurements
> to back it up. 

Totally agreed and this decision is being made on real world data even
if its perhaps not clearly being presented here.

$ cat buildstats/core-image-sato-qemux86/201106030912/eglibc-2.13-r1+svnr13356/do_package | grep time
eglibc-2.13-r1+svnr13356: do_package: Elapsed time: 828.01 seconds

As you can see, eglibc do_package takes about 14 minutes which is about
14% of our build time. That is a long time to block pretty much all
packaging activity, particularly if you have access to something with
several cores. When it does complete, even on my 4 core system you see a
"stampeding herd" of packaging happening on the build charts suggesting
a backlog does build up.

For those reasons I do think its a reasonable change.

Cheers,

Richard
Phil Blundell - June 9, 2011, 11:43 a.m.
On Thu, 2011-06-09 at 12:29 +0100, Richard Purdie wrote:
> As you can see, eglibc do_package takes about 14 minutes which is about
> 14% of our build time. That is a long time to block pretty much all
> packaging activity, particularly if you have access to something with
> several cores. When it does complete, even on my 4 core system you see a
> "stampeding herd" of packaging happening on the build charts suggesting
> a backlog does build up.

Yeah, I can imagine that a backlog of packaging activity does build up.
The thing I'm not entirely clear on is whether this is actually causing
some threads to get starved of work (and hence the total build time to
be longer than it needs to be) or whether we're really just shifting
things around in the timeline without making much/any difference to the
overall build duration.  (I'm not familiar enough with bitbake's
scheduler to know whether it will schedule tasks as early as possible,
or as late as possible, or something else.)

Just as a matter of interest, are you using qemu-based locale generation
or the cross localedef for your measurement?  14 minutes does sound like
an awfully long time and I wonder whether there is anything we could do
in absolute terms to just speed that process up.

p.
Richard Purdie - June 9, 2011, 1:15 p.m.
On Thu, 2011-06-09 at 12:43 +0100, Phil Blundell wrote:
> On Thu, 2011-06-09 at 12:29 +0100, Richard Purdie wrote:
> > As you can see, eglibc do_package takes about 14 minutes which is about
> > 14% of our build time. That is a long time to block pretty much all
> > packaging activity, particularly if you have access to something with
> > several cores. When it does complete, even on my 4 core system you see a
> > "stampeding herd" of packaging happening on the build charts suggesting
> > a backlog does build up.
> 
> Yeah, I can imagine that a backlog of packaging activity does build up.
> The thing I'm not entirely clear on is whether this is actually causing
> some threads to get starved of work (and hence the total build time to
> be longer than it needs to be) or whether we're really just shifting
> things around in the timeline without making much/any difference to the
> overall build duration.

It will certainly be a net win on large core systems which I know of
people running builds on as task starvation happens there. Last time I
tested this on a 4 core I think we saw a couple of minutes improvement
in build time so it appears beneficial there too.

>   (I'm not familiar enough with bitbake's
> scheduler to know whether it will schedule tasks as early as possible,
> or as late as possible, or something else.)

Roughly speaking, the default scheduler runs tasks with the most things
depending upon them first. This means it will fill any "spare" threads
with lower priority tasks such as packaging.

> Just as a matter of interest, are you using qemu-based locale generation
> or the cross localedef for your measurement?  14 minutes does sound like
> an awfully long time and I wonder whether there is anything we could do
> in absolute terms to just speed that process up.

I'm using cross localedef. Its faster than with qemu but still slow. I'm
very open to suggestions on how to speed it up as it does take an age.

Cheers,

Richard
Richard Purdie - June 9, 2011, 1:51 p.m.
On Thu, 2011-06-09 at 14:15 +0100, Richard Purdie wrote:
> On Thu, 2011-06-09 at 12:43 +0100, Phil Blundell wrote:
> > On Thu, 2011-06-09 at 12:29 +0100, Richard Purdie wrote:
> > > As you can see, eglibc do_package takes about 14 minutes which is about
> > > 14% of our build time. That is a long time to block pretty much all
> > > packaging activity, particularly if you have access to something with
> > > several cores. When it does complete, even on my 4 core system you see a
> > > "stampeding herd" of packaging happening on the build charts suggesting
> > > a backlog does build up.
> > 
> > Yeah, I can imagine that a backlog of packaging activity does build up.
> > The thing I'm not entirely clear on is whether this is actually causing
> > some threads to get starved of work (and hence the total build time to
> > be longer than it needs to be) or whether we're really just shifting
> > things around in the timeline without making much/any difference to the
> > overall build duration.
> 
> It will certainly be a net win on large core systems which I know of
> people running builds on as task starvation happens there. Last time I
> tested this on a 4 core I think we saw a couple of minutes improvement
> in build time so it appears beneficial there too.

Just as another data point, I'm running builds on my 4 core machine here
and it is scheduling many do_package tasks including long running ones
such as perl's in parallel with eglibc-locale's do_package task. It
would appear the bitbake scheduler believes there is benefit in
splitting these out at least.

Cheers,

Richard
Koen Kooi - June 9, 2011, 1:53 p.m.
Op 9 jun 2011, om 15:15 heeft Richard Purdie het volgende geschreven:

> On Thu, 2011-06-09 at 12:43 +0100, Phil Blundell wrote:
>> On Thu, 2011-06-09 at 12:29 +0100, Richard Purdie wrote:
>>> As you can see, eglibc do_package takes about 14 minutes which is about
>>> 14% of our build time. That is a long time to block pretty much all
>>> packaging activity, particularly if you have access to something with
>>> several cores. When it does complete, even on my 4 core system you see a
>>> "stampeding herd" of packaging happening on the build charts suggesting
>>> a backlog does build up.
>> 
>> Yeah, I can imagine that a backlog of packaging activity does build up.
>> The thing I'm not entirely clear on is whether this is actually causing
>> some threads to get starved of work (and hence the total build time to
>> be longer than it needs to be) or whether we're really just shifting
>> things around in the timeline without making much/any difference to the
>> overall build duration.
> 
> It will certainly be a net win on large core systems which I know of
> people running builds on as task starvation happens there. Last time I
> tested this on a 4 core I think we saw a couple of minutes improvement
> in build time so it appears beneficial there too.
> 
>>  (I'm not familiar enough with bitbake's
>> scheduler to know whether it will schedule tasks as early as possible,
>> or as late as possible, or something else.)
> 
> Roughly speaking, the default scheduler runs tasks with the most things
> depending upon them first. This means it will fill any "spare" threads
> with lower priority tasks such as packaging.
> 
>> Just as a matter of interest, are you using qemu-based locale generation
>> or the cross localedef for your measurement?  14 minutes does sound like
>> an awfully long time and I wonder whether there is anything we could do
>> in absolute terms to just speed that process up.
> 
> I'm using cross localedef. Its faster than with qemu but still slow. I'm
> very open to suggestions on how to speed it up as it does take an age.

Does cross localegen obey parallel make or does it only do a single locale at a time?

regards,

Koen
Phil Blundell - June 9, 2011, 1:55 p.m.
On Thu, 2011-06-09 at 15:53 +0200, Koen Kooi wrote:
> Does cross localegen obey parallel make or does it only do a single locale at a time?

Just one at a time.  It's run out of Python code in libc-package, not by
make, so there is no completely straightforward way to get it to respect
PARALLEL_MAKE.  I guess you could teach libc-package how to generate a
makefile and then run it; that'd probably be the easiest way to
parallelize that particular process.

p.

Patch

diff --git a/meta/classes/libc-common.bbclass b/meta/classes/libc-common.bbclass
new file mode 100644
index 0000000..bae0ace
--- /dev/null
+++ b/meta/classes/libc-common.bbclass
@@ -0,0 +1,23 @@ 
+do_install() {
+	oe_runmake install_root=${D} install
+	for r in ${rpcsvc}; do
+		h=`echo $r|sed -e's,\.x$,.h,'`
+		install -m 0644 ${S}/sunrpc/rpcsvc/$h ${D}/${includedir}/rpcsvc/
+	done
+	install -m 0644 ${WORKDIR}/etc/ld.so.conf ${D}/${sysconfdir}/
+	install -d ${D}${libdir}/locale
+	make -f ${WORKDIR}/generate-supported.mk IN="${S}/localedata/SUPPORTED" OUT="${WORKDIR}/SUPPORTED"
+	# get rid of some broken files...
+	for i in ${GLIBC_BROKEN_LOCALES}; do
+		grep -v $i ${WORKDIR}/SUPPORTED > ${WORKDIR}/SUPPORTED.tmp
+		mv ${WORKDIR}/SUPPORTED.tmp ${WORKDIR}/SUPPORTED
+	done
+	rm -f ${D}{sysconfdir}/rpc
+	rm -rf ${D}${datadir}/zoneinfo
+	rm -rf ${D}${libexecdir}/getconf
+}
+
+def get_libc_fpu_setting(bb, d):
+    if bb.data.getVar('TARGET_FPU', d, 1) in [ 'soft' ]:
+        return "--without-fp"
+    return ""
diff --git a/meta/classes/libc-package.bbclass b/meta/classes/libc-package.bbclass
index 55e3d48..5737af4 100644
--- a/meta/classes/libc-package.bbclass
+++ b/meta/classes/libc-package.bbclass
@@ -37,11 +37,6 @@  python __anonymous () {
                 break
 }
 
-def get_libc_fpu_setting(bb, d):
-    if bb.data.getVar('TARGET_FPU', d, 1) in [ 'soft' ]:
-        return "--without-fp"
-    return ""
-
 OVERRIDES_append = ":${TARGET_ARCH}-${TARGET_OS}"
 
 do_configure_prepend() {
@@ -84,25 +79,6 @@  rm -rf ${TMP_LOCALE}
 }
 
 
-do_install() {
-	oe_runmake install_root=${D} install
-	for r in ${rpcsvc}; do
-		h=`echo $r|sed -e's,\.x$,.h,'`
-		install -m 0644 ${S}/sunrpc/rpcsvc/$h ${D}/${includedir}/rpcsvc/
-	done
-	install -m 0644 ${WORKDIR}/etc/ld.so.conf ${D}/${sysconfdir}/
-	install -d ${D}${libdir}/locale
-	make -f ${WORKDIR}/generate-supported.mk IN="${S}/localedata/SUPPORTED" OUT="${WORKDIR}/SUPPORTED"
-	# get rid of some broken files...
-	for i in ${GLIBC_BROKEN_LOCALES}; do
-		grep -v $i ${WORKDIR}/SUPPORTED > ${WORKDIR}/SUPPORTED.tmp
-		mv ${WORKDIR}/SUPPORTED.tmp ${WORKDIR}/SUPPORTED
-	done
-	rm -f ${D}${sysconfdir}/rpc
-	rm -rf ${D}${datadir}/zoneinfo
-	rm -rf ${D}${libexecdir}/getconf
-}
-
 TMP_LOCALE="/tmp/locale${libdir}/locale"
 
 do_prep_locale_tree() {
diff --git a/meta/conf/distro/include/tclibc-eglibc.inc b/meta/conf/distro/include/tclibc-eglibc.inc
index a4c648e..8997d9e 100644
--- a/meta/conf/distro/include/tclibc-eglibc.inc
+++ b/meta/conf/distro/include/tclibc-eglibc.inc
@@ -10,7 +10,7 @@  TARGET_OS_powerpc = "linux${@['','-gnuspe'][bb.data.getVar('BASE_PACKAGE_ARCH',d
 # Add glibc overrides to the overrides for eglibc.
 OVERRIDES .= ":libc-glibc"
 
-PREFERRED_PROVIDER_virtual/libiconv ?= "eglibc"
+PREFERRED_PROVIDER_virtual/libiconv ?= "eglibc-locale"
 PREFERRED_PROVIDER_virtual/libiconv-nativesdk ?= "eglibc-nativesdk"
 PREFERRED_PROVIDER_virtual/libintl ?= "eglibc"
 PREFERRED_PROVIDER_virtual/libc ?= "eglibc"
@@ -24,10 +24,10 @@  LIBC_DEPENDENCIES = "libsegfault \
     		     eglibc-dev \
     		     eglibc-utils \
     		     eglibc-thread-db \
-    		     eglibc-localedata-i18n \
-    		     eglibc-gconv-ibm850 \
-    		     eglibc-gconv-cp1252 \
-    		     eglibc-gconv-iso8859-1 \
-    		     eglibc-gconv-iso8859-15 \
+    		     eglibc-locale-localedata-i18n \
+    		     eglibc-locale-gconv-ibm850 \
+    		     eglibc-locale-gconv-cp1252 \
+    		     eglibc-locale-gconv-iso8859-1 \
+    		     eglibc-locale-gconv-iso8859-15 \
     		     locale-base-en-us \
     		     locale-base-en-gb "
diff --git a/meta/conf/distro/include/tclibc-glibc.inc b/meta/conf/distro/include/tclibc-glibc.inc
index e5843b0..56516b4 100644
--- a/meta/conf/distro/include/tclibc-glibc.inc
+++ b/meta/conf/distro/include/tclibc-glibc.inc
@@ -10,7 +10,7 @@  TARGET_OS_powerpc = "linux${@['','-gnuspe'][bb.data.getVar('BASE_PACKAGE_ARCH',d
 # Add glibc to the overrides.
 OVERRIDES =. "libc-glibc:"
 
-PREFERRED_PROVIDER_virtual/libiconv ?= "glibc"
+PREFERRED_PROVIDER_virtual/libiconv ?= "glibc-locale"
 PREFERRED_PROVIDER_virtual/libiconv-nativesdk ?= "glibc-nativesdk"
 PREFERRED_PROVIDER_virtual/libintl ?= "glibc"
 PREFERRED_PROVIDER_virtual/libc ?= "glibc"
@@ -25,10 +25,10 @@  LIBC_DEPENDENCIES = "\
     glibc-dev \
     glibc-utils \
     glibc-thread-db \
-    glibc-localedata-i18n \
-    glibc-gconv-ibm850 \
-    glibc-gconv-cp1252 \
-    glibc-gconv-iso8859-1 \
-    glibc-gconv-iso8859-15 \
+    glibc-locale-localedata-i18n \
+    glibc-locale-gconv-ibm850 \
+    glibc-locale-gconv-cp1252 \
+    glibc-locale-gconv-iso8859-1 \
+    glibc-locale-gconv-iso8859-15 \
     locale-base-en-gb \
     "
diff --git a/meta/recipes-core/eglibc/eglibc-initial_2.13.bb b/meta/recipes-core/eglibc/eglibc-initial_2.13.bb
index 787c762..b934f5b 100644
--- a/meta/recipes-core/eglibc/eglibc-initial_2.13.bb
+++ b/meta/recipes-core/eglibc/eglibc-initial_2.13.bb
@@ -1,6 +1,10 @@ 
 require eglibc_${PV}.bb
 require eglibc-initial.inc
 
+do_install_locale() {
+	:
+}
+
 do_configure_prepend () {
         unset CFLAGS
 }
diff --git a/meta/recipes-core/eglibc/eglibc-locale_2.13.bb b/meta/recipes-core/eglibc/eglibc-locale_2.13.bb
new file mode 100644
index 0000000..64eb9e7
--- /dev/null
+++ b/meta/recipes-core/eglibc/eglibc-locale_2.13.bb
@@ -0,0 +1,56 @@ 
+INHIBIT_DEFAULT_DEPS = "1"
+LICENSE = "LGPL"
+
+do_fetch[noexec] = "1"
+do_unpack[noexec] = "1"
+do_patch[noexec] = "1"
+do_configure[noexec] = "1"
+do_compile[noexec] = "1"
+
+# Binary locales are generated at build time if ENABLE_BINARY_LOCALE_GENERATION
+# is set. The idea is to avoid running localedef on the target (at first boot)
+# to decrease initial boot time and avoid localedef being killed by the OOM
+# killer which used to effectively break i18n on machines with < 128MB RAM.
+
+# default to disabled 
+ENABLE_BINARY_LOCALE_GENERATION ?= "0"
+ENABLE_BINARY_LOCALE_GENERATION_pn-eglibc-locale-nativesdk = "0"
+
+#enable locale generation on these arches
+# BINARY_LOCALE_ARCHES is a space separated list of regular expressions
+BINARY_LOCALE_ARCHES ?= "arm.* i[3-6]86 x86_64 powerpc mips"
+
+# set "1" to use cross-localedef for locale generation
+# set "0" for qemu emulation of native localedef for locale generation
+LOCALE_GENERATION_WITH_CROSS-LOCALEDEF = "1"
+
+PR = "r0"
+
+PKGSUFFIX = ""
+PKGSUFFIX_virtclass-nativesdk = "-nativesdk"
+
+PACKAGES = "eglibc-locale localedef${PKGSUFFIX}"
+
+PACKAGES_DYNAMIC = "locale-base-* \
+                    eglibc-locale-gconv-* eglibc-locale-charmap-* eglibc-locale-localedata-* eglibc-locale-binary-localedata-* \
+                    glibc-locale-gconv-*${PKGSUFFIX}  glibc-locale-charmap-*  glibc-locale-localedata-*  glibc-locale-binary-localedata-*"
+
+PROVIDES = "virtual/libiconv"
+
+RPROVIDES_eglibc-locale = "glibc-locale"
+
+FILES_eglibc-locale-gconv = "${libdir}/gconv/*"
+FILES_localedef${PKGSUFFIX} = "${bindir}/localedef"
+
+do_install () {
+	cp -fpPR ${STAGING_INCDIR}/eglibc-locale-internal-${MULTIMACH_TARGET_SYS}/* ${D}
+	cp -fpPR ${D}/SUPPORTED ${WORKDIR}
+}
+
+DESCRIPTION_localedef = "eglibc: compile locale definition files"
+
+inherit libc-package
+
+do_install[depends] += "virtual/libc${PKGSUFFIX}:do_populate_sysroot"
+
+BBCLASSEXTEND = "nativesdk"
diff --git a/meta/recipes-core/eglibc/eglibc-package.inc b/meta/recipes-core/eglibc/eglibc-package.inc
index 76ebdc3..63a1de9 100644
--- a/meta/recipes-core/eglibc/eglibc-package.inc
+++ b/meta/recipes-core/eglibc/eglibc-package.inc
@@ -14,35 +14,13 @@  python __anonymous () {
                                    bb.data.getVar('TARGET_OS', d, 1))
 }
 
-# Binary locales are generated at build time if ENABLE_BINARY_LOCALE_GENERATION
-# is set. The idea is to avoid running localedef on the target (at first boot)
-# to decrease initial boot time and avoid localedef being killed by the OOM
-# killer which used to effectively break i18n on machines with < 128MB RAM.
-
-# default to disabled 
-ENABLE_BINARY_LOCALE_GENERATION ?= "0"
-ENABLE_BINARY_LOCALE_GENERATION_pn-eglibc-nativesdk = "0"
-
-#enable locale generation on these arches
-# BINARY_LOCALE_ARCHES is a space separated list of regular expressions
-BINARY_LOCALE_ARCHES ?= "arm.* i[3-6]86 x86_64 powerpc mips"
-
-# set "1" to use cross-localedef for locale generation
-# set "0" for qemu emulation of native localedef for locale generation
-LOCALE_GENERATION_WITH_CROSS-LOCALEDEF = "1"
-
 # Set this to zero if you don't want ldconfig in the output package
 USE_LDCONFIG ?= "1"
 
 PKGSUFFIX = ""
 PKGSUFFIX_virtclass-nativesdk = "-nativesdk"
 
-PACKAGES = "${PN}-dbg ${PN} catchsegv${PKGSUFFIX} sln${PKGSUFFIX} nscd${PKGSUFFIX} ldd${PKGSUFFIX} localedef${PKGSUFFIX} libcidn ${PN}-utils ${PN}-pic ${PN}-dev eglibc-doc eglibc-locale libmemusage libsegfault${PKGSUFFIX} eglibc-extra-nss eglibc-thread-db${PKGSUFFIX} eglibc-pcprofile"
-
-PACKAGES_DYNAMIC = " \
-		    glibc-gconv-*${PKGSUFFIX} glibc-charmap-* glibc-localedata-* glibc-binary-localedata-* \
-		    eglibc-gconv-* eglibc-charmap-* eglibc-localedata-* eglibc-binary-localedata-* \
-		    locale-base-*${PKGSUFFIX}"
+PACKAGES = "${PN}-dbg ${PN} catchsegv${PKGSUFFIX} sln${PKGSUFFIX} nscd${PKGSUFFIX} ldd${PKGSUFFIX} ${PN}-utils ${PN}-pic ${PN}-dev eglibc-doc libcidn libmemusage libsegfault${PKGSUFFIX} eglibc-extra-nss eglibc-thread-db${PKGSUFFIX} eglibc-pcprofile"
 
 # Create a eglibc-binaries
 ALLOW_EMPTY_${PN}-binaries = "1"
@@ -69,7 +47,6 @@  RPROVIDES_eglibc-utils = "glibc-utils"
 RPROVIDES_eglibc-pic = "glibc-pic"
 RPROVIDES_eglibc-dev = "glibc-dev"
 RPROVIDES_eglibc-doc = "glibc-doc"
-RPROVIDES_eglibc-locale = "glibc-locale"
 RPROVIDES_eglibc-extra-nss = "glibc-extra-nss"
 RPROVIDES_eglibc-thread-db = "glibc-thread-db"
 RPROVIDES_eglibc-pcprofile = "glibc-pcprofile"
@@ -88,13 +65,11 @@  FILES_eglibc-dev_append += "${bindir}/rpcgen ${libdir}/*.a \
 	${base_libdir}/*.a ${base_libdir}/*.o ${datadir}/aclocal"
 FILES_nscd${PKGSUFFIX} = "${sbindir}/nscd*"
 FILES_eglibc-utils = "${bindir}/* ${sbindir}/*"
-FILES_eglibc-gconv = "${libdir}/gconv/*"
-FILES_${PN}-dbg += "${libexecdir}/*/.debug ${libdir}/gconv/.debug"
+FILES_${PN}-dbg += "${libexecdir}/*/.debug"
 FILES_catchsegv${PKGSUFFIX} = "${bindir}/catchsegv"
 RDEPENDS_catchsegv${PKGSUFFIX} = "libsegfault"
 FILES_eglibc-pcprofile = "${base_libdir}/libpcprofile.so"
 FILES_eglibc-thread-db${PKGSUFFIX} = "${base_libdir}/libthread_db*"
-FILES_localedef${PKGSUFFIX} = "${bindir}/localedef"
 RPROVIDES_eglibc-dev += "libc-dev"
 
 SUMMARY_sln = "The static ln."
@@ -105,9 +80,8 @@  SUMMARY_eglibc-extra-nss = "hesiod, NIS and NIS+ nss libraries"
 DESCRIPTION_eglibc-extra-nss = "eglibc: nis, nisplus and hesiod search services."
 SUMMARY_ldd = "print shared library dependencies"
 DESCRIPTION_ldd = "/usr/bin/ldd prints shared library dependencies for each program or shared library specified on the command line."
-DESCRIPTION_localedef = "eglibc: compile locale definition files"
 SUMMARY_eglibc-utils = "Miscellaneous utilities provided by eglibc"
 DESCRIPTION_eglibc-utils = "Miscellaneous utilities including getconf, iconf, locale, gencat, tzselect, zic, rpcinfo, ..."
 
-inherit libc-package
+inherit libc-common
 
diff --git a/meta/recipes-core/eglibc/eglibc.inc b/meta/recipes-core/eglibc/eglibc.inc
index 46ffa82..0f5721d 100644
--- a/meta/recipes-core/eglibc/eglibc.inc
+++ b/meta/recipes-core/eglibc/eglibc.inc
@@ -26,7 +26,7 @@  DEPENDS = "virtual/${TARGET_PREFIX}gcc-intermediate linux-libc-headers"
 # nptl needs libgcc but dlopens it, so our shlibs code doesn't detect this
 #RDEPENDS_${PN} += "${@['','libgcc']['nptl' in '${GLIBC_ADDONS}']}"
 PROVIDES = "virtual/libc virtual/${TARGET_PREFIX}libc-for-gcc"
-PROVIDES += "virtual/libintl virtual/libiconv"
+PROVIDES += "virtual/libintl"
 inherit autotools
 
 LEAD_SONAME = "libc.so"
@@ -42,4 +42,5 @@  EGLIBCPARALLELISM := "PARALLELMFLAGS="${PARALLEL_MAKE}""
 EXTRA_OEMAKE += ${EGLIBCPARALLELISM}
 PARALLEL_MAKE = ""
 
-PACKAGES = "glibc catchsegv sln nscd ldd localedef glibc-utils glibc-dev glibc-doc glibc-locale libsegfault glibc-extra-nss glibc-thread-db glibc-pcprofile"
+PACKAGES = "glibc catchsegv sln nscd ldd glibc-utils glibc-dev glibc-doc libsegfault glibc-extra-nss glibc-thread-db glibc-pcprofile"
+
diff --git a/meta/recipes-core/eglibc/eglibc_2.13.bb b/meta/recipes-core/eglibc/eglibc_2.13.bb
index c77bf3c..4d43044 100644
--- a/meta/recipes-core/eglibc/eglibc_2.13.bb
+++ b/meta/recipes-core/eglibc/eglibc_2.13.bb
@@ -110,6 +110,18 @@  do_compile () {
 	)
 }
 
+do_install_locale () {
+	dest=${D}/${includedir}/eglibc-locale-internal-${MULTIMACH_TARGET_SYS}
+	install -d ${dest} ${dest}${bindir}
+	cp -fpPR ${D}${base_libdir} ${dest}${base_prefix}
+	cp -fpPR ${D}${libdir} ${dest}${exec_prefix}
+	cp -fpPR ${D}${datadir} ${dest}${exec_prefix}
+	cp -fpPR ${D}${bindir}/localedef ${dest}${bindir}
+	cp -fpPR ${WORKDIR}/SUPPORTED ${dest}
+}
+
+addtask do_install_locale after do_install before do_populate_sysroot
+
 require eglibc-package.inc
 
 BBCLASSEXTEND = "nativesdk"
diff --git a/meta/recipes-core/glibc/glibc-initial_2.10.1.bb b/meta/recipes-core/glibc/glibc-initial_2.10.1.bb
index d662970..bc3d7e4 100644
--- a/meta/recipes-core/glibc/glibc-initial_2.10.1.bb
+++ b/meta/recipes-core/glibc/glibc-initial_2.10.1.bb
@@ -1,6 +1,10 @@ 
 require glibc_${PV}.bb
 require glibc-initial.inc
 
+do_install_locale() {
+	:
+}
+
 do_configure_prepend () {
 	unset CFLAGS
 }
diff --git a/meta/recipes-core/glibc/glibc-locale_2.10.1.bb b/meta/recipes-core/glibc/glibc-locale_2.10.1.bb
new file mode 100644
index 0000000..719cd3b
--- /dev/null
+++ b/meta/recipes-core/glibc/glibc-locale_2.10.1.bb
@@ -0,0 +1,48 @@ 
+#INHIBIT_DEFAULT_DEPS = "1"
+LICENSE = "LGPL"
+
+do_fetch[noexec] = "1"
+do_unpack[noexec] = "1"
+do_patch[noexec] = "1"
+do_configure[noexec] = "1"
+do_compile[noexec] = "1"
+
+# Binary locales are generated at build time if ENABLE_BINARY_LOCALE_GENERATION
+# is set. The idea is to avoid running localedef on the target (at first boot)
+# to decrease initial boot time and avoid localedef being killed by the OOM
+# killer which used to effectively break i18n on machines with < 128MB RAM.
+
+# default to disabled 
+ENABLE_BINARY_LOCALE_GENERATION ?= "0"
+ENABLE_BINARY_LOCALE_GENERATION_pn-glibc-locale-nativesdk = "0"
+
+#enable locale generation on these arches
+# BINARY_LOCALE_ARCHES is a space separated list of regular expressions
+BINARY_LOCALE_ARCHES ?= "arm.* i[3-6]86 x86_64 powerpc mips"
+
+# set "1" to use cross-localedef for locale generation
+# set "0" for qemu emulation of native localedef for locale generation
+LOCALE_GENERATION_WITH_CROSS-LOCALEDEF = "1"
+
+PR = "r0"
+
+PKGSUFFIX = ""
+PKGSUFFIX_virtclass-nativesdk = "-nativesdk"
+
+PACKAGES = "glibc-locale localedef${PKGSUFFIX}"
+
+PACKAGES_DYNAMIC = "locale-base-* \
+                    glibc-locale-gconv-*${PKGSUFFIX}  glibc-locale-charmap-*  glibc-locale-localedata-*  glibc-locale-binary-localedata-*"
+
+
+PROVIDES = "virtual/libiconv"
+
+FILES_localedef${PKGSUFFIX} = "${bindir}/localedef"
+
+do_install () {
+	cp -fpPR ${STAGING_INCDIR}/glibc-locale-internal-${MULTIMACH_TARGET_SYS}/* ${D}
+	cp -fpPR ${D}/SUPPORTED ${WORKDIR}
+}
+
+do_install[depends] += "virtual/libc:do_populate_sysroot"
+do_install_virtclass-nativesdk[depends] += "virtual/libc-nativesdk:do_populate_sysroot"
diff --git a/meta/recipes-core/glibc/glibc-package.inc b/meta/recipes-core/glibc/glibc-package.inc
index 27dc427..4ea601c 100644
--- a/meta/recipes-core/glibc/glibc-package.inc
+++ b/meta/recipes-core/glibc/glibc-package.inc
@@ -14,34 +14,14 @@  python __anonymous () {
                                    bb.data.getVar('TARGET_OS', d, 1))
 }
 
-
-# Binary locales are generated at build time if ENABLE_BINARY_LOCALE_GENERATION
-# is set. The idea is to avoid running localedef on the target (at first boot)
-# to decrease initial boot time and avoid localedef being killed by the OOM
-# killer which used to effectively break i18n on machines with < 128MB RAM.
-
-# default to disabled until qemu works for everyone
-ENABLE_BINARY_LOCALE_GENERATION ?= "0"
-ENABLE_BINARY_LOCALE_GENERATION_pn-glibc-nativesdk = "0"
-
-# BINARY_LOCALE_ARCHES is a space separated list of regular expressions
-BINARY_LOCALE_ARCHES ?= "arm.* i[3-6]86 x86_64 powerpc mips"
-
-# set "1" to use cross-localedef for locale generation
-# set "0" for qemu emulation of native localedef for locale generation
-LOCALE_GENERATION_WITH_CROSS-LOCALEDEF = "1"
-
 PKGSUFFIX = ""
 PKGSUFFIX_virtclass-nativesdk = "-nativesdk"
 
 PACKAGES = "${PN}-dbg ${PN} catchsegv${PKGSUFFIX} sln${PKGSUFFIX} nscd${PKGSUFFIX} \
-	ldd${PKGSUFFIX} localedef${PKGSUFFIX} ${PN}-utils ${PN}-dev ${PN}-doc \
-	${PN}-locale libsegfault${PKGSUFFIX} ${PN}-extra-nss ${PN}-thread-db \
+	ldd${PKGSUFFIX} ${PN}-utils ${PN}-dev ${PN}-doc \
+	libsegfault${PKGSUFFIX} ${PN}-extra-nss ${PN}-thread-db \
 	${PN}-pcprofile"
 
-PACKAGES_DYNAMIC = "glibc-gconv-*${PKGSUFFIX} glibc-charmap-* glibc-localedata-* locale-base-* \
-	glibc-binary-localedata-*"
-
 libc_baselibs = "${base_libdir}/libc* ${base_libdir}/libm* ${base_libdir}/ld* \
 	${base_libdir}/libpthread* ${base_libdir}/libresolv* ${base_libdir}/librt* \
 	${base_libdir}/libutil* ${base_libdir}/libnsl* ${base_libdir}/libnss_files* \
@@ -72,12 +52,11 @@  FILES_glibc-dbg = "${glibcdbgfiles}"
 FILES_${PN}-dbg = "${glibcdbgfiles}"
 FILES_nscd${PKGSUFFIX} = "${sbindir}/nscd* ${sysconfdir}/nscd* ${sysconfdir}/init.d/nscd*"
 FILES_glibc-utils = "${bindir}/* ${sbindir}/*"
-FILES_glibc-gconv = "${libdir}/gconv/*"
+FILES_glibc-locale-gconv = "${libdir}/gconv/*"
 FILES_catchsegv${PKGSUFFIX} = "${bindir}/catchsegv"
 RDEPENDS_catchsegv${PKGSUFFIX} = "libsegfault"
 FILES_glibc-pcprofile = "${base_libdir}/libpcprofile.so"
 FILES_glibc-thread-db = "${base_libdir}/libthread_db*"
-FILES_localedef${PKGSUFFIX} = "${bindir}/localedef"
 RPROVIDES_glibc-dev += "libc-dev"
 
 SUMMARY_sln = "The static ln."
@@ -88,10 +67,10 @@  SUMMARY_glibc-extra-nss = "hesiod, NIS and NIS+ nss libraries"
 DESCRIPTION_glibc-extra-nss = "glibc: nis, nisplus and hesiod search services.
 SUMMARY_ldd = "print shared library dependencies"
 DESCRIPTION_ldd = "/usr/bin/ldd prints shared library dependencies for each program or shared library specified on the command line."
-DESCRIPTION_localedef = "glibc: compile locale definition files"
 SUMMARY_glibc-utils = "Miscellaneous utilities provided by glibc"
 DESCRIPTION_glibc-utils = "Miscellaneous utilities including getconf, iconf, locale, gencat, tzselect, zic, rpcinfo, ..."
 
 EXTRA_OEMAKE += "rootsbindir=${base_sbindir}"
 
-inherit libc-package
+inherit libc-common
+
diff --git a/meta/recipes-core/glibc/glibc.inc b/meta/recipes-core/glibc/glibc.inc
index 127ae23..a051161 100644
--- a/meta/recipes-core/glibc/glibc.inc
+++ b/meta/recipes-core/glibc/glibc.inc
@@ -15,7 +15,7 @@  RDEPENDS_${PN}-dev = "linux-libc-headers-dev"
 #RDEPENDS_ldd += " bash"
 #RDEPENDS_${PN} += "${@['','libgcc']['nptl' in '${GLIBC_ADDONS}']}"
 PROVIDES = "virtual/libc virtual/${TARGET_PREFIX}libc-for-gcc"
-PROVIDES += "virtual/libintl virtual/libiconv"
+PROVIDES += "virtual/libintl"
 
 STAGINGCC = "gcc-cross-intermediate"
 STAGINGCC_virtclass-nativesdk = "gcc-crosssdk-intermediate"
diff --git a/meta/recipes-core/glibc/glibc_2.10.1.bb b/meta/recipes-core/glibc/glibc_2.10.1.bb
index 33ebe24..a2e7b83 100644
--- a/meta/recipes-core/glibc/glibc_2.10.1.bb
+++ b/meta/recipes-core/glibc/glibc_2.10.1.bb
@@ -154,6 +154,18 @@  do_compile () {
 	)
 }
 
+do_install_locale () {
+	dest=${D}/${includedir}/glibc-locale-internal-${MULTIMACH_TARGET_SYS}
+	install -d ${dest} ${dest}${bindir}
+	cp -fpPR ${D}${base_libdir} ${dest}${base_prefix}
+	cp -fpPR ${D}${libdir} ${dest}${exec_prefix}
+	cp -fpPR ${D}${datadir} ${dest}${exec_prefix}
+	cp -fpPR ${D}${bindir}/localedef ${dest}${bindir}
+	cp -fpPR ${WORKDIR}/SUPPORTED ${dest}
+}
+
+addtask do_install_locale after do_install before do_populate_sysroot
+
 require glibc-stage.inc
 
 require glibc-package.inc
diff --git a/meta/recipes-core/meta/external-csl-toolchain_2008q3-72.bb b/meta/recipes-core/meta/external-csl-toolchain_2008q3-72.bb
index f714928..8630467 100644
--- a/meta/recipes-core/meta/external-csl-toolchain_2008q3-72.bb
+++ b/meta/recipes-core/meta/external-csl-toolchain_2008q3-72.bb
@@ -17,7 +17,7 @@  PROVIDES = "\
 	glibc-thread-db \
 	virtual/linux-libc-headers "
 RPROVIDES = "glibc-utils libsegfault glibc-thread-db"
-PACKAGES_DYNAMIC = "glibc-gconv-*"
+PACKAGES_DYNAMIC = "glibc-locale-gconv-*"
 PR = "r1"
 
 #SRC_URI = "http://www.codesourcery.com/public/gnu_toolchain/arm-none-linux-gnueabi/arm-${PV}-arm-none-linux-gnueabi-i686-pc-linux-gnu.tar.bz2"
@@ -40,6 +40,7 @@  do_install() {
 
 GLIBC_INTERNAL_USE_BINARY_LOCALE ?= "compile"
 
+inherit libc-common
 inherit libc-package
 
 PACKAGES += "libgcc libgcc-dev libstdc++ libstdc++-dev linux-libc-headers"
diff --git a/meta/recipes-core/meta/external-poky-toolchain.bb b/meta/recipes-core/meta/external-poky-toolchain.bb
index 5507746..e812209 100644
--- a/meta/recipes-core/meta/external-poky-toolchain.bb
+++ b/meta/recipes-core/meta/external-poky-toolchain.bb
@@ -10,7 +10,7 @@  PROVIDES = "\
     virtual/libiconv \
     virtual/linux-libc-headers "
 RPROVIDES = "glibc-utils libsegfault glibc-thread-db libgcc-dev libstdc++-dev libstdc++"
-PACKAGES_DYNAMIC = "glibc-gconv-*"
+PACKAGES_DYNAMIC = "glibc-locale-gconv-*"
 INHIBIT_DEFAULT_DEPS = "1"
 PR = "r1"
 
diff --git a/meta/recipes-gnome/gtk+/gtk+.inc b/meta/recipes-gnome/gtk+/gtk+.inc
index 9e2523a..22d36b7 100644
--- a/meta/recipes-gnome/gtk+/gtk+.inc
+++ b/meta/recipes-gnome/gtk+/gtk+.inc
@@ -37,7 +37,7 @@  FILES_${PN}-dbg += " \
 	${libdir}/gtk-2.0/${LIBV}/printbackends/.debug/*"
 
 GTKBASE_RRECOMMENDS ?= "liberation-fonts gdk-pixbuf-loader-png gdk-pixbuf-loader-jpeg gdk-pixbuf-loader-gif gdk-pixbuf-loader-xpm shared-mime-info"
-GTKGLIBC_RRECOMMENDS ?= "${GTKBASE_RRECOMMENDS} glibc-gconv-iso8859-1"
+GTKGLIBC_RRECOMMENDS ?= "${GTKBASE_RRECOMMENDS} glibc-locale-gconv-iso8859-1"
 
 RRECOMMENDS_${PN} = "${GTKBASE_RRECOMMENDS}"
 RRECOMMENDS_${PN}_libc-glibc = "${GTKGLIBC_RRECOMMENDS}"