Patchwork [2/2] tclibc-uclibc.inc: Append -uclibc only to target recipes

login
mail settings
Submitter Khem Raj
Date May 18, 2011, 7:49 a.m.
Message ID <20110518074909.GA31650@sakrah.homelinux.org>
Download mbox | patch
Permalink /patch/4289/
State New, archived
Headers show

Comments

Khem Raj - May 18, 2011, 7:49 a.m.
On (17/05/11 01:13), Richard Purdie wrote:
> On Sun, 2011-05-15 at 23:04 -0700, Khem Raj wrote:
> > Do not define DEPLOY_DIR_IMAGE
> > Append -uclibc to STAGING_DIR_TARGET only for target recipe and cross
> > recipes
> > Append -uclibc to STAGING_DIR_HOST only for target recipes.
> > 
> > These changes make sure that we still share the native sysroot
> > 
> > Signed-off-by: Khem Raj <raj.khem@gmail.com>
> 
> The more I looked at that patch, the more holes I could see in what we
> were doing (and what Angstrom currently does). I started playing around
> with the patch below which tried to improve on that idea.
> 
> I then concluded that we might be able to do something like:
> 
> MACHINEOVERRIDES := "${MACHINE}"
> MACHINE_append = "-uclibc"
> 
> since what we're really trying to do in the uclibc case is replace
> anything MACHINE specific with something containing uclibc?
> 
> Thoughts?
> 
> Cheers,
> 
> Richard
> 
> 
> diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
> index 553c6a2..354668f 100644
> --- a/meta/classes/sstate.bbclass
> +++ b/meta/classes/sstate.bbclass
> @@ -23,7 +23,7 @@ python () {
>          bb.data.setVar('SSTATE_PKGARCH', bb.data.getVar('BUILD_ARCH', d), d)
>      elif bb.data.inherits_class('cross', d):
>          bb.data.setVar('SSTATE_PKGARCH', bb.data.expand("${BUILD_ARCH}_${BASE_PACKAGE_ARCH}", d), d)
> -        bb.data.setVar('SSTATE_MANMACH', bb.data.expand("${BUILD_ARCH}_${MACHINE}", d), d)
> +        bb.data.setVar('SSTATE_MANMACH', bb.data.expand("${BUILD_ARCH}_${STAGING_MACHNAME}", d), d)
>      elif bb.data.inherits_class('crosssdk', d):
>          bb.data.setVar('SSTATE_PKGARCH', bb.data.expand("${BUILD_ARCH}_${BASE_PACKAGE_ARCH}", d), d)
>      elif bb.data.inherits_class('nativesdk', d):
> @@ -31,7 +31,7 @@ python () {
>      elif bb.data.inherits_class('cross-canadian', d):
>          bb.data.setVar('SSTATE_PKGARCH', bb.data.expand("${SDK_ARCH}_${BASE_PACKAGE_ARCH}", d), d)
>      else:
> -        bb.data.setVar('SSTATE_MANMACH', bb.data.expand("${MACHINE}", d), d)
> +        bb.data.setVar('SSTATE_MANMACH', bb.data.expand("${STAGING_MACHNAME}", d), d)
>  
>      # These classes encode staging paths into their scripts data so can only be
>      # reused if we manipulate the paths
> diff --git a/meta/classes/staging.bbclass b/meta/classes/staging.bbclass
> index fef6457..c2e4e50 100644
> --- a/meta/classes/staging.bbclass
> +++ b/meta/classes/staging.bbclass
> @@ -91,7 +91,7 @@ SSTATETASKS += "do_populate_sysroot"
>  do_populate_sysroot[sstate-name] = "populate-sysroot"
>  do_populate_sysroot[sstate-inputdirs] = "${SYSROOT_DESTDIR}"
>  do_populate_sysroot[sstate-outputdirs] = "${STAGING_DIR_HOST}/"
> -do_populate_sysroot[stamp-extra-info] = "${MACHINE}"
> +do_populate_sysroot[stamp-extra-info] = "${STAGING_MACHNAME}"
>  
>  python do_populate_sysroot_setscene () {
>  	sstate_setscene(d)
> diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
> index 8b6236e..141b942 100644
> --- a/meta/conf/bitbake.conf
> +++ b/meta/conf/bitbake.conf
> @@ -300,7 +300,8 @@ STAGING_DATADIR_NATIVE = "${STAGING_DIR_NATIVE}${datadir_native}"
>  
>  # This should really be MULTIMACH_HOST_SYS but that breaks "all" and machine 
>  # specific packages - hack around it for now.
> -STAGING_DIR_HOST = "${STAGING_DIR}/${MACHINE}"
> +STAGING_MACHNAME ??= "${MACHINE}"
> +STAGING_DIR_HOST = "${STAGING_DIR}/${STAGING_MACHNAME}"
>  STAGING_BINDIR = "${STAGING_DIR_HOST}${bindir}"
>  STAGING_LIBDIR = "${STAGING_DIR_HOST}${libdir}"
>  STAGING_INCDIR = "${STAGING_DIR_HOST}${includedir}"
> @@ -312,7 +313,7 @@ STAGING_PYDIR = "${STAGING_DIR}/lib/python2.4"
>  
>  # This should really be MULTIMACH_TARGET_SYS but that breaks "all" and machine 
>  # specific packages - hack around it for now.
> -STAGING_DIR_TARGET = "${STAGING_DIR}/${MACHINE}"
> +STAGING_DIR_TARGET = "${STAGING_DIR}/${STAGING_MACHNAME}"
>  STAGING_DIR_TCBOOTSTRAP = "${STAGING_DIR_TARGET}-tcbootstrap"
>  
>  # Setting DEPLOY_DIR outside of TMPDIR is helpful, when you are using
> @@ -344,8 +345,8 @@ STAGING_KERNEL_DIR = "${STAGING_DIR_HOST}/kernel"
>  
>  IMAGE_ROOTFS = "${WORKDIR}/rootfs"
>  IMAGE_BASENAME = "${PN}"
> -IMAGE_NAME = "${IMAGE_BASENAME}-${MACHINE}-${DATETIME}"
> -IMAGE_LINK_NAME = "${IMAGE_BASENAME}-${MACHINE}"
> +IMAGE_NAME = "${IMAGE_BASENAME}-${STAGING_MACHNAME}-${DATETIME}"
> +IMAGE_LINK_NAME = "${IMAGE_BASENAME}-${STAGING_MACHNAME}"
>  
>  # This option allows for a precentage overage of the actaul image size rather than a
>  # fixed extra space
> diff --git a/meta/conf/distro/include/tclibc-uclibc.inc b/meta/conf/distro/include/tclibc-uclibc.inc
> index 27f6ec6..e84da81 100644
> --- a/meta/conf/distro/include/tclibc-uclibc.inc
> +++ b/meta/conf/distro/include/tclibc-uclibc.inc
> @@ -20,11 +20,7 @@ CXXFLAGS += "-fvisibility-inlines-hidden"
>  
>  IMAGE_LINGUAS = ""
>  
> -DEPLOY_DIR_IMAGE = "${TMPDIR}/deploy/images"
> -DEPLOY_DIR_append = "-uclibc"
> -STAGING_DIR_TARGET_append = "-uclibc"
> -STAGING_DIR_HOST_append = "-uclibc"
> -SSTATE_MANIFESTS_append = "-uclibc"
> +STAGING_MACHNAME = "${MACHINE}-uclibc"
>  
>  LIBC_DEPENDENCIES = "\
>      uclibc \
> 

I think we need to globalize libc variable like MACHINE or MULTIMACHINE
and we could check if PREFERRED_PROVIDER_virtual/libc == TCLIBC or not
in base.bbclass for sanity. Something like attached patch might be
able to do it
Richard Purdie - May 18, 2011, 7:57 a.m.
On Wed, 2011-05-18 at 00:49 -0700, Khem Raj wrote:
> I think we need to globalize libc variable like MACHINE or MULTIMACHINE
> and we could check if PREFERRED_PROVIDER_virtual/libc == TCLIBC or not
> in base.bbclass for sanity. Something like attached patch might be
> able to do it

Having assessed how many places in the system we'd need to make this
kind of change I'm now of the opinion its too great. I think
manipulating TMPDIR is a better cleaner solution and we should inject
this there...

Cheers,

Richard
Khem Raj - May 18, 2011, 8:12 a.m.
On (18/05/11 08:57), Richard Purdie wrote:
> On Wed, 2011-05-18 at 00:49 -0700, Khem Raj wrote:
> > I think we need to globalize libc variable like MACHINE or MULTIMACHINE
> > and we could check if PREFERRED_PROVIDER_virtual/libc == TCLIBC or not
> > in base.bbclass for sanity. Something like attached patch might be
> > able to do it
> 
> Having assessed how many places in the system we'd need to make this
> kind of change I'm now of the opinion its too great. I think
> manipulating TMPDIR is a better cleaner solution and we should inject
> this there...

hmmm ok. So we delete these definitions from tclibc-uclibc.inc

DEPLOY_DIR_IMAGE = "${TMPDIR}/deploy/images"
DEPLOY_DIR_append = "-uclibc"
STAGING_DIR_TARGET_append = "-uclibc"
STAGING_DIR_HOST_append = "-uclibc"
SSTATE_MANIFESTS_append = "-uclibc"

and append define
TMPDIR ?= "${TOPDIR}/tmp-${TCLIBC}"

should be enough
> 
> Cheers,
> 
> Richard
>
Koen Kooi - May 18, 2011, 8:47 a.m.
Op 18 mei 2011, om 10:12 heeft Khem Raj het volgende geschreven:

> On (18/05/11 08:57), Richard Purdie wrote:
>> On Wed, 2011-05-18 at 00:49 -0700, Khem Raj wrote:
>>> I think we need to globalize libc variable like MACHINE or MULTIMACHINE
>>> and we could check if PREFERRED_PROVIDER_virtual/libc == TCLIBC or not
>>> in base.bbclass for sanity. Something like attached patch might be
>>> able to do it
>> 
>> Having assessed how many places in the system we'd need to make this
>> kind of change I'm now of the opinion its too great. I think
>> manipulating TMPDIR is a better cleaner solution and we should inject
>> this there...
> 
> hmmm ok. So we delete these definitions from tclibc-uclibc.inc
> 
> DEPLOY_DIR_IMAGE = "${TMPDIR}/deploy/images"
> DEPLOY_DIR_append = "-uclibc"
> STAGING_DIR_TARGET_append = "-uclibc"
> STAGING_DIR_HOST_append = "-uclibc"
> SSTATE_MANIFESTS_append = "-uclibc"
> 
> and append define
> TMPDIR ?= "${TOPDIR}/tmp-${TCLIBC}"
> 
> should be enough

Like this: http://patches.openembedded.org/patch/4229/ ?
Frans Meulenbroeks - May 18, 2011, 8:48 a.m.
2011/5/18 Richard Purdie <richard.purdie@linuxfoundation.org>

> On Wed, 2011-05-18 at 00:49 -0700, Khem Raj wrote:
> > I think we need to globalize libc variable like MACHINE or MULTIMACHINE
> > and we could check if PREFERRED_PROVIDER_virtual/libc == TCLIBC or not
> > in base.bbclass for sanity. Something like attached patch might be
> > able to do it
>
> Having assessed how many places in the system we'd need to make this
> kind of change I'm now of the opinion its too great. I think
> manipulating TMPDIR is a better cleaner solution and we should inject
> this there...
>
>
> Note that this introduces  additional build time and space to rebuild
native recipes for the different TMPDIR variants. Something which is not
needed with the solution from Khem, if I properly understand things.

From a conceptual point of view it seems better to mangle the target subdir
name  in work (e.g. ppce300c3-oe-linux-uclibc) or introduce a subdir in
TMPDIR for  (e.g. ppce300c3-oe-linux/uclibc )

(actually I seem to recall that we have or had something in OE as well,
something like deploy/glibc)

Frans
Khem Raj - May 18, 2011, 9:55 a.m.
On Wed, May 18, 2011 at 1:47 AM, Koen Kooi <koen@dominion.thruhere.net> wrote:
>
> Op 18 mei 2011, om 10:12 heeft Khem Raj het volgende geschreven:
>
>> On (18/05/11 08:57), Richard Purdie wrote:
>>> On Wed, 2011-05-18 at 00:49 -0700, Khem Raj wrote:
>>>> I think we need to globalize libc variable like MACHINE or MULTIMACHINE
>>>> and we could check if PREFERRED_PROVIDER_virtual/libc == TCLIBC or not
>>>> in base.bbclass for sanity. Something like attached patch might be
>>>> able to do it
>>>
>>> Having assessed how many places in the system we'd need to make this
>>> kind of change I'm now of the opinion its too great. I think
>>> manipulating TMPDIR is a better cleaner solution and we should inject
>>> this there...
>>
>> hmmm ok. So we delete these definitions from tclibc-uclibc.inc
>>
>> DEPLOY_DIR_IMAGE = "${TMPDIR}/deploy/images"
>> DEPLOY_DIR_append = "-uclibc"
>> STAGING_DIR_TARGET_append = "-uclibc"
>> STAGING_DIR_HOST_append = "-uclibc"
>> SSTATE_MANIFESTS_append = "-uclibc"
>>
>> and append define
>> TMPDIR ?= "${TOPDIR}/tmp-${TCLIBC}"
>>
>> should be enough
>
> Like this: http://patches.openembedded.org/patch/4229/ ?


yes exactly looks good to me.

>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>
Richard Purdie - May 18, 2011, 10:09 a.m.
On Wed, 2011-05-18 at 10:48 +0200, Frans Meulenbroeks wrote:
> 2011/5/18 Richard Purdie <richard.purdie@linuxfoundation.org>
> 
> > On Wed, 2011-05-18 at 00:49 -0700, Khem Raj wrote:
> > > I think we need to globalize libc variable like MACHINE or MULTIMACHINE
> > > and we could check if PREFERRED_PROVIDER_virtual/libc == TCLIBC or not
> > > in base.bbclass for sanity. Something like attached patch might be
> > > able to do it
> >
> > Having assessed how many places in the system we'd need to make this
> > kind of change I'm now of the opinion its too great. I think
> > manipulating TMPDIR is a better cleaner solution and we should inject
> > this there...
> >
> >
> > Note that this introduces  additional build time and space to rebuild
> native recipes for the different TMPDIR variants. Something which is not
> needed with the solution from Khem, if I properly understand things.

I mentioned this elsewhere in the thread but the sstate cache directory
is in TOPDIR, not TMPDIR so its shared between the builds. This means
the -native recipes should not need any additional build time. There is
some disk space usage but this should be minimal and disk space is
comparatively cheap anyway.

> From a conceptual point of view it seems better to mangle the target subdir
> name  in work (e.g. ppce300c3-oe-linux-uclibc) or introduce a subdir in
> TMPDIR for  (e.g. ppce300c3-oe-linux/uclibc )

This turns out to be quite nasty mangling as the deploy directories
don't include this and would need mangling in a certain style, the
sysroots need a different mangling and so does workdir, the cache
directories and so on. What these all have in common is they're in
TMPDIR and the native directories are pretty much the only thing we
don't mangle. The exceptions to the mangling seem to work out quite ugly
as in the cross case we have to "half-mangle" directories too.

> (actually I seem to recall that we have or had something in OE as 
> well, something like deploy/glibc)

Some of it worked ok in OE, some of it didn't and the move to a sysroot
directory per machine was the straw that broke the camel's back so to
speak. We need to take a step back and consider what we're trying to
achieve from an overall architecture perspective and which solution
makes sense.

Cheers,

Richard

Patch

diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
index 23095ec..76642be 100644
--- a/meta/classes/base.bbclass
+++ b/meta/classes/base.bbclass
@@ -179,7 +179,11 @@  python base_eventhandler() {
 				pesteruser.append(v)
 		if pesteruser:
 			bb.fatal('The following variable(s) were not set: %s\nPlease set them directly, or choose a MACHINE or DISTRO that sets them.' % ', '.join(pesteruser))
-
+	if name == "ParseCompleted":
+		libc = bb.data.getVar('TCLIBC', e.data, True)
+		virtual_libc = bb.data.getVar('PREFERRED_PROVIDER_virtual/libc', e.data, True)
+		if not libc or libc != virtual_libc:
+			bb.fatal('PREFERRED_PROVIDER_virtual/libc = %s and TCLIBC = %s, they should be same.' % (virtual_libc, libc))
         if name == "ConfigParsed":
                 generate_git_config(e)
 
diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
index 553c6a2..dbd046a 100644
--- a/meta/classes/sstate.bbclass
+++ b/meta/classes/sstate.bbclass
@@ -23,7 +23,7 @@  python () {
         bb.data.setVar('SSTATE_PKGARCH', bb.data.getVar('BUILD_ARCH', d), d)
     elif bb.data.inherits_class('cross', d):
         bb.data.setVar('SSTATE_PKGARCH', bb.data.expand("${BUILD_ARCH}_${BASE_PACKAGE_ARCH}", d), d)
-        bb.data.setVar('SSTATE_MANMACH', bb.data.expand("${BUILD_ARCH}_${MACHINE}", d), d)
+        bb.data.setVar('SSTATE_MANMACH', bb.data.expand("${BUILD_ARCH}_${MACHINE}_${TCLIBC}", d), d)
     elif bb.data.inherits_class('crosssdk', d):
         bb.data.setVar('SSTATE_PKGARCH', bb.data.expand("${BUILD_ARCH}_${BASE_PACKAGE_ARCH}", d), d)
     elif bb.data.inherits_class('nativesdk', d):
@@ -31,7 +31,7 @@  python () {
     elif bb.data.inherits_class('cross-canadian', d):
         bb.data.setVar('SSTATE_PKGARCH', bb.data.expand("${SDK_ARCH}_${BASE_PACKAGE_ARCH}", d), d)
     else:
-        bb.data.setVar('SSTATE_MANMACH', bb.data.expand("${MACHINE}", d), d)
+        bb.data.setVar('SSTATE_MANMACH', bb.data.expand("${MACHINE}_${TCLIBC}", d), d)
 
     # These classes encode staging paths into their scripts data so can only be
     # reused if we manipulate the paths
diff --git a/meta/classes/staging.bbclass b/meta/classes/staging.bbclass
index fef6457..856efd8 100644
--- a/meta/classes/staging.bbclass
+++ b/meta/classes/staging.bbclass
@@ -92,6 +92,7 @@  do_populate_sysroot[sstate-name] = "populate-sysroot"
 do_populate_sysroot[sstate-inputdirs] = "${SYSROOT_DESTDIR}"
 do_populate_sysroot[sstate-outputdirs] = "${STAGING_DIR_HOST}/"
 do_populate_sysroot[stamp-extra-info] = "${MACHINE}"
+do_populate_sysroot[stamp-extra-info] = "${TCLIBC}"
 
 python do_populate_sysroot_setscene () {
 	sstate_setscene(d)
diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index c292759..6f2d64a 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -300,7 +300,7 @@  STAGING_DATADIR_NATIVE = "${STAGING_DIR_NATIVE}${datadir_native}"
 
 # This should really be MULTIMACH_HOST_SYS but that breaks "all" and machine 
 # specific packages - hack around it for now.
-STAGING_DIR_HOST = "${STAGING_DIR}/${MACHINE}"
+STAGING_DIR_HOST = "${STAGING_DIR}/${MACHINE}-${TCLIBC}"
 STAGING_BINDIR = "${STAGING_DIR_HOST}${bindir}"
 STAGING_LIBDIR = "${STAGING_DIR_HOST}${libdir}"
 STAGING_INCDIR = "${STAGING_DIR_HOST}${includedir}"
@@ -312,7 +312,7 @@  STAGING_PYDIR = "${STAGING_DIR}/lib/python2.4"
 
 # This should really be MULTIMACH_TARGET_SYS but that breaks "all" and machine 
 # specific packages - hack around it for now.
-STAGING_DIR_TARGET = "${STAGING_DIR}/${MACHINE}"
+STAGING_DIR_TARGET = "${STAGING_DIR}/${MACHINE}-${TCLIBC}"
 STAGING_DIR_TCBOOTSTRAP = "${STAGING_DIR_TARGET}-tcbootstrap"
 
 # Setting DEPLOY_DIR outside of TMPDIR is helpful, when you are using
@@ -344,8 +344,8 @@  STAGING_KERNEL_DIR = "${STAGING_DIR_HOST}/kernel"
 
 IMAGE_ROOTFS = "${WORKDIR}/rootfs"
 IMAGE_BASENAME = "${PN}"
-IMAGE_NAME = "${IMAGE_BASENAME}-${MACHINE}-${DATETIME}"
-IMAGE_LINK_NAME = "${IMAGE_BASENAME}-${MACHINE}"
+IMAGE_NAME = "${IMAGE_BASENAME}-${MACHINE}-${TCLIBC}-${DATETIME}"
+IMAGE_LINK_NAME = "${IMAGE_BASENAME}-${MACHINE}-${TCLIBC}"
 
 # This option allows for a precentage overage of the actaul image size rather than a
 # fixed extra space