| Submitter | Steffen Sledz |
|---|---|
| Date | March 11, 2011, 11:04 a.m. |
| Message ID | <1299841477-23487-1-git-send-email-sledz@dresearch.de> |
| Download | mbox | patch |
| Permalink | /patch/1297/ |
| State | Accepted |
| Headers | show |
Comments
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 11-03-11 12:04, Steffen Sledz wrote: > Fixes compile error > > | ./.libs/libnetsnmpmibs.so: undefined reference to `nl_handle_destroy' > | ./.libs/libnetsnmpmibs.so: undefined reference to `nl_handle_alloc' > > which occurred sometimes depending on build order (not in clean package > only builds). What features do we loose when disabling netlink support? I'd rather get the libnl recipe fixed (e.g. static lib only) instead of hacking the dependants. regards, Koen > > Signed-off-by: Steffen Sledz <sledz@dresearch.de> > --- > recipes/net-snmp/net-snmp_svn.bb | 4 ++-- > 1 files changed, 2 insertions(+), 2 deletions(-) > > diff --git a/recipes/net-snmp/net-snmp_svn.bb b/recipes/net-snmp/net-snmp_svn.bb > index 2a75722..c53c91f 100644 > --- a/recipes/net-snmp/net-snmp_svn.bb > +++ b/recipes/net-snmp/net-snmp_svn.bb > @@ -1,7 +1,7 @@ > require net-snmp.inc > DEPENDS += "libtool libtool-native libtool-cross" > SRCREV = "20069" > -PR = "${INC_PR}.0" > +PR = "${INC_PR}.1" > > S = "${WORKDIR}/net-snmp" > > @@ -10,7 +10,7 @@ SRC_URI = "svn://net-snmp.svn.sourceforge.net/svnroot/net-snmp/trunk;module=net- > file://snmpd.conf \ > file://snmptrapd.conf" > > -EXTRA_OECONF += "--disable-embedded-perl --with-perl-modules=no GREP=/bin/grep SED=/bin/sed" > +EXTRA_OECONF += "--disable-embedded-perl --with-perl-modules=no --without-nl GREP=/bin/grep SED=/bin/sed" > EXTRA_OEMAKE = "INSTALL_PREFIX=${D}" > > CCACHE = "" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iD8DBQFNeh4JMkyGM64RGpERAoHZAJ9qMCGgy94d0SRjWn8BYytVOgRTxwCgj+GR tioY8TZybHS74EH8FMVubQ0= =euaE -----END PGP SIGNATURE-----
On (11/03/11 16:53), Steffen Sledz wrote: > On 03/11/2011 02:05 PM, Koen Kooi wrote: > > On 11-03-11 12:04, Steffen Sledz wrote: > >> Fixes compile error > > > >> | ./.libs/libnetsnmpmibs.so: undefined reference to `nl_handle_destroy' > >> | ./.libs/libnetsnmpmibs.so: undefined reference to `nl_handle_alloc' > > > >> which occurred sometimes depending on build order (not in clean package > >> only builds). > > > > What features do we loose when disabling netlink support? I'd rather get > > the libnl recipe fixed (e.g. static lib only) instead of hacking the > > dependants. > > I'm not sure. > > But there is no dependency to libnl. If *libnl was not built before net-snmp* the autotools/configure does not find it. So in that case libnl wasn't used anyway. > > Just in case *it was built before* net-snmp tries to use it and hits the mentioned error. So assuming that successful builds before did not link in libnl this seems ok to me. > > Regards, > Steffen > > -- > DResearch Fahrzeugelektronik GmbH > Otto-Schmirgal-Str. 3, 10319 Berlin, Germany > Tel: +49 30 515932-237 mailto:sledz@DResearch.de > Fax: +49 30 515932-299 > Geschäftsführer: Dr. Michael Weber, Werner Mögle; > Amtsgericht Berlin Charlottenburg; HRB 130120 B; > Ust.-IDNr. DE273952058 > > _______________________________________________ > Openembedded-devel mailing list > Openembedded-devel@lists.openembedded.org > http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel
Am 12.03.2011 01:05, schrieb Khem Raj: > On (11/03/11 16:53), Steffen Sledz wrote: >> On 03/11/2011 02:05 PM, Koen Kooi wrote: >>> On 11-03-11 12:04, Steffen Sledz wrote: >>>> Fixes compile error >>> >>>> | ./.libs/libnetsnmpmibs.so: undefined reference to `nl_handle_destroy' >>>> | ./.libs/libnetsnmpmibs.so: undefined reference to `nl_handle_alloc' >>> >>>> which occurred sometimes depending on build order (not in clean package >>>> only builds). >>> >>> What features do we loose when disabling netlink support? I'd rather get >>> the libnl recipe fixed (e.g. static lib only) instead of hacking the >>> dependants. >> >> I'm not sure. >> >> But there is no dependency to libnl. If *libnl was not built before net-snmp* the autotools/configure does not find it. So in that case libnl wasn't used anyway. >> >> Just in case *it was built before* net-snmp tries to use it and hits the mentioned error. > > So assuming that successful builds before did not link in libnl this > seems ok to me. Can i take that as an ACK? Steffen
On (14/03/11 08:06), Steffen Sledz wrote: > Am 12.03.2011 01:05, schrieb Khem Raj: > > On (11/03/11 16:53), Steffen Sledz wrote: > >> On 03/11/2011 02:05 PM, Koen Kooi wrote: > >>> On 11-03-11 12:04, Steffen Sledz wrote: > >>>> Fixes compile error > >>> > >>>> | ./.libs/libnetsnmpmibs.so: undefined reference to `nl_handle_destroy' > >>>> | ./.libs/libnetsnmpmibs.so: undefined reference to `nl_handle_alloc' > >>> > >>>> which occurred sometimes depending on build order (not in clean package > >>>> only builds). > >>> > >>> What features do we loose when disabling netlink support? I'd rather get > >>> the libnl recipe fixed (e.g. static lib only) instead of hacking the > >>> dependants. > >> > >> I'm not sure. > >> > >> But there is no dependency to libnl. If *libnl was not built before net-snmp* the autotools/configure does not find it. So in that case libnl wasn't used anyway. > >> > >> Just in case *it was built before* net-snmp tries to use it and hits the mentioned error. > > > > So assuming that successful builds before did not link in libnl this > > seems ok to me. > > Can i take that as an ACK? Acked-by: Khem Raj <raj.khem@gmail.com> > > Steffen > > -- > DResearch Fahrzeugelektronik GmbH > Otto-Schmirgal-Str. 3, 10319 Berlin, Germany > Tel: +49 30 515932-237 mailto:sledz@DResearch.de > Fax: +49 30 515932-299 > Geschäftsführer: Dr. Michael Weber, Werner Mögle; > Amtsgericht Berlin Charlottenburg; HRB 130120 B; > Ust.-IDNr. DE273952058 >
On Fri, 2011-03-11 at 12:04 +0100, Steffen Sledz wrote: > which occurred sometimes depending on build order (not in clean > package only builds). I would like to raise awareness of the underlying problem here. The current dependency/staging model of OE basically has this feature that a build can be influenced not only by it's own dependencies, but also what has been build before it (or not). I strongly believe that this has to be fixed on the architectural level, and not just on a case-by-case level as is currently needed. I haven't received much feedback on the preivous posting about the per-recipe staging principle implemented in OE-lite, but I decided to take this opportunity to re-iterate the fact that the OE-lite implementation of staging and build dependencies eliminates this problem. I am still very much interested in discussing how to move this technology from OE-lite to OE, but as it impacts all recipe metadata (build dependencies has to be redefined), OE community at a large really needs to value the benefits of solving this problem. Best regards, Esben
On Tue, Mar 15, 2011 at 10:08:37AM +0100, Esben Haabendal wrote: > On Fri, 2011-03-11 at 12:04 +0100, Steffen Sledz wrote: > > which occurred sometimes depending on build order (not in clean > > package only builds). > > I would like to raise awareness of the underlying problem here. > > The current dependency/staging model of OE basically has this feature > that a build can be influenced not only by it's own dependencies, but > also what has been build before it (or not). > > I strongly believe that this has to be fixed on the architectural level, > and not just on a case-by-case level as is currently needed. > > I haven't received much feedback on the preivous posting about the > per-recipe staging principle implemented in OE-lite, but I decided to > take this opportunity to re-iterate the fact that the OE-lite > implementation of staging and build dependencies eliminates this > problem. > > I am still very much interested in discussing how to move this > technology from OE-lite to OE, but as it impacts all recipe metadata > (build dependencies has to be redefined), OE community at a large really > needs to value the benefits of solving this problem. Esben, Thanks for raising this issue and making people aware of it! This has previously been discussed several times, including last time during the Yocto Summit in December, where I brought up the question of per-package staging/dependency to Richard's attention, when he introduced the new Shared State (sstate). While per-package dependency wasn't considered a critical must-have feature right away, it was acknowledged as something worthwhile looking at and, according to Richard, should be easy to accomplish with the new sstate functionality...
On 03/15/2011 07:38 PM, Esben Haabendal wrote: > On Fri, 2011-03-11 at 12:04 +0100, Steffen Sledz wrote: >> which occurred sometimes depending on build order (not in clean >> package only builds). > > I would like to raise awareness of the underlying problem here. > > The current dependency/staging model of OE basically has this feature > that a build can be influenced not only by it's own dependencies, but > also what has been build before it (or not). > > I strongly believe that this has to be fixed on the architectural level, > and not just on a case-by-case level as is currently needed. > > I haven't received much feedback on the preivous posting about the > per-recipe staging principle implemented in OE-lite, but I decided to > take this opportunity to re-iterate the fact that the OE-lite > implementation of staging and build dependencies eliminates this > problem. > > I am still very much interested in discussing how to move this > technology from OE-lite to OE, but as it impacts all recipe metadata > (build dependencies has to be redefined), OE community at a large really > needs to value the benefits of solving this problem. > > Best regards, > Esben > > > _______________________________________________ > Openembedded-devel mailing list > Openembedded-devel@lists.openembedded.org > http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel I did some work last year to fix some of the low hanging fruit (on a case by case basis). My methodology was to do two runs of bitbake -k world, then compare the output. In case anyone is interested, I used the script below to identify the problematic recipes. Last time I ran it was in November, by which time the output had reduced to a list of 5-10 items most of the time. #!/bin/sh # # Quick and dirty script to determine which tasks fail in one world.log, # where they succeeded in another. world.log should be generated with, e.g. # $ bitbake -k world | tee world.log # # If there are too many failures, this script will take too long as grep -f # is slooow. if [ $# != 2 ]; then echo "usage: $0 world.log.1 world.log.2" exit 1 fi tasks="do_setscene do_fetch do_distribute_sources do_unpack do_prepsources do_patch do_configure do_qa_configure do_compile do_install do_populate_sysroot do_package do_package_write_ipk do_package_write do_package_stage do_package_stage_all do_build do_qa_staging" tmp1=`mktemp /tmp/world_regress.XXXXXX` tmp2=`mktemp /tmp/world_regress.XXXXXX` for task in $tasks; do grep "task $task: Failed" $2 \ | sed -e 's/^.*NOTE: package //' \ -e 's/task \(.*\): Failed/\1/' \ > $tmp1 grep "task $task: Succeeded" $1 \ | sed -e 's/^.*NOTE: package //' \ -e 's/task \(.*\): Succeeded/\1/' \ > $tmp2 grep -f $tmp1 $tmp2 \ | sort -n done rm -f $tmp1 $tmp2
On Tue, 2011-03-15 at 18:03 -0400, Denys Dmytriyenko wrote: > On Tue, Mar 15, 2011 at 10:08:37AM +0100, Esben Haabendal wrote: > > On Fri, 2011-03-11 at 12:04 +0100, Steffen Sledz wrote: > > > which occurred sometimes depending on build order (not in clean > > > package only builds). > > > > I would like to raise awareness of the underlying problem here. > > > > The current dependency/staging model of OE basically has this feature > > that a build can be influenced not only by it's own dependencies, but > > also what has been build before it (or not). > > > > I strongly believe that this has to be fixed on the architectural level, > > and not just on a case-by-case level as is currently needed. > > > > I haven't received much feedback on the preivous posting about the > > per-recipe staging principle implemented in OE-lite, but I decided to > > take this opportunity to re-iterate the fact that the OE-lite > > implementation of staging and build dependencies eliminates this > > problem. > > > > I am still very much interested in discussing how to move this > > technology from OE-lite to OE, but as it impacts all recipe metadata > > (build dependencies has to be redefined), OE community at a large really > > needs to value the benefits of solving this problem. > > Esben, > > Thanks for raising this issue and making people aware of it! > > This has previously been discussed several times, including last time during > the Yocto Summit in December, where I brought up the question of per-package > staging/dependency to Richard's attention, when he introduced the new Shared > State (sstate). While per-package dependency wasn't considered a critical > must-have feature right away, it was acknowledged as something worthwhile > looking at and, according to Richard, should be easy to accomplish with the > new sstate functionality... There is two different objectives in this. Per-recipe staging and per-package build-time dependencies. While I assume per-recipe staging will be doable to build on top of sstate, per-package build-time dependencies is not really related to sstate, but is mostly a meta-data issue (man-hours wise), while also requiring changes to tbe BitBake cooker. The final thing that I have found while solving the above issues, is how simple (KISS!) all this can be addressed when going from per-recipe build-time dependencies to per-package. And I really think that we need more KISS in OE ;-) /Esben
Hello, I was looking at the libxml2 package. While going through the recipes I came to know that its native package depends on python-native package as well. This dependency is defined in libxml2.inc file. DEPENDS_virtclass-native = "python-native" This package build fine after removing this dependency. I am not clear why this dependency is being added in libxml2 package. Can somebody clear that to me. While looking at the git log I came to know that this change is coming from libxml2-native.inc file. But I am not clear why this dependency was added as this package build fine without it. Please help me in understanding this dependency. Thanks. Regards, Noor
2011/3/16 Esben Haabendal <eha@doredevelopment.dk>: > On Tue, 2011-03-15 at 18:03 -0400, Denys Dmytriyenko wrote: >> On Tue, Mar 15, 2011 at 10:08:37AM +0100, Esben Haabendal wrote: >> > On Fri, 2011-03-11 at 12:04 +0100, Steffen Sledz wrote: >> > > which occurred sometimes depending on build order (not in clean >> > > package only builds). >> > >> > I would like to raise awareness of the underlying problem here. >> > >> > The current dependency/staging model of OE basically has this feature >> > that a build can be influenced not only by it's own dependencies, but >> > also what has been build before it (or not). >> > >> > I strongly believe that this has to be fixed on the architectural level, >> > and not just on a case-by-case level as is currently needed. >> > >> > I haven't received much feedback on the preivous posting about the >> > per-recipe staging principle implemented in OE-lite, but I decided to >> > take this opportunity to re-iterate the fact that the OE-lite >> > implementation of staging and build dependencies eliminates this >> > problem. >> > >> > I am still very much interested in discussing how to move this >> > technology from OE-lite to OE, but as it impacts all recipe metadata >> > (build dependencies has to be redefined), OE community at a large really >> > needs to value the benefits of solving this problem. >> >> Esben, >> >> Thanks for raising this issue and making people aware of it! >> >> This has previously been discussed several times, including last time during >> the Yocto Summit in December, where I brought up the question of per-package >> staging/dependency to Richard's attention, when he introduced the new Shared >> State (sstate). While per-package dependency wasn't considered a critical >> must-have feature right away, it was acknowledged as something worthwhile >> looking at and, according to Richard, should be easy to accomplish with the >> new sstate functionality... > > There is two different objectives in this. > > Per-recipe staging and per-package build-time dependencies. While I > assume per-recipe staging will be doable to build on top of sstate, > per-package build-time dependencies is not really related to sstate, but > is mostly a meta-data issue (man-hours wise), while also requiring > changes to tbe BitBake cooker. > > The final thing that I have found while solving the above issues, is how > simple (KISS!) all this can be addressed when going from per-recipe > build-time dependencies to per-package. And I really think that we need > more KISS in OE ;-) > > /Esben From a consistency point of view per package build-time dependencies are highly desirable to get a consistent and reproducible system as you are always getting the same output irrespective of build order. Currently this is not the case because configure might pick up whatever package is build first. So for (deeply) embedded product development this is a very desirable feature, QA wise, if not a must have. Frans
On 3/15/2011 11:22 PM, Ahsan, Noor wrote: > Hello, > > I was looking at the libxml2 package. While going through the recipes I > came to know that its native package depends on python-native package as > well. This dependency is defined in libxml2.inc file. > > DEPENDS_virtclass-native = "python-native" > > This package build fine after removing this dependency. I am not clear > why this dependency is being added in libxml2 package. Can somebody > clear that to me. While looking at the git log I came to know that this > change is coming from libxml2-native.inc file. But I am not clear why > this dependency was added as this package build fine without it. > > Please help me in understanding this dependency. Thanks. Try to build this recipe from scratch and see if it still works then we can remove the dep > > Regards, > Noor > > _______________________________________________ > Openembedded-devel mailing list > Openembedded-devel@lists.openembedded.org > http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel
2011/3/16 Khem Raj <raj.khem@gmail.com>: > On 3/15/2011 11:22 PM, Ahsan, Noor wrote: >> >> Hello, >> >> I was looking at the libxml2 package. While going through the recipes I >> came to know that its native package depends on python-native package as >> well. This dependency is defined in libxml2.inc file. >> >> DEPENDS_virtclass-native = "python-native" >> >> This package build fine after removing this dependency. I am not clear >> why this dependency is being added in libxml2 package. Can somebody >> clear that to me. While looking at the git log I came to know that this >> change is coming from libxml2-native.inc file. But I am not clear why >> this dependency was added as this package build fine without it. >> >> Please help me in understanding this dependency. Thanks. > > Try to build this recipe from scratch and see if it still works then we can > remove the dep I'm not sure if that is the right way. It might well be that typically python-native is already there in a scratch build because some earlier package needed it (or even because of the scheduling of build steps. Probably a better way is to build with packaged staging (with staging dir outside TMPDIR) then rm TMPDIR and bake the package. Alternately peek into the package to see if it needs python. If it is needed then the DEPENDS should stay. Looking at the issue at hand maybe the best person to ask is Martin Jansa as he last touched that line according to git blame: 3cd9ddbf recipes/libxml/libxml2.inc (Martin Jansa 2010-02-26 12:32:53 +0100 5) DEPENDS_virtclass-native = "python-native" The other thing is that further on in the .inc file I see: XPY_virtclass-native = " --with-python=${PYTHON_DIR}" This strongly suggests a hard dependency on python, so removing the DEPENDS "because it builds without it" does not seem to be a good plan to me. My strong feeling is that recipes should be self-contained (so should list all packages they need to build, minus the ones that are inherited from classes (and maybe a few implicit ones like maybe make) Frans
On 3/16/2011 12:28 AM, Frans Meulenbroeks wrote: > 2011/3/16 Khem Raj<raj.khem@gmail.com>: >> On 3/15/2011 11:22 PM, Ahsan, Noor wrote: >>> >>> Hello, >>> >>> I was looking at the libxml2 package. While going through the recipes I >>> came to know that its native package depends on python-native package as >>> well. This dependency is defined in libxml2.inc file. >>> >>> DEPENDS_virtclass-native = "python-native" >>> >>> This package build fine after removing this dependency. I am not clear >>> why this dependency is being added in libxml2 package. Can somebody >>> clear that to me. While looking at the git log I came to know that this >>> change is coming from libxml2-native.inc file. But I am not clear why >>> this dependency was added as this package build fine without it. >>> >>> Please help me in understanding this dependency. Thanks. >> >> Try to build this recipe from scratch and see if it still works then we can >> remove the dep > > I'm not sure if that is the right way. > It might well be that typically python-native is already there in a > scratch build because some earlier package needed it (or even because > of the scheduling of build steps. > > Probably a better way is to build with packaged staging (with staging > dir outside TMPDIR) then rm TMPDIR and bake the package. > Alternately peek into the package to see if it needs python. If it is > needed then the DEPENDS should stay. Did you read carefully what I wrote ?
2011/3/16 Khem Raj <raj.khem@gmail.com>: > On 3/16/2011 12:28 AM, Frans Meulenbroeks wrote: >> >> 2011/3/16 Khem Raj<raj.khem@gmail.com>: >>> >>> On 3/15/2011 11:22 PM, Ahsan, Noor wrote: >>>> >>>> Hello, >>>> >>>> I was looking at the libxml2 package. While going through the recipes I >>>> came to know that its native package depends on python-native package as >>>> well. This dependency is defined in libxml2.inc file. >>>> >>>> DEPENDS_virtclass-native = "python-native" >>>> >>>> This package build fine after removing this dependency. I am not clear >>>> why this dependency is being added in libxml2 package. Can somebody >>>> clear that to me. While looking at the git log I came to know that this >>>> change is coming from libxml2-native.inc file. But I am not clear why >>>> this dependency was added as this package build fine without it. >>>> >>>> Please help me in understanding this dependency. Thanks. >>> >>> Try to build this recipe from scratch and see if it still works then we >>> can >>> remove the dep >> >> I'm not sure if that is the right way. >> It might well be that typically python-native is already there in a >> scratch build because some earlier package needed it (or even because >> of the scheduling of build steps. >> >> Probably a better way is to build with packaged staging (with staging >> dir outside TMPDIR) then rm TMPDIR and bake the package. >> Alternately peek into the package to see if it needs python. If it is >> needed then the DEPENDS should stay. > > > Did you read carefully what I wrote ? Yes. From scratch for me (and maybe also for others) means rm TMPDIR (including pstaging). In that case you might pick up stray packages. You probably meant keeping pstaging, but at least for some of the readers that is not obvious. Frans
On Wed, Mar 16, 2011 at 08:28:39AM +0100, Frans Meulenbroeks wrote: > 2011/3/16 Khem Raj <raj.khem@gmail.com>: > > On 3/15/2011 11:22 PM, Ahsan, Noor wrote: > >> > >> Hello, > >> > >> I was looking at the libxml2 package. While going through the recipes I > >> came to know that its native package depends on python-native package as > >> well. This dependency is defined in libxml2.inc file. > >> > >> DEPENDS_virtclass-native = "python-native" > >> > >> This package build fine after removing this dependency. I am not clear > >> why this dependency is being added in libxml2 package. Can somebody > >> clear that to me. While looking at the git log I came to know that this > >> change is coming from libxml2-native.inc file. But I am not clear why > >> this dependency was added as this package build fine without it. > >> > >> Please help me in understanding this dependency. Thanks. > > > > Try to build this recipe from scratch and see if it still works then we can > > remove the dep That's not enough, libxml2-native builds fine.. but then xml2po will fail if there isn't libxml2-native built with python support. see a25f07ca6159e1cd5b8a215ca5da8405fb1b9053 So yes.. for such tests you should build recipe from scratch _and_ check if resulting package/sysroot is still the same. Regards, > I'm not sure if that is the right way. > It might well be that typically python-native is already there in a > scratch build because some earlier package needed it (or even because > of the scheduling of build steps. > > Probably a better way is to build with packaged staging (with staging > dir outside TMPDIR) then rm TMPDIR and bake the package. > Alternately peek into the package to see if it needs python. If it is > needed then the DEPENDS should stay. > > Looking at the issue at hand maybe the best person to ask is Martin > Jansa as he last touched that line according to git blame: > 3cd9ddbf recipes/libxml/libxml2.inc (Martin Jansa > 2010-02-26 12:32:53 +0100 5) DEPENDS_virtclass-native = > "python-native" > > The other thing is that further on in the .inc file I see: > XPY_virtclass-native = " --with-python=${PYTHON_DIR}" > > This strongly suggests a hard dependency on python, so removing the > DEPENDS "because it builds without it" does not seem to be a good plan > to me. > > My strong feeling is that recipes should be self-contained (so should > list all packages they need to build, minus the ones that are > inherited from classes (and maybe a few implicit ones like maybe make) > > Frans > > _______________________________________________ > Openembedded-devel mailing list > Openembedded-devel@lists.openembedded.org > http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel
Hello,
I made 2 changes, first one was to remove python-native from DEPENDS and
second was commented out XPY_virtclass-native = "
--with-python=${PYTHON_DIR}". Moreover I removed all the stuff from
build dir, so there were not python-native for me. After kicking this
build libxml2-native build fine.
I was looking for xml2po recipe, I couldn't find it. Can you tell me how
to build xml2po package so that I can reproduce the problem.
Regards,
Noor
-----Original Message-----
From: openembedded-devel-bounces@lists.openembedded.org
[mailto:openembedded-devel-bounces@lists.openembedded.org] On Behalf Of
Martin Jansa
Sent: Wednesday, March 16, 2011 1:05 PM
To: openembedded-devel@lists.openembedded.org
Subject: Re: [oe] Python-native dependency in libxml2
On Wed, Mar 16, 2011 at 08:28:39AM +0100, Frans Meulenbroeks wrote:
> 2011/3/16 Khem Raj <raj.khem@gmail.com>:
> > On 3/15/2011 11:22 PM, Ahsan, Noor wrote:
> >>
> >> Hello,
> >>
> >> I was looking at the libxml2 package. While going through the
recipes I
> >> came to know that its native package depends on python-native
package as
> >> well. This dependency is defined in libxml2.inc file.
> >>
> >> DEPENDS_virtclass-native = "python-native"
> >>
> >> This package build fine after removing this dependency. I am not
clear
> >> why this dependency is being added in libxml2 package. Can somebody
> >> clear that to me. While looking at the git log I came to know that
this
> >> change is coming from libxml2-native.inc file. But I am not clear
why
> >> this dependency was added as this package build fine without it.
> >>
> >> Please help me in understanding this dependency. Thanks.
> >
> > Try to build this recipe from scratch and see if it still works then
we can
> > remove the dep
That's not enough, libxml2-native builds fine.. but then xml2po
will fail if there isn't libxml2-native built with python support.
see a25f07ca6159e1cd5b8a215ca5da8405fb1b9053
So yes.. for such tests you should build recipe from scratch _and_ check
if resulting package/sysroot is still the same.
Regards,
> I'm not sure if that is the right way.
> It might well be that typically python-native is already there in a
> scratch build because some earlier package needed it (or even because
> of the scheduling of build steps.
>
> Probably a better way is to build with packaged staging (with staging
> dir outside TMPDIR) then rm TMPDIR and bake the package.
> Alternately peek into the package to see if it needs python. If it is
> needed then the DEPENDS should stay.
>
> Looking at the issue at hand maybe the best person to ask is Martin
> Jansa as he last touched that line according to git blame:
> 3cd9ddbf recipes/libxml/libxml2.inc (Martin Jansa
> 2010-02-26 12:32:53 +0100 5) DEPENDS_virtclass-native =
> "python-native"
>
> The other thing is that further on in the .inc file I see:
> XPY_virtclass-native = " --with-python=${PYTHON_DIR}"
>
> This strongly suggests a hard dependency on python, so removing the
> DEPENDS "because it builds without it" does not seem to be a good plan
> to me.
>
> My strong feeling is that recipes should be self-contained (so should
> list all packages they need to build, minus the ones that are
> inherited from classes (and maybe a few implicit ones like maybe make)
>
> Frans
>
> _______________________________________________
> Openembedded-devel mailing list
> Openembedded-devel@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel
Hello,
Today I got the fresh stuff and tried to build gnome-doc-utils-native
(to build xml2po package). I am building on Ubuntu host 9.10. First it
gave me an error that it could not find intltoolize command. You can see
the attached build log "build_intltool_error,log". So I added intltool
package in gnome-doc-utils-native DEPENDS so it built successfully. You
can see the build log "build_intltool_fix.log". Then I removed all the
build stuff from build DIR and commented DEPENDS_virtclass-native =
"python-native" and XPY_virtclass-native = "
--with-python=${PYTHON_DIR}" in libxml2.inc file. Rebuild
gnome-doc-utils-native. It built successfully again. The build log is
also attached "build_no_python.log". I think there is no need of python
dependency in libxml2 package. Kindly correct me if I am doing anything
wrong.
We need to fix gnome-doc-utils-native recipe, so that it does not show
intltoolize error.
Regards,
Noor
-----Original Message-----
From: openembedded-devel-bounces@lists.openembedded.org
[mailto:openembedded-devel-bounces@lists.openembedded.org] On Behalf Of
Ahsan, Noor
Sent: Wednesday, March 16, 2011 1:39 PM
To: openembedded-devel@lists.openembedded.org
Subject: Re: [oe] Python-native dependency in libxml2
Hello,
I made 2 changes, first one was to remove python-native from DEPENDS and
second was commented out XPY_virtclass-native = "
--with-python=${PYTHON_DIR}". Moreover I removed all the stuff from
build dir, so there were not python-native for me. After kicking this
build libxml2-native build fine.
I was looking for xml2po recipe, I couldn't find it. Can you tell me how
to build xml2po package so that I can reproduce the problem.
Regards,
Noor
-----Original Message-----
From: openembedded-devel-bounces@lists.openembedded.org
[mailto:openembedded-devel-bounces@lists.openembedded.org] On Behalf Of
Martin Jansa
Sent: Wednesday, March 16, 2011 1:05 PM
To: openembedded-devel@lists.openembedded.org
Subject: Re: [oe] Python-native dependency in libxml2
On Wed, Mar 16, 2011 at 08:28:39AM +0100, Frans Meulenbroeks wrote:
> 2011/3/16 Khem Raj <raj.khem@gmail.com>:
> > On 3/15/2011 11:22 PM, Ahsan, Noor wrote:
> >>
> >> Hello,
> >>
> >> I was looking at the libxml2 package. While going through the
recipes I
> >> came to know that its native package depends on python-native
package as
> >> well. This dependency is defined in libxml2.inc file.
> >>
> >> DEPENDS_virtclass-native = "python-native"
> >>
> >> This package build fine after removing this dependency. I am not
clear
> >> why this dependency is being added in libxml2 package. Can somebody
> >> clear that to me. While looking at the git log I came to know that
this
> >> change is coming from libxml2-native.inc file. But I am not clear
why
> >> this dependency was added as this package build fine without it.
> >>
> >> Please help me in understanding this dependency. Thanks.
> >
> > Try to build this recipe from scratch and see if it still works then
we can
> > remove the dep
That's not enough, libxml2-native builds fine.. but then xml2po
will fail if there isn't libxml2-native built with python support.
see a25f07ca6159e1cd5b8a215ca5da8405fb1b9053
So yes.. for such tests you should build recipe from scratch _and_ check
if resulting package/sysroot is still the same.
Regards,
> I'm not sure if that is the right way.
> It might well be that typically python-native is already there in a
> scratch build because some earlier package needed it (or even because
> of the scheduling of build steps.
>
> Probably a better way is to build with packaged staging (with staging
> dir outside TMPDIR) then rm TMPDIR and bake the package.
> Alternately peek into the package to see if it needs python. If it is
> needed then the DEPENDS should stay.
>
> Looking at the issue at hand maybe the best person to ask is Martin
> Jansa as he last touched that line according to git blame:
> 3cd9ddbf recipes/libxml/libxml2.inc (Martin Jansa
> 2010-02-26 12:32:53 +0100 5) DEPENDS_virtclass-native =
> "python-native"
>
> The other thing is that further on in the .inc file I see:
> XPY_virtclass-native = " --with-python=${PYTHON_DIR}"
>
> This strongly suggests a hard dependency on python, so removing the
> DEPENDS "because it builds without it" does not seem to be a good plan
> to me.
>
> My strong feeling is that recipes should be self-contained (so should
> list all packages they need to build, minus the ones that are
> inherited from classes (and maybe a few implicit ones like maybe make)
>
> Frans
>
> _______________________________________________
> Openembedded-devel mailing list
> Openembedded-devel@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel
On Tue, 2011-03-15 at 10:08 +0100, Esben Haabendal wrote: > The current dependency/staging model of OE basically has this feature > that a build can be influenced not only by it's own dependencies, but > also what has been build before it (or not). > > I strongly believe that this has to be fixed on the architectural level, > and not just on a case-by-case level as is currently needed. Yes, agreed. This clearly sucks, and indeed I think it is probably the largest single area of architectural suckage in OE today. > I am still very much interested in discussing how to move this > technology from OE-lite to OE, but as it impacts all recipe metadata > (build dependencies has to be redefined), OE community at a large really > needs to value the benefits of solving this problem. The benefits of solving the problem are clearly very great, but I don't think OE itself is really in a position to embrace a "big bang" kind of change which would require redefining all the build dependencies. I think we need to find a technological solution which will work with our current DEPENDS scheme. I did do some work in that direction about a year ago, but I got sidetracked into dealing with some toolchain suckage (which was making it difficult to construct the per-recipe sysroots in the way that I wanted) and then ran out of time. I would like to get back to working on this stuff but right now it is difficult for me to set aside the time for it. p.
On Thu, 2011-03-17 at 11:18 +0000, Phil Blundell wrote: > > > I am still very much interested in discussing how to move this > > technology from OE-lite to OE, but as it impacts all recipe metadata > > (build dependencies has to be redefined), OE community at a large > really > > needs to value the benefits of solving this problem. > > The benefits of solving the problem are clearly very great, but I > don't think OE itself is really in a position to embrace a "big bang" > kind of change which would require redefining all the build > dependencies. I think we need to find a technological solution which > will work with our current DEPENDS scheme. I know that this seems to be the general consensus in OE, but I do not agree with it. Sometimes architectural work requires working out the details on a branch, and then when it is mostly done, do the "big bang" which will then not really be that "big" anymore, as most of the issues have been resolved. Is OE really in a position to permantly settle for something suboptimal in such a central area? /Esben
On 17/03/2011 14:43, Esben Haabendal wrote: > On Thu, 2011-03-17 at 11:18 +0000, Phil Blundell wrote: >>> I am still very much interested in discussing how to move this >>> technology from OE-lite to OE, but as it impacts all recipe metadata >>> (build dependencies has to be redefined), OE community at a large >> really >>> needs to value the benefits of solving this problem. >> The benefits of solving the problem are clearly very great, but I >> don't think OE itself is really in a position to embrace a "big bang" >> kind of change which would require redefining all the build >> dependencies. I think we need to find a technological solution which >> will work with our current DEPENDS scheme. > I know that this seems to be the general consensus in OE, but I do not > agree with it. Sometimes architectural work requires working out the > details on a branch, and then when it is mostly done, do the "big bang" > which will then not really be that "big" anymore, as most of the issues > have been resolved. > > Is OE really in a position to permantly settle for something suboptimal > in such a central area? > I would suggest a branch of oe-core is the ideal place to do this work. It can be proved on oe-core then rolled out to meta-oe at a later date. Graeme
On Thu, 2011-03-17 at 15:43 +0100, Esben Haabendal wrote: > Is OE really in a position to permantly settle for something suboptimal > in such a central area? No, but rejecting the big bang doesn't mean that we can't make the change; it just means that we need to find a way to make the old and new arrangements coexist for a transition period. This is the way every other architectural change in OE has been done and I don't see any reason that this one needs to be different. If we really did want to go ahead and have a metadata flag day then no doubt there are plenty of other things we would like to change at the same time. But, thus far, this has never seemed to justify the disruption of breaking every recipe in both the main OE repo and third party overlays. I must admit that I'm also slightly unclear about why the change in build dependency specifications is a prerequisite for doing per-recipe sysroots. Is that just an artifact of the way you implemented it in OE-lite or is there some fundamental constraint that means it needs to be this way? p.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 17-03-11 15:52, Graeme Gregory wrote: > On 17/03/2011 14:43, Esben Haabendal wrote: >> On Thu, 2011-03-17 at 11:18 +0000, Phil Blundell wrote: >>>> I am still very much interested in discussing how to move this >>>> technology from OE-lite to OE, but as it impacts all recipe metadata >>>> (build dependencies has to be redefined), OE community at a large >>> really >>>> needs to value the benefits of solving this problem. >>> The benefits of solving the problem are clearly very great, but I >>> don't think OE itself is really in a position to embrace a "big bang" >>> kind of change which would require redefining all the build >>> dependencies. I think we need to find a technological solution which >>> will work with our current DEPENDS scheme. >> I know that this seems to be the general consensus in OE, but I do not >> agree with it. Sometimes architectural work requires working out the >> details on a branch, and then when it is mostly done, do the "big bang" >> which will then not really be that "big" anymore, as most of the issues >> have been resolved. >> >> Is OE really in a position to permantly settle for something suboptimal >> in such a central area? >> > I would suggest a branch of oe-core is the ideal place to do this work. > It can be proved on oe-core then rolled out to meta-oe at a later date. OE-core already features per-machine sysroots, so the main problem we had, relocatability, has already been solved. regards, Koen -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iD8DBQFNgielMkyGM64RGpERAqcIAKCyT44a9MzDF+esO6DgRbtBXi4L8ACdGzpN 4vN5d5EtUwbo3cNHfMl9XQ4= =eFCq -----END PGP SIGNATURE-----
On Thu, 2011-03-17 at 15:07 +0000, Phil Blundell wrote: > On Thu, 2011-03-17 at 15:43 +0100, Esben Haabendal wrote: > > Is OE really in a position to permantly settle for something suboptimal > > in such a central area? > > No, but rejecting the big bang doesn't mean that we can't make the > change; it just means that we need to find a way to make the old and new > arrangements coexist for a transition period. This is the way every > other architectural change in OE has been done and I don't see any > reason that this one needs to be different. Exactly > > If we really did want to go ahead and have a metadata flag day then no > doubt there are plenty of other things we would like to change at the > same time. But, thus far, this has never seemed to justify the > disruption of breaking every recipe in both the main OE repo and third > party overlays. Well, it might be possible to minimize the disruption for a transitional period by carefully specifying some catch-all build-time package dependencies pulling in all packages for recipes not ported yet. I think it would be much more worthwhile to put effort into this than to push for per-recipe sysroot with the current sstate solution. Having package-based build-time dependencies, using the same packages as when building run-time images, is much simpler than what is currently done in OE, while at the same time giving increased flexibility in specifying the build-time dependencies needed for a recipe and it's packages. But you might have to try it out before really understanding why you cannot live without it ;-) > I must admit that I'm also slightly unclear about why the change in > build dependency specifications is a prerequisite for doing per-recipe > sysroots. It might not be, but the result will of-course not be the same. >Is that just an artifact of the way you implemented it in > OE-lite or is there some fundamental constraint that means it needs to > be this way? I don't think there are any fundamental constraint. Sometimes doing things right are simply so much easier. As a result, I already do have a solution with both per-recipe staging/sysroot and buld-time dependencies using target packages. I feel pretty confident that this was the fast path, which is sort of backed by the fact that it is done, whereas the path (detour?) currently being worked on in OE is still just on it's way. /Esben
On Thu, 2011-03-17 at 18:52 +0100, Esben Haabendal wrote: > Well, it might be possible to minimize the disruption for a transitional > period by carefully specifying some catch-all build-time package > dependencies pulling in all packages for recipes not ported yet. Yes, that's the sort of thing I was thinking of. It isn't even totally obvious to me that specifying individual packages is any better than "all packages from recipe X", since with the former you then have a potential maintenance headache if files get moved from one package to another. p.
On Thu, 2011-03-17 at 18:05 +0000, Phil Blundell wrote: > On Thu, 2011-03-17 at 18:52 +0100, Esben Haabendal wrote: > > Well, it might be possible to minimize the disruption for a transitional > > period by carefully specifying some catch-all build-time package > > dependencies pulling in all packages for recipes not ported yet. > > Yes, that's the sort of thing I was thinking of. It isn't even totally > obvious to me that specifying individual packages is any better than > "all packages from recipe X", since with the former you then have a > potential maintenance headache if files get moved from one package to > another. There is a number of ways that I believe package based build dependencies are better than recipe based. a) It is possible to depend on parts of a recipe, which fx. is useful when a recipe provides more than 1 library, where you might not want all of them. b) To build a recipe, you depend on some stuff which you don't need to, or perhaps even really don't want to pass on to recipes depending on this recipe. c) KISS. Using the actual target packages for satisfying build-time dependencies are a very simple approach, which I strongly believe will make OE a better tool in the long run, by reducing complexity, and thus lowering the bar for contributing to OE archictural work. The maintenance headache you talk about is already there. In OE-lite build-time dependency 95% of the time just follow the run-time dependencies, perhaps making it easier to maintain than OE, as we don't have to think about another type of "item names" to depend on. /Esben
On Thu, 2011-03-17 at 20:58 +0100, Esben Haabendal wrote: > On Thu, 2011-03-17 at 18:05 +0000, Phil Blundell wrote: > > On Thu, 2011-03-17 at 18:52 +0100, Esben Haabendal wrote: > > > Well, it might be possible to minimize the disruption for a transitional > > > period by carefully specifying some catch-all build-time package > > > dependencies pulling in all packages for recipes not ported yet. > > > > Yes, that's the sort of thing I was thinking of. It isn't even totally > > obvious to me that specifying individual packages is any better than > > "all packages from recipe X", since with the former you then have a > > potential maintenance headache if files get moved from one package to > > another. > > There is a number of ways that I believe package based build > dependencies are better than recipe based. > > a) It is possible to depend on parts of a recipe, which fx. is useful > when a recipe provides more than 1 library, where you might not want all > of them. You'd still need to have built all of them (at least as far as the end of do_compile) though, right? In that case it doesn't seem as though installing the unnecessary ones in the sysroot is all that big a hardship. > b) To build a recipe, you depend on some stuff which you don't need to, > or perhaps even really don't want to pass on to recipes depending on > this recipe. Sorry, I don't understand what you're saying here. > c) KISS. Using the actual target packages for satisfying build-time > dependencies are a very simple approach, which I strongly believe will > make OE a better tool in the long run, by reducing complexity, and thus > lowering the bar for contributing to OE archictural work. I agree with this, but it seems unrelated to the question of whether build dependencies should be specified in terms of recipes or output packages. > The maintenance headache you talk about is already there. In OE-lite > build-time dependency 95% of the time just follow the run-time > dependencies, perhaps making it easier to maintain than OE, as we don't > have to think about another type of "item names" to depend on. (How) do you deal with library package renaming in OE-lite? p.
On Thu, 2011-03-17 at 21:00 +0000, Phil Blundell wrote: > On Thu, 2011-03-17 at 20:58 +0100, Esben Haabendal wrote: > > There is a number of ways that I believe package based build > > dependencies are better than recipe based. > > > > a) It is possible to depend on parts of a recipe, which fx. is useful > > when a recipe provides more than 1 library, where you might not want all > > of them. > > You'd still need to have built all of them (at least as far as the end > of do_compile) though, right? Of-course. All recipes providing something for a do_stage must be build up till the do_package stage, so I can unpack the actual target packages needed. > In that case it doesn't seem as though > installing the unnecessary ones in the sysroot is all that big a > hardship. It is not to save time, it is about controlling what goes into a build. As many software packages scans for presence of various components through the availability of headers and libraries, I believe it is important to decide not just what goes into the stage directory, but also what does not. So in OE-lite, if I say I depend on libfoo, I do not get foobar, just because it was built by the same recipe. > > b) To build a recipe, you depend on some stuff which you don't need to, > > or perhaps even really don't want to pass on to recipes depending on > > this recipe. > > Sorry, I don't understand what you're saying here. Example: I build recipe A. Recipe A depends on package B-b from recipe B. Recipe B depends on package C-c from recipe C, and D-d-native from recipe D. Recipe A only get's package B-b staged, unless the B-b package has specified some build-time dependencies. This is relevant both for target library and native tools dependencies. > > c) KISS. Using the actual target packages for satisfying build-time > > dependencies are a very simple approach, which I strongly believe will > > make OE a better tool in the long run, by reducing complexity, and thus > > lowering the bar for contributing to OE archictural work. > > I agree with this, but it seems unrelated to the question of whether > build dependencies should be specified in terms of recipes or output > packages. Yes, I get your point. You could of-course still specify build dependencies using recipe names/provides, and then just unpack all target packages build by that recipe. This would give you the major part of the KISS win I see, except for in the dependency handling code of BitBake (and in the learning curve of OE developers). Keep in mind that by using packages for build-time dependencies, the mechanism is exactly identical for run-time and build-time dependencies. Exactly the same code in BitBake handles both, and the same mechanism needs to be understood. This is also part of the KISS win of doing it this way. > > The maintenance headache you talk about is already there. In OE-lite > > build-time dependency 95% of the time just follow the run-time > > dependencies, perhaps making it easier to maintain than OE, as we don't > > have to think about another type of "item names" to depend on. > > (How) do you deal with library package renaming in OE-lite? What exactly do you mean? (We are doing several things with library package naming...) /Esben
Hello Martin/Frans and Khem, Any recommendation after following analysis that how should I proceed. Should I send the patches to community? Regards, Noor -----Original Message----- From: openembedded-devel-bounces@lists.openembedded.org [mailto:openembedded-devel-bounces@lists.openembedded.org] On Behalf Of Ahsan, Noor Sent: Thursday, March 17, 2011 3:41 PM To: openembedded-devel@lists.openembedded.org Subject: Re: [oe] Python-native dependency in libxml2 Hello, Today I got the fresh stuff and tried to build gnome-doc-utils-native (to build xml2po package). I am building on Ubuntu host 9.10. First it gave me an error that it could not find intltoolize command. You can see the attached build log "build_intltool_error,log". So I added intltool package in gnome-doc-utils-native DEPENDS so it built successfully. You can see the build log "build_intltool_fix.log". Then I removed all the build stuff from build DIR and commented DEPENDS_virtclass-native = "python-native" and XPY_virtclass-native = " --with-python=${PYTHON_DIR}" in libxml2.inc file. Rebuild gnome-doc-utils-native. It built successfully again. The build log is also attached "build_no_python.log". I think there is no need of python dependency in libxml2 package. Kindly correct me if I am doing anything wrong. We need to fix gnome-doc-utils-native recipe, so that it does not show intltoolize error. Regards, Noor -----Original Message----- From: openembedded-devel-bounces@lists.openembedded.org [mailto:openembedded-devel-bounces@lists.openembedded.org] On Behalf Of Ahsan, Noor Sent: Wednesday, March 16, 2011 1:39 PM To: openembedded-devel@lists.openembedded.org Subject: Re: [oe] Python-native dependency in libxml2 Hello, I made 2 changes, first one was to remove python-native from DEPENDS and second was commented out XPY_virtclass-native = " --with-python=${PYTHON_DIR}". Moreover I removed all the stuff from build dir, so there were not python-native for me. After kicking this build libxml2-native build fine. I was looking for xml2po recipe, I couldn't find it. Can you tell me how to build xml2po package so that I can reproduce the problem. Regards, Noor -----Original Message----- From: openembedded-devel-bounces@lists.openembedded.org [mailto:openembedded-devel-bounces@lists.openembedded.org] On Behalf Of Martin Jansa Sent: Wednesday, March 16, 2011 1:05 PM To: openembedded-devel@lists.openembedded.org Subject: Re: [oe] Python-native dependency in libxml2 On Wed, Mar 16, 2011 at 08:28:39AM +0100, Frans Meulenbroeks wrote: > 2011/3/16 Khem Raj <raj.khem@gmail.com>: > > On 3/15/2011 11:22 PM, Ahsan, Noor wrote: > >> > >> Hello, > >> > >> I was looking at the libxml2 package. While going through the recipes I > >> came to know that its native package depends on python-native package as > >> well. This dependency is defined in libxml2.inc file. > >> > >> DEPENDS_virtclass-native = "python-native" > >> > >> This package build fine after removing this dependency. I am not clear > >> why this dependency is being added in libxml2 package. Can somebody > >> clear that to me. While looking at the git log I came to know that this > >> change is coming from libxml2-native.inc file. But I am not clear why > >> this dependency was added as this package build fine without it. > >> > >> Please help me in understanding this dependency. Thanks. > > > > Try to build this recipe from scratch and see if it still works then we can > > remove the dep That's not enough, libxml2-native builds fine.. but then xml2po will fail if there isn't libxml2-native built with python support. see a25f07ca6159e1cd5b8a215ca5da8405fb1b9053 So yes.. for such tests you should build recipe from scratch _and_ check if resulting package/sysroot is still the same. Regards, > I'm not sure if that is the right way. > It might well be that typically python-native is already there in a > scratch build because some earlier package needed it (or even because > of the scheduling of build steps. > > Probably a better way is to build with packaged staging (with staging > dir outside TMPDIR) then rm TMPDIR and bake the package. > Alternately peek into the package to see if it needs python. If it is > needed then the DEPENDS should stay. > > Looking at the issue at hand maybe the best person to ask is Martin > Jansa as he last touched that line according to git blame: > 3cd9ddbf recipes/libxml/libxml2.inc (Martin Jansa > 2010-02-26 12:32:53 +0100 5) DEPENDS_virtclass-native = > "python-native" > > The other thing is that further on in the .inc file I see: > XPY_virtclass-native = " --with-python=${PYTHON_DIR}" > > This strongly suggests a hard dependency on python, so removing the > DEPENDS "because it builds without it" does not seem to be a good plan > to me. > > My strong feeling is that recipes should be self-contained (so should > list all packages they need to build, minus the ones that are > inherited from classes (and maybe a few implicit ones like maybe make) > > Frans > > _______________________________________________ > Openembedded-devel mailing list > Openembedded-devel@lists.openembedded.org > http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel
On Fri, Mar 18, 2011 at 08:41:40AM +0100, Ahsan, Noor wrote: > Hello Martin/Frans and Khem, > Any recommendation after following analysis that how should I proceed. > Should I send the patches to community? Maybe koen will reply with more info about xml2po issue he found, imho it wasn't in xml2po build but while using xml2to to generate docs in some other recipe. IIRC I had similar issue in some other recipe, but unfortunatelly don't remember which one. Regards, > > Regards, > Noor > > -----Original Message----- > From: openembedded-devel-bounces@lists.openembedded.org > [mailto:openembedded-devel-bounces@lists.openembedded.org] On Behalf Of > Ahsan, Noor > Sent: Thursday, March 17, 2011 3:41 PM > To: openembedded-devel@lists.openembedded.org > Subject: Re: [oe] Python-native dependency in libxml2 > > Hello, > > Today I got the fresh stuff and tried to build gnome-doc-utils-native > (to build xml2po package). I am building on Ubuntu host 9.10. First it > gave me an error that it could not find intltoolize command. You can see > the attached build log "build_intltool_error,log". So I added intltool > package in gnome-doc-utils-native DEPENDS so it built successfully. You > can see the build log "build_intltool_fix.log". Then I removed all the > build stuff from build DIR and commented DEPENDS_virtclass-native = > "python-native" and XPY_virtclass-native = " > --with-python=${PYTHON_DIR}" in libxml2.inc file. Rebuild > gnome-doc-utils-native. It built successfully again. The build log is > also attached "build_no_python.log". I think there is no need of python > dependency in libxml2 package. Kindly correct me if I am doing anything > wrong. > > We need to fix gnome-doc-utils-native recipe, so that it does not show > intltoolize error. > > Regards, > Noor > > -----Original Message----- > From: openembedded-devel-bounces@lists.openembedded.org > [mailto:openembedded-devel-bounces@lists.openembedded.org] On Behalf Of > Ahsan, Noor > Sent: Wednesday, March 16, 2011 1:39 PM > To: openembedded-devel@lists.openembedded.org > Subject: Re: [oe] Python-native dependency in libxml2 > > Hello, > > I made 2 changes, first one was to remove python-native from DEPENDS and > second was commented out XPY_virtclass-native = " > --with-python=${PYTHON_DIR}". Moreover I removed all the stuff from > build dir, so there were not python-native for me. After kicking this > build libxml2-native build fine. > > I was looking for xml2po recipe, I couldn't find it. Can you tell me how > to build xml2po package so that I can reproduce the problem. > > Regards, > Noor > > -----Original Message----- > From: openembedded-devel-bounces@lists.openembedded.org > [mailto:openembedded-devel-bounces@lists.openembedded.org] On Behalf Of > Martin Jansa > Sent: Wednesday, March 16, 2011 1:05 PM > To: openembedded-devel@lists.openembedded.org > Subject: Re: [oe] Python-native dependency in libxml2 > > On Wed, Mar 16, 2011 at 08:28:39AM +0100, Frans Meulenbroeks wrote: > > 2011/3/16 Khem Raj <raj.khem@gmail.com>: > > > On 3/15/2011 11:22 PM, Ahsan, Noor wrote: > > >> > > >> Hello, > > >> > > >> I was looking at the libxml2 package. While going through the > recipes I > > >> came to know that its native package depends on python-native > package as > > >> well. This dependency is defined in libxml2.inc file. > > >> > > >> DEPENDS_virtclass-native = "python-native" > > >> > > >> This package build fine after removing this dependency. I am not > clear > > >> why this dependency is being added in libxml2 package. Can somebody > > >> clear that to me. While looking at the git log I came to know that > this > > >> change is coming from libxml2-native.inc file. But I am not clear > why > > >> this dependency was added as this package build fine without it. > > >> > > >> Please help me in understanding this dependency. Thanks. > > > > > > Try to build this recipe from scratch and see if it still works then > we can > > > remove the dep > > That's not enough, libxml2-native builds fine.. but then xml2po > will fail if there isn't libxml2-native built with python support. > > see a25f07ca6159e1cd5b8a215ca5da8405fb1b9053 > > So yes.. for such tests you should build recipe from scratch _and_ check > if resulting package/sysroot is still the same. > > Regards, > > > I'm not sure if that is the right way. > > It might well be that typically python-native is already there in a > > scratch build because some earlier package needed it (or even because > > of the scheduling of build steps. > > > > Probably a better way is to build with packaged staging (with staging > > dir outside TMPDIR) then rm TMPDIR and bake the package. > > Alternately peek into the package to see if it needs python. If it is > > needed then the DEPENDS should stay. > > > > Looking at the issue at hand maybe the best person to ask is Martin > > Jansa as he last touched that line according to git blame: > > 3cd9ddbf recipes/libxml/libxml2.inc (Martin Jansa > > 2010-02-26 12:32:53 +0100 5) DEPENDS_virtclass-native = > > "python-native" > > > > The other thing is that further on in the .inc file I see: > > XPY_virtclass-native = " --with-python=${PYTHON_DIR}" > > > > This strongly suggests a hard dependency on python, so removing the > > DEPENDS "because it builds without it" does not seem to be a good plan > > to me. > > > > My strong feeling is that recipes should be self-contained (so should > > list all packages they need to build, minus the ones that are > > inherited from classes (and maybe a few implicit ones like maybe make) > > > > Frans > > > > _______________________________________________ > > Openembedded-devel mailing list > > Openembedded-devel@lists.openembedded.org > > http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel > > -- > Martin 'JaMa' Jansa jabber: Martin.Jansa@gmail.com > > _______________________________________________ > Openembedded-devel mailing list > Openembedded-devel@lists.openembedded.org > http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel > > _______________________________________________ > Openembedded-devel mailing list > Openembedded-devel@lists.openembedded.org > http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel
2011/3/18 Ahsan, Noor <Noor_Ahsan@mentor.com>: > Hello Martin/Frans and Khem, > > Any recommendation after following analysis that how should I proceed. > Should I send the patches to community? > > Regards, > Noor I think Martin is in a better position to judge this than I am. Did you also verify that the generated output (the packages and the contents of the packages) is identical in the python and no-python case? Best regards, Frans
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 18-03-11 09:38, Martin Jansa wrote: > On Fri, Mar 18, 2011 at 08:41:40AM +0100, Ahsan, Noor wrote: >> Hello Martin/Frans and Khem, >> Any recommendation after following analysis that how should I proceed. >> Should I send the patches to community? > > Maybe koen will reply with more info about xml2po issue he found, imho > it wasn't in xml2po build but while using xml2to to generate docs in some > other recipe. The issue I had that I needed xml2{t,p}o to work on the target as well for native compilation. If that still works, Noors changes is fine by me. regards, Koen > > IIRC I had similar issue in some other recipe, but unfortunatelly don't > remember which one. > > Regards, > >> >> Regards, >> Noor >> >> -----Original Message----- >> From: openembedded-devel-bounces@lists.openembedded.org >> [mailto:openembedded-devel-bounces@lists.openembedded.org] On Behalf Of >> Ahsan, Noor >> Sent: Thursday, March 17, 2011 3:41 PM >> To: openembedded-devel@lists.openembedded.org >> Subject: Re: [oe] Python-native dependency in libxml2 >> >> Hello, >> >> Today I got the fresh stuff and tried to build gnome-doc-utils-native >> (to build xml2po package). I am building on Ubuntu host 9.10. First it >> gave me an error that it could not find intltoolize command. You can see >> the attached build log "build_intltool_error,log". So I added intltool >> package in gnome-doc-utils-native DEPENDS so it built successfully. You >> can see the build log "build_intltool_fix.log". Then I removed all the >> build stuff from build DIR and commented DEPENDS_virtclass-native = >> "python-native" and XPY_virtclass-native = " >> --with-python=${PYTHON_DIR}" in libxml2.inc file. Rebuild >> gnome-doc-utils-native. It built successfully again. The build log is >> also attached "build_no_python.log". I think there is no need of python >> dependency in libxml2 package. Kindly correct me if I am doing anything >> wrong. >> >> We need to fix gnome-doc-utils-native recipe, so that it does not show >> intltoolize error. >> >> Regards, >> Noor >> >> -----Original Message----- >> From: openembedded-devel-bounces@lists.openembedded.org >> [mailto:openembedded-devel-bounces@lists.openembedded.org] On Behalf Of >> Ahsan, Noor >> Sent: Wednesday, March 16, 2011 1:39 PM >> To: openembedded-devel@lists.openembedded.org >> Subject: Re: [oe] Python-native dependency in libxml2 >> >> Hello, >> >> I made 2 changes, first one was to remove python-native from DEPENDS and >> second was commented out XPY_virtclass-native = " >> --with-python=${PYTHON_DIR}". Moreover I removed all the stuff from >> build dir, so there were not python-native for me. After kicking this >> build libxml2-native build fine. >> >> I was looking for xml2po recipe, I couldn't find it. Can you tell me how >> to build xml2po package so that I can reproduce the problem. >> >> Regards, >> Noor >> >> -----Original Message----- >> From: openembedded-devel-bounces@lists.openembedded.org >> [mailto:openembedded-devel-bounces@lists.openembedded.org] On Behalf Of >> Martin Jansa >> Sent: Wednesday, March 16, 2011 1:05 PM >> To: openembedded-devel@lists.openembedded.org >> Subject: Re: [oe] Python-native dependency in libxml2 >> >> On Wed, Mar 16, 2011 at 08:28:39AM +0100, Frans Meulenbroeks wrote: >>> 2011/3/16 Khem Raj <raj.khem@gmail.com>: >>>> On 3/15/2011 11:22 PM, Ahsan, Noor wrote: >>>>> >>>>> Hello, >>>>> >>>>> I was looking at the libxml2 package. While going through the >> recipes I >>>>> came to know that its native package depends on python-native >> package as >>>>> well. This dependency is defined in libxml2.inc file. >>>>> >>>>> DEPENDS_virtclass-native = "python-native" >>>>> >>>>> This package build fine after removing this dependency. I am not >> clear >>>>> why this dependency is being added in libxml2 package. Can somebody >>>>> clear that to me. While looking at the git log I came to know that >> this >>>>> change is coming from libxml2-native.inc file. But I am not clear >> why >>>>> this dependency was added as this package build fine without it. >>>>> >>>>> Please help me in understanding this dependency. Thanks. >>>> >>>> Try to build this recipe from scratch and see if it still works then >> we can >>>> remove the dep >> >> That's not enough, libxml2-native builds fine.. but then xml2po >> will fail if there isn't libxml2-native built with python support. >> >> see a25f07ca6159e1cd5b8a215ca5da8405fb1b9053 >> >> So yes.. for such tests you should build recipe from scratch _and_ check >> if resulting package/sysroot is still the same. >> >> Regards, >> >>> I'm not sure if that is the right way. >>> It might well be that typically python-native is already there in a >>> scratch build because some earlier package needed it (or even because >>> of the scheduling of build steps. >>> >>> Probably a better way is to build with packaged staging (with staging >>> dir outside TMPDIR) then rm TMPDIR and bake the package. >>> Alternately peek into the package to see if it needs python. If it is >>> needed then the DEPENDS should stay. >>> >>> Looking at the issue at hand maybe the best person to ask is Martin >>> Jansa as he last touched that line according to git blame: >>> 3cd9ddbf recipes/libxml/libxml2.inc (Martin Jansa >>> 2010-02-26 12:32:53 +0100 5) DEPENDS_virtclass-native = >>> "python-native" >>> >>> The other thing is that further on in the .inc file I see: >>> XPY_virtclass-native = " --with-python=${PYTHON_DIR}" >>> >>> This strongly suggests a hard dependency on python, so removing the >>> DEPENDS "because it builds without it" does not seem to be a good plan >>> to me. >>> >>> My strong feeling is that recipes should be self-contained (so should >>> list all packages they need to build, minus the ones that are >>> inherited from classes (and maybe a few implicit ones like maybe make) >>> >>> Frans >>> >>> _______________________________________________ >>> Openembedded-devel mailing list >>> Openembedded-devel@lists.openembedded.org >>> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel >> >> -- >> Martin 'JaMa' Jansa jabber: Martin.Jansa@gmail.com >> >> _______________________________________________ >> Openembedded-devel mailing list >> Openembedded-devel@lists.openembedded.org >> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel >> >> _______________________________________________ >> Openembedded-devel mailing list >> Openembedded-devel@lists.openembedded.org >> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel > > > > _______________________________________________ > Openembedded-devel mailing list > Openembedded-devel@lists.openembedded.org > http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iD8DBQFNgypFMkyGM64RGpERAka3AJ91Dpu6+SCx3MH0HtK6LUyEnaH7tACeM4EP b0nPj51+QTs4wdONs4BCP7Y= =Zxcy -----END PGP SIGNATURE-----
On Fri, 2011-03-18 at 06:29 +0100, Esben Haabendal wrote: > Yes, I get your point. You could of-course still specify build > dependencies using recipe names/provides, and then just unpack all > target packages build by that recipe. > > This would give you the major part of the KISS win I see, except for in > the dependency handling code of BitBake (and in the learning curve of OE > developers). Keep in mind that by using packages for build-time > dependencies, the mechanism is exactly identical for run-time and > build-time dependencies. Exactly the same code in BitBake handles both, > and the same mechanism needs to be understood. This is also part of the > KISS win of doing it this way. Right. So, it seems to me that the appropriate way to attack the problem in OE is to start by doing exactly that: leave the syntax of DEPENDS alone for now, treat each entry in DEPENDS as meaning "all packages built by the named recipe", and then implement the per-recipe sysroot construction that I think we are agreed is highly desirable. Then, as a later refinement, we could introduce a mechanism for specifying dependencies more precisely, at the package level rather than the recipe level. We could do that either by extending the syntax of DEPENDS, something like: DEPENDS = "boost:boost-iostreams" (to say that you wanted just the boost-iostreams package from the boost recipe) or, alternately, by introducing a new variable which would either supplement or replace DEPENDS. Either of those would allow us to migrate to fine-grained dependencies without breaking existing recipes. > > (How) do you deal with library package renaming in OE-lite? > > What exactly do you mean? (We are doing several things with library > package naming...) I was thinking of situations like the Debian package autonamer, ie the thing that causes glibc-dev to come out named "libc6-dev.ipk" or similar depending on the soname of the library that was built. It seems like this would be a bit awkward to deal with if your dependencies were specified purely in terms of output packages. p.
Phil Blundell <philb@gnu.org> writes: > On Fri, 2011-03-18 at 06:29 +0100, Esben Haabendal wrote: >> Yes, I get your point. You could of-course still specify build >> dependencies using recipe names/provides, and then just unpack all >> target packages build by that recipe. >> >> This would give you the major part of the KISS win I see, except for in >> the dependency handling code of BitBake (and in the learning curve of OE >> developers). Keep in mind that by using packages for build-time >> dependencies, the mechanism is exactly identical for run-time and >> build-time dependencies. Exactly the same code in BitBake handles both, >> and the same mechanism needs to be understood. This is also part of the >> KISS win of doing it this way. > > Right. So, it seems to me that the appropriate way to attack the > problem in OE is to start by doing exactly that: leave the syntax of > DEPENDS alone for now, treat each entry in DEPENDS as meaning "all > packages built by the named recipe", and then implement the per-recipe > sysroot construction that I think we are agreed is highly desirable. By doing it this way, I don't think we are getting much closer to the second step (package-based build-time dependencies). The OE-lite implementation of per-recipe staging builds on top of package-based build-time dependencies, so to do it the other way around, would imply taking the sstate concept and extend it to do per-recipe sysroots. After that, I believe you have exactly the same amount of development left to get where OE-lite is in respect to package-based build-time depencies and per-recipe staging. As I see it, package-based build-time dependencies practically obsoletes sstate. So if we for a moment imagine that we have moved OE to have sstate based per-recipe staging, we still have the same amount of work to do in the following areas: * Modifying BitBake and OE metadata to handle package-based build-time dependencies * Modifying existing recipes to include package-based build-time dependency information You can see the per-recipe staging as a postive side-effect of the package-based build-time dependency implementation. So why take the detour instead of just doing it? > Then, as a later refinement, we could introduce a mechanism for > specifying dependencies more precisely, at the package level rather than > the recipe level. We could do that either by extending the syntax of > DEPENDS, something like: > > DEPENDS = "boost:boost-iostreams" > > (to say that you wanted just the boost-iostreams package from the boost > recipe) That is IMO just adding more cruft to OE. Not really what we need. > or, alternately, by introducing a new variable which would either > supplement or replace DEPENDS. Either of those would allow us to > migrate to fine-grained dependencies without breaking existing recipes. Sounds better. Also, just as for run-time dependencies, the build-time package dependencies is not necesarely the actual package names, but some provides names, so having a "recipe:package" syntax really does not make much sense. >> > (How) do you deal with library package renaming in OE-lite? >> >> What exactly do you mean? (We are doing several things with library >> package naming...) > > I was thinking of situations like the Debian package autonamer, ie the > thing that causes glibc-dev to come out named "libc6-dev.ipk" or similar > depending on the soname of the library that was built. It seems like > this would be a bit awkward to deal with if your dependencies were > specified purely in terms of output packages. As OE-lite is not OE, we are currently not supporting such type of package renaming. Let's leave that to Debian guys to fight with ;-) But as we build-time dependencies are typically in the form of some package "provides", it doesn't really matter if the package is renamed. /Esben
On Thu, 2011-03-17 at 20:58 +0100, Esben Haabendal wrote: > On Thu, 2011-03-17 at 18:05 +0000, Phil Blundell wrote: > > On Thu, 2011-03-17 at 18:52 +0100, Esben Haabendal wrote: > > > Well, it might be possible to minimize the disruption for a transitional > > > period by carefully specifying some catch-all build-time package > > > dependencies pulling in all packages for recipes not ported yet. > > > > Yes, that's the sort of thing I was thinking of. It isn't even totally > > obvious to me that specifying individual packages is any better than > > "all packages from recipe X", since with the former you then have a > > potential maintenance headache if files get moved from one package to > > another. > > There is a number of ways that I believe package based build > dependencies are better than recipe based. > > a) It is possible to depend on parts of a recipe, which fx. is useful > when a recipe provides more than 1 library, where you might not want all > of them. > > b) To build a recipe, you depend on some stuff which you don't need to, > or perhaps even really don't want to pass on to recipes depending on > this recipe. > > c) KISS. Using the actual target packages for satisfying build-time > dependencies are a very simple approach, which I strongly believe will > make OE a better tool in the long run, by reducing complexity, and thus > lowering the bar for contributing to OE archictural work. I'd like to step in here and point out that your approach is not as simple as you claim. It certainly is a really nice solution for one very specific use case. There are however many other problems which appear when you step away from your isolated usecase. Take the case of different package managers, how is that handled? Can you only have one enabled at a given time? What happens if you switch package manager half way through a build. Currently package managers and package format are abstracted, I suspect we lose that with your implementation. What if the package managers have slightly different behaviours? Secondly, how about task output that isn't packages? The output of do_deploy for example(kernel/firmware/bootloader)? The pkgdata generated by do_package? A generic extra task added by the user which outputs some data? What you're proposing is to totally actually totally collapse our current generic task structure into something entirely governed by target packages and their specific requirements. Of course you could say the existing mechanisms are still available but then where is the simplification? sstate is very simple in concept and very simple in operation compared to anything we've previously had. It is generic being task based and can apply to any task as required. It doesn't require radical changes to the existing model, there is a clear migration path and it also embraces things like the checksum/signature generation which I believe are going to play a significant role in the future. This isn't to say what you're doing is wrong or that we shouldn't be looking at it for ideas but I believe it needs to be something incremental on top of what we already have and functionality we have now cannot simply be lost as people use it. Cheers, Richard
On Fri, 2011-03-18 at 13:14 +0100, Esben Haabendal wrote: > By doing it this way, I don't think we are getting much closer to the > second step (package-based build-time dependencies). The OE-lite > implementation of per-recipe staging builds on top of package-based > build-time dependencies, so to do it the other way around, would imply > taking the sstate concept and extend it to do per-recipe sysroots. > After that, I believe you have exactly the same amount of development > left to get where OE-lite is in respect to package-based build-time > depencies and per-recipe staging. > > As I see it, package-based build-time dependencies practically obsoletes > sstate. This is where I think you misunderstand why sstate has been written and what its role is. Its a generic abstraction of taking the output from a task, saving it away somewhere and then being able to reuse a prebuilt version of it. There is no dependency on a package manager, package format or special dependency formats. sstate simply looks for a prebuilt version (matching a checksum), uses it if present, otherwise builds from "scratch". > > I was thinking of situations like the Debian package autonamer, ie the > > thing that causes glibc-dev to come out named "libc6-dev.ipk" or similar > > depending on the soname of the library that was built. It seems like > > this would be a bit awkward to deal with if your dependencies were > > specified purely in terms of output packages. > > As OE-lite is not OE, we are currently not supporting such type of > package renaming. Let's leave that to Debian guys to fight with ;-) What other features are you not supporting? Are you prepared to think about them or is that someone else's problem? Cheers, Richard
Richard Purdie <richard.purdie@linuxfoundation.org> writes: > On Thu, 2011-03-17 at 20:58 +0100, Esben Haabendal wrote: >> On Thu, 2011-03-17 at 18:05 +0000, Phil Blundell wrote: >> > On Thu, 2011-03-17 at 18:52 +0100, Esben Haabendal wrote: >> > > Well, it might be possible to minimize the disruption for a transitional >> > > period by carefully specifying some catch-all build-time package >> > > dependencies pulling in all packages for recipes not ported yet. >> > >> > Yes, that's the sort of thing I was thinking of. It isn't even totally >> > obvious to me that specifying individual packages is any better than >> > "all packages from recipe X", since with the former you then have a >> > potential maintenance headache if files get moved from one package to >> > another. >> >> There is a number of ways that I believe package based build >> dependencies are better than recipe based. >> >> a) It is possible to depend on parts of a recipe, which fx. is useful >> when a recipe provides more than 1 library, where you might not want all >> of them. >> >> b) To build a recipe, you depend on some stuff which you don't need to, >> or perhaps even really don't want to pass on to recipes depending on >> this recipe. >> >> c) KISS. Using the actual target packages for satisfying build-time >> dependencies are a very simple approach, which I strongly believe will >> make OE a better tool in the long run, by reducing complexity, and thus >> lowering the bar for contributing to OE archictural work. > > I'd like to step in here and point out that your approach is not as > simple as you claim. It certainly is a really nice solution for one very > specific use case. > > There are however many other problems which appear when you step away > from your isolated usecase. > > Take the case of different package managers, how is that handled? > Can you only have one enabled at a given time? What happens if you > switch package manager half way through a build. Currently package > managers and package format are abstracted, I suspect we lose that > with your implementation. What if the package managers have slightly > different behaviours? In OE-lite, which is designed for a much more specific use-case than OE, it is handled very simple. OE-lite (currently) does not support package managers at all. The package format used is simple flat tarballs. The way I see package managers handled would be to add them on top of that. With that I mean that the meta-data contains the necessary packaging information, this is stored in some standardized meta-data container stored together with the tarballs (but propbably in separate files), and packages (RPM/dpkg/ipkg) generation would be implemented on top of that. I don't believe it is worth the added complexity to support package managers internally in OE-lite. But if OE were to implement something similar, then I don't see any real showstoppers to extending the do_stage/do_rstage functions to handle multiple package managers. But I believe it would be best to let BitBake handle all the dependencie stuff as OE-lite of-course does. > Secondly, how about task output that isn't packages? As of now, there is no other type of output. But I don't see any real blockers for having non-package output. That said, in OE-lite the goal is to only have packages as task output. > The output of do_deploy for example(kernel/firmware/bootloader)? The output from do_deploy is never used by other tasks. Instead, the recipes that uses do_deploy can just arrange for a package to generated with the stuff that needs by other tasks (like the things you mentioned above). > The pkgdata generated by do_package? As OE-lite does not generate packages for real package managers (just tarballs), this is currently not handled. But I would just store it in some way together with the tarball (fx. *.tar and *.pkgdata). > A generic extra task added by the user which outputs some data? Like what? > What you're proposing is to totally actually totally collapse our > current generic task structure into something entirely governed by > target packages and their specific requirements. I partly agree. The proposal is to let target packages have a much more central role than in current OE. But the task structure in OE-lite is actually very much similar to in OE, so I don't think really agree with your wording (totally collapse). > Of course you could say the existing mechanisms are still available And they are. > but then where is the simplification? The simplification I have been trying to explain here is the fact that I don't have to handle two different ways if dependency-handling for build- and run-time dependencies. They are completely identical. Also, I don't have to care about crafting any kind of special sysroot/sstate packages, just target packages (tarballs). > sstate is very simple in concept and very simple in operation compared > to anything we've previously had. Could be. But my main point is that it still builds on what I believe is a fundamentally flawed concept, ie. the idea of sharing anything between recipes other than clearly defined entities (ie. packages). > It is generic being task based and can apply to any task as > required. It doesn't require radical changes to the existing model, > there is a clear migration path I agree on that. > and it also embraces things like the checksum/signature generation > which I believe are going to play a significant role in the future. Yes, that will definitely be important in the future. But I don't see how checksum/signature generation is made much easier by the design concepts being discussed here. > This isn't to say what you're doing is wrong or that we shouldn't be > looking at it for ideas but I believe it needs to be something > incremental on top of what we already have and functionality we have now > cannot simply be lost as people use it. And that is probably the core of this discussion. I am happy to see that there seems to be some kind of concensus that what I am doing/showing in OE-lite has some clear merits. So the important choice is if the absolute driving force for development has to be that everything has to be "incremental on top of what we already have", or it is sometimes better to replace something. I basically does not see as much value in "incremental on top of ..." as you do, but understand and appreciate the need for a transitional phase for meta-data. But these are really two different issues. If something (like this proposal for "real" package-based staging/sysroot) is chosen for the OE future, a plan for transition needs to be made and executed. But that is not the same as to say that we cannot at the same time replace some core parts/concepts. /Esben
Richard Purdie <richard.purdie@linuxfoundation.org> writes: > On Fri, 2011-03-18 at 13:14 +0100, Esben Haabendal wrote: >> By doing it this way, I don't think we are getting much closer to the >> second step (package-based build-time dependencies). The OE-lite >> implementation of per-recipe staging builds on top of package-based >> build-time dependencies, so to do it the other way around, would imply >> taking the sstate concept and extend it to do per-recipe sysroots. >> After that, I believe you have exactly the same amount of development >> left to get where OE-lite is in respect to package-based build-time >> depencies and per-recipe staging. >> >> As I see it, package-based build-time dependencies practically obsoletes >> sstate. > > This is where I think you misunderstand why sstate has been written and > what its role is. Please note my use of "practically", which should have been quoted or something... > Its a generic abstraction of taking the output from a task, saving it > away somewhere and then being able to reuse a prebuilt version of it. I know. And this is clearly different to what package-based build-time dependencies give. Without sstate, you will not be able to reuse prebuilt versions of any random task. You can only reuse prebuilt versions of packages (ie. do_package tasks). In practice, I believe that is sufficient for most needs. But I don't see any showstoppers for using something sstate together with package-based build-time dependencies. I am just saying that what I believes is the most important value of sstate, more robus builds, is already provided the package-based build-time dependencies. > There is no dependency on a package manager, package format or special > dependency formats. sstate simply looks for a prebuilt version (matching > a checksum), uses it if present, otherwise builds from "scratch". > >> > I was thinking of situations like the Debian package autonamer, ie the >> > thing that causes glibc-dev to come out named "libc6-dev.ipk" or similar >> > depending on the soname of the library that was built. It seems like >> > this would be a bit awkward to deal with if your dependencies were >> > specified purely in terms of output packages. >> >> As OE-lite is not OE, we are currently not supporting such type of >> package renaming. Let's leave that to Debian guys to fight with ;-) > > What other features are you not supporting? Are you prepared to think > about them or is that someone else's problem? I am not sure what direction you are trying to push the discussion now. I am not prepared to single-handedly (without a large $$$ sponsor) take over complete responsibility for all of OE. I am though prepared to think about them, and also prepared to work with the OE community to find good solution to everybody else's problems. I don't think it is totally fair to imply that I am being completely ignorant to other people's problems. I know we are ignoring a lot of OE features in OE-lite, but that is the reason it is done as OE-lite and not in OE. I did not have time and manpower to implement what I needed in the timeframes I had without ignoring those features. I am though now approach you (OE community) trying to find out if we can find a way to get some of the work done in OE-lite back to OE, just as you have done several times with features developed in Poky going back to OE. Talking about that, with Poky being much more directly coupled with OE, where will new features be developed in the future? This discussion seems to indicate that it is not going to be in OE as such. /Esben
Dear all, pb and I had a discussion about this on irc earlier today. Actually we feel there are three different issues in the proposal: 1. only stage those recipes that are in DEPENDS. 2. stage based on packages instead of recipes (so e.g. you can stage only a lib package) 3. use tar (or a different tool) to stage things; or in other words, decouple the storage format and the package format. I think each of these has its merits. I also understand that incremental introduction is desired above a big bang approach. Let us look at the issues in detail: 1. only stage those recipes that are in DEPENDS. This seems pretty useful as it gives consistent builds not depending on whatever you have build before (and so making things deterministic) This is especially useful when doing incremental builds. It can also solve races. Introducing it would mean changes in staging; incremental introduction is possible if e.g. we add a flag PACKAGE_BASED_STAGING or so to the recipes we want this to have (or add a special class or so). Once all recipes are moved, the flag can be removed again. Question is how efficient this can be. It mostly depends on how we want to achieve the staging. I'm definitely no expert, but we could e.g. - create the staging area from earlier package images each time we build it. Does not seem overly efficient - stage each package in its own dir and create symlinks in the required staging area. - stage each pacakge in its own dir and add to -I and -L to only stage the packages that are needed. The staging area for a package to be build should probably be within the work dir of the recipe otherwise we'll loose the capability to build things in parallel 2. stage based on packages instead of recipes (so e.g. you can stage only a lib package) This gives more fine grained control. Not sure how much it brings and what it will cost. It might be possible to do it incrementally by e.g. having a keyword PACKAGE_DEPENDS. Question is if it is worth the effort 3. use tar (or a different tool) to stage things; or in other words, decouple the storage format and the package format. To me that is a completely different question only loosely related to the above. If staging is from packaged images, it would help though to have a format that is fast to unpack. Another, perhaps more interesting reason, would be to use a package manager agnostic format and use that to build packages from (for the package manager you want to have it). I don't oversee the impact of this one though. Your opinion on this proposal, improvement suggestions, discussion on the best way forward etc is appreciated. Frans
Patch
diff --git a/recipes/net-snmp/net-snmp_svn.bb b/recipes/net-snmp/net-snmp_svn.bb index 2a75722..c53c91f 100644 --- a/recipes/net-snmp/net-snmp_svn.bb +++ b/recipes/net-snmp/net-snmp_svn.bb @@ -1,7 +1,7 @@ require net-snmp.inc DEPENDS += "libtool libtool-native libtool-cross" SRCREV = "20069" -PR = "${INC_PR}.0" +PR = "${INC_PR}.1" S = "${WORKDIR}/net-snmp" @@ -10,7 +10,7 @@ SRC_URI = "svn://net-snmp.svn.sourceforge.net/svnroot/net-snmp/trunk;module=net- file://snmpd.conf \ file://snmptrapd.conf" -EXTRA_OECONF += "--disable-embedded-perl --with-perl-modules=no GREP=/bin/grep SED=/bin/sed" +EXTRA_OECONF += "--disable-embedded-perl --with-perl-modules=no --without-nl GREP=/bin/grep SED=/bin/sed" EXTRA_OEMAKE = "INSTALL_PREFIX=${D}" CCACHE = ""
Fixes compile error | ./.libs/libnetsnmpmibs.so: undefined reference to `nl_handle_destroy' | ./.libs/libnetsnmpmibs.so: undefined reference to `nl_handle_alloc' which occurred sometimes depending on build order (not in clean package only builds). Signed-off-by: Steffen Sledz <sledz@dresearch.de> --- recipes/net-snmp/net-snmp_svn.bb | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-)