Patchwork [1/2] make: add make-replacement-native 3.81

login
register
mail settings
Submitter Ross Burton
Date Aug. 15, 2012, 4:18 p.m.
Message ID <10192d7fd47ea041f73511eaa8b5e56c13152951.1345047378.git.ross.burton@intel.com>
Download mbox | patch
Permalink /patch/34651/
State New
Headers show

Comments

Ross Burton - Aug. 15, 2012, 4:18 p.m.
GNU Make 3.82 has some regressions regarding parallel processing that causes
some packages to fail, specifically WebKitGTK+.  Add make-replacement-native
3.81 so that those packages can depend on a Make which is known to work.

Signed-off-by: Ross Burton <ross.burton@intel.com>
---
 meta/recipes-devtools/make/make-replacement-native_3.81.bb |    6 ++++++
 1 file changed, 6 insertions(+)
 create mode 100644 meta/recipes-devtools/make/make-replacement-native_3.81.bb
Martin Jansa - Aug. 15, 2012, 9:12 p.m.
On Wed, Aug 15, 2012 at 05:18:49PM +0100, Ross Burton wrote:
> GNU Make 3.82 has some regressions regarding parallel processing that causes
> some packages to fail, specifically WebKitGTK+.  Add make-replacement-native
> 3.81 so that those packages can depend on a Make which is known to work.
> 
> Signed-off-by: Ross Burton <ross.burton@intel.com>
> ---
>  meta/recipes-devtools/make/make-replacement-native_3.81.bb |    6 ++++++
>  1 file changed, 6 insertions(+)
>  create mode 100644 meta/recipes-devtools/make/make-replacement-native_3.81.bb
> 
> diff --git a/meta/recipes-devtools/make/make-replacement-native_3.81.bb b/meta/recipes-devtools/make/make-replacement-native_3.81.bb
> new file mode 100644
> index 0000000..716a8b5
> --- /dev/null
> +++ b/meta/recipes-devtools/make/make-replacement-native_3.81.bb
> @@ -0,0 +1,6 @@
> +require make_${PV}.bb
> +
> +inherit native
> +
> +BPN = "make"
> +EXTRAINSTALL = ""

eglibc was failing to build with make-3.81
https://bugzilla.yoctoproject.org/show_bug.cgi?id=2314

so I guess once this is built (as webkit-gtk dependency), eglibc will maybe sometimes fail again..

Cheers,
Ross Burton - Aug. 15, 2012, 9:17 p.m.
On 15 August 2012 22:12, Martin Jansa <martin.jansa@gmail.com> wrote:
> eglibc was failing to build with make-3.81
> https://bugzilla.yoctoproject.org/show_bug.cgi?id=2314
>
> so I guess once this is built (as webkit-gtk dependency), eglibc will maybe sometimes fail again..

Interesting, this is the first time I've encountered 3.81 having
problems.  So we can't use 3.81 and we can't use 3.82 either. :)

Ross
Richard Purdie - Aug. 15, 2012, 9:39 p.m.
On Wed, 2012-08-15 at 17:18 +0100, Ross Burton wrote:
> GNU Make 3.82 has some regressions regarding parallel processing that causes
> some packages to fail, specifically WebKitGTK+.  Add make-replacement-native
> 3.81 so that those packages can depend on a Make which is known to work.
> 
> Signed-off-by: Ross Burton <ross.burton@intel.com>
> ---
>  meta/recipes-devtools/make/make-replacement-native_3.81.bb |    6 ++++++
>  1 file changed, 6 insertions(+)
>  create mode 100644 meta/recipes-devtools/make/make-replacement-native_3.81.bb
> 
> diff --git a/meta/recipes-devtools/make/make-replacement-native_3.81.bb b/meta/recipes-devtools/make/make-replacement-native_3.81.bb
> new file mode 100644
> index 0000000..716a8b5
> --- /dev/null
> +++ b/meta/recipes-devtools/make/make-replacement-native_3.81.bb
> @@ -0,0 +1,6 @@
> +require make_${PV}.bb
> +
> +inherit native
> +
> +BPN = "make"
> +EXTRAINSTALL = ""

Unfortunately we have some experience with these replacement recipes and
they need some "special" handling. You need to install make into a
subdirectory off ${bindir} and then add it to PATH in the webkit recipe.
There is precedent for this:

http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=3f05622bac998f351168eb49a5ca96e7473f51be

So BBCLASSEXTEND the make recipe, add a PROVIDES, then do a
EXTRANATIVEPATH += "make-native" in the webkit recipe.

The reason for this elaborate dance is so we avoid races, nothing should
be executing the make binary when we do anything to it with sstate
(install or remove it). Admittedly this is much more critical when there
is an associated library like bzip but we need to set the right example.

Cheers,

Richard
Phil Blundell - Aug. 15, 2012, 9:40 p.m.
On Wed, 2012-08-15 at 22:17 +0100, Burton, Ross wrote:
> On 15 August 2012 22:12, Martin Jansa <martin.jansa@gmail.com> wrote:
> > eglibc was failing to build with make-3.81
> > https://bugzilla.yoctoproject.org/show_bug.cgi?id=2314
> >
> > so I guess once this is built (as webkit-gtk dependency), eglibc will maybe sometimes fail again..
> 
> Interesting, this is the first time I've encountered 3.81 having
> problems.  So we can't use 3.81 and we can't use 3.82 either. :)

You can use 3.81, you just need to install it either under a different
name (e.g. call it "make-3.81" and set $MAKE) or in a different
directory (e.g. ${bindir}/make-3.81/make and set $PATH) so that other
recipes don't pick it up unexpectedly.

p.
Mark Hatle - Aug. 15, 2012, 9:44 p.m.
On 8/15/12 4:39 PM, Richard Purdie wrote:
> On Wed, 2012-08-15 at 17:18 +0100, Ross Burton wrote:
>> GNU Make 3.82 has some regressions regarding parallel processing that causes
>> some packages to fail, specifically WebKitGTK+.  Add make-replacement-native
>> 3.81 so that those packages can depend on a Make which is known to work.
>>
>> Signed-off-by: Ross Burton <ross.burton@intel.com>
>> ---
>>   meta/recipes-devtools/make/make-replacement-native_3.81.bb |    6 ++++++
>>   1 file changed, 6 insertions(+)
>>   create mode 100644 meta/recipes-devtools/make/make-replacement-native_3.81.bb
>>
>> diff --git a/meta/recipes-devtools/make/make-replacement-native_3.81.bb b/meta/recipes-devtools/make/make-replacement-native_3.81.bb
>> new file mode 100644
>> index 0000000..716a8b5
>> --- /dev/null
>> +++ b/meta/recipes-devtools/make/make-replacement-native_3.81.bb
>> @@ -0,0 +1,6 @@
>> +require make_${PV}.bb
>> +
>> +inherit native
>> +
>> +BPN = "make"
>> +EXTRAINSTALL = ""
>
> Unfortunately we have some experience with these replacement recipes and
> they need some "special" handling. You need to install make into a
> subdirectory off ${bindir} and then add it to PATH in the webkit recipe.
> There is precedent for this:

I was looking at the staging code the other day.. I suspect we could eliminate 
some of these problems with a few changes.

The order the items are staged is the first.  Right now bin and sbin are staged 
before libs.. so if we change that, it would resolve some of the problems we've 
seen.  The second thing is the way the copy itself occurs.. We should really do 
this in an atomic fashion..  use install, or copy to an alternative name, and 
then 'mv' to the actual name.

Only concern w/ the change to copy could be performance, but I'm really not sure 
how severe that is.... might also be possible to do that only in the 'native' cases.

--Mark

> http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=3f05622bac998f351168eb49a5ca96e7473f51be
>
> So BBCLASSEXTEND the make recipe, add a PROVIDES, then do a
> EXTRANATIVEPATH += "make-native" in the webkit recipe.
>
> The reason for this elaborate dance is so we avoid races, nothing should
> be executing the make binary when we do anything to it with sstate
> (install or remove it). Admittedly this is much more critical when there
> is an associated library like bzip but we need to set the right example.
>
> Cheers,
>
> Richard
>
>
>
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>
Andreas Müller - Aug. 16, 2012, 8:30 p.m.
On Wed, Aug 15, 2012 at 11:40 PM, Phil Blundell <philb@gnu.org> wrote:
> On Wed, 2012-08-15 at 22:17 +0100, Burton, Ross wrote:
>> On 15 August 2012 22:12, Martin Jansa <martin.jansa@gmail.com> wrote:
>> > eglibc was failing to build with make-3.81
>> > https://bugzilla.yoctoproject.org/show_bug.cgi?id=2314
>> >
>> > so I guess once this is built (as webkit-gtk dependency), eglibc will maybe sometimes fail again..
>>
>> Interesting, this is the first time I've encountered 3.81 having
>> problems.  So we can't use 3.81 and we can't use 3.82 either. :)
>
> You can use 3.81, you just need to install it either under a different
> name (e.g. call it "make-3.81" and set $MAKE) or in a different
> directory (e.g. ${bindir}/make-3.81/make and set $PATH) so that other
> recipes don't pick it up unexpectedly.
>
> p.
>
The following is really dirty but at least to me it succeeds and to me
it seems to have less influences than the 'make 3.81 dance':

# dirty hack for gnu make 3.82 race condition
do_compile() {
    if [ x"$MAKE" = x ]; then MAKE=make; fi
    bbnote make ${PARALLEL_MAKE} "$@"
    for error_count in 1 2 3; do
        bbnote "Number of trials: $error_count"
        exit_code=0
        make ${PARALLEL_MAKE} "$@" || exit_code=1
        if [ $exit_code = 0 ]; then
            break
        fi
    done
    if [ ! $exit_code = 0 ]; then
        die "oe_runmake failed"
    fi
}

I have make 3.82 and before this I had 100% error at first do_compile.
For rerun (without cleanup) I had 100% success. Now I get 100% success
with make called twice. Would something like have a chance to be
accepted?

Andreas
Martin Jansa - Aug. 16, 2012, 9:55 p.m.
On Thu, Aug 16, 2012 at 10:30:37PM +0200, Andreas Müller wrote:
> On Wed, Aug 15, 2012 at 11:40 PM, Phil Blundell <philb@gnu.org> wrote:
> > On Wed, 2012-08-15 at 22:17 +0100, Burton, Ross wrote:
> >> On 15 August 2012 22:12, Martin Jansa <martin.jansa@gmail.com> wrote:
> >> > eglibc was failing to build with make-3.81
> >> > https://bugzilla.yoctoproject.org/show_bug.cgi?id=2314
> >> >
> >> > so I guess once this is built (as webkit-gtk dependency), eglibc will maybe sometimes fail again..
> >>
> >> Interesting, this is the first time I've encountered 3.81 having
> >> problems.  So we can't use 3.81 and we can't use 3.82 either. :)
> >
> > You can use 3.81, you just need to install it either under a different
> > name (e.g. call it "make-3.81" and set $MAKE) or in a different
> > directory (e.g. ${bindir}/make-3.81/make and set $PATH) so that other
> > recipes don't pick it up unexpectedly.
> >
> > p.
> >
> The following is really dirty but at least to me it succeeds and to me
> it seems to have less influences than the 'make 3.81 dance':
> 
> # dirty hack for gnu make 3.82 race condition
> do_compile() {
>     if [ x"$MAKE" = x ]; then MAKE=make; fi
>     bbnote make ${PARALLEL_MAKE} "$@"
>     for error_count in 1 2 3; do
>         bbnote "Number of trials: $error_count"
>         exit_code=0
>         make ${PARALLEL_MAKE} "$@" || exit_code=1
>         if [ $exit_code = 0 ]; then
>             break
>         fi
>     done
>     if [ ! $exit_code = 0 ]; then
>         die "oe_runmake failed"
>     fi
> }
> 
> I have make 3.82 and before this I had 100% error at first do_compile.
> For rerun (without cleanup) I had 100% success. Now I get 100% success
> with make called twice. Would something like have a chance to be
> accepted?

Same for me, except I didn't go that far to reexecute make from recipe,
just executing feed build twice in for loop to be sure it will succeed.

Cheers,

Patch

diff --git a/meta/recipes-devtools/make/make-replacement-native_3.81.bb b/meta/recipes-devtools/make/make-replacement-native_3.81.bb
new file mode 100644
index 0000000..716a8b5
--- /dev/null
+++ b/meta/recipes-devtools/make/make-replacement-native_3.81.bb
@@ -0,0 +1,6 @@ 
+require make_${PV}.bb
+
+inherit native
+
+BPN = "make"
+EXTRAINSTALL = ""