Patchwork [1/2] sstatesig: include native/cross/nativesdk deps in target signatures

login
register
mail settings
Submitter Martin Jansa
Date Jan. 18, 2014, 2:02 p.m.
Message ID <981d02338666c368a371adb1bc0a5417b4a39138.1390052619.git.Martin.Jansa@gmail.com>
Download mbox | patch
Permalink /patch/65189/
State Accepted
Commit 9abc0028fc2b8d7d79b280d30bdbd987fca2b6d8
Headers show

Comments

Martin Jansa - Jan. 18, 2014, 2:02 p.m.
* I don't have any real evidence or good statistics for this, but when
  comparing signature dumps from my big bitbake world builds I usually
  see a lot of rebuilds caused by changes in .bbclasses and only very
  rare would be the case where oe-core upgrade brings changes in -native
  recipes and no change in .bbclasses used from target recipes
* changing the default to include them shouldn't cause significant
  increase in rebuilds and sstate reuse a bit safer
* people working on toolchain (e.g. using gcc from AUTOREV) can easily
  extend sstate_rundepfilter to ignore them again (it's easier than
  removing existing filter), example how add own signature handler in
  your layer is here:
  https://github.com/openwebos/meta-webos/commit/9ac3a7c803e7793b3274e4998f167b6278db8042

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
---
 meta/lib/oe/sstatesig.py | 4 ----
 1 file changed, 4 deletions(-)
Richard Purdie - Jan. 27, 2014, 2:51 p.m.
On Sat, 2014-01-18 at 15:02 +0100, Martin Jansa wrote:
> * I don't have any real evidence or good statistics for this, but when
>   comparing signature dumps from my big bitbake world builds I usually
>   see a lot of rebuilds caused by changes in .bbclasses and only very
>   rare would be the case where oe-core upgrade brings changes in -native
>   recipes and no change in .bbclasses used from target recipes
> * changing the default to include them shouldn't cause significant
>   increase in rebuilds and sstate reuse a bit safer
> * people working on toolchain (e.g. using gcc from AUTOREV) can easily
>   extend sstate_rundepfilter to ignore them again (it's easier than
>   removing existing filter), example how add own signature handler in
>   your layer is here:
>   https://github.com/openwebos/meta-webos/commit/9ac3a7c803e7793b3274e4998f167b6278db8042
> 
> Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
> ---
>  meta/lib/oe/sstatesig.py | 4 ----
>  1 file changed, 4 deletions(-)

Whilst I merged this, we do have a problem here. This change means that
the sstate is locked to the build system architecture.

For example, if you have a 32 bit machine, it can't use any of the
sstate built on a 64 bit machine and vice versa.

I'm not sure what we do about this but its a problem...

Cheers,

Richard

> diff --git a/meta/lib/oe/sstatesig.py b/meta/lib/oe/sstatesig.py
> index 3011f16..8b0e344 100644
> --- a/meta/lib/oe/sstatesig.py
> +++ b/meta/lib/oe/sstatesig.py
> @@ -36,10 +36,6 @@ def sstate_rundepfilter(siggen, fn, recipename, task, dep, depname, dataCache):
>      if isPackageGroup(fn):
>          return False  
>  
> -    # Drop native/cross/nativesdk dependencies from target recipes
> -    if isNative(depname) or isCross(depname) or isNativeSDK(depname):
> -        return False
> -
>      # Exclude well defined machine specific configurations which don't change ABI
>      if depname in siggen.abisaferecipes and not isImage(fn):
>          return False
Martin Jansa - Jan. 27, 2014, 3:53 p.m.
On Mon, Jan 27, 2014 at 02:51:15PM +0000, Richard Purdie wrote:
> On Sat, 2014-01-18 at 15:02 +0100, Martin Jansa wrote:
> > * I don't have any real evidence or good statistics for this, but when
> >   comparing signature dumps from my big bitbake world builds I usually
> >   see a lot of rebuilds caused by changes in .bbclasses and only very
> >   rare would be the case where oe-core upgrade brings changes in -native
> >   recipes and no change in .bbclasses used from target recipes
> > * changing the default to include them shouldn't cause significant
> >   increase in rebuilds and sstate reuse a bit safer
> > * people working on toolchain (e.g. using gcc from AUTOREV) can easily
> >   extend sstate_rundepfilter to ignore them again (it's easier than
> >   removing existing filter), example how add own signature handler in
> >   your layer is here:
> >   https://github.com/openwebos/meta-webos/commit/9ac3a7c803e7793b3274e4998f167b6278db8042
> > 
> > Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
> > ---
> >  meta/lib/oe/sstatesig.py | 4 ----
> >  1 file changed, 4 deletions(-)
> 
> Whilst I merged this, we do have a problem here. This change means that
> the sstate is locked to the build system architecture.
> 
> For example, if you have a 32 bit machine, it can't use any of the
> sstate built on a 64 bit machine and vice versa.
> 
> I'm not sure what we do about this but its a problem...

Agreed, that's bad, I was testing it in setup where we publish the
sstate archive from the builder with the same architecture and have
similar setup building just -native recipes on 32bit for 32bit devs.

So this problem should have hit us as well, I'll recheck.

> Cheers,
> 
> Richard
> 
> > diff --git a/meta/lib/oe/sstatesig.py b/meta/lib/oe/sstatesig.py
> > index 3011f16..8b0e344 100644
> > --- a/meta/lib/oe/sstatesig.py
> > +++ b/meta/lib/oe/sstatesig.py
> > @@ -36,10 +36,6 @@ def sstate_rundepfilter(siggen, fn, recipename, task, dep, depname, dataCache):
> >      if isPackageGroup(fn):
> >          return False  
> >  
> > -    # Drop native/cross/nativesdk dependencies from target recipes
> > -    if isNative(depname) or isCross(depname) or isNativeSDK(depname):
> > -        return False
> > -
> >      # Exclude well defined machine specific configurations which don't change ABI
> >      if depname in siggen.abisaferecipes and not isImage(fn):
> >          return False
> 
>

Patch

diff --git a/meta/lib/oe/sstatesig.py b/meta/lib/oe/sstatesig.py
index 3011f16..8b0e344 100644
--- a/meta/lib/oe/sstatesig.py
+++ b/meta/lib/oe/sstatesig.py
@@ -36,10 +36,6 @@  def sstate_rundepfilter(siggen, fn, recipename, task, dep, depname, dataCache):
     if isPackageGroup(fn):
         return False  
 
-    # Drop native/cross/nativesdk dependencies from target recipes
-    if isNative(depname) or isCross(depname) or isNativeSDK(depname):
-        return False
-
     # Exclude well defined machine specific configurations which don't change ABI
     if depname in siggen.abisaferecipes and not isImage(fn):
         return False