Patchwork lib/oe/sstatesig: Ensure packagegroups don't continually rebuild

login
register
mail settings
Submitter Richard Purdie
Date Nov. 5, 2013, 10:15 p.m.
Message ID <1383689722.6271.184.camel@ted>
Download mbox | patch
Permalink /patch/61179/
State New
Headers show

Comments

Richard Purdie - Nov. 5, 2013, 10:15 p.m.
packagegroups are allarch and shouldn't change depending on the target
or machine selected. In general they should have good stable namespaces
for their dependencies. As such we can exclude them from rebuilding when
dependency checksums change.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
Otavio Salvador - Nov. 6, 2013, 6:51 p.m.
On Tue, Nov 5, 2013 at 8:15 PM, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> packagegroups are allarch and shouldn't change depending on the target
> or machine selected. In general they should have good stable namespaces
> for their dependencies. As such we can exclude them from rebuilding when
> dependency checksums change.
>
> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

What about machine-base and core-boot? this may change depending on machine.
Richard Purdie - Nov. 6, 2013, 11:42 p.m.
On Wed, 2013-11-06 at 16:51 -0200, Otavio Salvador wrote:
> On Tue, Nov 5, 2013 at 8:15 PM, Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> > packagegroups are allarch and shouldn't change depending on the target
> > or machine selected. In general they should have good stable namespaces
> > for their dependencies. As such we can exclude them from rebuilding when
> > dependency checksums change.
> >
> > Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> 
> What about machine-base and core-boot? this may change depending on machine.

Those already get marked as machine specific. I'd not expect
dependencies to be changing package names there for example so I'm not
sure this is an issue.

Cheers,

Richard
Otavio Salvador - Nov. 7, 2013, 3:58 p.m.
On Wed, Nov 6, 2013 at 9:42 PM, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> On Wed, 2013-11-06 at 16:51 -0200, Otavio Salvador wrote:
>> On Tue, Nov 5, 2013 at 8:15 PM, Richard Purdie
>> <richard.purdie@linuxfoundation.org> wrote:
>> > packagegroups are allarch and shouldn't change depending on the target
>> > or machine selected. In general they should have good stable namespaces
>> > for their dependencies. As such we can exclude them from rebuilding when
>> > dependency checksums change.
>> >
>> > Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
>>
>> What about machine-base and core-boot? this may change depending on machine.
>
> Those already get marked as machine specific. I'd not expect
> dependencies to be changing package names there for example so I'm not
> sure this is an issue.

In:

+    def isPackageGroup(fn):
+        inherits = " ".join(dataCache.inherits[fn])
+        return "packagegroup.bbclass" in inherits

We must check PACKAGE_ARCH, no?
Richard Purdie - Nov. 7, 2013, 10:57 p.m.
On Thu, 2013-11-07 at 13:58 -0200, Otavio Salvador wrote:
> On Wed, Nov 6, 2013 at 9:42 PM, Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> > On Wed, 2013-11-06 at 16:51 -0200, Otavio Salvador wrote:
> >> On Tue, Nov 5, 2013 at 8:15 PM, Richard Purdie
> >> <richard.purdie@linuxfoundation.org> wrote:
> >> > packagegroups are allarch and shouldn't change depending on the target
> >> > or machine selected. In general they should have good stable namespaces
> >> > for their dependencies. As such we can exclude them from rebuilding when
> >> > dependency checksums change.
> >> >
> >> > Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> >>
> >> What about machine-base and core-boot? this may change depending on machine.
> >
> > Those already get marked as machine specific. I'd not expect
> > dependencies to be changing package names there for example so I'm not
> > sure this is an issue.
> 
> In:
> 
> +    def isPackageGroup(fn):
> +        inherits = " ".join(dataCache.inherits[fn])
> +        return "packagegroup.bbclass" in inherits
> 
> We must check PACKAGE_ARCH, no?

No. The fact they're marked as MACHINE_ARCH ensures they change when
they need to, we don't need sstate to do that as well.

Cheers,

Richard
Otavio Salvador - Nov. 8, 2013, 11:55 a.m.
On Thu, Nov 7, 2013 at 8:57 PM, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> On Thu, 2013-11-07 at 13:58 -0200, Otavio Salvador wrote:
>> On Wed, Nov 6, 2013 at 9:42 PM, Richard Purdie
>> <richard.purdie@linuxfoundation.org> wrote:
>> > On Wed, 2013-11-06 at 16:51 -0200, Otavio Salvador wrote:
>> >> On Tue, Nov 5, 2013 at 8:15 PM, Richard Purdie
>> >> <richard.purdie@linuxfoundation.org> wrote:
>> >> > packagegroups are allarch and shouldn't change depending on the target
>> >> > or machine selected. In general they should have good stable namespaces
>> >> > for their dependencies. As such we can exclude them from rebuilding when
>> >> > dependency checksums change.
>> >> >
>> >> > Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
>> >>
>> >> What about machine-base and core-boot? this may change depending on machine.
>> >
>> > Those already get marked as machine specific. I'd not expect
>> > dependencies to be changing package names there for example so I'm not
>> > sure this is an issue.
>>
>> In:
>>
>> +    def isPackageGroup(fn):
>> +        inherits = " ".join(dataCache.inherits[fn])
>> +        return "packagegroup.bbclass" in inherits
>>
>> We must check PACKAGE_ARCH, no?
>
> No. The fact they're marked as MACHINE_ARCH ensures they change when
> they need to, we don't need sstate to do that as well.

Fine :)

Patch

diff --git a/meta/lib/oe/sstatesig.py b/meta/lib/oe/sstatesig.py
index 852fb7e..33ff892 100644
--- a/meta/lib/oe/sstatesig.py
+++ b/meta/lib/oe/sstatesig.py
@@ -11,6 +11,9 @@  def sstate_rundepfilter(siggen, fn, recipename, task, dep, depname, dataCache):
     def isKernel(fn):
         inherits = " ".join(dataCache.inherits[fn])
         return inherits.find("module-base.bbclass") != -1 or inherits.find("linux-kernel-base.bbclass") != -1
+    def isPackageGroup(fn):
+        inherits = " ".join(dataCache.inherits[fn])
+        return "packagegroup.bbclass" in inherits
 
     # Always include our own inter-task dependencies
     if recipename == depname:
@@ -27,6 +30,10 @@  def sstate_rundepfilter(siggen, fn, recipename, task, dep, depname, dataCache):
 
     # Only target packages beyond here
 
+    # packagegroups are assumed to have well behaved names which don't change between architecures/tunes
+    if isPackageGroup(fn):
+        return False  
+
     # Drop native/cross/nativesdk dependencies from target recipes
     if isNative(depname) or isCross(depname) or isNativeSDK(depname):
         return False