[bitbake-devel] runqueue: add mc to pending hash index

Submitted by Kang Kai on Nov. 28, 2019, 9:27 a.m. | Patch ID: 167483

Details

Message ID 20191128092718.13336-1-kai.kang@windriver.com
State New
Headers show

Commit Message

Kang Kai Nov. 28, 2019, 9:27 a.m.
From: Kai Kang <kai.kang@windriver.com>

When build multiconfig target by following steps in section "Enabling
Multiple Configuration Build Dependencies" in yocto dev-manual which
adds dependency between multiconfig targets:

do_image[mcdepends] = "mc:x86:arm:core-image-minimal:do_rootfs"

it shows a lot of warnings, such as:

| WARNING: Deferring
|   mc:x86:virtual:native:/path/to/meta/recipes-support/nss/nss_3.45.bb:do_populate_sysroot
|   after
|   mc:arm:virtual:native:/path/to/meta/recipes-support/nss/nss_3.45.bb:do_populate_sysroot

In function pending_hash_index it only checks PN, taskname and hash. But
they are same for multiconfig native and allarch packages. Add multiconfig
to pending hash index too to make the index unique.

Ref:
https://www.yoctoproject.org/docs/latest/dev-manual/dev-manual.html#dev-enabling-multiple-configuration-build-dependencies

Signed-off-by: Kai Kang <kai.kang@windriver.com>
---
 bitbake/lib/bb/runqueue.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

Patch hide | download patch | download mbox

diff --git a/bitbake/lib/bb/runqueue.py b/bitbake/lib/bb/runqueue.py
index 485d9341f5..059eb9e283 100644
--- a/bitbake/lib/bb/runqueue.py
+++ b/bitbake/lib/bb/runqueue.py
@@ -69,12 +69,12 @@  def build_tid(mc, fn, taskname):
     return fn + ":" + taskname
 
 # Index used to pair up potentially matching multiconfig tasks
-# We match on PN, taskname and hash being equal
+# We match on mc, PN, taskname and hash being equal
 def pending_hash_index(tid, rqdata):
     (mc, fn, taskname, taskfn) = split_tid_mcfn(tid)
     pn = rqdata.dataCaches[mc].pkg_fn[taskfn]
     h = rqdata.runtaskentries[tid].unihash
-    return pn + ":" + "taskname" + h
+    return mc + ":" + pn + ":" + "taskname" + h
 
 class RunQueueStats:
     """

Comments

Richard Purdie Nov. 28, 2019, 5:51 p.m.
On Thu, 2019-11-28 at 17:27 +0800, kai.kang@windriver.com wrote:
> From: Kai Kang <kai.kang@windriver.com>
> 
> When build multiconfig target by following steps in section "Enabling
> Multiple Configuration Build Dependencies" in yocto dev-manual which
> adds dependency between multiconfig targets:
> 
> do_image[mcdepends] = "mc:x86:arm:core-image-minimal:do_rootfs"
> 
> it shows a lot of warnings, such as:
> 
> > WARNING: Deferring
> >   mc:x86:virtual:native:/path/to/meta/recipes-
> > support/nss/nss_3.45.bb:do_populate_sysroot
> >   after
> >   mc:arm:virtual:native:/path/to/meta/recipes-
> > support/nss/nss_3.45.bb:do_populate_sysroot
> 
> In function pending_hash_index it only checks PN, taskname and hash.
> But
> they are same for multiconfig native and allarch packages. Add
> multiconfig
> to pending hash index too to make the index unique.
> 
> Ref:
> https://www.yoctoproject.org/docs/latest/dev-manual/dev-manual.html#dev-enabling-multiple-configuration-build-dependencies
> 
> Signed-off-by: Kai Kang <kai.kang@windriver.com>

This patch completes defeats the point of this code, disabling it. Its
the wrong thing to do, we want this code to be active.

The bug is that this should not be a warning, probably just a note.

We need it to be a note so that we can tell the mechanism is in action
from build output logs.

Cheers,

Richard

> ---
>  bitbake/lib/bb/runqueue.py | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/bitbake/lib/bb/runqueue.py b/bitbake/lib/bb/runqueue.py
> index 485d9341f5..059eb9e283 100644
> --- a/bitbake/lib/bb/runqueue.py
> +++ b/bitbake/lib/bb/runqueue.py
> @@ -69,12 +69,12 @@ def build_tid(mc, fn, taskname):
>      return fn + ":" + taskname
>  
>  # Index used to pair up potentially matching multiconfig tasks
> -# We match on PN, taskname and hash being equal
> +# We match on mc, PN, taskname and hash being equal
>  def pending_hash_index(tid, rqdata):
>      (mc, fn, taskname, taskfn) = split_tid_mcfn(tid)
>      pn = rqdata.dataCaches[mc].pkg_fn[taskfn]
>      h = rqdata.runtaskentries[tid].unihash
> -    return pn + ":" + "taskname" + h
> +    return mc + ":" + pn + ":" + "taskname" + h
>  
>  class RunQueueStats:
>      """
> -- 
> 2.17.1
>
Kang Kai Nov. 29, 2019, 1:11 a.m.
On 2019/11/29 上午1:51, Richard Purdie wrote:
> On Thu, 2019-11-28 at 17:27 +0800, kai.kang@windriver.com wrote:
>> From: Kai Kang <kai.kang@windriver.com>
>>
>> When build multiconfig target by following steps in section "Enabling
>> Multiple Configuration Build Dependencies" in yocto dev-manual which
>> adds dependency between multiconfig targets:
>>
>> do_image[mcdepends] = "mc:x86:arm:core-image-minimal:do_rootfs"
>>
>> it shows a lot of warnings, such as:
>>
>>> WARNING: Deferring
>>>    mc:x86:virtual:native:/path/to/meta/recipes-
>>> support/nss/nss_3.45.bb:do_populate_sysroot
>>>    after
>>>    mc:arm:virtual:native:/path/to/meta/recipes-
>>> support/nss/nss_3.45.bb:do_populate_sysroot
>> In function pending_hash_index it only checks PN, taskname and hash.
>> But
>> they are same for multiconfig native and allarch packages. Add
>> multiconfig
>> to pending hash index too to make the index unique.
>>
>> Ref:
>> https://www.yoctoproject.org/docs/latest/dev-manual/dev-manual.html#dev-enabling-multiple-configuration-build-dependencies
>>
>> Signed-off-by: Kai Kang <kai.kang@windriver.com>
> This patch completes defeats the point of this code, disabling it. Its
> the wrong thing to do, we want this code to be active.
>
> The bug is that this should not be a warning, probably just a note.
>
> We need it to be a note so that we can tell the mechanism is in action
> from build output logs.

I have sent a patch to change from warning to note. But without 
comment,  I thought it should be fixed in this function.
Would you like to review the patch "runqueue.py: not show warning for 
deferred multiconfig task" on Oct 17, please?

Thanks,
Kai



>
> Cheers,
>
> Richard
>
>> ---
>>   bitbake/lib/bb/runqueue.py | 4 ++--
>>   1 file changed, 2 insertions(+), 2 deletions(-)
>>
>> diff --git a/bitbake/lib/bb/runqueue.py b/bitbake/lib/bb/runqueue.py
>> index 485d9341f5..059eb9e283 100644
>> --- a/bitbake/lib/bb/runqueue.py
>> +++ b/bitbake/lib/bb/runqueue.py
>> @@ -69,12 +69,12 @@ def build_tid(mc, fn, taskname):
>>       return fn + ":" + taskname
>>   
>>   # Index used to pair up potentially matching multiconfig tasks
>> -# We match on PN, taskname and hash being equal
>> +# We match on mc, PN, taskname and hash being equal
>>   def pending_hash_index(tid, rqdata):
>>       (mc, fn, taskname, taskfn) = split_tid_mcfn(tid)
>>       pn = rqdata.dataCaches[mc].pkg_fn[taskfn]
>>       h = rqdata.runtaskentries[tid].unihash
>> -    return pn + ":" + "taskname" + h
>> +    return mc + ":" + pn + ":" + "taskname" + h
>>   
>>   class RunQueueStats:
>>       """
>> -- 
>> 2.17.1
>>
>
Richard Purdie Nov. 29, 2019, 11:31 a.m.
On Fri, 2019-11-29 at 09:11 +0800, Kang Kai wrote:
> On 2019/11/29 上午1:51, Richard Purdie wrote:
> > On Thu, 2019-11-28 at 17:27 +0800, kai.kang@windriver.com wrote:
> > > From: Kai Kang <kai.kang@windriver.com>
> > > 
> > > When build multiconfig target by following steps in section
> > > "Enabling
> > > Multiple Configuration Build Dependencies" in yocto dev-manual
> > > which
> > > adds dependency between multiconfig targets:
> > > 
> > > do_image[mcdepends] = "mc:x86:arm:core-image-minimal:do_rootfs"
> > > 
> > > it shows a lot of warnings, such as:
> > > 
> > > > WARNING: Deferring
> > > >    mc:x86:virtual:native:/path/to/meta/recipes-
> > > > support/nss/nss_3.45.bb:do_populate_sysroot
> > > >    after
> > > >    mc:arm:virtual:native:/path/to/meta/recipes-
> > > > support/nss/nss_3.45.bb:do_populate_sysroot
> > > In function pending_hash_index it only checks PN, taskname and
> > > hash.
> > > But
> > > they are same for multiconfig native and allarch packages. Add
> > > multiconfig
> > > to pending hash index too to make the index unique.
> > > 
> > > Ref:
> > > https://www.yoctoproject.org/docs/latest/dev-manual/dev-manual.html#dev-enabling-multiple-configuration-build-dependencies
> > > 
> > > Signed-off-by: Kai Kang <kai.kang@windriver.com>
> > This patch completes defeats the point of this code, disabling it.
> > Its
> > the wrong thing to do, we want this code to be active.
> > 
> > The bug is that this should not be a warning, probably just a note.
> > 
> > We need it to be a note so that we can tell the mechanism is in
> > action
> > from build output logs.
> 
> I have sent a patch to change from warning to note. But without 
> comment,  I thought it should be fixed in this function.
> Would you like to review the patch "runqueue.py: not show warning
> for 
> deferred multiconfig task" on Oct 17, please?

I thought I'd seen a proper fix!

That patch is correct, not sure why it wasn't merged but I'd thought it
should be. Its merged now, thanks.

Cheers,

Richard