[1/2] staging: Add extra hash handling code

Target build output, stored in do_populate_sysroot or do_package can depend
not only upon direct dependencies but also indirect ones. A good example is
linux-libc-headers. The toolchain depends on this but most target recipes do
not. There are some headers which are not used by the toolchain build and do
not change the toolchain task output, hence the task hashes can change without
changing the sysroot output of that recipe yet they can influence others.

A specific example is rtc.h which can change rtcwake.c in util-linux but is not
used in the glibc or gcc build. To account for this, we need to account for the
populate_sysroot hashes in the task output hashes.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
 meta/classes/sstate.bbclass  |  2 +-
 meta/classes/staging.bbclass | 35 ++++++++++++++++++++++++++++++++++-
 2 files changed, 35 insertions(+), 2 deletions(-)

On Thu, Feb 3, 2022 at 3:25 AM Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
>
> Target build output, stored in do_populate_sysroot or do_package can depend
> not only upon direct dependencies but also indirect ones. A good example is
> linux-libc-headers. The toolchain depends on this but most target recipes do
> not. There are some headers which are not used by the toolchain build and do
> not change the toolchain task output, hence the task hashes can change without
> changing the sysroot output of that recipe yet they can influence others.
>
> A specific example is rtc.h which can change rtcwake.c in util-linux but is not
> used in the glibc or gcc build. To account for this, we need to account for the
> populate_sysroot hashes in the task output hashes.
>
> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> ---
>  meta/classes/sstate.bbclass  |  2 +-
>  meta/classes/staging.bbclass | 35 ++++++++++++++++++++++++++++++++++-
>  2 files changed, 35 insertions(+), 2 deletions(-)
>
> diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
> index 17dcf4cc175..fd9acbab9b4 100644
> --- a/meta/classes/sstate.bbclass
> +++ b/meta/classes/sstate.bbclass
> @@ -94,7 +94,7 @@ SSTATE_ARCHS[vardepsexclude] = "ORIGNATIVELSBSTRING"
>
>  SSTATE_MANMACH ?= "${SSTATE_PKGARCH}"
>
> -SSTATECREATEFUNCS = "sstate_hardcode_path"
> +SSTATECREATEFUNCS += "sstate_hardcode_path"
>  SSTATECREATEFUNCS[vardeps] = "SSTATE_SCAN_FILES"
>  SSTATEPOSTCREATEFUNCS = ""
>  SSTATEPREINSTFUNCS = ""
> diff --git a/meta/classes/staging.bbclass b/meta/classes/staging.bbclass
> index 25f77c77350..3ea9c974eb5 100644
> --- a/meta/classes/staging.bbclass
> +++ b/meta/classes/staging.bbclass
> @@ -21,7 +21,7 @@ SYSROOT_DIRS_NATIVE = " \
>  "
>  SYSROOT_DIRS:append:class-native = " ${SYSROOT_DIRS_NATIVE}"
>  SYSROOT_DIRS:append:class-cross = " ${SYSROOT_DIRS_NATIVE}"
> -SYSROOT_DIRS:append:class-crosssdk = " ${SYSROOT_DIRS_NATIVE}"
> +SYSROOT_DIRS:app:class-crosssdk = " ${SYSROOT_DIRS_NATIVE}"

Did you intend to drop the "end" from "append" here?

-M

>
>  # These directories will not be staged in the sysroot
>  SYSROOT_DIRS_BLACKLIST = " \
> @@ -624,3 +624,36 @@ python staging_taskhandler() {
>  }
>  staging_taskhandler[eventmask] = "bb.event.RecipeTaskPreProcess"
>  addhandler staging_taskhandler
> +
> +
> +#
> +# Target build output, stored in do_populate_sysroot or do_package can depend
> +# not only upon direct dependencies but also indirect ones. A good example is
> +# linux-libc-headers. The toolchain depends on this but most target recipes do
> +# not. There are some headers which are not used by the toolchain build and do
> +# not change the toolchain task output, hence the task hashes can change without
> +# changing the sysroot output of that recipe yet they can influence others.
> +#
> +# A specific example is rtc.h which can change rtcwake.c in util-linux but is not
> +# used in the glibc or gcc build. To account for this, we need to account for the
> +# populate_sysroot hashes in the task output hashes.
> +#
> +python target_add_sysroot_deps () {
> +    current_task = "do_" + d.getVar("BB_CURRENTTASK")
> +    if current_task not in ["do_populate_sysroot", "do_package"]:
> +        return
> +
> +    pn = d.getVar("PN")
> +    if pn.endswith("-native"):
> +        return
> +
> +    taskdepdata = d.getVar("BB_TASKDEPDATA", False)
> +    deps = {}
> +    for dep in taskdepdata.values():
> +        if dep[1] == "do_populate_sysroot" and not dep[0].endswith(("-native", "-initial")) and "-cross-" not in dep[0]:
> +            deps[dep[0]] = dep[6]
> +
> +    d.setVar("HASHEQUIV_EXTRA_SIGDATA", "\n".join("%s: %s" % (k, deps[k]) for k in sorted(deps.keys())))
> +}
> +SSTATECREATEFUNCS += "target_add_sysroot_deps"
> +
> --
> 2.32.0
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#161273): https://lists.openembedded.org/g/openembedded-core/message/161273
> Mute This Topic: https://lists.openembedded.org/mt/88880595/3618418
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [matt@madison.systems]
> -=-=-=-=-=-=-=-=-=-=-=-
>

On Thu, 2022-02-03 at 03:31 -0800, Matt Madison wrote:
> On Thu, Feb 3, 2022 at 3:25 AM Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> > 
> > Target build output, stored in do_populate_sysroot or do_package can depend
> > not only upon direct dependencies but also indirect ones. A good example is
> > linux-libc-headers. The toolchain depends on this but most target recipes do
> > not. There are some headers which are not used by the toolchain build and do
> > not change the toolchain task output, hence the task hashes can change without
> > changing the sysroot output of that recipe yet they can influence others.
> > 
> > A specific example is rtc.h which can change rtcwake.c in util-linux but is not
> > used in the glibc or gcc build. To account for this, we need to account for the
> > populate_sysroot hashes in the task output hashes.
> > 
> > Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> > ---
> >  meta/classes/sstate.bbclass  |  2 +-
> >  meta/classes/staging.bbclass | 35 ++++++++++++++++++++++++++++++++++-
> >  2 files changed, 35 insertions(+), 2 deletions(-)
> > 
> > diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
> > index 17dcf4cc175..fd9acbab9b4 100644
> > --- a/meta/classes/sstate.bbclass
> > +++ b/meta/classes/sstate.bbclass
> > @@ -94,7 +94,7 @@ SSTATE_ARCHS[vardepsexclude] = "ORIGNATIVELSBSTRING"
> > 
> >  SSTATE_MANMACH ?= "${SSTATE_PKGARCH}"
> > 
> > -SSTATECREATEFUNCS = "sstate_hardcode_path"
> > +SSTATECREATEFUNCS += "sstate_hardcode_path"
> >  SSTATECREATEFUNCS[vardeps] = "SSTATE_SCAN_FILES"
> >  SSTATEPOSTCREATEFUNCS = ""
> >  SSTATEPREINSTFUNCS = ""
> > diff --git a/meta/classes/staging.bbclass b/meta/classes/staging.bbclass
> > index 25f77c77350..3ea9c974eb5 100644
> > --- a/meta/classes/staging.bbclass
> > +++ b/meta/classes/staging.bbclass
> > @@ -21,7 +21,7 @@ SYSROOT_DIRS_NATIVE = " \
> >  "
> >  SYSROOT_DIRS:append:class-native = " ${SYSROOT_DIRS_NATIVE}"
> >  SYSROOT_DIRS:append:class-cross = " ${SYSROOT_DIRS_NATIVE}"
> > -SYSROOT_DIRS:append:class-crosssdk = " ${SYSROOT_DIRS_NATIVE}"
> > +SYSROOT_DIRS:app:class-crosssdk = " ${SYSROOT_DIRS_NATIVE}"
> 
> Did you intend to drop the "end" from "append" here?

Definitely not, well spotted. I think that happened as I was improving comments
before sending to the list and this is why we have review. I've fixed in master-
next, thanks!

Cheers,

Richard