sstate.bbclass: Fetch non-existing local .sig files if needed

For the case of a global shared state cache and a local sstate cache
with a mix of signed and un-signed entries, there is a case where
a .sig is missing locally, but may exist in the global sstate cache.

For this case, do not just fail sstate fetch, but rather backfill
the .sig file from the global sstate cache.

If this case is detected, re-run the fetch operation to get the
global .sig file (if it exists).

Change-Id: Id0491af1d3a0d29b2773d1607be9edfb7fa5ad70
Signed-off-by: Tobias Hagelborn <tobias.hagelborn@axis.com>
---
 meta/classes-global/sstate.bbclass | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

On Thu, 2023-01-26 at 14:51 +0100, Tobias Hagelborn wrote:
> For the case of a global shared state cache and a local sstate cache
> with a mix of signed and un-signed entries, there is a case where
> a .sig is missing locally, but may exist in the global sstate cache.
> 
> For this case, do not just fail sstate fetch, but rather backfill
> the .sig file from the global sstate cache.
> 
> If this case is detected, re-run the fetch operation to get the
> global .sig file (if it exists).
> 
> Change-Id: Id0491af1d3a0d29b2773d1607be9edfb7fa5ad70
> Signed-off-by: Tobias Hagelborn <tobias.hagelborn@axis.com>
> ---
>  meta/classes-global/sstate.bbclass | 6 +++++-
>  1 file changed, 5 insertions(+), 1 deletion(-)
> 
> diff --git a/meta/classes-global/sstate.bbclass b/meta/classes-global/sstate.bbclass
> index 77e3ea34e1..584d86ee2f 100644
> --- a/meta/classes-global/sstate.bbclass
> +++ b/meta/classes-global/sstate.bbclass
> @@ -365,6 +365,7 @@ def sstate_installpkg(ss, d):
>      d.setVar("SSTATE_CURRTASK", ss['task'])
>      sstatefetch = d.getVar('SSTATE_PKGNAME')
>      sstatepkg = d.getVar('SSTATE_PKG')
> +    verify_sig = bb.utils.to_boolean(d.getVar("SSTATE_VERIFY_SIG"), False)
>  
>      if not os.path.exists(sstatepkg):
>          pstaging_fetch(sstatefetch, d)
> @@ -373,11 +374,14 @@ def sstate_installpkg(ss, d):
>          bb.note("Sstate package %s does not exist" % sstatepkg)
>          return False
>  
> +    if verify_sig and not os.path.isfile(sstatepkg + '.sig'):
> +        pstaging_fetch(sstatefetch, d)
> +
>      sstate_clean(ss, d)
>  
>      d.setVar('SSTATE_INSTDIR', sstateinst)
>  
> -    if bb.utils.to_boolean(d.getVar("SSTATE_VERIFY_SIG"), False):
> +    if verify_sig:
>          if not os.path.isfile(sstatepkg + '.sig'):
>              bb.warn("No signature file for sstate package %s, skipping acceleration..." % sstatepkg)
>              return False


I think this is going to be clearer to maintain if the code looks like:

    if not os.path.exists(sstatepkg) or (verify_sig and not os.path.isfile(sstatepkg + '.sig')):
        pstaging_fetch(sstatefetch, d)

and then we just call pstaging_fetch() once? It might also be more
consistent to use exists() for both cases.

Does that still address your use case?

Cheers,

Richard