sanity: Add warning for local hasheqiv server with remote sstate mirrors

Message ID 20220323142411.1277228-1-richard.purdie@linuxfoundation.org
State Accepted, archived
Commit ae4eb33b5a6a037348e8f66e3d4f19b58dba8fa2
Headers show
Series sanity: Add warning for local hasheqiv server with remote sstate mirrors | expand

Commit Message

Richard Purdie March 23, 2022, 2:24 p.m. UTC
We're seeing a lot of users configuring an sstate mirror but not realising that
the default hash equivalenve setting will make this ineffective. Add a warning
to highlight this to the user for the common case.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
 meta/classes/sanity.bbclass | 5 +++++
 1 file changed, 5 insertions(+)

Comments

Jose Quaresma March 25, 2022, 5:53 p.m. UTC | #1
Hi Steve,

I see this patch on dunfell-nut but afaik dunfell don't support upstream
hash equivalence server
https://git.yoctoproject.org/poky-contrib/commit/?h=stable/dunfell-nut&id=1d33ad9c8ea2388f51ff00ba3a3442f15f29e663

Jose

Richard Purdie <richard.purdie@linuxfoundation.org> escreveu no dia quarta,
23/03/2022 à(s) 14:24:

> We're seeing a lot of users configuring an sstate mirror but not realising
> that
> the default hash equivalenve setting will make this ineffective. Add a
> warning
> to highlight this to the user for the common case.
>
> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> ---
>  meta/classes/sanity.bbclass | 5 +++++
>  1 file changed, 5 insertions(+)
>
> diff --git a/meta/classes/sanity.bbclass b/meta/classes/sanity.bbclass
> index 92807dc88e9..be956fbcd6c 100644
> --- a/meta/classes/sanity.bbclass
> +++ b/meta/classes/sanity.bbclass
> @@ -914,6 +914,11 @@ def check_sanity_everybuild(status, d):
>                      mirror_base =
> urllib.parse.urlparse(mirror[:-1*len('/PATH')]).path
>                      check_symlink(mirror_base, d)
>
> +    # Check sstate mirrors aren't being used with a local hash server and
> no remote
> +    hashserv = d.getVar("BB_HASHSERVE")
> +    if d.getVar("SSTATE_MIRRORS") and hashserv and
> hashserv.startswith("unix://") and not d.getVar("BB_HASHSERVE_UPSTREAM"):
> +        bb.warn("You are using a local hash equivalence server but have
> configured an sstate mirror. This will likely mean no sstate will match
> from the mirror. You may wish to disable the hash equivalence use
> (BB_HASHSERVE), or use a hash equivalence server alongside the sstate
> mirror.")
> +
>      # Check that TMPDIR hasn't changed location since the last time we
> were run
>      tmpdir = d.getVar('TMPDIR')
>      checkfile = os.path.join(tmpdir, "saved_tmpdir")
> --
> 2.32.0
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#163584):
> https://lists.openembedded.org/g/openembedded-core/message/163584
> Mute This Topic: https://lists.openembedded.org/mt/89976515/5052612
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
> quaresma.jose@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>
Steve Sakoman March 25, 2022, 6:08 p.m. UTC | #2
On Fri, Mar 25, 2022 at 7:53 AM Jose Quaresma <quaresma.jose@gmail.com> wrote:
>
> Hi Steve,
>
> I see this patch on dunfell-nut but afaik dunfell don't support upstream hash equivalence server
> https://git.yoctoproject.org/poky-contrib/commit/?h=stable/dunfell-nut&id=1d33ad9c8ea2388f51ff00ba3a3442f15f29e663

Yes, a moment of temporary insanity :-)

If you look again you'll see it's gone.  The stable/dunfell-nut branch
is not to be trusted, it gets changed and rebased multiple times per
day.

Once I send patches to the list for review, stable/dunfell-next can be
trusted a bit more, but even that occasionally gets rebased if there
is some last minute feedback before I send the pull request.

Steve

> Richard Purdie <richard.purdie@linuxfoundation.org> escreveu no dia quarta, 23/03/2022 à(s) 14:24:
>>
>> We're seeing a lot of users configuring an sstate mirror but not realising that
>> the default hash equivalenve setting will make this ineffective. Add a warning
>> to highlight this to the user for the common case.
>>
>> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
>> ---
>>  meta/classes/sanity.bbclass | 5 +++++
>>  1 file changed, 5 insertions(+)
>>
>> diff --git a/meta/classes/sanity.bbclass b/meta/classes/sanity.bbclass
>> index 92807dc88e9..be956fbcd6c 100644
>> --- a/meta/classes/sanity.bbclass
>> +++ b/meta/classes/sanity.bbclass
>> @@ -914,6 +914,11 @@ def check_sanity_everybuild(status, d):
>>                      mirror_base = urllib.parse.urlparse(mirror[:-1*len('/PATH')]).path
>>                      check_symlink(mirror_base, d)
>>
>> +    # Check sstate mirrors aren't being used with a local hash server and no remote
>> +    hashserv = d.getVar("BB_HASHSERVE")
>> +    if d.getVar("SSTATE_MIRRORS") and hashserv and hashserv.startswith("unix://") and not d.getVar("BB_HASHSERVE_UPSTREAM"):
>> +        bb.warn("You are using a local hash equivalence server but have configured an sstate mirror. This will likely mean no sstate will match from the mirror. You may wish to disable the hash equivalence use (BB_HASHSERVE), or use a hash equivalence server alongside the sstate mirror.")
>> +
>>      # Check that TMPDIR hasn't changed location since the last time we were run
>>      tmpdir = d.getVar('TMPDIR')
>>      checkfile = os.path.join(tmpdir, "saved_tmpdir")
>> --
>> 2.32.0
>>
>>
>> -=-=-=-=-=-=-=-=-=-=-=-
>> Links: You receive all messages sent to this group.
>> View/Reply Online (#163584): https://lists.openembedded.org/g/openembedded-core/message/163584
>> Mute This Topic: https://lists.openembedded.org/mt/89976515/5052612
>> Group Owner: openembedded-core+owner@lists.openembedded.org
>> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [quaresma.jose@gmail.com]
>> -=-=-=-=-=-=-=-=-=-=-=-
>>
>
>
> --
> Best regards,
>
> José Quaresma

Patch

diff --git a/meta/classes/sanity.bbclass b/meta/classes/sanity.bbclass
index 92807dc88e9..be956fbcd6c 100644
--- a/meta/classes/sanity.bbclass
+++ b/meta/classes/sanity.bbclass
@@ -914,6 +914,11 @@  def check_sanity_everybuild(status, d):
                     mirror_base = urllib.parse.urlparse(mirror[:-1*len('/PATH')]).path
                     check_symlink(mirror_base, d)
 
+    # Check sstate mirrors aren't being used with a local hash server and no remote
+    hashserv = d.getVar("BB_HASHSERVE")
+    if d.getVar("SSTATE_MIRRORS") and hashserv and hashserv.startswith("unix://") and not d.getVar("BB_HASHSERVE_UPSTREAM"):
+        bb.warn("You are using a local hash equivalence server but have configured an sstate mirror. This will likely mean no sstate will match from the mirror. You may wish to disable the hash equivalence use (BB_HASHSERVE), or use a hash equivalence server alongside the sstate mirror.")
+
     # Check that TMPDIR hasn't changed location since the last time we were run
     tmpdir = d.getVar('TMPDIR')
     checkfile = os.path.join(tmpdir, "saved_tmpdir")