diff mbox series

sstatesig: Improve output hash calculation

Message ID 20230208235011.40486-1-mateuszmar2@gmail.com
State Accepted, archived
Commit ee729163f31f26b1462a47e1e53f7a0f9de9b464
Headers show
Series sstatesig: Improve output hash calculation | expand

Commit Message

Mateusz Marciniec Feb. 8, 2023, 11:50 p.m. UTC
From: Mateusz Marciniec <mateuszmar2@gmail.com>

Symbolic links to the files are included during the output hash
calculation but symlinks to the directories are missed.
So if the new symlink to a directory was the only change made,
then the output hash won't change,
and the Hash Equivalence server may change unihash.
In the next run bitbake may use an older package from sstate-cache.

To fix this followlinks=True flag could be set for os.walk
but it can lead to infinite recursion if link points
to a parent directory of itself.
Also, all files from a directory to which symlink points
would be included in depsig file.
Therefore another solution was applied, I added code that will loop
through directories and process those that are symlinks.

Signed-off-by: Mateusz Marciniec <mateuszmar2@gmail.com>
Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
---
 meta/lib/oe/sstatesig.py | 5 +++++
 1 file changed, 5 insertions(+)

Comments

Peter Kjellerstedt Feb. 9, 2023, 9:43 a.m. UTC | #1
> -----Original Message-----
> From: openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org> On Behalf Of Mateusz Marciniec
> Sent: den 9 februari 2023 00:50
> To: openembedded-core@lists.openembedded.org
> Cc: Mateusz Marciniec <mateuszmar2@gmail.com>; Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
> Subject: [OE-core][PATCH] sstatesig: Improve output hash calculation
> 
> From: Mateusz Marciniec <mateuszmar2@gmail.com>
> 
> Symbolic links to the files are included during the output hash
> calculation but symlinks to the directories are missed.
> So if the new symlink to a directory was the only change made,
> then the output hash won't change,
> and the Hash Equivalence server may change unihash.
> In the next run bitbake may use an older package from sstate-cache.
> 
> To fix this followlinks=True flag could be set for os.walk
> but it can lead to infinite recursion if link points
> to a parent directory of itself.
> Also, all files from a directory to which symlink points
> would be included in depsig file.
> Therefore another solution was applied, I added code that will loop
> through directories and process those that are symlinks.
> 
> Signed-off-by: Mateusz Marciniec <mateuszmar2@gmail.com>
> Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
> ---
>  meta/lib/oe/sstatesig.py | 5 +++++
>  1 file changed, 5 insertions(+)
> 
> diff --git a/meta/lib/oe/sstatesig.py b/meta/lib/oe/sstatesig.py
> index f0224454c9..a5bc030f58 100644
> --- a/meta/lib/oe/sstatesig.py
> +++ b/meta/lib/oe/sstatesig.py
> @@ -652,6 +652,11 @@ def OEOuthashBasic(path, sigfile, task, d):
>                  if f == 'fixmepath':
>                      continue
>                  process(os.path.join(root, f))
> +
> +            for d in dirs:

Don't use `d` as a local variable for the directory, it is universally 
used as reference to the datastore (also in this function as the function 
declaration above indicates). Use `dir` instead.

> +                if not os.path.islink(os.path.join(root, d)):
> +                    continue
> +                process(os.path.join(root, d))
>      finally:
>          os.chdir(prev_dir)
> 
> --
> 2.39.1

//Peter
diff mbox series

Patch

diff --git a/meta/lib/oe/sstatesig.py b/meta/lib/oe/sstatesig.py
index f0224454c9..a5bc030f58 100644
--- a/meta/lib/oe/sstatesig.py
+++ b/meta/lib/oe/sstatesig.py
@@ -652,6 +652,11 @@  def OEOuthashBasic(path, sigfile, task, d):
                 if f == 'fixmepath':
                     continue
                 process(os.path.join(root, f))
+
+            for d in dirs:
+                if not os.path.islink(os.path.join(root, d)):
+                    continue
+                process(os.path.join(root, d))
     finally:
         os.chdir(prev_dir)