Patchwork [bitbake-devel] fetch2: Sort file checksums by value, not path

login
register
mail settings
Submitter Tyler Hall
Date Dec. 19, 2012, 11:26 p.m.
Message ID <1355959610-29470-1-git-send-email-tylerwhall@gmail.com>
Download mbox | patch
Permalink /patch/41603/
State Accepted
Commit 22bd19d208f0251f5a1f9b98f3cac66181f3fc07
Headers show

Comments

Tyler Hall - Dec. 19, 2012, 11:26 p.m.
Changing the path to a file could change the task hash even if the file
still has the same checksum. This occurs when the task depends on
multiple files and the sort order of their paths changes. Usually the
sorting is consistent because layers tend to have the same relative
paths, but this should take care of other configuations.

The problem arose when using a .bbappend to add files to a recipe in
another layer. If the layer is located alongside the other layers and
their parent directory is moved, the hash does not change. However,
moving the .bbappend layer outside of the common directory can change
the path sort order and the task hash.

Signed-off-by: Tyler Hall <tylerwhall@gmail.com>
---
 lib/bb/fetch2/__init__.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
Richard Purdie - Jan. 18, 2013, 12:45 p.m.
On Wed, 2012-12-19 at 18:26 -0500, Tyler Hall wrote:
> Changing the path to a file could change the task hash even if the file
> still has the same checksum. This occurs when the task depends on
> multiple files and the sort order of their paths changes. Usually the
> sorting is consistent because layers tend to have the same relative
> paths, but this should take care of other configuations.
> 
> The problem arose when using a .bbappend to add files to a recipe in
> another layer. If the layer is located alongside the other layers and
> their parent directory is moved, the hash does not change. However,
> moving the .bbappend layer outside of the common directory can change
> the path sort order and the task hash.
> 
> Signed-off-by: Tyler Hall <tylerwhall@gmail.com>
> ---
>  lib/bb/fetch2/__init__.py | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)

Good catch, merged to master, thanks.

Richard

Patch

diff --git a/lib/bb/fetch2/__init__.py b/lib/bb/fetch2/__init__.py
index 150dc3c..4746356 100644
--- a/lib/bb/fetch2/__init__.py
+++ b/lib/bb/fetch2/__init__.py
@@ -30,6 +30,7 @@  from __future__ import print_function
 import os, re
 import logging
 import urllib
+import operator
 import bb.persist_data, bb.utils
 import bb.checksum
 from bb import data
@@ -729,7 +730,7 @@  def get_file_checksums(filelist, pn):
         if checksum:
             checksums.append((pth, checksum))
 
-    checksums.sort()
+    checksums.sort(key=operator.itemgetter(1))
     return checksums