diff mbox series

selftest/reproducible: Allow packages exclusion via config

Message ID 20231114235332.112208-1-yoann.congal@smile.fr
State Accepted, archived
Commit bea0e3f5b8b383ae8590b874a5878f67832d6929
Headers show
Series selftest/reproducible: Allow packages exclusion via config | expand

Commit Message

Yoann Congal Nov. 14, 2023, 11:53 p.m. UTC
From: Richard Purdie <richard.purdie@linuxfoundation.org>

OEQA_REPRODUCIBLE_EXCLUDED_PACKAGES can be used to prevent known
not-reproducible packages to make the reproducible test fail.

For example, in local.conf:
  OEQA_REPRODUCIBLE_EXCLUDED_PACKAGES = "example-not-reproducible"

To quote the original exclusion commit (4864764667097b32a3ee1935e2c52df732d55632)
> selftest/reproducible: add an exclusion list for items that are not yet reproducible
> [...]
> Non-reproducible excluded packages are not given to diffoscope and do not cause a
> failure, but still saved side-by-side with non-reproducible failing ones to make
> investigation easier.

NB: Patch was written by Richard with small fixes and a commit message
from Yoann.
Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
---
 meta/lib/oeqa/selftest/cases/reproducible.py | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

Comments

Yoann Congal Nov. 15, 2023, midnight UTC | #1
Le mer. 15 nov. 2023 à 00:53, Yoann Congal <yoann.congal@smile.fr> a écrit :
>
> From: Richard Purdie <richard.purdie@linuxfoundation.org>
>
> OEQA_REPRODUCIBLE_EXCLUDED_PACKAGES can be used to prevent known
> not-reproducible packages to make the reproducible test fail.
>
> For example, in local.conf:
>   OEQA_REPRODUCIBLE_EXCLUDED_PACKAGES = "example-not-reproducible"
>
> To quote the original exclusion commit (4864764667097b32a3ee1935e2c52df732d55632)
> > selftest/reproducible: add an exclusion list for items that are not yet reproducible
> > [...]
> > Non-reproducible excluded packages are not given to diffoscope and do not cause a
> > failure, but still saved side-by-side with non-reproducible failing ones to make
> > investigation easier.
>
> NB: Patch was written by Richard with small fixes and a commit message
> from Yoann.
> Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
> ---
>  meta/lib/oeqa/selftest/cases/reproducible.py | 7 ++++---
>  1 file changed, 4 insertions(+), 3 deletions(-)

NB: This does clash with Richard's patch :
[PATCH 1/2] oeqa/selftest/reproducible: Add
OEQA_REPRODUCIBLE_EXCLUDED_PACKAGES support
https://lists.openembedded.org/g/openembedded-core/message/190523
diff mbox series

Patch

diff --git a/meta/lib/oeqa/selftest/cases/reproducible.py b/meta/lib/oeqa/selftest/cases/reproducible.py
index 84c6c3a05f..dac0dce346 100644
--- a/meta/lib/oeqa/selftest/cases/reproducible.py
+++ b/meta/lib/oeqa/selftest/cases/reproducible.py
@@ -45,13 +45,14 @@  class CompareResult(object):
         return (self.status, self.test) < (other.status, other.test)
 
 class PackageCompareResults(object):
-    def __init__(self):
+    def __init__(self, exclusions):
         self.total = []
         self.missing = []
         self.different = []
         self.different_excluded = []
         self.same = []
         self.active_exclusions = set()
+        exclude_packages.extend((exclusions or "").split())
 
     def add_result(self, r):
         self.total.append(r)
@@ -153,7 +154,7 @@  class ReproducibleTests(OESelftestTestCase):
 
     def setUpLocal(self):
         super().setUpLocal()
-        needed_vars = ['TOPDIR', 'TARGET_PREFIX', 'BB_NUMBER_THREADS', 'BB_HASHSERVE', 'OEQA_REPRODUCIBLE_TEST_PACKAGE', 'OEQA_REPRODUCIBLE_TEST_TARGET', 'OEQA_REPRODUCIBLE_TEST_SSTATE_TARGETS']
+        needed_vars = ['TOPDIR', 'TARGET_PREFIX', 'BB_NUMBER_THREADS', 'BB_HASHSERVE', 'OEQA_REPRODUCIBLE_TEST_PACKAGE', 'OEQA_REPRODUCIBLE_TEST_TARGET', 'OEQA_REPRODUCIBLE_TEST_SSTATE_TARGETS', 'OEQA_REPRODUCIBLE_EXCLUDED_PACKAGES']
         bb_vars = get_bb_vars(needed_vars)
         for v in needed_vars:
             setattr(self, v.lower(), bb_vars[v])
@@ -175,7 +176,7 @@  class ReproducibleTests(OESelftestTestCase):
         self.extraresults['reproducible.rawlogs']['log'] += msg
 
     def compare_packages(self, reference_dir, test_dir, diffutils_sysroot):
-        result = PackageCompareResults()
+        result = PackageCompareResults(self.oeqa_reproducible_excluded_packages)
 
         old_cwd = os.getcwd()
         try: