mbox series

[v2,0/1] Fix dependency handling bug of remove operator

Message ID 20230919065938.1317052-1-insu0.park@gmail.com
Headers show
Series Fix dependency handling bug of remove operator | expand

Message

Insu Park Sept. 19, 2023, 6:59 a.m. UTC 
Unlike the meta layers under the Yocto projects, custom layers frequently use following style
of overriding to disable something default in the base layer without touching it:
    bb in the base layer:
        OPTIONS = "foo bar"
    bbappend in the custom layer:
        OPTIONS:remove = "${@bb.utils.contains('CUSTOM_FEATURES', 'no-foo', 'foo', '', d)}"

The problem is that the value of "CUSTOM_FEATURES" is not shown in the task signature.
Here is a test example.

test-override-operators.bb:
    ANOTHERVAR = "testval"
    ANOTHERVAR:prepend = "${@bb.utils.filter('TESTVAR1', 'testval1', d)} "
    ANOTHERVAR:append = " ${@bb.utils.filter('TESTVAR2', 'testval2', d)}"
    ANOTHERVAR:remove = "${@bb.utils.contains('TESTVAR3', 'no-testval', 'testval', '', d)}"

local.conf:
    TESTVAR1 = "blah"
    TESTVAR2 = "testval2"
    TESTVAR3 = "blah"

Signature dump (Bug):
    Variable ANOTHERVAR value is ${@bb.utils.filter('TESTVAR1', 'testval1', d)} testval ${@bb.utils.filter('TESTVAR2', 'testval2', d)}
    TESTVAR1{testval1} = Unset
    TESTVAR2{testval2} = Set
    _remove of ${@bb.utils.contains('TESTVAR3', 'no-testval', 'testval', '', d)}

Note that the signature dump above doesn't take the value of TESTVAR3 into account.
Therefore, even if the local.conf changes the TESTVAR3 value to "no-testval",
the two different configs produce exactly same signature, so the recipe won't be re-built,
even worse, its sstate-cache output is not deterministic (depend on the config it first met).

With the fixed code, the new signature recognizes the TESTVAR3, and the problem solved.
Signature dump (Fixed):
    (Same as above)
    TESTVAR3{no-testval} = Unset

Now, the new test case fails without the fix:
|$ bitbake-selftest bb.tests.codeparser.DependencyReferenceTest.test_contains_vardeps_override_operators
|F
|======================================================================
|FAIL: test_contains_vardeps_override_operators (bb.tests.codeparser.DependencyReferenceTest)
|----------------------------------------------------------------------
|Traceback (most recent call last):
|  File "/home/worker/work/poky-test/bitbake/lib/bb/tests/codeparser.py", line 454, in test_contains_vardeps_override_operators
|    self.assertEqual(sorted(values.splitlines()),
|AssertionError: Lists differ: ['${@[155 chars]t', '_remove of ${@bb.utils.contains("TESTVAR3[32 chars]d)}'] != ['${@[155 chars]t', 'TESTVAR3{no-testval} = Set', '_remove of [62 chars]d)}']
|
|First differing element 3:
|'_remove of ${@bb.utils.contains("TESTVAR3", "no-testval", "testval", "", d)}'
|'TESTVAR3{no-testval} = Set'
|
|Second list contains 1 additional elements.
|First extra element 4:
|'_remove of ${@bb.utils.contains("TESTVAR3", "no-testval", "testval", "", d)}'
|
|  ['${@bb.utils.filter("TESTVAR1", "testval1", d)} testval '
|   '${@bb.utils.filter("TESTVAR2", "testval2", d)}',
|   'TESTVAR1{testval1} = Unset',
|   'TESTVAR2{testval2} = Set',
|+  'TESTVAR3{no-testval} = Set',
|   '_remove of ${@bb.utils.contains("TESTVAR3", "no-testval", "testval", "", d)}']
|
|----------------------------------------------------------------------
|Ran 1 test in 0.002s
|
|FAILED (failures=1)

Insu Park (1):
  bitbake: data: Add missing dependency handling of remove operator

 lib/bb/data.py             |  1 +
 lib/bb/tests/codeparser.py | 26 ++++++++++++++++++++++++++
 2 files changed, 27 insertions(+)