[2/3] selftest/sstatetests: add tests for 'bitbake -S printdiff'

'bitbake -S printdiff' is a useful diagnostic facility for finding out
why sstate is not being reused, but until now it had no tests that would
ensure it works. This commit adds three basic scenarios:

1. make a change in a really basic, common recipe that is at the very root
of dependency trees (quilt-native), and ensure that change is correctly discovered when
building an image.

2. make a change in gcc-source recipe, which is somewhat special
(operates in work-shared), and ensure that gcc-runtime builds track
that down as well.

3. make a change in base_do_configure() definition from base.bbclass,
which is not recipe-specific, but affects many basic recipes, and ensure that
is correctly reported as well.

The test itself actually runs twice:
- first against a fully populated build directory, where
the printdiff code is guaranteed to find the correct previous
stamp that can be compared with in a predictable manner.

- then in an empty build directory where the printdiff code
goes to look in the sstate cache, and so the existence of the
previous signature can be tested, but not the difference with it
(what the exact difference would be is unpredictable as the
sstate cache is indeed shared between many builds).

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
---
 .../base-do-configure-modified.bbclass        |   3 +
 .../gcc-source/gcc-source_%.bbappend          |   2 +
 .../quilt-native/quilt-native_%.bbappend      |   2 +
 meta/lib/oeqa/selftest/cases/sstatetests.py   | 104 ++++++++++++++++++
 4 files changed, 111 insertions(+)
 create mode 100644 meta-selftest/classes/base-do-configure-modified.bbclass
 create mode 100644 meta-selftest/recipes-test/gcc-source/gcc-source_%.bbappend
 create mode 100644 meta-selftest/recipes-test/quilt-native/quilt-native_%.bbappend

Alexander Kanavin <alex.kanavin@gmail.com> escreveu no dia terça,
17/10/2023 à(s) 14:31:

> 'bitbake -S printdiff' is a useful diagnostic facility for finding out
> why sstate is not being reused, but until now it had no tests that would
> ensure it works. This commit adds three basic scenarios:
>
> 1. make a change in a really basic, common recipe that is at the very root
> of dependency trees (quilt-native), and ensure that change is correctly
> discovered when
> building an image.
>
> 2. make a change in gcc-source recipe, which is somewhat special
> (operates in work-shared), and ensure that gcc-runtime builds track
> that down as well.
>
> 3. make a change in base_do_configure() definition from base.bbclass,
> which is not recipe-specific, but affects many basic recipes, and ensure
> that
> is correctly reported as well.
>
> The test itself actually runs twice:
> - first against a fully populated build directory, where
> the printdiff code is guaranteed to find the correct previous
> stamp that can be compared with in a predictable manner.
>
> - then in an empty build directory where the printdiff code
> goes to look in the sstate cache, and so the existence of the
> previous signature can be tested, but not the difference with it
> (what the exact difference would be is unpredictable as the
> sstate cache is indeed shared between many builds).
>
> Signed-off-by: Alexander Kanavin <alex@linutronix.de>
> ---
>  .../base-do-configure-modified.bbclass        |   3 +
>  .../gcc-source/gcc-source_%.bbappend          |   2 +
>  .../quilt-native/quilt-native_%.bbappend      |   2 +
>  meta/lib/oeqa/selftest/cases/sstatetests.py   | 104 ++++++++++++++++++
>  4 files changed, 111 insertions(+)
>  create mode 100644
> meta-selftest/classes/base-do-configure-modified.bbclass
>  create mode 100644
> meta-selftest/recipes-test/gcc-source/gcc-source_%.bbappend
>  create mode 100644
> meta-selftest/recipes-test/quilt-native/quilt-native_%.bbappend
>
> diff --git a/meta-selftest/classes/base-do-configure-modified.bbclass
> b/meta-selftest/classes/base-do-configure-modified.bbclass
> new file mode 100644
> index 00000000000..3f96827a428
> --- /dev/null
> +++ b/meta-selftest/classes/base-do-configure-modified.bbclass
> @@ -0,0 +1,3 @@
> +base_do_configure:append () {
> +       echo "this changes base_do_configure() definiton"
> +}
> diff --git a/meta-selftest/recipes-test/gcc-source/gcc-source_%.bbappend
> b/meta-selftest/recipes-test/gcc-source/gcc-source_%.bbappend
> new file mode 100644
> index 00000000000..205720982cb
> --- /dev/null
> +++ b/meta-selftest/recipes-test/gcc-source/gcc-source_%.bbappend
> @@ -0,0 +1,2 @@
> +# This bbappend is used to alter the recipe using the test_recipe.inc
> file created by tests.
> +include test_recipe.inc
> diff --git
> a/meta-selftest/recipes-test/quilt-native/quilt-native_%.bbappend
> b/meta-selftest/recipes-test/quilt-native/quilt-native_%.bbappend
> new file mode 100644
> index 00000000000..205720982cb
> --- /dev/null
> +++ b/meta-selftest/recipes-test/quilt-native/quilt-native_%.bbappend
> @@ -0,0 +1,2 @@
> +# This bbappend is used to alter the recipe using the test_recipe.inc
> file created by tests.
> +include test_recipe.inc
> diff --git a/meta/lib/oeqa/selftest/cases/sstatetests.py
> b/meta/lib/oeqa/selftest/cases/sstatetests.py
> index bdad9088d37..29e31f2ef9f 100644
> --- a/meta/lib/oeqa/selftest/cases/sstatetests.py
> +++ b/meta/lib/oeqa/selftest/cases/sstatetests.py
> @@ -773,3 +773,107 @@ addtask tmptask2 before do_tmptask1
>                  latestfiles = sorted(filedates.keys(), key=lambda f:
> filedates[f])[-2:]
>                  bb.siggen.compare_sigfiles(latestfiles[-2],
> latestfiles[-1], recursecb)
>                  self.assertEqual(recursecb_count,1)
> +
> +class SStatePrintdiff(SStateBase):
> +    def run_test_printdiff_changerecipe(self, target, change_recipe,
> change_bbtask, change_content, expected_sametmp_output,
> expected_difftmp_output):
> +        self.write_config("""
> +TMPDIR = "${TOPDIR}/tmp-sstateprintdiff"
> +""")
> +        self.track_for_cleanup(self.topdir + "/tmp-sstateprintdiff")
> +        bitbake(target)
> +        bitbake("-S none {}".format(target))
> +        bitbake(change_bbtask)
> +        self.write_recipeinc(change_recipe, change_content)
> +        result_sametmp = bitbake("-S printdiff {}".format(target))
> +
> +        self.write_config("""
> +TMPDIR = "${TOPDIR}/tmp-sstateprintdiff-2"
> +""")
> +        self.track_for_cleanup(self.topdir + "/tmp-sstateprintdiff-2")
> +        result_difftmp = bitbake("-S printdiff {}".format(target))
> +
> +        self.delete_recipeinc(change_recipe)
> +        for item in expected_sametmp_output:
> +            self.assertIn(item, result_sametmp.output)
> +        for item in expected_difftmp_output:
> +            self.assertIn(item, result_difftmp.output)
> +
> +    def run_test_printdiff_changeconfig(self, target, change_content,
> expected_sametmp_output, expected_difftmp_output):
> +        self.write_config("""
> +TMPDIR = "${TOPDIR}/tmp-sstateprintdiff"
> +""")
> +        self.track_for_cleanup(self.topdir + "/tmp-sstateprintdiff")
> +        bitbake(target)
> +        bitbake("-S none {}".format(target))
> +        self.append_config(change_content)
> +        result_sametmp = bitbake("-S printdiff {}".format(target))
> +
> +        self.write_config("""
> +TMPDIR = "${TOPDIR}/tmp-sstateprintdiff-2"
> +""")
> +        self.append_config(change_content)
> +        self.track_for_cleanup(self.topdir + "/tmp-sstateprintdiff-2")
> +        result_difftmp = bitbake("-S printdiff {}".format(target))
> +
> +        for item in expected_sametmp_output:
> +            self.assertIn(item, result_sametmp.output)
> +        for item in expected_difftmp_output:
> +            self.assertIn(item, result_difftmp.output)
> +
> +
> +    # Check if printdiff walks the full dependency chain from the image
> target to where the change is in a specific recipe
> +    def test_image_minimal_vs_quilt(self):
> +        expected_output = ("Task quilt-native:do_install couldn't be used
> from the cache because:",
> +"We need hash",
> +"most recent matching task was")
> +        expected_sametmp_output = expected_output + ("Variable do_install
> value changed",'+    echo "this changes the task signature"')
> +        expected_difftmp_output = expected_output
> +
> +        self.run_test_printdiff_changerecipe("core-image-minimal",
> "quilt-native", "-c do_install quilt-native",
> +"""
> +do_install:append() {
> +    echo "this changes the task signature"
> +}
> +""",
> +expected_sametmp_output, expected_difftmp_output)
> +
> +    # Check if changes to gcc-source (which uses tmp/work-shared) are
> correctly discovered
> +    def test_gcc_runtime_vs_gcc_source(self):
> +        expected_output = ("Task gcc-source-13.2.0:do_preconfigure
> couldn't be used from the cache because:",
>

We should avoid using the PV=13.2.0 explicitly imo otherwise it will need
to be updated with the recipe forever.

The archiver selftest have a gcc-source-PV example
https://git.openembedded.org/openembedded-core/tree/meta/lib/oeqa/selftest/cases/archiver.py#n141

+"We need hash",
> +"most recent matching task was")
> +        expected_sametmp_output = expected_output + ("Variable
> do_preconfigure value changed",'+    print("this changes the task
> signature")')
> +        #FIXME: printdiff is supposed to find at least one preconfigure
> task signature in the sstate cache, but isn't able to
> +        #expected_difftmp_output = expected_output
> +        expected_difftmp_output = ()
> +
> +        self.run_test_printdiff_changerecipe("gcc-runtime", "gcc-source",
> "-c do_preconfigure gcc-source-13.2.0",
>

Same here

Jose


> +"""
> +python do_preconfigure:append() {
> +    print("this changes the task signature")
> +}
> +""",
> +expected_sametmp_output, expected_difftmp_output)
> +
> +    # Check if changing a really base task definiton is reported against
> multiple core recipes using it
> +    def test_image_minimal_vs_base_do_configure(self):
> +        expected_output = ("Task zstd-native:do_configure couldn't be
> used from the cache because:",
> +"Task texinfo-dummy-native:do_configure couldn't be used from the cache
> because:",
> +"Task ldconfig-native:do_configure couldn't be used from the cache
> because:",
> +"Task gettext-minimal-native:do_configure couldn't be used from the cache
> because:",
> +"Task tzcode-native:do_configure couldn't be used from the cache
> because:",
> +"Task makedevs-native:do_configure couldn't be used from the cache
> because:",
> +"Task pigz-native:do_configure couldn't be used from the cache because:",
> +"Task update-rc.d-native:do_configure couldn't be used from the cache
> because:",
> +"Task unzip-native:do_configure couldn't be used from the cache because:",
> +"Task gnu-config-native:do_configure couldn't be used from the cache
> because:",
> +"We need hash",
> +"most recent matching task was")
> +        expected_sametmp_output = expected_output + ("Variable
> base_do_configure value changed",'+     echo "this changes
> base_do_configure() definiton"')
> +        expected_difftmp_output = expected_output
> +
> +        self.run_test_printdiff_changeconfig("core-image-minimal",
> +"""
> +INHERIT += "base-do-configure-modified"
> +""",
> +expected_sametmp_output, expected_difftmp_output)
> --
> 2.39.2
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#189332):
> https://lists.openembedded.org/g/openembedded-core/message/189332
> Mute This Topic: https://lists.openembedded.org/mt/102017475/5052612
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
> quaresma.jose@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>

Thanks, I was completely sure in my head I had fixed this, but either
I haven't, or it has been somehow lost.

Alex

On Tue, 17 Oct 2023 at 16:06, Jose Quaresma <quaresma.jose@gmail.com> wrote:
>
>
>
> Alexander Kanavin <alex.kanavin@gmail.com> escreveu no dia terça, 17/10/2023 à(s) 14:31:
>>
>> 'bitbake -S printdiff' is a useful diagnostic facility for finding out
>> why sstate is not being reused, but until now it had no tests that would
>> ensure it works. This commit adds three basic scenarios:
>>
>> 1. make a change in a really basic, common recipe that is at the very root
>> of dependency trees (quilt-native), and ensure that change is correctly discovered when
>> building an image.
>>
>> 2. make a change in gcc-source recipe, which is somewhat special
>> (operates in work-shared), and ensure that gcc-runtime builds track
>> that down as well.
>>
>> 3. make a change in base_do_configure() definition from base.bbclass,
>> which is not recipe-specific, but affects many basic recipes, and ensure that
>> is correctly reported as well.
>>
>> The test itself actually runs twice:
>> - first against a fully populated build directory, where
>> the printdiff code is guaranteed to find the correct previous
>> stamp that can be compared with in a predictable manner.
>>
>> - then in an empty build directory where the printdiff code
>> goes to look in the sstate cache, and so the existence of the
>> previous signature can be tested, but not the difference with it
>> (what the exact difference would be is unpredictable as the
>> sstate cache is indeed shared between many builds).
>>
>> Signed-off-by: Alexander Kanavin <alex@linutronix.de>
>> ---
>>  .../base-do-configure-modified.bbclass        |   3 +
>>  .../gcc-source/gcc-source_%.bbappend          |   2 +
>>  .../quilt-native/quilt-native_%.bbappend      |   2 +
>>  meta/lib/oeqa/selftest/cases/sstatetests.py   | 104 ++++++++++++++++++
>>  4 files changed, 111 insertions(+)
>>  create mode 100644 meta-selftest/classes/base-do-configure-modified.bbclass
>>  create mode 100644 meta-selftest/recipes-test/gcc-source/gcc-source_%.bbappend
>>  create mode 100644 meta-selftest/recipes-test/quilt-native/quilt-native_%.bbappend
>>
>> diff --git a/meta-selftest/classes/base-do-configure-modified.bbclass b/meta-selftest/classes/base-do-configure-modified.bbclass
>> new file mode 100644
>> index 00000000000..3f96827a428
>> --- /dev/null
>> +++ b/meta-selftest/classes/base-do-configure-modified.bbclass
>> @@ -0,0 +1,3 @@
>> +base_do_configure:append () {
>> +       echo "this changes base_do_configure() definiton"
>> +}
>> diff --git a/meta-selftest/recipes-test/gcc-source/gcc-source_%.bbappend b/meta-selftest/recipes-test/gcc-source/gcc-source_%.bbappend
>> new file mode 100644
>> index 00000000000..205720982cb
>> --- /dev/null
>> +++ b/meta-selftest/recipes-test/gcc-source/gcc-source_%.bbappend
>> @@ -0,0 +1,2 @@
>> +# This bbappend is used to alter the recipe using the test_recipe.inc file created by tests.
>> +include test_recipe.inc
>> diff --git a/meta-selftest/recipes-test/quilt-native/quilt-native_%.bbappend b/meta-selftest/recipes-test/quilt-native/quilt-native_%.bbappend
>> new file mode 100644
>> index 00000000000..205720982cb
>> --- /dev/null
>> +++ b/meta-selftest/recipes-test/quilt-native/quilt-native_%.bbappend
>> @@ -0,0 +1,2 @@
>> +# This bbappend is used to alter the recipe using the test_recipe.inc file created by tests.
>> +include test_recipe.inc
>> diff --git a/meta/lib/oeqa/selftest/cases/sstatetests.py b/meta/lib/oeqa/selftest/cases/sstatetests.py
>> index bdad9088d37..29e31f2ef9f 100644
>> --- a/meta/lib/oeqa/selftest/cases/sstatetests.py
>> +++ b/meta/lib/oeqa/selftest/cases/sstatetests.py
>> @@ -773,3 +773,107 @@ addtask tmptask2 before do_tmptask1
>>                  latestfiles = sorted(filedates.keys(), key=lambda f: filedates[f])[-2:]
>>                  bb.siggen.compare_sigfiles(latestfiles[-2], latestfiles[-1], recursecb)
>>                  self.assertEqual(recursecb_count,1)
>> +
>> +class SStatePrintdiff(SStateBase):
>> +    def run_test_printdiff_changerecipe(self, target, change_recipe, change_bbtask, change_content, expected_sametmp_output, expected_difftmp_output):
>> +        self.write_config("""
>> +TMPDIR = "${TOPDIR}/tmp-sstateprintdiff"
>> +""")
>> +        self.track_for_cleanup(self.topdir + "/tmp-sstateprintdiff")
>> +        bitbake(target)
>> +        bitbake("-S none {}".format(target))
>> +        bitbake(change_bbtask)
>> +        self.write_recipeinc(change_recipe, change_content)
>> +        result_sametmp = bitbake("-S printdiff {}".format(target))
>> +
>> +        self.write_config("""
>> +TMPDIR = "${TOPDIR}/tmp-sstateprintdiff-2"
>> +""")
>> +        self.track_for_cleanup(self.topdir + "/tmp-sstateprintdiff-2")
>> +        result_difftmp = bitbake("-S printdiff {}".format(target))
>> +
>> +        self.delete_recipeinc(change_recipe)
>> +        for item in expected_sametmp_output:
>> +            self.assertIn(item, result_sametmp.output)
>> +        for item in expected_difftmp_output:
>> +            self.assertIn(item, result_difftmp.output)
>> +
>> +    def run_test_printdiff_changeconfig(self, target, change_content, expected_sametmp_output, expected_difftmp_output):
>> +        self.write_config("""
>> +TMPDIR = "${TOPDIR}/tmp-sstateprintdiff"
>> +""")
>> +        self.track_for_cleanup(self.topdir + "/tmp-sstateprintdiff")
>> +        bitbake(target)
>> +        bitbake("-S none {}".format(target))
>> +        self.append_config(change_content)
>> +        result_sametmp = bitbake("-S printdiff {}".format(target))
>> +
>> +        self.write_config("""
>> +TMPDIR = "${TOPDIR}/tmp-sstateprintdiff-2"
>> +""")
>> +        self.append_config(change_content)
>> +        self.track_for_cleanup(self.topdir + "/tmp-sstateprintdiff-2")
>> +        result_difftmp = bitbake("-S printdiff {}".format(target))
>> +
>> +        for item in expected_sametmp_output:
>> +            self.assertIn(item, result_sametmp.output)
>> +        for item in expected_difftmp_output:
>> +            self.assertIn(item, result_difftmp.output)
>> +
>> +
>> +    # Check if printdiff walks the full dependency chain from the image target to where the change is in a specific recipe
>> +    def test_image_minimal_vs_quilt(self):
>> +        expected_output = ("Task quilt-native:do_install couldn't be used from the cache because:",
>> +"We need hash",
>> +"most recent matching task was")
>> +        expected_sametmp_output = expected_output + ("Variable do_install value changed",'+    echo "this changes the task signature"')
>> +        expected_difftmp_output = expected_output
>> +
>> +        self.run_test_printdiff_changerecipe("core-image-minimal", "quilt-native", "-c do_install quilt-native",
>> +"""
>> +do_install:append() {
>> +    echo "this changes the task signature"
>> +}
>> +""",
>> +expected_sametmp_output, expected_difftmp_output)
>> +
>> +    # Check if changes to gcc-source (which uses tmp/work-shared) are correctly discovered
>> +    def test_gcc_runtime_vs_gcc_source(self):
>> +        expected_output = ("Task gcc-source-13.2.0:do_preconfigure couldn't be used from the cache because:",
>
>
> We should avoid using the PV=13.2.0 explicitly imo otherwise it will need to be updated with the recipe forever.
>
> The archiver selftest have a gcc-source-PV example
> https://git.openembedded.org/openembedded-core/tree/meta/lib/oeqa/selftest/cases/archiver.py#n141
>
>> +"We need hash",
>> +"most recent matching task was")
>> +        expected_sametmp_output = expected_output + ("Variable do_preconfigure value changed",'+    print("this changes the task signature")')
>> +        #FIXME: printdiff is supposed to find at least one preconfigure task signature in the sstate cache, but isn't able to
>> +        #expected_difftmp_output = expected_output
>> +        expected_difftmp_output = ()
>> +
>> +        self.run_test_printdiff_changerecipe("gcc-runtime", "gcc-source", "-c do_preconfigure gcc-source-13.2.0",
>
>
> Same here
>
> Jose
>
>>
>> +"""
>> +python do_preconfigure:append() {
>> +    print("this changes the task signature")
>> +}
>> +""",
>> +expected_sametmp_output, expected_difftmp_output)
>> +
>> +    # Check if changing a really base task definiton is reported against multiple core recipes using it
>> +    def test_image_minimal_vs_base_do_configure(self):
>> +        expected_output = ("Task zstd-native:do_configure couldn't be used from the cache because:",
>> +"Task texinfo-dummy-native:do_configure couldn't be used from the cache because:",
>> +"Task ldconfig-native:do_configure couldn't be used from the cache because:",
>> +"Task gettext-minimal-native:do_configure couldn't be used from the cache because:",
>> +"Task tzcode-native:do_configure couldn't be used from the cache because:",
>> +"Task makedevs-native:do_configure couldn't be used from the cache because:",
>> +"Task pigz-native:do_configure couldn't be used from the cache because:",
>> +"Task update-rc.d-native:do_configure couldn't be used from the cache because:",
>> +"Task unzip-native:do_configure couldn't be used from the cache because:",
>> +"Task gnu-config-native:do_configure couldn't be used from the cache because:",
>> +"We need hash",
>> +"most recent matching task was")
>> +        expected_sametmp_output = expected_output + ("Variable base_do_configure value changed",'+     echo "this changes base_do_configure() definiton"')
>> +        expected_difftmp_output = expected_output
>> +
>> +        self.run_test_printdiff_changeconfig("core-image-minimal",
>> +"""
>> +INHERIT += "base-do-configure-modified"
>> +""",
>> +expected_sametmp_output, expected_difftmp_output)
>> --
>> 2.39.2
>>
>>
>> -=-=-=-=-=-=-=-=-=-=-=-
>> Links: You receive all messages sent to this group.
>> View/Reply Online (#189332): https://lists.openembedded.org/g/openembedded-core/message/189332
>> Mute This Topic: https://lists.openembedded.org/mt/102017475/5052612
>> Group Owner: openembedded-core+owner@lists.openembedded.org
>> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [quaresma.jose@gmail.com]
>> -=-=-=-=-=-=-=-=-=-=-=-
>>
>
>
> --
> Best regards,
>
> José Quaresma

I sent a v2 of this patch where this issue is corrected.

Alex

On Tue, 17 Oct 2023 at 16:15, Alexander Kanavin via
lists.openembedded.org <alex.kanavin=gmail.com@lists.openembedded.org>
wrote:
>
> Thanks, I was completely sure in my head I had fixed this, but either
> I haven't, or it has been somehow lost.
>
> Alex
>
> On Tue, 17 Oct 2023 at 16:06, Jose Quaresma <quaresma.jose@gmail.com> wrote:
> >
> >
> >
> > Alexander Kanavin <alex.kanavin@gmail.com> escreveu no dia terça, 17/10/2023 à(s) 14:31:
> >>
> >> 'bitbake -S printdiff' is a useful diagnostic facility for finding out
> >> why sstate is not being reused, but until now it had no tests that would
> >> ensure it works. This commit adds three basic scenarios:
> >>
> >> 1. make a change in a really basic, common recipe that is at the very root
> >> of dependency trees (quilt-native), and ensure that change is correctly discovered when
> >> building an image.
> >>
> >> 2. make a change in gcc-source recipe, which is somewhat special
> >> (operates in work-shared), and ensure that gcc-runtime builds track
> >> that down as well.
> >>
> >> 3. make a change in base_do_configure() definition from base.bbclass,
> >> which is not recipe-specific, but affects many basic recipes, and ensure that
> >> is correctly reported as well.
> >>
> >> The test itself actually runs twice:
> >> - first against a fully populated build directory, where
> >> the printdiff code is guaranteed to find the correct previous
> >> stamp that can be compared with in a predictable manner.
> >>
> >> - then in an empty build directory where the printdiff code
> >> goes to look in the sstate cache, and so the existence of the
> >> previous signature can be tested, but not the difference with it
> >> (what the exact difference would be is unpredictable as the
> >> sstate cache is indeed shared between many builds).
> >>
> >> Signed-off-by: Alexander Kanavin <alex@linutronix.de>
> >> ---
> >>  .../base-do-configure-modified.bbclass        |   3 +
> >>  .../gcc-source/gcc-source_%.bbappend          |   2 +
> >>  .../quilt-native/quilt-native_%.bbappend      |   2 +
> >>  meta/lib/oeqa/selftest/cases/sstatetests.py   | 104 ++++++++++++++++++
> >>  4 files changed, 111 insertions(+)
> >>  create mode 100644 meta-selftest/classes/base-do-configure-modified.bbclass
> >>  create mode 100644 meta-selftest/recipes-test/gcc-source/gcc-source_%.bbappend
> >>  create mode 100644 meta-selftest/recipes-test/quilt-native/quilt-native_%.bbappend
> >>
> >> diff --git a/meta-selftest/classes/base-do-configure-modified.bbclass b/meta-selftest/classes/base-do-configure-modified.bbclass
> >> new file mode 100644
> >> index 00000000000..3f96827a428
> >> --- /dev/null
> >> +++ b/meta-selftest/classes/base-do-configure-modified.bbclass
> >> @@ -0,0 +1,3 @@
> >> +base_do_configure:append () {
> >> +       echo "this changes base_do_configure() definiton"
> >> +}
> >> diff --git a/meta-selftest/recipes-test/gcc-source/gcc-source_%.bbappend b/meta-selftest/recipes-test/gcc-source/gcc-source_%.bbappend
> >> new file mode 100644
> >> index 00000000000..205720982cb
> >> --- /dev/null
> >> +++ b/meta-selftest/recipes-test/gcc-source/gcc-source_%.bbappend
> >> @@ -0,0 +1,2 @@
> >> +# This bbappend is used to alter the recipe using the test_recipe.inc file created by tests.
> >> +include test_recipe.inc
> >> diff --git a/meta-selftest/recipes-test/quilt-native/quilt-native_%.bbappend b/meta-selftest/recipes-test/quilt-native/quilt-native_%.bbappend
> >> new file mode 100644
> >> index 00000000000..205720982cb
> >> --- /dev/null
> >> +++ b/meta-selftest/recipes-test/quilt-native/quilt-native_%.bbappend
> >> @@ -0,0 +1,2 @@
> >> +# This bbappend is used to alter the recipe using the test_recipe.inc file created by tests.
> >> +include test_recipe.inc
> >> diff --git a/meta/lib/oeqa/selftest/cases/sstatetests.py b/meta/lib/oeqa/selftest/cases/sstatetests.py
> >> index bdad9088d37..29e31f2ef9f 100644
> >> --- a/meta/lib/oeqa/selftest/cases/sstatetests.py
> >> +++ b/meta/lib/oeqa/selftest/cases/sstatetests.py
> >> @@ -773,3 +773,107 @@ addtask tmptask2 before do_tmptask1
> >>                  latestfiles = sorted(filedates.keys(), key=lambda f: filedates[f])[-2:]
> >>                  bb.siggen.compare_sigfiles(latestfiles[-2], latestfiles[-1], recursecb)
> >>                  self.assertEqual(recursecb_count,1)
> >> +
> >> +class SStatePrintdiff(SStateBase):
> >> +    def run_test_printdiff_changerecipe(self, target, change_recipe, change_bbtask, change_content, expected_sametmp_output, expected_difftmp_output):
> >> +        self.write_config("""
> >> +TMPDIR = "${TOPDIR}/tmp-sstateprintdiff"
> >> +""")
> >> +        self.track_for_cleanup(self.topdir + "/tmp-sstateprintdiff")
> >> +        bitbake(target)
> >> +        bitbake("-S none {}".format(target))
> >> +        bitbake(change_bbtask)
> >> +        self.write_recipeinc(change_recipe, change_content)
> >> +        result_sametmp = bitbake("-S printdiff {}".format(target))
> >> +
> >> +        self.write_config("""
> >> +TMPDIR = "${TOPDIR}/tmp-sstateprintdiff-2"
> >> +""")
> >> +        self.track_for_cleanup(self.topdir + "/tmp-sstateprintdiff-2")
> >> +        result_difftmp = bitbake("-S printdiff {}".format(target))
> >> +
> >> +        self.delete_recipeinc(change_recipe)
> >> +        for item in expected_sametmp_output:
> >> +            self.assertIn(item, result_sametmp.output)
> >> +        for item in expected_difftmp_output:
> >> +            self.assertIn(item, result_difftmp.output)
> >> +
> >> +    def run_test_printdiff_changeconfig(self, target, change_content, expected_sametmp_output, expected_difftmp_output):
> >> +        self.write_config("""
> >> +TMPDIR = "${TOPDIR}/tmp-sstateprintdiff"
> >> +""")
> >> +        self.track_for_cleanup(self.topdir + "/tmp-sstateprintdiff")
> >> +        bitbake(target)
> >> +        bitbake("-S none {}".format(target))
> >> +        self.append_config(change_content)
> >> +        result_sametmp = bitbake("-S printdiff {}".format(target))
> >> +
> >> +        self.write_config("""
> >> +TMPDIR = "${TOPDIR}/tmp-sstateprintdiff-2"
> >> +""")
> >> +        self.append_config(change_content)
> >> +        self.track_for_cleanup(self.topdir + "/tmp-sstateprintdiff-2")
> >> +        result_difftmp = bitbake("-S printdiff {}".format(target))
> >> +
> >> +        for item in expected_sametmp_output:
> >> +            self.assertIn(item, result_sametmp.output)
> >> +        for item in expected_difftmp_output:
> >> +            self.assertIn(item, result_difftmp.output)
> >> +
> >> +
> >> +    # Check if printdiff walks the full dependency chain from the image target to where the change is in a specific recipe
> >> +    def test_image_minimal_vs_quilt(self):
> >> +        expected_output = ("Task quilt-native:do_install couldn't be used from the cache because:",
> >> +"We need hash",
> >> +"most recent matching task was")
> >> +        expected_sametmp_output = expected_output + ("Variable do_install value changed",'+    echo "this changes the task signature"')
> >> +        expected_difftmp_output = expected_output
> >> +
> >> +        self.run_test_printdiff_changerecipe("core-image-minimal", "quilt-native", "-c do_install quilt-native",
> >> +"""
> >> +do_install:append() {
> >> +    echo "this changes the task signature"
> >> +}
> >> +""",
> >> +expected_sametmp_output, expected_difftmp_output)
> >> +
> >> +    # Check if changes to gcc-source (which uses tmp/work-shared) are correctly discovered
> >> +    def test_gcc_runtime_vs_gcc_source(self):
> >> +        expected_output = ("Task gcc-source-13.2.0:do_preconfigure couldn't be used from the cache because:",
> >
> >
> > We should avoid using the PV=13.2.0 explicitly imo otherwise it will need to be updated with the recipe forever.
> >
> > The archiver selftest have a gcc-source-PV example
> > https://git.openembedded.org/openembedded-core/tree/meta/lib/oeqa/selftest/cases/archiver.py#n141
> >
> >> +"We need hash",
> >> +"most recent matching task was")
> >> +        expected_sametmp_output = expected_output + ("Variable do_preconfigure value changed",'+    print("this changes the task signature")')
> >> +        #FIXME: printdiff is supposed to find at least one preconfigure task signature in the sstate cache, but isn't able to
> >> +        #expected_difftmp_output = expected_output
> >> +        expected_difftmp_output = ()
> >> +
> >> +        self.run_test_printdiff_changerecipe("gcc-runtime", "gcc-source", "-c do_preconfigure gcc-source-13.2.0",
> >
> >
> > Same here
> >
> > Jose
> >
> >>
> >> +"""
> >> +python do_preconfigure:append() {
> >> +    print("this changes the task signature")
> >> +}
> >> +""",
> >> +expected_sametmp_output, expected_difftmp_output)
> >> +
> >> +    # Check if changing a really base task definiton is reported against multiple core recipes using it
> >> +    def test_image_minimal_vs_base_do_configure(self):
> >> +        expected_output = ("Task zstd-native:do_configure couldn't be used from the cache because:",
> >> +"Task texinfo-dummy-native:do_configure couldn't be used from the cache because:",
> >> +"Task ldconfig-native:do_configure couldn't be used from the cache because:",
> >> +"Task gettext-minimal-native:do_configure couldn't be used from the cache because:",
> >> +"Task tzcode-native:do_configure couldn't be used from the cache because:",
> >> +"Task makedevs-native:do_configure couldn't be used from the cache because:",
> >> +"Task pigz-native:do_configure couldn't be used from the cache because:",
> >> +"Task update-rc.d-native:do_configure couldn't be used from the cache because:",
> >> +"Task unzip-native:do_configure couldn't be used from the cache because:",
> >> +"Task gnu-config-native:do_configure couldn't be used from the cache because:",
> >> +"We need hash",
> >> +"most recent matching task was")
> >> +        expected_sametmp_output = expected_output + ("Variable base_do_configure value changed",'+     echo "this changes base_do_configure() definiton"')
> >> +        expected_difftmp_output = expected_output
> >> +
> >> +        self.run_test_printdiff_changeconfig("core-image-minimal",
> >> +"""
> >> +INHERIT += "base-do-configure-modified"
> >> +""",
> >> +expected_sametmp_output, expected_difftmp_output)
> >> --
> >> 2.39.2
> >>
> >>
> >>
> >>
> >
> >
> > --
> > Best regards,
> >
> > José Quaresma
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#189338): https://lists.openembedded.org/g/openembedded-core/message/189338
> Mute This Topic: https://lists.openembedded.org/mt/102017475/1686489
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>

Alexander Kanavin <alex.kanavin@gmail.com> escreveu no dia quarta,
18/10/2023 à(s) 09:48:

> I sent a v2 of this patch where this issue is corrected.
>

Thanks


>
> Alex
>
> On Tue, 17 Oct 2023 at 16:15, Alexander Kanavin via
> lists.openembedded.org <alex.kanavin=gmail.com@lists.openembedded.org>
> wrote:
> >
> > Thanks, I was completely sure in my head I had fixed this, but either
> > I haven't, or it has been somehow lost.
> >
> > Alex
> >
> > On Tue, 17 Oct 2023 at 16:06, Jose Quaresma <quaresma.jose@gmail.com>
> wrote:
> > >
> > >
> > >
> > > Alexander Kanavin <alex.kanavin@gmail.com> escreveu no dia terça,
> 17/10/2023 à(s) 14:31:
> > >>
> > >> 'bitbake -S printdiff' is a useful diagnostic facility for finding out
> > >> why sstate is not being reused, but until now it had no tests that
> would
> > >> ensure it works. This commit adds three basic scenarios:
> > >>
> > >> 1. make a change in a really basic, common recipe that is at the very
> root
> > >> of dependency trees (quilt-native), and ensure that change is
> correctly discovered when
> > >> building an image.
> > >>
> > >> 2. make a change in gcc-source recipe, which is somewhat special
> > >> (operates in work-shared), and ensure that gcc-runtime builds track
> > >> that down as well.
> > >>
> > >> 3. make a change in base_do_configure() definition from base.bbclass,
> > >> which is not recipe-specific, but affects many basic recipes, and
> ensure that
> > >> is correctly reported as well.
> > >>
> > >> The test itself actually runs twice:
> > >> - first against a fully populated build directory, where
> > >> the printdiff code is guaranteed to find the correct previous
> > >> stamp that can be compared with in a predictable manner.
> > >>
> > >> - then in an empty build directory where the printdiff code
> > >> goes to look in the sstate cache, and so the existence of the
> > >> previous signature can be tested, but not the difference with it
> > >> (what the exact difference would be is unpredictable as the
> > >> sstate cache is indeed shared between many builds).
> > >>
> > >> Signed-off-by: Alexander Kanavin <alex@linutronix.de>
> > >> ---
> > >>  .../base-do-configure-modified.bbclass        |   3 +
> > >>  .../gcc-source/gcc-source_%.bbappend          |   2 +
> > >>  .../quilt-native/quilt-native_%.bbappend      |   2 +
> > >>  meta/lib/oeqa/selftest/cases/sstatetests.py   | 104
> ++++++++++++++++++
> > >>  4 files changed, 111 insertions(+)
> > >>  create mode 100644
> meta-selftest/classes/base-do-configure-modified.bbclass
> > >>  create mode 100644
> meta-selftest/recipes-test/gcc-source/gcc-source_%.bbappend
> > >>  create mode 100644
> meta-selftest/recipes-test/quilt-native/quilt-native_%.bbappend
> > >>
> > >> diff --git a/meta-selftest/classes/base-do-configure-modified.bbclass
> b/meta-selftest/classes/base-do-configure-modified.bbclass
> > >> new file mode 100644
> > >> index 00000000000..3f96827a428
> > >> --- /dev/null
> > >> +++ b/meta-selftest/classes/base-do-configure-modified.bbclass
> > >> @@ -0,0 +1,3 @@
> > >> +base_do_configure:append () {
> > >> +       echo "this changes base_do_configure() definiton"
> > >> +}
> > >> diff --git
> a/meta-selftest/recipes-test/gcc-source/gcc-source_%.bbappend
> b/meta-selftest/recipes-test/gcc-source/gcc-source_%.bbappend
> > >> new file mode 100644
> > >> index 00000000000..205720982cb
> > >> --- /dev/null
> > >> +++ b/meta-selftest/recipes-test/gcc-source/gcc-source_%.bbappend
> > >> @@ -0,0 +1,2 @@
> > >> +# This bbappend is used to alter the recipe using the
> test_recipe.inc file created by tests.
> > >> +include test_recipe.inc
> > >> diff --git
> a/meta-selftest/recipes-test/quilt-native/quilt-native_%.bbappend
> b/meta-selftest/recipes-test/quilt-native/quilt-native_%.bbappend
> > >> new file mode 100644
> > >> index 00000000000..205720982cb
> > >> --- /dev/null
> > >> +++ b/meta-selftest/recipes-test/quilt-native/quilt-native_%.bbappend
> > >> @@ -0,0 +1,2 @@
> > >> +# This bbappend is used to alter the recipe using the
> test_recipe.inc file created by tests.
> > >> +include test_recipe.inc
> > >> diff --git a/meta/lib/oeqa/selftest/cases/sstatetests.py
> b/meta/lib/oeqa/selftest/cases/sstatetests.py
> > >> index bdad9088d37..29e31f2ef9f 100644
> > >> --- a/meta/lib/oeqa/selftest/cases/sstatetests.py
> > >> +++ b/meta/lib/oeqa/selftest/cases/sstatetests.py
> > >> @@ -773,3 +773,107 @@ addtask tmptask2 before do_tmptask1
> > >>                  latestfiles = sorted(filedates.keys(), key=lambda f:
> filedates[f])[-2:]
> > >>                  bb.siggen.compare_sigfiles(latestfiles[-2],
> latestfiles[-1], recursecb)
> > >>                  self.assertEqual(recursecb_count,1)
> > >> +
> > >> +class SStatePrintdiff(SStateBase):
> > >> +    def run_test_printdiff_changerecipe(self, target, change_recipe,
> change_bbtask, change_content, expected_sametmp_output,
> expected_difftmp_output):
> > >> +        self.write_config("""
> > >> +TMPDIR = "${TOPDIR}/tmp-sstateprintdiff"
> > >> +""")
> > >> +        self.track_for_cleanup(self.topdir + "/tmp-sstateprintdiff")
> > >> +        bitbake(target)
> > >> +        bitbake("-S none {}".format(target))
> > >> +        bitbake(change_bbtask)
> > >> +        self.write_recipeinc(change_recipe, change_content)
> > >> +        result_sametmp = bitbake("-S printdiff {}".format(target))
> > >> +
> > >> +        self.write_config("""
> > >> +TMPDIR = "${TOPDIR}/tmp-sstateprintdiff-2"
> > >> +""")
> > >> +        self.track_for_cleanup(self.topdir +
> "/tmp-sstateprintdiff-2")
> > >> +        result_difftmp = bitbake("-S printdiff {}".format(target))
> > >> +
> > >> +        self.delete_recipeinc(change_recipe)
> > >> +        for item in expected_sametmp_output:
> > >> +            self.assertIn(item, result_sametmp.output)
> > >> +        for item in expected_difftmp_output:
> > >> +            self.assertIn(item, result_difftmp.output)
> > >> +
> > >> +    def run_test_printdiff_changeconfig(self, target,
> change_content, expected_sametmp_output, expected_difftmp_output):
> > >> +        self.write_config("""
> > >> +TMPDIR = "${TOPDIR}/tmp-sstateprintdiff"
> > >> +""")
> > >> +        self.track_for_cleanup(self.topdir + "/tmp-sstateprintdiff")
> > >> +        bitbake(target)
> > >> +        bitbake("-S none {}".format(target))
> > >> +        self.append_config(change_content)
> > >> +        result_sametmp = bitbake("-S printdiff {}".format(target))
> > >> +
> > >> +        self.write_config("""
> > >> +TMPDIR = "${TOPDIR}/tmp-sstateprintdiff-2"
> > >> +""")
> > >> +        self.append_config(change_content)
> > >> +        self.track_for_cleanup(self.topdir +
> "/tmp-sstateprintdiff-2")
> > >> +        result_difftmp = bitbake("-S printdiff {}".format(target))
> > >> +
> > >> +        for item in expected_sametmp_output:
> > >> +            self.assertIn(item, result_sametmp.output)
> > >> +        for item in expected_difftmp_output:
> > >> +            self.assertIn(item, result_difftmp.output)
> > >> +
> > >> +
> > >> +    # Check if printdiff walks the full dependency chain from the
> image target to where the change is in a specific recipe
> > >> +    def test_image_minimal_vs_quilt(self):
> > >> +        expected_output = ("Task quilt-native:do_install couldn't be
> used from the cache because:",
> > >> +"We need hash",
> > >> +"most recent matching task was")
> > >> +        expected_sametmp_output = expected_output + ("Variable
> do_install value changed",'+    echo "this changes the task signature"')
> > >> +        expected_difftmp_output = expected_output
> > >> +
> > >> +        self.run_test_printdiff_changerecipe("core-image-minimal",
> "quilt-native", "-c do_install quilt-native",
> > >> +"""
> > >> +do_install:append() {
> > >> +    echo "this changes the task signature"
> > >> +}
> > >> +""",
> > >> +expected_sametmp_output, expected_difftmp_output)
> > >> +
> > >> +    # Check if changes to gcc-source (which uses tmp/work-shared)
> are correctly discovered
> > >> +    def test_gcc_runtime_vs_gcc_source(self):
> > >> +        expected_output = ("Task gcc-source-13.2.0:do_preconfigure
> couldn't be used from the cache because:",
> > >
> > >
> > > We should avoid using the PV=13.2.0 explicitly imo otherwise it will
> need to be updated with the recipe forever.
> > >
> > > The archiver selftest have a gcc-source-PV example
> > >
> https://git.openembedded.org/openembedded-core/tree/meta/lib/oeqa/selftest/cases/archiver.py#n141
> > >
> > >> +"We need hash",
> > >> +"most recent matching task was")
> > >> +        expected_sametmp_output = expected_output + ("Variable
> do_preconfigure value changed",'+    print("this changes the task
> signature")')
> > >> +        #FIXME: printdiff is supposed to find at least one
> preconfigure task signature in the sstate cache, but isn't able to
> > >> +        #expected_difftmp_output = expected_output
> > >> +        expected_difftmp_output = ()
> > >> +
> > >> +        self.run_test_printdiff_changerecipe("gcc-runtime",
> "gcc-source", "-c do_preconfigure gcc-source-13.2.0",
> > >
> > >
> > > Same here
> > >
> > > Jose
> > >
> > >>
> > >> +"""
> > >> +python do_preconfigure:append() {
> > >> +    print("this changes the task signature")
> > >> +}
> > >> +""",
> > >> +expected_sametmp_output, expected_difftmp_output)
> > >> +
> > >> +    # Check if changing a really base task definiton is reported
> against multiple core recipes using it
> > >> +    def test_image_minimal_vs_base_do_configure(self):
> > >> +        expected_output = ("Task zstd-native:do_configure couldn't
> be used from the cache because:",
> > >> +"Task texinfo-dummy-native:do_configure couldn't be used from the
> cache because:",
> > >> +"Task ldconfig-native:do_configure couldn't be used from the cache
> because:",
> > >> +"Task gettext-minimal-native:do_configure couldn't be used from the
> cache because:",
> > >> +"Task tzcode-native:do_configure couldn't be used from the cache
> because:",
> > >> +"Task makedevs-native:do_configure couldn't be used from the cache
> because:",
> > >> +"Task pigz-native:do_configure couldn't be used from the cache
> because:",
> > >> +"Task update-rc.d-native:do_configure couldn't be used from the
> cache because:",
> > >> +"Task unzip-native:do_configure couldn't be used from the cache
> because:",
> > >> +"Task gnu-config-native:do_configure couldn't be used from the cache
> because:",
> > >> +"We need hash",
> > >> +"most recent matching task was")
> > >> +        expected_sametmp_output = expected_output + ("Variable
> base_do_configure value changed",'+     echo "this changes
> base_do_configure() definiton"')
> > >> +        expected_difftmp_output = expected_output
> > >> +
> > >> +        self.run_test_printdiff_changeconfig("core-image-minimal",
> > >> +"""
> > >> +INHERIT += "base-do-configure-modified"
> > >> +""",
> > >> +expected_sametmp_output, expected_difftmp_output)
> > >> --
> > >> 2.39.2
> > >>
> > >>
> > >>
> > >>
> > >
> > >
> > > --
> > > Best regards,
> > >
> > > José Quaresma
> >
> > -=-=-=-=-=-=-=-=-=-=-=-
> > Links: You receive all messages sent to this group.
> > View/Reply Online (#189338):
> https://lists.openembedded.org/g/openembedded-core/message/189338
> > Mute This Topic: https://lists.openembedded.org/mt/102017475/1686489
> > Group Owner: openembedded-core+owner@lists.openembedded.org
> > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
> alex.kanavin@gmail.com]
> > -=-=-=-=-=-=-=-=-=-=-=-
> >
>