| Message ID | e2ad5c51a48a8c85bae1c9c7e21912e8fb412c77.camel@infinera.com |
|---|---|
| State | New, archived |
| Delegated to: | Steve Sakoman |
| Headers | show |
| Series | kirkstone: oe-selftest -v -r reproducible.ReproducibleTests.test_reproducible_builds broken ? | expand |
The short execution time is suspicious. The test is written so that build B doesn't reuse the cache from build A, and if it does because of your local tweaks, then you're probably just comparing the A package with itself. Alex On Mon, 18 Dec 2023 at 12:20, Joakim Tjernlund via lists.openembedded.org <Joakim.Tjernlund=infinera.com@lists.openembedded.org> wrote: > > I setup above test for reproducible builds and build one pkg(u-boot) and try to force a > non-reproducible build to test the framework: > --- a/meta/recipes-bsp/u-boot/u-boot.inc > +++ b/meta/recipes-bsp/u-boot/u-boot.inc > @@ -9,7 +9,7 @@ inherit uboot-config uboot-extlinux-config uboot-sign deploy python3native kerne > > DEPENDS += "swig-native" > > -EXTRA_OEMAKE = 'CROSS_COMPILE=${TARGET_PREFIX} CC="${TARGET_PREFIX}gcc ${TOOLCHAIN_OPTIONS}" V=1' > +EXTRA_OEMAKE = 'CROSS_COMPILE=${TARGET_PREFIX} CC="${TARGET_PREFIX}gcc ${TOOLCHAIN_OPTIONS}" V=1 SOURCE_DATE_EPOCH="$(date +%s)"' > EXTRA_OEMAKE += 'HOSTCC="${BUILD_CC} ${BUILD_CFLAGS} ${BUILD_LDFLAGS}"' > EXTRA_OEMAKE += 'STAGING_INCDIR=${STAGING_INCDIR_NATIVE} STAGING_LIBDIR=${STAGING_LIBDIR_NATIVE}' > > forcing EPOCH to current build date but I still get an success report: > > oe-selftest -v -r reproducible.ReproducibleTests.test_reproducible_builds > 2023-12-18 12:05:22,355 - oe-selftest - INFO - Adding layer libraries: > 2023-12-18 12:05:22,356 - oe-selftest - INFO - /home/jocke/yocto-xr/3pp/poky/meta/lib > 2023-12-18 12:05:22,356 - oe-selftest - INFO - /home/jocke/yocto-xr/3pp/poky/meta-yocto-bsp/lib > 2023-12-18 12:05:22,356 - oe-selftest - INFO - /home/jocke/yocto-xr/3pp/meta-openembedded/meta-oe/lib > 2023-12-18 12:05:22,356 - oe-selftest - INFO - /home/jocke/yocto-xr/meta-xr/lib > 2023-12-18 12:05:22,356 - oe-selftest - INFO - /home/jocke/yocto-xr/3pp/poky/meta-selftest/lib > 2023-12-18 12:05:22,359 - oe-selftest - INFO - Running bitbake -e to test the configuration is valid/parsable > 2023-12-18 12:05:27,248 - oe-selftest - INFO - Adding: "include selftest.inc" in /home/jocke/yocto-xr/build/infn-xr/gmcu-st/conf/local.conf > 2023-12-18 12:05:27,248 - oe-selftest - INFO - Adding: "include bblayers.inc" in bblayers.conf > 2023-12-18 12:05:27,248 - oe-selftest - INFO - test_reproducible_builds (reproducible.ReproducibleTests) > 2023-12-18 12:05:29,724 - oe-selftest - DEBUG - Writing to: /home/jocke/yocto-xr/build/infn-xr/gmcu-st/conf/selftest.inc > > > 2023-12-18 12:08:26,539 - oe-selftest - INFO - Building reproducibleA (sstate allowed)... > 2023-12-18 12:08:26,545 - oe-selftest - DEBUG - Writing to: /home/jocke/yocto-xr/build/infn-xr/gmcu-st/conf/selftest.inc > PACKAGE_CLASSES = "package_ipk" > INHIBIT_PACKAGE_STRIP = "1" > TMPDIR = "/home/jocke/yocto-xr/build/infn-xr/reproducibleA/tmp" > LICENSE_FLAGS_ACCEPTED = "commercial" > DISTRO_FEATURES:append = ' systemd pam' > USERADDEXTENSION = "useradd-staticids" > USERADD_ERROR_DYNAMIC = "skip" > USERADD_UID_TABLES += "files/static-passwd" > USERADD_GID_TABLES += "files/static-group" > > > 2023-12-18 12:09:08,936 - oe-selftest - INFO - Building reproducibleB (sstate NOT allowed)... > 2023-12-18 12:09:08,937 - oe-selftest - DEBUG - Writing to: /home/jocke/yocto-xr/build/infn-xr/gmcu-st/conf/selftest.inc > PACKAGE_CLASSES = "package_ipk" > INHIBIT_PACKAGE_STRIP = "1" > TMPDIR = "/home/jocke/yocto-xr/build/infn-xr/reproducibleB/tmp" > LICENSE_FLAGS_ACCEPTED = "commercial" > DISTRO_FEATURES:append = ' systemd pam' > USERADDEXTENSION = "useradd-staticids" > USERADD_ERROR_DYNAMIC = "skip" > USERADD_UID_TABLES += "files/static-passwd" > USERADD_GID_TABLES += "files/static-group" > SSTATE_DIR = "${TMPDIR}/sstate" > SSTATE_MIRRORS = "file://.*/.*-native.* http://sstate.yoctoproject.org/all/PATH;downloadfilename=PATH file://.*/.*-cross.* > http://sstate.yoctoproject.org/all/PATH;downloadfilename=PATH" > > > 2023-12-18 12:09:24,772 - oe-selftest - INFO - Checking ipk packages for differences... > 2023-12-18 12:09:24,843 - oe-selftest - INFO - Reproducibility summary for ipk: same=4 different=0 different_excluded=0 missing=0 total=4 > unused_exclusions=[] > 2023-12-18 12:09:24,847 - oe-selftest - INFO - ... ok > 2023-12-18 12:09:26,044 - oe-selftest - INFO - ---------------------------------------------------------------------- > 2023-12-18 12:09:26,044 - oe-selftest - INFO - Ran 1 test in 241.335s > 2023-12-18 12:09:26,045 - oe-selftest - INFO - OK > 2023-12-18 12:09:32,355 - oe-selftest - INFO - RESULTS: > 2023-12-18 12:09:32,356 - oe-selftest - INFO - RESULTS - reproducible.ReproducibleTests.test_reproducible_builds: PASSED (237.60s) > 2023-12-18 12:09:32,361 - oe-selftest - INFO - SUMMARY: > 2023-12-18 12:09:32,362 - oe-selftest - INFO - oe-selftest () - Ran 1 test in 241.336s > 2023-12-18 12:09:32,362 - oe-selftest - INFO - oe-selftest - OK - All required tests passed (successes=1, skipped=0, failures=0, errors=0) > > > Looking harder I don't see oe-seftest ever creating reproducibleA/reproducibleB directories. > I have these set > OEQA_REPRODUCIBLE_TEST_PACKAGE = "ipk" > OEQA_REPRODUCIBLE_TEST_TARGET = "xr-u-boot" > > I am missing something? > > Jocke > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#192631): https://lists.openembedded.org/g/openembedded-core/message/192631 > Mute This Topic: https://lists.openembedded.org/mt/103240689/1686489 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- >
Sounds plausible but how is that possible? I don't understand what those tweaks are(unless you mean the SOURCE_DATE_EPOCH="$(date +%s)" tweak) Maybe I should try without SSTATE cache at all just to see if that make a difference? Jocke On Mon, 2023-12-18 at 14:11 +0100, Alexander Kanavin wrote: > The short execution time is suspicious. The test is written so that > build B doesn't reuse the cache from build A, and if it does because > of your local tweaks, then you're probably just comparing the A > package with itself. > > Alex > > On Mon, 18 Dec 2023 at 12:20, Joakim Tjernlund via > lists.openembedded.org > <Joakim.Tjernlund=infinera.com@lists.openembedded.org> wrote: > > > > I setup above test for reproducible builds and build one pkg(u-boot) and try to force a > > non-reproducible build to test the framework: > > --- a/meta/recipes-bsp/u-boot/u-boot.inc > > +++ b/meta/recipes-bsp/u-boot/u-boot.inc > > @@ -9,7 +9,7 @@ inherit uboot-config uboot-extlinux-config uboot-sign deploy python3native kerne > > > > DEPENDS += "swig-native" > > > > -EXTRA_OEMAKE = 'CROSS_COMPILE=${TARGET_PREFIX} CC="${TARGET_PREFIX}gcc ${TOOLCHAIN_OPTIONS}" V=1' > > +EXTRA_OEMAKE = 'CROSS_COMPILE=${TARGET_PREFIX} CC="${TARGET_PREFIX}gcc ${TOOLCHAIN_OPTIONS}" V=1 SOURCE_DATE_EPOCH="$(date +%s)"' > > EXTRA_OEMAKE += 'HOSTCC="${BUILD_CC} ${BUILD_CFLAGS} ${BUILD_LDFLAGS}"' > > EXTRA_OEMAKE += 'STAGING_INCDIR=${STAGING_INCDIR_NATIVE} STAGING_LIBDIR=${STAGING_LIBDIR_NATIVE}' > > > > forcing EPOCH to current build date but I still get an success report: > > > > oe-selftest -v -r reproducible.ReproducibleTests.test_reproducible_builds > > 2023-12-18 12:05:22,355 - oe-selftest - INFO - Adding layer libraries: > > 2023-12-18 12:05:22,356 - oe-selftest - INFO - /home/jocke/yocto-xr/3pp/poky/meta/lib > > 2023-12-18 12:05:22,356 - oe-selftest - INFO - /home/jocke/yocto-xr/3pp/poky/meta-yocto-bsp/lib > > 2023-12-18 12:05:22,356 - oe-selftest - INFO - /home/jocke/yocto-xr/3pp/meta-openembedded/meta-oe/lib > > 2023-12-18 12:05:22,356 - oe-selftest - INFO - /home/jocke/yocto-xr/meta-xr/lib > > 2023-12-18 12:05:22,356 - oe-selftest - INFO - /home/jocke/yocto-xr/3pp/poky/meta-selftest/lib > > 2023-12-18 12:05:22,359 - oe-selftest - INFO - Running bitbake -e to test the configuration is valid/parsable > > 2023-12-18 12:05:27,248 - oe-selftest - INFO - Adding: "include selftest.inc" in /home/jocke/yocto-xr/build/infn-xr/gmcu-st/conf/local.conf > > 2023-12-18 12:05:27,248 - oe-selftest - INFO - Adding: "include bblayers.inc" in bblayers.conf > > 2023-12-18 12:05:27,248 - oe-selftest - INFO - test_reproducible_builds (reproducible.ReproducibleTests) > > 2023-12-18 12:05:29,724 - oe-selftest - DEBUG - Writing to: /home/jocke/yocto-xr/build/infn-xr/gmcu-st/conf/selftest.inc > > > > > > 2023-12-18 12:08:26,539 - oe-selftest - INFO - Building reproducibleA (sstate allowed)... > > 2023-12-18 12:08:26,545 - oe-selftest - DEBUG - Writing to: /home/jocke/yocto-xr/build/infn-xr/gmcu-st/conf/selftest.inc > > PACKAGE_CLASSES = "package_ipk" > > INHIBIT_PACKAGE_STRIP = "1" > > TMPDIR = "/home/jocke/yocto-xr/build/infn-xr/reproducibleA/tmp" > > LICENSE_FLAGS_ACCEPTED = "commercial" > > DISTRO_FEATURES:append = ' systemd pam' > > USERADDEXTENSION = "useradd-staticids" > > USERADD_ERROR_DYNAMIC = "skip" > > USERADD_UID_TABLES += "files/static-passwd" > > USERADD_GID_TABLES += "files/static-group" > > > > > > 2023-12-18 12:09:08,936 - oe-selftest - INFO - Building reproducibleB (sstate NOT allowed)... > > 2023-12-18 12:09:08,937 - oe-selftest - DEBUG - Writing to: /home/jocke/yocto-xr/build/infn-xr/gmcu-st/conf/selftest.inc > > PACKAGE_CLASSES = "package_ipk" > > INHIBIT_PACKAGE_STRIP = "1" > > TMPDIR = "/home/jocke/yocto-xr/build/infn-xr/reproducibleB/tmp" > > LICENSE_FLAGS_ACCEPTED = "commercial" > > DISTRO_FEATURES:append = ' systemd pam' > > USERADDEXTENSION = "useradd-staticids" > > USERADD_ERROR_DYNAMIC = "skip" > > USERADD_UID_TABLES += "files/static-passwd" > > USERADD_GID_TABLES += "files/static-group" > > SSTATE_DIR = "${TMPDIR}/sstate" > > SSTATE_MIRRORS = "file://.*/.*-native.* http://sstate.yoctoproject.org/all/PATH;downloadfilename=PATH file://.*/.*-cross.* > > http://sstate.yoctoproject.org/all/PATH;downloadfilename=PATH" > > > > > > 2023-12-18 12:09:24,772 - oe-selftest - INFO - Checking ipk packages for differences... > > 2023-12-18 12:09:24,843 - oe-selftest - INFO - Reproducibility summary for ipk: same=4 different=0 different_excluded=0 missing=0 total=4 > > unused_exclusions=[] > > 2023-12-18 12:09:24,847 - oe-selftest - INFO - ... ok > > 2023-12-18 12:09:26,044 - oe-selftest - INFO - ---------------------------------------------------------------------- > > 2023-12-18 12:09:26,044 - oe-selftest - INFO - Ran 1 test in 241.335s > > 2023-12-18 12:09:26,045 - oe-selftest - INFO - OK > > 2023-12-18 12:09:32,355 - oe-selftest - INFO - RESULTS: > > 2023-12-18 12:09:32,356 - oe-selftest - INFO - RESULTS - reproducible.ReproducibleTests.test_reproducible_builds: PASSED (237.60s) > > 2023-12-18 12:09:32,361 - oe-selftest - INFO - SUMMARY: > > 2023-12-18 12:09:32,362 - oe-selftest - INFO - oe-selftest () - Ran 1 test in 241.336s > > 2023-12-18 12:09:32,362 - oe-selftest - INFO - oe-selftest - OK - All required tests passed (successes=1, skipped=0, failures=0, errors=0) > > > > > > Looking harder I don't see oe-seftest ever creating reproducibleA/reproducibleB directories. > > I have these set > > OEQA_REPRODUCIBLE_TEST_PACKAGE = "ipk" > > OEQA_REPRODUCIBLE_TEST_TARGET = "xr-u-boot" > > > > I am missing something? > > > > Jocke > > > > -=-=-=-=-=-=-=-=-=-=-=- > > Links: You receive all messages sent to this group. > > View/Reply Online (#192631): https://lists.openembedded.org/g/openembedded-core/message/192631 > > Mute This Topic: https://lists.openembedded.org/mt/103240689/1686489 > > Group Owner: openembedded-core+owner@lists.openembedded.org > > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com] > > -=-=-=-=-=-=-=-=-=-=-=- > >
I suppose you can run oe-selftest again with -K, then its build directories (a and b ones) will be preserved and you can inspect them. Alex On Mon, 18 Dec 2023 at 14:58, Joakim Tjernlund <Joakim.Tjernlund@infinera.com> wrote: > > Sounds plausible but how is that possible? I don't understand what those tweaks are(unless you > mean the SOURCE_DATE_EPOCH="$(date +%s)" tweak) > Maybe I should try without SSTATE cache at all just to see if that make a difference? > > Jocke > > On Mon, 2023-12-18 at 14:11 +0100, Alexander Kanavin wrote: > > The short execution time is suspicious. The test is written so that > > build B doesn't reuse the cache from build A, and if it does because > > of your local tweaks, then you're probably just comparing the A > > package with itself. > > > > Alex > > > > On Mon, 18 Dec 2023 at 12:20, Joakim Tjernlund via > > lists.openembedded.org > > <Joakim.Tjernlund=infinera.com@lists.openembedded.org> wrote: > > > > > > I setup above test for reproducible builds and build one pkg(u-boot) and try to force a > > > non-reproducible build to test the framework: > > > --- a/meta/recipes-bsp/u-boot/u-boot.inc > > > +++ b/meta/recipes-bsp/u-boot/u-boot.inc > > > @@ -9,7 +9,7 @@ inherit uboot-config uboot-extlinux-config uboot-sign deploy python3native kerne > > > > > > DEPENDS += "swig-native" > > > > > > -EXTRA_OEMAKE = 'CROSS_COMPILE=${TARGET_PREFIX} CC="${TARGET_PREFIX}gcc ${TOOLCHAIN_OPTIONS}" V=1' > > > +EXTRA_OEMAKE = 'CROSS_COMPILE=${TARGET_PREFIX} CC="${TARGET_PREFIX}gcc ${TOOLCHAIN_OPTIONS}" V=1 SOURCE_DATE_EPOCH="$(date +%s)"' > > > EXTRA_OEMAKE += 'HOSTCC="${BUILD_CC} ${BUILD_CFLAGS} ${BUILD_LDFLAGS}"' > > > EXTRA_OEMAKE += 'STAGING_INCDIR=${STAGING_INCDIR_NATIVE} STAGING_LIBDIR=${STAGING_LIBDIR_NATIVE}' > > > > > > forcing EPOCH to current build date but I still get an success report: > > > > > > oe-selftest -v -r reproducible.ReproducibleTests.test_reproducible_builds > > > 2023-12-18 12:05:22,355 - oe-selftest - INFO - Adding layer libraries: > > > 2023-12-18 12:05:22,356 - oe-selftest - INFO - /home/jocke/yocto-xr/3pp/poky/meta/lib > > > 2023-12-18 12:05:22,356 - oe-selftest - INFO - /home/jocke/yocto-xr/3pp/poky/meta-yocto-bsp/lib > > > 2023-12-18 12:05:22,356 - oe-selftest - INFO - /home/jocke/yocto-xr/3pp/meta-openembedded/meta-oe/lib > > > 2023-12-18 12:05:22,356 - oe-selftest - INFO - /home/jocke/yocto-xr/meta-xr/lib > > > 2023-12-18 12:05:22,356 - oe-selftest - INFO - /home/jocke/yocto-xr/3pp/poky/meta-selftest/lib > > > 2023-12-18 12:05:22,359 - oe-selftest - INFO - Running bitbake -e to test the configuration is valid/parsable > > > 2023-12-18 12:05:27,248 - oe-selftest - INFO - Adding: "include selftest.inc" in /home/jocke/yocto-xr/build/infn-xr/gmcu-st/conf/local.conf > > > 2023-12-18 12:05:27,248 - oe-selftest - INFO - Adding: "include bblayers.inc" in bblayers.conf > > > 2023-12-18 12:05:27,248 - oe-selftest - INFO - test_reproducible_builds (reproducible.ReproducibleTests) > > > 2023-12-18 12:05:29,724 - oe-selftest - DEBUG - Writing to: /home/jocke/yocto-xr/build/infn-xr/gmcu-st/conf/selftest.inc > > > > > > > > > 2023-12-18 12:08:26,539 - oe-selftest - INFO - Building reproducibleA (sstate allowed)... > > > 2023-12-18 12:08:26,545 - oe-selftest - DEBUG - Writing to: /home/jocke/yocto-xr/build/infn-xr/gmcu-st/conf/selftest.inc > > > PACKAGE_CLASSES = "package_ipk" > > > INHIBIT_PACKAGE_STRIP = "1" > > > TMPDIR = "/home/jocke/yocto-xr/build/infn-xr/reproducibleA/tmp" > > > LICENSE_FLAGS_ACCEPTED = "commercial" > > > DISTRO_FEATURES:append = ' systemd pam' > > > USERADDEXTENSION = "useradd-staticids" > > > USERADD_ERROR_DYNAMIC = "skip" > > > USERADD_UID_TABLES += "files/static-passwd" > > > USERADD_GID_TABLES += "files/static-group" > > > > > > > > > 2023-12-18 12:09:08,936 - oe-selftest - INFO - Building reproducibleB (sstate NOT allowed)... > > > 2023-12-18 12:09:08,937 - oe-selftest - DEBUG - Writing to: /home/jocke/yocto-xr/build/infn-xr/gmcu-st/conf/selftest.inc > > > PACKAGE_CLASSES = "package_ipk" > > > INHIBIT_PACKAGE_STRIP = "1" > > > TMPDIR = "/home/jocke/yocto-xr/build/infn-xr/reproducibleB/tmp" > > > LICENSE_FLAGS_ACCEPTED = "commercial" > > > DISTRO_FEATURES:append = ' systemd pam' > > > USERADDEXTENSION = "useradd-staticids" > > > USERADD_ERROR_DYNAMIC = "skip" > > > USERADD_UID_TABLES += "files/static-passwd" > > > USERADD_GID_TABLES += "files/static-group" > > > SSTATE_DIR = "${TMPDIR}/sstate" > > > SSTATE_MIRRORS = "file://.*/.*-native.* http://sstate.yoctoproject.org/all/PATH;downloadfilename=PATH file://.*/.*-cross.* > > > http://sstate.yoctoproject.org/all/PATH;downloadfilename=PATH" > > > > > > > > > 2023-12-18 12:09:24,772 - oe-selftest - INFO - Checking ipk packages for differences... > > > 2023-12-18 12:09:24,843 - oe-selftest - INFO - Reproducibility summary for ipk: same=4 different=0 different_excluded=0 missing=0 total=4 > > > unused_exclusions=[] > > > 2023-12-18 12:09:24,847 - oe-selftest - INFO - ... ok > > > 2023-12-18 12:09:26,044 - oe-selftest - INFO - ---------------------------------------------------------------------- > > > 2023-12-18 12:09:26,044 - oe-selftest - INFO - Ran 1 test in 241.335s > > > 2023-12-18 12:09:26,045 - oe-selftest - INFO - OK > > > 2023-12-18 12:09:32,355 - oe-selftest - INFO - RESULTS: > > > 2023-12-18 12:09:32,356 - oe-selftest - INFO - RESULTS - reproducible.ReproducibleTests.test_reproducible_builds: PASSED (237.60s) > > > 2023-12-18 12:09:32,361 - oe-selftest - INFO - SUMMARY: > > > 2023-12-18 12:09:32,362 - oe-selftest - INFO - oe-selftest () - Ran 1 test in 241.336s > > > 2023-12-18 12:09:32,362 - oe-selftest - INFO - oe-selftest - OK - All required tests passed (successes=1, skipped=0, failures=0, errors=0) > > > > > > > > > Looking harder I don't see oe-seftest ever creating reproducibleA/reproducibleB directories. > > > I have these set > > > OEQA_REPRODUCIBLE_TEST_PACKAGE = "ipk" > > > OEQA_REPRODUCIBLE_TEST_TARGET = "xr-u-boot" > > > > > > I am missing something? > > > > > > Jocke > > > > > > -=-=-=-=-=-=-=-=-=-=-=- > > > Links: You receive all messages sent to this group. > > > View/Reply Online (#192631): https://lists.openembedded.org/g/openembedded-core/message/192631 > > > Mute This Topic: https://lists.openembedded.org/mt/103240689/1686489 > > > Group Owner: openembedded-core+owner@lists.openembedded.org > > > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com] > > > -=-=-=-=-=-=-=-=-=-=-=- > > > >
I have tried that and there are no A/B dirs in there. I can find u-boot...ipk and some other stuff but there is no evidence that the A/B dirs ever existed. Jocke On Mon, 2023-12-18 at 17:20 +0100, Alexander Kanavin wrote: > I suppose you can run oe-selftest again with -K, then its build > directories (a and b ones) will be preserved and you can inspect them. > > Alex > > On Mon, 18 Dec 2023 at 14:58, Joakim Tjernlund > <Joakim.Tjernlund@infinera.com> wrote: > > > > Sounds plausible but how is that possible? I don't understand what those tweaks are(unless you > > mean the SOURCE_DATE_EPOCH="$(date +%s)" tweak) > > Maybe I should try without SSTATE cache at all just to see if that make a difference? > > > > Jocke > > > > On Mon, 2023-12-18 at 14:11 +0100, Alexander Kanavin wrote: > > > The short execution time is suspicious. The test is written so that > > > build B doesn't reuse the cache from build A, and if it does because > > > of your local tweaks, then you're probably just comparing the A > > > package with itself. > > > > > > Alex > > > > > > On Mon, 18 Dec 2023 at 12:20, Joakim Tjernlund via > > > lists.openembedded.org > > > <Joakim.Tjernlund=infinera.com@lists.openembedded.org> wrote: > > > > > > > > I setup above test for reproducible builds and build one pkg(u-boot) and try to force a > > > > non-reproducible build to test the framework: > > > > --- a/meta/recipes-bsp/u-boot/u-boot.inc > > > > +++ b/meta/recipes-bsp/u-boot/u-boot.inc > > > > @@ -9,7 +9,7 @@ inherit uboot-config uboot-extlinux-config uboot-sign deploy python3native kerne > > > > > > > > DEPENDS += "swig-native" > > > > > > > > -EXTRA_OEMAKE = 'CROSS_COMPILE=${TARGET_PREFIX} CC="${TARGET_PREFIX}gcc ${TOOLCHAIN_OPTIONS}" V=1' > > > > +EXTRA_OEMAKE = 'CROSS_COMPILE=${TARGET_PREFIX} CC="${TARGET_PREFIX}gcc ${TOOLCHAIN_OPTIONS}" V=1 SOURCE_DATE_EPOCH="$(date +%s)"' > > > > EXTRA_OEMAKE += 'HOSTCC="${BUILD_CC} ${BUILD_CFLAGS} ${BUILD_LDFLAGS}"' > > > > EXTRA_OEMAKE += 'STAGING_INCDIR=${STAGING_INCDIR_NATIVE} STAGING_LIBDIR=${STAGING_LIBDIR_NATIVE}' > > > > > > > > forcing EPOCH to current build date but I still get an success report: > > > > > > > > oe-selftest -v -r reproducible.ReproducibleTests.test_reproducible_builds > > > > 2023-12-18 12:05:22,355 - oe-selftest - INFO - Adding layer libraries: > > > > 2023-12-18 12:05:22,356 - oe-selftest - INFO - /home/jocke/yocto-xr/3pp/poky/meta/lib > > > > 2023-12-18 12:05:22,356 - oe-selftest - INFO - /home/jocke/yocto-xr/3pp/poky/meta-yocto-bsp/lib > > > > 2023-12-18 12:05:22,356 - oe-selftest - INFO - /home/jocke/yocto-xr/3pp/meta-openembedded/meta-oe/lib > > > > 2023-12-18 12:05:22,356 - oe-selftest - INFO - /home/jocke/yocto-xr/meta-xr/lib > > > > 2023-12-18 12:05:22,356 - oe-selftest - INFO - /home/jocke/yocto-xr/3pp/poky/meta-selftest/lib > > > > 2023-12-18 12:05:22,359 - oe-selftest - INFO - Running bitbake -e to test the configuration is valid/parsable > > > > 2023-12-18 12:05:27,248 - oe-selftest - INFO - Adding: "include selftest.inc" in /home/jocke/yocto-xr/build/infn-xr/gmcu-st/conf/local.conf > > > > 2023-12-18 12:05:27,248 - oe-selftest - INFO - Adding: "include bblayers.inc" in bblayers.conf > > > > 2023-12-18 12:05:27,248 - oe-selftest - INFO - test_reproducible_builds (reproducible.ReproducibleTests) > > > > 2023-12-18 12:05:29,724 - oe-selftest - DEBUG - Writing to: /home/jocke/yocto-xr/build/infn-xr/gmcu-st/conf/selftest.inc > > > > > > > > > > > > 2023-12-18 12:08:26,539 - oe-selftest - INFO - Building reproducibleA (sstate allowed)... > > > > 2023-12-18 12:08:26,545 - oe-selftest - DEBUG - Writing to: /home/jocke/yocto-xr/build/infn-xr/gmcu-st/conf/selftest.inc > > > > PACKAGE_CLASSES = "package_ipk" > > > > INHIBIT_PACKAGE_STRIP = "1" > > > > TMPDIR = "/home/jocke/yocto-xr/build/infn-xr/reproducibleA/tmp" > > > > LICENSE_FLAGS_ACCEPTED = "commercial" > > > > DISTRO_FEATURES:append = ' systemd pam' > > > > USERADDEXTENSION = "useradd-staticids" > > > > USERADD_ERROR_DYNAMIC = "skip" > > > > USERADD_UID_TABLES += "files/static-passwd" > > > > USERADD_GID_TABLES += "files/static-group" > > > > > > > > > > > > 2023-12-18 12:09:08,936 - oe-selftest - INFO - Building reproducibleB (sstate NOT allowed)... > > > > 2023-12-18 12:09:08,937 - oe-selftest - DEBUG - Writing to: /home/jocke/yocto-xr/build/infn-xr/gmcu-st/conf/selftest.inc > > > > PACKAGE_CLASSES = "package_ipk" > > > > INHIBIT_PACKAGE_STRIP = "1" > > > > TMPDIR = "/home/jocke/yocto-xr/build/infn-xr/reproducibleB/tmp" > > > > LICENSE_FLAGS_ACCEPTED = "commercial" > > > > DISTRO_FEATURES:append = ' systemd pam' > > > > USERADDEXTENSION = "useradd-staticids" > > > > USERADD_ERROR_DYNAMIC = "skip" > > > > USERADD_UID_TABLES += "files/static-passwd" > > > > USERADD_GID_TABLES += "files/static-group" > > > > SSTATE_DIR = "${TMPDIR}/sstate" > > > > SSTATE_MIRRORS = "file://.*/.*-native.* http://sstate.yoctoproject.org/all/PATH;downloadfilename=PATH file://.*/.*-cross.* > > > > http://sstate.yoctoproject.org/all/PATH;downloadfilename=PATH" > > > > > > > > > > > > 2023-12-18 12:09:24,772 - oe-selftest - INFO - Checking ipk packages for differences... > > > > 2023-12-18 12:09:24,843 - oe-selftest - INFO - Reproducibility summary for ipk: same=4 different=0 different_excluded=0 missing=0 total=4 > > > > unused_exclusions=[] > > > > 2023-12-18 12:09:24,847 - oe-selftest - INFO - ... ok > > > > 2023-12-18 12:09:26,044 - oe-selftest - INFO - ---------------------------------------------------------------------- > > > > 2023-12-18 12:09:26,044 - oe-selftest - INFO - Ran 1 test in 241.335s > > > > 2023-12-18 12:09:26,045 - oe-selftest - INFO - OK > > > > 2023-12-18 12:09:32,355 - oe-selftest - INFO - RESULTS: > > > > 2023-12-18 12:09:32,356 - oe-selftest - INFO - RESULTS - reproducible.ReproducibleTests.test_reproducible_builds: PASSED (237.60s) > > > > 2023-12-18 12:09:32,361 - oe-selftest - INFO - SUMMARY: > > > > 2023-12-18 12:09:32,362 - oe-selftest - INFO - oe-selftest () - Ran 1 test in 241.336s > > > > 2023-12-18 12:09:32,362 - oe-selftest - INFO - oe-selftest - OK - All required tests passed (successes=1, skipped=0, failures=0, errors=0) > > > > > > > > > > > > Looking harder I don't see oe-seftest ever creating reproducibleA/reproducibleB directories. > > > > I have these set > > > > OEQA_REPRODUCIBLE_TEST_PACKAGE = "ipk" > > > > OEQA_REPRODUCIBLE_TEST_TARGET = "xr-u-boot" > > > > > > > > I am missing something? > > > > > > > > Jocke > > > > > > > > -=-=-=-=-=-=-=-=-=-=-=- > > > > Links: You receive all messages sent to this group. > > > > View/Reply Online (#192631): https://lists.openembedded.org/g/openembedded-core/message/192631 > > > > Mute This Topic: https://lists.openembedded.org/mt/103240689/1686489 > > > > Group Owner: openembedded-core+owner@lists.openembedded.org > > > > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com] > > > > -=-=-=-=-=-=-=-=-=-=-=- > > > > > >
On Mon, 18 Dec 2023 at 17:33, Joakim Tjernlund <Joakim.Tjernlund@infinera.com> wrote: > > I have tried that and there are no A/B dirs in there. I can find u-boot...ipk and some other > stuff but there is no evidence that the A/B dirs ever existed. You need to read the test itself as well to see where the directories are. If the code path is going through their creation and running a builds with them, then there's no way they can't exist. Alex
Note that selftest makes a separate build directory, usually
../{current_build_dir}-st.
Alex
On Mon, 18 Dec 2023 at 17:40, Alexander Kanavin via
lists.openembedded.org <alex.kanavin=gmail.com@lists.openembedded.org>
wrote:
>
> On Mon, 18 Dec 2023 at 17:33, Joakim Tjernlund
> <Joakim.Tjernlund@infinera.com> wrote:
> >
> > I have tried that and there are no A/B dirs in there. I can find u-boot...ipk and some other
> > stuff but there is no evidence that the A/B dirs ever existed.
>
> You need to read the test itself as well to see where the directories
> are. If the code path is going through their creation and running a
> builds with them, then there's no way they can't exist.
>
> Alex
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#192664): https://lists.openembedded.org/g/openembedded-core/message/192664
> Mute This Topic: https://lists.openembedded.org/mt/103240689/1686489
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
Yes, there is an xxx-st where the reproducible build is performed and there is no A/B dies in there.
I added some logging:
--- a/meta/lib/oeqa/selftest/cases/reproducible.py
+++ b/meta/lib/oeqa/selftest/cases/reproducible.py
@@ -205,8 +205,8 @@ class ReproducibleTests(OESelftestTestCase):
capture_vars = ['DEPLOY_DIR_' + c.upper() for c in self.package_classes]
tmpdir = os.path.join(self.topdir, name, 'tmp')
- if os.path.exists(tmpdir):
- bb.utils.remove(tmpdir, recurse=True)
+ #if os.path.exists(tmpdir):
+ #bb.utils.remove(tmpdir, recurse=True)
config = textwrap.dedent('''\
PACKAGE_CLASSES = "{package_classes}"
@@ -229,6 +229,7 @@ class ReproducibleTests(OESelftestTestCase):
# This config fragment will disable using shared and the sstate
# mirror, forcing a complete build from scratch
+ #SSTATE_MIRRORS = "https://se-artif-prd.infinera.com/artifactory/list/icex-sstate"
config += textwrap.dedent('''\
SSTATE_DIR = "${TMPDIR}/sstate"
SSTATE_MIRRORS = "file://.*/.*-native.* http://sstate.yoctoproject.org/all/PATH;downloadfilename=PATH file://.*/.*-cross.*
http://sstate.yoctoproject.org/all/PATH;downloadfilename=PATH"
@@ -269,13 +270,17 @@ class ReproducibleTests(OESelftestTestCase):
# kept after the build so it can be diffed for debugging.
fails = []
-
+ self.logger.info('vars_A:%s' % vars_A)
+ self.logger.info('vars_B:%s' % vars_B)
for c in self.package_classes:
+ self.logger.info('c in package_classes:%s' % c)
with self.subTest(package_class=c):
package_class = 'package_' + c
deploy_A = vars_A['DEPLOY_DIR_' + c.upper()]
deploy_B = vars_B['DEPLOY_DIR_' + c.upper()]
+ self.logger.info('deploy_A:%s' % deploy_A)
+ self.logger.info('deplay_B:%s' % deploy_B)
self.logger.info('Checking %s packages for differences...' % c)
result = self.compare_packages(deploy_A, deploy_B, diffutils_sysroot)
and got this output:
...
2023-12-18 18:35:08,912 - oe-selftest - INFO - vars_A:{'DEPLOY_DIR_IPK': '/home/jocke/yocto-xr/build/infn-xr/gmcu-st/tmp/deploy/ipk'}
2023-12-18 18:35:08,913 - oe-selftest - INFO - vars_B:{'DEPLOY_DIR_IPK': '/home/jocke/yocto-xr/build/infn-xr/gmcu-st/tmp/deploy/ipk'}
2023-12-18 18:35:08,913 - oe-selftest - INFO - c in package_classes:ipk
2023-12-18 18:35:08,913 - oe-selftest - INFO - deploy_A:/home/jocke/yocto-xr/build/infn-xr/gmcu-st/tmp/deploy/ipk
2023-12-18 18:35:08,913 - oe-selftest - INFO - deplay_B:/home/jocke/yocto-xr/build/infn-xr/gmcu-st/tmp/deploy/ipk
Notice how both A and B variants are the same. I wonder what setting I got that does that ?
Jocke
On Mon, 2023-12-18 at 17:42 +0100, Alexander Kanavin wrote:
> Note that selftest makes a separate build directory, usually
> ../{current_build_dir}-st.
>
> Alex
>
> On Mon, 18 Dec 2023 at 17:40, Alexander Kanavin via
> lists.openembedded.org <alex.kanavin=gmail.com@lists.openembedded.org>
> wrote:
> >
> > On Mon, 18 Dec 2023 at 17:33, Joakim Tjernlund
> > <Joakim.Tjernlund@infinera.com> wrote:
> > >
> > > I have tried that and there are no A/B dirs in there. I can find u-boot...ipk and some other
> > > stuff but there is no evidence that the A/B dirs ever existed.
> >
> > You need to read the test itself as well to see where the directories
> > are. If the code path is going through their creation and running a
> > builds with them, then there's no way they can't exist.
> >
> > Alex
> >
> > -=-=-=-=-=-=-=-=-=-=-=-
> > Links: You receive all messages sent to this group.
> > View/Reply Online (#192664): https://lists.openembedded.org/g/openembedded-core/message/192664
> > Mute This Topic: https://lists.openembedded.org/mt/103240689/1686489
> > Group Owner: openembedded-core+owner@lists.openembedded.org
> > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com]
> > -=-=-=-=-=-=-=-=-=-=-=-
> >
--- a/meta/recipes-bsp/u-boot/u-boot.inc +++ b/meta/recipes-bsp/u-boot/u-boot.inc @@ -9,7 +9,7 @@ inherit uboot-config uboot-extlinux-config uboot-sign deploy python3native kerne DEPENDS += "swig-native" -EXTRA_OEMAKE = 'CROSS_COMPILE=${TARGET_PREFIX} CC="${TARGET_PREFIX}gcc ${TOOLCHAIN_OPTIONS}" V=1' +EXTRA_OEMAKE = 'CROSS_COMPILE=${TARGET_PREFIX} CC="${TARGET_PREFIX}gcc ${TOOLCHAIN_OPTIONS}" V=1 SOURCE_DATE_EPOCH="$(date +%s)"' EXTRA_OEMAKE += 'HOSTCC="${BUILD_CC} ${BUILD_CFLAGS} ${BUILD_LDFLAGS}"' EXTRA_OEMAKE += 'STAGING_INCDIR=${STAGING_INCDIR_NATIVE} STAGING_LIBDIR=${STAGING_LIBDIR_NATIVE}'
I setup above test for reproducible builds and build one pkg(u-boot) and try to force a non-reproducible build to test the framework: forcing EPOCH to current build date but I still get an success report: oe-selftest -v -r reproducible.ReproducibleTests.test_reproducible_builds 2023-12-18 12:05:22,355 - oe-selftest - INFO - Adding layer libraries: 2023-12-18 12:05:22,356 - oe-selftest - INFO - /home/jocke/yocto-xr/3pp/poky/meta/lib 2023-12-18 12:05:22,356 - oe-selftest - INFO - /home/jocke/yocto-xr/3pp/poky/meta-yocto-bsp/lib 2023-12-18 12:05:22,356 - oe-selftest - INFO - /home/jocke/yocto-xr/3pp/meta-openembedded/meta-oe/lib 2023-12-18 12:05:22,356 - oe-selftest - INFO - /home/jocke/yocto-xr/meta-xr/lib 2023-12-18 12:05:22,356 - oe-selftest - INFO - /home/jocke/yocto-xr/3pp/poky/meta-selftest/lib 2023-12-18 12:05:22,359 - oe-selftest - INFO - Running bitbake -e to test the configuration is valid/parsable 2023-12-18 12:05:27,248 - oe-selftest - INFO - Adding: "include selftest.inc" in /home/jocke/yocto-xr/build/infn-xr/gmcu-st/conf/local.conf 2023-12-18 12:05:27,248 - oe-selftest - INFO - Adding: "include bblayers.inc" in bblayers.conf 2023-12-18 12:05:27,248 - oe-selftest - INFO - test_reproducible_builds (reproducible.ReproducibleTests) 2023-12-18 12:05:29,724 - oe-selftest - DEBUG - Writing to: /home/jocke/yocto-xr/build/infn-xr/gmcu-st/conf/selftest.inc 2023-12-18 12:08:26,539 - oe-selftest - INFO - Building reproducibleA (sstate allowed)... 2023-12-18 12:08:26,545 - oe-selftest - DEBUG - Writing to: /home/jocke/yocto-xr/build/infn-xr/gmcu-st/conf/selftest.inc PACKAGE_CLASSES = "package_ipk" INHIBIT_PACKAGE_STRIP = "1" TMPDIR = "/home/jocke/yocto-xr/build/infn-xr/reproducibleA/tmp" LICENSE_FLAGS_ACCEPTED = "commercial" DISTRO_FEATURES:append = ' systemd pam' USERADDEXTENSION = "useradd-staticids" USERADD_ERROR_DYNAMIC = "skip" USERADD_UID_TABLES += "files/static-passwd" USERADD_GID_TABLES += "files/static-group" 2023-12-18 12:09:08,936 - oe-selftest - INFO - Building reproducibleB (sstate NOT allowed)... 2023-12-18 12:09:08,937 - oe-selftest - DEBUG - Writing to: /home/jocke/yocto-xr/build/infn-xr/gmcu-st/conf/selftest.inc PACKAGE_CLASSES = "package_ipk" INHIBIT_PACKAGE_STRIP = "1" TMPDIR = "/home/jocke/yocto-xr/build/infn-xr/reproducibleB/tmp" LICENSE_FLAGS_ACCEPTED = "commercial" DISTRO_FEATURES:append = ' systemd pam' USERADDEXTENSION = "useradd-staticids" USERADD_ERROR_DYNAMIC = "skip" USERADD_UID_TABLES += "files/static-passwd" USERADD_GID_TABLES += "files/static-group" SSTATE_DIR = "${TMPDIR}/sstate" SSTATE_MIRRORS = "file://.*/.*-native.* http://sstate.yoctoproject.org/all/PATH;downloadfilename=PATH file://.*/.*-cross.* http://sstate.yoctoproject.org/all/PATH;downloadfilename=PATH" 2023-12-18 12:09:24,772 - oe-selftest - INFO - Checking ipk packages for differences... 2023-12-18 12:09:24,843 - oe-selftest - INFO - Reproducibility summary for ipk: same=4 different=0 different_excluded=0 missing=0 total=4 unused_exclusions=[] 2023-12-18 12:09:24,847 - oe-selftest - INFO - ... ok 2023-12-18 12:09:26,044 - oe-selftest - INFO - ---------------------------------------------------------------------- 2023-12-18 12:09:26,044 - oe-selftest - INFO - Ran 1 test in 241.335s 2023-12-18 12:09:26,045 - oe-selftest - INFO - OK 2023-12-18 12:09:32,355 - oe-selftest - INFO - RESULTS: 2023-12-18 12:09:32,356 - oe-selftest - INFO - RESULTS - reproducible.ReproducibleTests.test_reproducible_builds: PASSED (237.60s) 2023-12-18 12:09:32,361 - oe-selftest - INFO - SUMMARY: 2023-12-18 12:09:32,362 - oe-selftest - INFO - oe-selftest () - Ran 1 test in 241.336s 2023-12-18 12:09:32,362 - oe-selftest - INFO - oe-selftest - OK - All required tests passed (successes=1, skipped=0, failures=0, errors=0) Looking harder I don't see oe-seftest ever creating reproducibleA/reproducibleB directories. I have these set OEQA_REPRODUCIBLE_TEST_PACKAGE = "ipk" OEQA_REPRODUCIBLE_TEST_TARGET = "xr-u-boot" I am missing something? Jocke