| Message ID | DB7PR07MB4887C89CABD41DD04CDF9886D34DA@DB7PR07MB4887.eurprd07.prod.outlook.com |
|---|---|
| State | New |
| Headers | show |
| Series | fix: reproducible builds for initramfs and UKI img | expand |
On Mon, 2023-06-05 at 09:00 +0000, Frieder Paape wrote: > I've encountered issues reproducing initramfs and UKI image builds, > which will be fixed with this patch. > > 1. initramfs > There's a symbolic link to /sbin/init, which is appended to the cpio archive after creation. > The links timestamp needs to be static and the cpio append command needs the '--reproducible' flag to produce deterministic outcomes. > > 2. Unified Kernel Image > '--preserve-dates' is required for a static 'Time/Date' entry. > I've added '--enable-deterministic-archives' although in my case this > didn't change anything. > > Signed-off-by: Frieder Paape <frieder@konvera.io> > --- > meta/classes-recipe/image_types.bbclass | 5 +++-- > scripts/lib/wic/plugins/source/bootimg-efi.py | 2 ++ > 2 files changed, 5 insertions(+), 2 deletions(-) > > diff --git a/meta/classes-recipe/image_types.bbclass b/meta/classes-recipe/image_types.bbclass > index bbddfaf272..f73b4d965e 100644 > --- a/meta/classes-recipe/image_types.bbclass > +++ b/meta/classes-recipe/image_types.bbclass > @@ -148,10 +148,11 @@ IMAGE_CMD:cpio () { > if [ ! -L ${IMAGE_ROOTFS}/init ] && [ ! -e ${IMAGE_ROOTFS}/init ]; then > if [ -L ${IMAGE_ROOTFS}/sbin/init ] || [ -e ${IMAGE_ROOTFS}/sbin/init ]; then > ln -sf /sbin/init ${WORKDIR}/cpio_append/init > + touch -h -r ${IMAGE_ROOTFS}/sbin/init ${WORKDIR}/cpio_append/init > else > - touch ${WORKDIR}/cpio_append/init > + touch -h -r ${IMAGE_ROOTFS} ${WORKDIR}/cpio_append/init > fi > - (cd ${WORKDIR}/cpio_append && echo ./init | cpio -oA -H newc -F ${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.cpio) > + (cd ${WORKDIR}/cpio_append && echo ./init | cpio --reproducible -oA -H newc -F ${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.cpio) > fi > fi > } > diff --git a/scripts/lib/wic/plugins/source/bootimg-efi.py b/scripts/lib/wic/plugins/source/bootimg-efi.py > index 43c6fd94d9..2bf7375887 100644 > --- a/scripts/lib/wic/plugins/source/bootimg-efi.py > +++ b/scripts/lib/wic/plugins/source/bootimg-efi.py > @@ -351,6 +351,8 @@ class BootimgEFIPlugin(SourcePlugin): > > # https://www.freedesktop.org/software/systemd/man/systemd-stub.html > objcopy_cmd = "%s-objcopy" % target_sys > + objcopy_cmd += " --enable-deterministic-archives" > + objcopy_cmd += " --preserve-dates" > objcopy_cmd += " --add-section .osrel=%s/usr/lib/os-release" % staging_dir_host > objcopy_cmd += " --change-section-vma .osrel=0x20000" > objcopy_cmd += " --add-section .cmdline=%s" % cmdline.name I like the idea of this and agree it is something we should improve. Unfortunately it caused testing failures: https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/5236/steps/14/logs/stdio oe-selftest -r fitimage.FitImageTests.test_initramfs_bundle should reproduce. Cheers, Richard
The failure happens because `touch` doesn't create a file if called with the no-dereference option `-h`.
Removing `-h` from affected touch command.
Signed-off-by: Frieder Paape <frieder@konvera.io>
---
meta/classes-recipe/image_types.bbclass | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/meta/classes-recipe/image_types.bbclass b/meta/classes-recipe/image_types.bbclass
index f73b4d965e..023eb87537 100644
--- a/meta/classes-recipe/image_types.bbclass
+++ b/meta/classes-recipe/image_types.bbclass
@@ -150,7 +150,7 @@ IMAGE_CMD:cpio () {
ln -sf /sbin/init ${WORKDIR}/cpio_append/init
touch -h -r ${IMAGE_ROOTFS}/sbin/init ${WORKDIR}/cpio_append/init
else
- touch -h -r ${IMAGE_ROOTFS} ${WORKDIR}/cpio_append/init
+ touch -r ${IMAGE_ROOTFS} ${WORKDIR}/cpio_append/init
fi
(cd ${WORKDIR}/cpio_append && echo ./init | cpio --reproducible -oA -H newc -F ${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.cpio)
fi
--
2.39.2 (Apple Git-143)
It's better to resend the whole patchset as v2. Alex On Tue, 6 Jun 2023 at 10:01, Frieder Paape <frieder@konvera.io> wrote: > > The failure happens because `touch` doesn't create a file if called with the no-dereference option `-h`. > Removing `-h` from affected touch command. > > Signed-off-by: Frieder Paape <frieder@konvera.io> > --- > meta/classes-recipe/image_types.bbclass | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/meta/classes-recipe/image_types.bbclass b/meta/classes-recipe/image_types.bbclass > index f73b4d965e..023eb87537 100644 > --- a/meta/classes-recipe/image_types.bbclass > +++ b/meta/classes-recipe/image_types.bbclass > @@ -150,7 +150,7 @@ IMAGE_CMD:cpio () { > ln -sf /sbin/init ${WORKDIR}/cpio_append/init > touch -h -r ${IMAGE_ROOTFS}/sbin/init ${WORKDIR}/cpio_append/init > else > - touch -h -r ${IMAGE_ROOTFS} ${WORKDIR}/cpio_append/init > + touch -r ${IMAGE_ROOTFS} ${WORKDIR}/cpio_append/init > fi > (cd ${WORKDIR}/cpio_append && echo ./init | cpio --reproducible -oA -H newc -F ${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.cpio) > fi > -- > 2.39.2 (Apple Git-143) > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#182416): https://lists.openembedded.org/g/openembedded-core/message/182416 > Mute This Topic: https://lists.openembedded.org/mt/99359051/1686489 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- >
On Wed, 2023-06-07 at 11:00 +0200, Alexander Kanavin wrote:
> It's better to resend the whole patchset as v2.
I did squash this one into the original patch.
For future reference, I did also tweak the shortlog to mention
"image_types:" as the prefix so the area of code changing was clearer
and matches the format we usually use.
It is nice to see work on reproducibility!
Cheers,
Richard
diff --git a/meta/classes-recipe/image_types.bbclass b/meta/classes-recipe/image_types.bbclass index bbddfaf272..f73b4d965e 100644 --- a/meta/classes-recipe/image_types.bbclass +++ b/meta/classes-recipe/image_types.bbclass @@ -148,10 +148,11 @@ IMAGE_CMD:cpio () { if [ ! -L ${IMAGE_ROOTFS}/init ] && [ ! -e ${IMAGE_ROOTFS}/init ]; then if [ -L ${IMAGE_ROOTFS}/sbin/init ] || [ -e ${IMAGE_ROOTFS}/sbin/init ]; then ln -sf /sbin/init ${WORKDIR}/cpio_append/init + touch -h -r ${IMAGE_ROOTFS}/sbin/init ${WORKDIR}/cpio_append/init else - touch ${WORKDIR}/cpio_append/init + touch -h -r ${IMAGE_ROOTFS} ${WORKDIR}/cpio_append/init fi - (cd ${WORKDIR}/cpio_append && echo ./init | cpio -oA -H newc -F ${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.cpio) + (cd ${WORKDIR}/cpio_append && echo ./init | cpio --reproducible -oA -H newc -F ${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.cpio) fi fi } diff --git a/scripts/lib/wic/plugins/source/bootimg-efi.py b/scripts/lib/wic/plugins/source/bootimg-efi.py index 43c6fd94d9..2bf7375887 100644 --- a/scripts/lib/wic/plugins/source/bootimg-efi.py +++ b/scripts/lib/wic/plugins/source/bootimg-efi.py @@ -351,6 +351,8 @@ class BootimgEFIPlugin(SourcePlugin): # https://www.freedesktop.org/software/systemd/man/systemd-stub.html objcopy_cmd = "%s-objcopy" % target_sys + objcopy_cmd += " --enable-deterministic-archives" + objcopy_cmd += " --preserve-dates" objcopy_cmd += " --add-section .osrel=%s/usr/lib/os-release" % staging_dir_host objcopy_cmd += " --change-section-vma .osrel=0x20000" objcopy_cmd += " --add-section .cmdline=%s" % cmdline.name
I've encountered issues reproducing initramfs and UKI image builds, which will be fixed with this patch. 1. initramfs There's a symbolic link to /sbin/init, which is appended to the cpio archive after creation. The links timestamp needs to be static and the cpio append command needs the '--reproducible' flag to produce deterministic outcomes. 2. Unified Kernel Image '--preserve-dates' is required for a static 'Time/Date' entry. I've added '--enable-deterministic-archives' although in my case this didn't change anything. Signed-off-by: Frieder Paape <frieder@konvera.io> --- meta/classes-recipe/image_types.bbclass | 5 +++-- scripts/lib/wic/plugins/source/bootimg-efi.py | 2 ++ 2 files changed, 5 insertions(+), 2 deletions(-)