| Message ID | 20240415060203.2961-1-johannes.schneider@leica-geosystems.com |
|---|---|
| Headers | show |
| Series | pkg-database and systemd-sysext image | expand |
On 15/04/2024 08:02:00+0200, Johannes Schneider via lists.openembedded.org wrote: > systemd-sysext allows to overlay another image (or multiple) ontop of > a "base-image" = the current rootfs, via the use of overlayfs; to add > tools and features meant for development purposes. > > To quote the documentation on systemd-sysext: > " ...addition in order to make debugging/development easier). System > extension images should not be misunderstood as a generic software > packaging framework, ..." > > To build a lean image, that only holds packages that are not already > part of the base-image, a snapshot of the package-database is taken > after the installation of the base-rootfs is done, and picked up again > when collecting the rootfs of such a extension image. > > with all this in place an example usage could look like this: > some-core-image.bb > inherit core-image > IMAGE_GEN_PKGDBFS = "1" > > extending-image.bb > inherit image-sysext > IMAGE_FSTYPES = "squashfs" > IMAGE_BASE_PKGDB = "some-core-image" > # the above pointing at a package-db similar to: > # build/deploy/images/$MACHINE/some-core-image-$MACHINE-20240210172305-pkgdb.rootfs.tar.gz > > then on the device, running some-core-image, with the extension image placed at FN: > $> ln -s "$FN" /run/extensions/$(basename $FN).raw > $> systemd-sysext list > $> SYSTEMD_LOG_LEVEL=debug systemd-sysext merge > > As long as the VERSION_ID of the extension image matches the os-release > in the base image, the above commands return sucessfully; > for details on the compativility check see the docs for systemd-sysext. > > ========= > > changes with v2: > rebase from 'kirkstone' onto 'master' > > changes with v3: > incorporate review suggestions for simplification > add task dependency handling > add oe-selftest for the pkgdb handling > add variable documentation and > some more comments, and examples in the commit-msg > > changes with v4: > rebase onto 'master' => no functional changes > fixed patchtest findings > > changes with v5: > rebase onto 'master' > add '.sysext' to the deployed symlink name > sidenote on the tests and autobuilder failure: run locally they succeed, e.g.: > #> oe-selftest --verbose -r imagefeatures.ImageFeatures.test_image_gen_pkgdbfs > > changes with v6: > restructure test to call 'bitbake' only once in the testcase itself > (in hopes of solving the autobuilder problem; local test runs succeed) This still fails on the autobuilders: https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/6607/steps/14/logs/stdio Reached login banner in 3.26 seconds (04/18/24 18:54:41) Couldn't login into serial console as root using blank password The output: root Password: Login timed out after 60 seconds. Poky (Yocto Project Reference Distro) 5.0+snapshot-0ee872ff1baf43e5436ccbd4c9ae4d733289e05e qemux86-64 /dev/ttyS1 qemux86-64 login: <<< run_serial(): command timed out after 120 seconds without output >>> 2024-04-18 19:17:59,061 - oe-selftest - INFO - 6: 20/46 379/588 (131.80s) (0 failed) (imagefeatures.ImageFeatures.test_image_gen_pkgdbfs) 2024-04-18 19:17:59,062 - oe-selftest - INFO - testtools.testresult.real._StringException: Traceback (most recent call last): File "/home/pokybuild/yocto-worker/oe-selftest-debian/build/meta/lib/oeqa/selftest/cases/imagefeatures.py", line 342, in test_image_gen_pkgdbfs self.assertTrue(os.path.exists(os.path.join(img_vars['DEPLOY_DIR_IMAGE'], 'var/lib/opkg/status')), 'opkg\'s status file was not present in: %s' % tar_file) File "/usr/lib/python3.9/unittest/case.py", line 682, in assertTrue raise self.failureException(msg) AssertionError: False is not true : opkg's status file was not present in: /home/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3638382/tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.rootfs-pkgdb.tar.gz > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#198219): https://lists.openembedded.org/g/openembedded-core/message/198219 > Mute This Topic: https://lists.openembedded.org/mt/105530718/3617179 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alexandre.belloni@bootlin.com] > -=-=-=-=-=-=-=-=-=-=-=- >
Hoi Alexandre, and thanks again for pointing out the build-failure. but since the recent test restructuring didn't help either... i could use help to reproduce what the autobuilder is *exactly* doing :-S because when running the tests directly, they pass (see below) my guess is that in the whole/complete test-run the autobuilder pulls sth out of the (sstate) cache instead of actually building the image with the different feature-set? or could it be some other side-effect when the whole test-set is run? i had based the testcases for this patch-set on the others already present - so the same logical flow of building images is used (or did i miss sth?) any hints/pointers/thoughts would be greatly appreciated :-) gruß Johannes $> oe-selftest --verbose -r imagefeatures.ImageFeatures.test_image_gen_pkgdbfs 2024-04-15 07:50:39,668 - oe-selftest - INFO - Adding layer libraries: 2024-04-15 07:50:39,668 - oe-selftest - INFO - /bsp-user-workspaces/scjoh/upstream-patches/openembedded-core/meta/lib 2024-04-15 07:50:39,668 - oe-selftest - INFO - /bsp-user-workspaces/scjoh/upstream-patches/openembedded-core/meta-selftest/lib 2024-04-15 07:50:39,669 - oe-selftest - INFO - Checking base configuration is valid/parsable NOTE: Starting bitbake server... 2024-04-15 07:50:40,549 - oe-selftest - INFO - Adding: "include selftest.inc" in /bsp-user-workspaces/scjoh/upstream-patches/openembedded-core/build-st/conf/local.conf 2024-04-15 07:50:40,549 - oe-selftest - INFO - Adding: "include bblayers.inc" in bblayers.conf 2024-04-15 07:51:03,338 - oe-selftest - INFO - test_image_gen_pkgdbfs (imagefeatures.ImageFeatures) 2024-04-15 07:51:03,344 - oe-selftest - DEBUG - Writing to: /bsp-user-workspaces/scjoh/upstream-patches/openembedded-core/build-st/conf/selftest.inc IMAGE_GEN_PKGDBFS = "1" 2024-04-15 07:51:56,862 - oe-selftest - DEBUG - running cmd: /bsp-user-workspaces/scjoh/upstream-patches/openembedded-core/build-st/tmp-glibc/sysroots/x86_64/usr/bin/opkg --volatile-cache --offline-root=/bsp-user-workspaces/scjoh/upstream-patches/openembedded-core/build-st/tmp-glibc/deploy/images/qemux86-64 --add-arch qemux86-64:11 --add-arch qemux86_64:12 list-installed 2024-04-15 07:51:56,875 - oe-selftest - DEBUG - list-installed: base-files - 3.0.14-r0 init-ifupdown - 1.0-r0 kernel-6.6.23-yocto-standard - 6.6.23+git0+f7f00b22ef_2d01bc1d4e-r0 kernel-image-6.6.23-yocto-standard - 6.6.23+git0+f7f00b22ef_2d01bc1d4e-r0 kernel-image-bzimage-6.6.23-yocto-standard - 6.6.23+git0+f7f00b22ef_2d01bc1d4e-r0 kernel-module-uvesafb-6.6.23-yocto-standard - 6.6.23+git0+f7f00b22ef_2d01bc1d4e-r0 packagegroup-core-boot - 1.0-r0 sysvinit-inittab - 2.88dsf-r0 v86d - 0.1.10-r0 2024-04-15 07:51:56,875 - oe-selftest - DEBUG - running cmd: /bsp-user-workspaces/scjoh/upstream-patches/openembedded-core/build-st/tmp-glibc/sysroots/x86_64/usr/bin/opkg --volatile-cache --offline-root=/bsp-user-workspaces/scjoh/upstream-patches/openembedded-core/build-st/tmp-glibc/deploy/images/qemux86-64 --add-arch core2-64:11 --add-arch core2_64:12 list-installed 2024-04-15 07:51:56,888 - oe-selftest - DEBUG - list-installed: base-passwd - 3.6.3-r0 busybox - 1.36.1-r0 busybox-hwclock - 1.36.1-r0 busybox-syslog - 1.36.1-r0 busybox-udhcpc - 1.36.1-r0 eudev - 3.2.14-r0 init-system-helpers-service - 1.66-r0 initscripts - 1.0-r0 initscripts-functions - 1.0-r0 kmod - 31-r0 ldconfig - 2.39+git0+1b9c1a0047-r0 libblkid1 - 2.39.3-r0 libc6 - 2.39+git0+1b9c1a0047-r0 libcrypto3 - 3.2.1-r0 libkmod2 - 31-r0 liblzma5 - 5.4.6-r0 libz1 - 1.3.1-r0 modutils-initscripts - 1.0-r0 openssl-conf - 3.2.1-r0 openssl-ossl-module-legacy - 3.2.1-r0 sysvinit - 3.04-r0 sysvinit-pidof - 3.04-r0 ttyrun - 2.31.0-r0 update-alternatives-opkg - 0.6.3-r0 util-linux-fcntl-lock - 2.39.3-r0 2024-04-15 07:51:56,888 - oe-selftest - INFO - ... ok 2024-04-15 07:51:58,138 - oe-selftest - INFO - ---------------------------------------------------------------------- 2024-04-15 07:51:58,138 - oe-selftest - INFO - Ran 1 test in 77.808s 2024-04-15 07:51:58,139 - oe-selftest - INFO - OK 2024-04-15 07:52:01,880 - oe-selftest - INFO - RESULTS: 2024-04-15 07:52:01,880 - oe-selftest - INFO - RESULTS - imagefeatures.ImageFeatures.test_image_gen_pkgdbfs: PASSED (53.55s) 2024-04-15 07:52:01,881 - oe-selftest - INFO - SUMMARY: 2024-04-15 07:52:01,882 - oe-selftest - INFO - oe-selftest () - Ran 1 test in 77.809s 2024-04-15 07:52:01,882 - oe-selftest - INFO - oe-selftest - OK - All required tests passed (successes=1, skipped=0, failures=0, errors=0)
systemd-sysext allows to overlay another image (or multiple) ontop of a "base-image" = the current rootfs, via the use of overlayfs; to add tools and features meant for development purposes. To quote the documentation on systemd-sysext: " ...addition in order to make debugging/development easier). System extension images should not be misunderstood as a generic software packaging framework, ..." To build a lean image, that only holds packages that are not already part of the base-image, a snapshot of the package-database is taken after the installation of the base-rootfs is done, and picked up again when collecting the rootfs of such a extension image. with all this in place an example usage could look like this: some-core-image.bb inherit core-image IMAGE_GEN_PKGDBFS = "1" extending-image.bb inherit image-sysext IMAGE_FSTYPES = "squashfs" IMAGE_BASE_PKGDB = "some-core-image" # the above pointing at a package-db similar to: # build/deploy/images/$MACHINE/some-core-image-$MACHINE-20240210172305-pkgdb.rootfs.tar.gz then on the device, running some-core-image, with the extension image placed at FN: $> ln -s "$FN" /run/extensions/$(basename $FN).raw $> systemd-sysext list $> SYSTEMD_LOG_LEVEL=debug systemd-sysext merge As long as the VERSION_ID of the extension image matches the os-release in the base image, the above commands return sucessfully; for details on the compativility check see the docs for systemd-sysext. ========= changes with v2: rebase from 'kirkstone' onto 'master' changes with v3: incorporate review suggestions for simplification add task dependency handling add oe-selftest for the pkgdb handling add variable documentation and some more comments, and examples in the commit-msg changes with v4: rebase onto 'master' => no functional changes fixed patchtest findings changes with v5: rebase onto 'master' add '.sysext' to the deployed symlink name sidenote on the tests and autobuilder failure: run locally they succeed, e.g.: #> oe-selftest --verbose -r imagefeatures.ImageFeatures.test_image_gen_pkgdbfs changes with v6: restructure test to call 'bitbake' only once in the testcase itself (in hopes of solving the autobuilder problem; local test runs succeed)