| Message ID | 1560cc968f5f93a925cc3ba9436f13210dc4e4c5.1701952927.git.pidge@baylibre.com |
|---|---|
| State | Accepted, archived |
| Commit | b47f2352376bd16b7e7087b4dab143403e67e094 |
| Headers | show |
| Series | useradd fixes, tests and documentation | expand |
On Thu, 7 Dec 2023 at 13:45, Eilís 'pidge' Ní Fhlannagáin <pidge@baylibre.com> wrote: > -USERADDSETSCENEDEPS:class-target = "${MLPREFIX}base-passwd:do_populate_sysroot_setscene pseudo-native:do_populate_sysroot_setscene shadow-native:do_populate_sysroot_setscene ${MLPREFIX}shadow-sysroot:do_populate_sysroot_setscene" > +USERADDSETSCENEDEPS:class-target = "${MLPREFIX}base-passwd:do_populate_sysroot_setscene pseudo-native:do_populate_sysroot_setscene shadow-native:do_populate_sysroot_setscene ${MLPREFIX}shadow-sysroot:do_populate_sysroot_setscene ${@' '.join(['%s:do_populate_sysroot_setscene' % pkg for pkg in d.getVar("USERADD_DEPENDS").split()])}" I'd like to clarify: do all these recent fixes mean that we can also add dynamic library dependencies of shadow-native into this list (it becomes an issue with latest versions of shadow currently not in oe-core)? Specifically, this commit https://git.yoctoproject.org/poky-contrib/commit/?h=akanavin/package-version-updates&id=9fdbe321fa53785cc35bd1bb4054366383ef20e5 doing -USERADDSETSCENEDEPS:class-target = "${MLPREFIX}base-passwd:do_populate_sysroot_setscene pseudo-native:do_populate_sysroot_setscene shadow-native:do_populate_sysroot_setscene ${MLPREFIX}shadow-sysroot:do_populate_sysroot_setscene" +USERADDSETSCENEDEPS:class-target = "${MLPREFIX}base-passwd:do_populate_sysroot_setscene \ + pseudo-native:do_populate_sysroot_setscene \ + shadow-native:do_populate_sysroot_setscene \ + attr-native:do_populate_sysroot_setscene \ + libbsd-native:do_populate_sysroot_setscene \ + libmd-native:do_populate_sysroot_setscene \ + ${MLPREFIX}shadow-sysroot:do_populate_sysroot_setscene" has been rejected by RP with the suggestion that shadow should be linked statically, but now I'm not sure if the argument still holds. Alex
On Thu, 2023-12-07 at 13:58 +0100, Alexander Kanavin wrote: > On Thu, 7 Dec 2023 at 13:45, Eilís 'pidge' Ní Fhlannagáin > <pidge@baylibre.com> wrote: > > -USERADDSETSCENEDEPS:class-target = "${MLPREFIX}base-passwd:do_populate_sysroot_setscene pseudo-native:do_populate_sysroot_setscene shadow-native:do_populate_sysroot_setscene ${MLPREFIX}shadow-sysroot:do_populate_sysroot_setscene" > > +USERADDSETSCENEDEPS:class-target = "${MLPREFIX}base-passwd:do_populate_sysroot_setscene pseudo-native:do_populate_sysroot_setscene shadow-native:do_populate_sysroot_setscene ${MLPREFIX}shadow-sysroot:do_populate_sysroot_setscene ${@' '.join(['%s:do_populate_sysroot_setscene' % pkg for pkg in d.getVar("USERADD_DEPENDS").split()])}" > > I'd like to clarify: do all these recent fixes mean that we can also > add dynamic library dependencies of shadow-native into this list (it > becomes an issue with latest versions of shadow currently not in > oe-core)? > > Specifically, this commit > > https://git.yoctoproject.org/poky-contrib/commit/?h=akanavin/package-version-updates&id=9fdbe321fa53785cc35bd1bb4054366383ef20e5 > > doing > > -USERADDSETSCENEDEPS:class-target = > "${MLPREFIX}base-passwd:do_populate_sysroot_setscene > pseudo-native:do_populate_sysroot_setscene > shadow-native:do_populate_sysroot_setscene > ${MLPREFIX}shadow-sysroot:do_populate_sysroot_setscene" > +USERADDSETSCENEDEPS:class-target = > "${MLPREFIX}base-passwd:do_populate_sysroot_setscene \ > + pseudo-native:do_populate_sysroot_setscene \ > + shadow-native:do_populate_sysroot_setscene \ > + attr-native:do_populate_sysroot_setscene \ > + libbsd-native:do_populate_sysroot_setscene \ > + libmd-native:do_populate_sysroot_setscene \ > + ${MLPREFIX}shadow-sysroot:do_populate_sysroot_setscene" > > has been rejected by RP with the suggestion that shadow should be > linked statically, but now I'm not sure if the argument still holds. After my recent change, we could in theory do this as bitbake might be able to make it work. It does run the risk of circular dependencies and these setscene inter-task dependencies are *horrible* in general from an sstate perspective though as my other comments about these dependencies leading to poor behaviour still stand. To be clear about what you're saying above, you're saying that if any of those native dependencies are missing, we cannot use/install the sstate object. We have to defer installing that sstate until all the others are downloaded and available too. My preference is still to statically link shadow to avoid this pretty horrible side effect. Cheers, Richard
> If recipe A requires the useradd actions of recipe B we need to > ensure that recipe B is part of the recipe A dependancy chain. In > order to do that, we introduce USERADD_DEPENDS. This makes sure > that the do_populate_sysroot_setscene of recipe B exists for > recipe A in case of a missing TMPDIR. This requires changes made in > runqueue.py by RP. I was wondering if all required changes including these would be considered suitable to backport to e.g. kirkstone? Or am I going to have to look forward to scarthgap for these :) Sincerely, Jermain
diff --git a/meta-selftest/recipes-test/selftest-users/creategroup1.bb b/meta-selftest/recipes-test/selftest-users/creategroup1.bb new file mode 100644 index 00000000000..ebbbfaa83d3 --- /dev/null +++ b/meta-selftest/recipes-test/selftest-users/creategroup1.bb @@ -0,0 +1,32 @@ +SUMMARY = "creategroup pt 1" +LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302" + +LICENSE = "MIT" + +DEPENDS = "coreutils-native" + +S = "${WORKDIR}" + +inherit useradd allarch + +USERADD_PACKAGES = "${PN}" +USERADD_PARAM:${PN} = "-u 5555 --gid grouptest gt1" +GROUPADD_PARAM:${PN} = "-r grouptest" + +TESTDIR = "${D}${sysconfdir}/creategroup" + +do_install() { + install -d ${TESTDIR} + install -d ${TESTDIR}/dir + touch ${TESTDIR}/file + ln -s ./file ${TESTDIR}/symlink + install -d ${TESTDIR}/fifotest + mkfifo ${TESTDIR}/fifotest/fifo + + chown gt1:grouptest ${TESTDIR}/file + chown -R gt1:grouptest ${TESTDIR}/dir + chown -h gt1:grouptest ${TESTDIR}/symlink + chown -R gt1:grouptest ${TESTDIR}/fifotest +} + +FILES:${PN} = "${sysconfdir}/creategroup/*" diff --git a/meta-selftest/recipes-test/selftest-users/creategroup2.bb b/meta-selftest/recipes-test/selftest-users/creategroup2.bb new file mode 100644 index 00000000000..ef697f09b4d --- /dev/null +++ b/meta-selftest/recipes-test/selftest-users/creategroup2.bb @@ -0,0 +1,33 @@ +SUMMARY = "creategroup pt 2" +LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302" + +LICENSE = "MIT" + +DEPENDS = "coreutils-native" +USERADD_DEPENDS = "creategroup1" + +S = "${WORKDIR}" + +inherit useradd allarch + +USERADD_PACKAGES = "${PN}" +USERADD_PARAM:${PN} = "-u 5556 --gid grouptest gt2" + +TESTDIR = "${D}${sysconfdir}/creategroup" + +do_install() { + install -d ${TESTDIR} + install -d ${TESTDIR}/dir + touch ${TESTDIR}/file + ln -s ./file ${TESTDIR}/symlink + install -d ${TESTDIR}/fifotest + mkfifo ${TESTDIR}/fifotest/fifo + + chown gt2:grouptest ${TESTDIR}/file + chown -R gt2:grouptest ${TESTDIR}/dir + chown -h gt2:grouptest ${TESTDIR}/symlink + chown -R gt2:grouptest ${TESTDIR}/fifotest +} + +FILES:${PN} = "${sysconfdir}/creategroup/*" + diff --git a/meta-selftest/recipes-test/selftest-users/useraddbadtask.bb b/meta-selftest/recipes-test/selftest-users/useraddbadtask.bb new file mode 100644 index 00000000000..99e04a80b34 --- /dev/null +++ b/meta-selftest/recipes-test/selftest-users/useraddbadtask.bb @@ -0,0 +1,20 @@ +SUMMARY = "UserAddBadTask" +LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302" + +LICENSE = "MIT" + +DEPENDS:append = "coreutils-native" + +S = "${WORKDIR}" + +inherit useradd allarch + +USERADD_PACKAGES = "${PN}" +USERADD_PARAM:${PN} = "-u 5555 --gid groupaddtask useraddtask" +GROUPADD_PARAM:${PN} = "-r groupaddtask" + +do_badthingshappen() { + echo "foo" +} + +addtask badthingshappen after do_populate_sysroot before do_package diff --git a/meta/classes/useradd.bbclass b/meta/classes/useradd.bbclass index 4d3bd9a5f56..a4b8a2d6d60 100644 --- a/meta/classes/useradd.bbclass +++ b/meta/classes/useradd.bbclass @@ -177,9 +177,11 @@ SYSROOT_PREPROCESS_FUNCS += "${SYSROOTFUNC}" SSTATEPREINSTFUNCS:append:class-target = " useradd_sysroot_sstate" +USERADD_DEPENDS ??= "" +DEPENDS += "${USERADD_DEPENDS}" do_package_setscene[depends] += "${USERADDSETSCENEDEPS}" do_populate_sysroot_setscene[depends] += "${USERADDSETSCENEDEPS}" -USERADDSETSCENEDEPS:class-target = "${MLPREFIX}base-passwd:do_populate_sysroot_setscene pseudo-native:do_populate_sysroot_setscene shadow-native:do_populate_sysroot_setscene ${MLPREFIX}shadow-sysroot:do_populate_sysroot_setscene" +USERADDSETSCENEDEPS:class-target = "${MLPREFIX}base-passwd:do_populate_sysroot_setscene pseudo-native:do_populate_sysroot_setscene shadow-native:do_populate_sysroot_setscene ${MLPREFIX}shadow-sysroot:do_populate_sysroot_setscene ${@' '.join(['%s:do_populate_sysroot_setscene' % pkg for pkg in d.getVar("USERADD_DEPENDS").split()])}" USERADDSETSCENEDEPS = "" # Recipe parse-time sanity checks diff --git a/meta/lib/oeqa/selftest/cases/usergrouptests.py b/meta/lib/oeqa/selftest/cases/usergrouptests.py new file mode 100644 index 00000000000..a63fce54dc9 --- /dev/null +++ b/meta/lib/oeqa/selftest/cases/usergrouptests.py @@ -0,0 +1,26 @@ +# +# Copyright OpenEmbedded Contributors +# +# SPDX-License-Identifier: MIT +# + +from oeqa.selftest.case import OESelftestTestCase +from oeqa.utils.commands import bitbake, get_bb_var +import bb.utils +import os + +class UserGroupTests(OESelftestTestCase): + def test_group_from_dep_package(self): + self.logger.info("Building creategroup2") + bitbake(' creategroup2 creategroup1') + bitbake(' creategroup2 creategroup1 -c clean') + self.logger.info("Packaging creategroup2") + self.assertTrue(bitbake(' creategroup2 -c package')) + + def _test_add_task_between_p_sysroot_and_package(self): + self.logger.info("Cleaning sstate for useraddbadtask") + #bitbake(' useraddbadtask -f -c cleansstate') + self.logger.info("Building useraddbadtask") + # This is expected to fail due to bug #14961 + self.assertTrue(bitbake(' useraddbadtask -C fetch')) +
If recipe A requires the useradd actions of recipe B we need to ensure that recipe B is part of the recipe A dependancy chain. In order to do that, we introduce USERADD_DEPENDS. This makes sure that the do_populate_sysroot_setscene of recipe B exists for recipe A in case of a missing TMPDIR. This requires changes made in runqueue.py by RP. This commit along with the runqueue fixes effects: Bug 13419 - recipes that add users to groups cannot rely on other recipes creating those groups (when population from sstate happens) Bug 13904 - do_prepare_recipe_sysroot: postinst-useradd-* does not run in order of dependency and sometimes fails Bug 13279 - Make sure users/groups exist for package_write_* tasks Bug 15084 - For some reason using of same user in two recipes does not work properly I've included the start of self-testing for useradd by adding tests for 13419 (which ends up testing 13904, 13279, 15084 by virtue of them all having the same root cause) I've also added (and disabled), a test for 14961 - addtask between do_populate_sysroot and do_package breaks useradd class. A fix is still needed for this, but that fix is a better error message that stops people from doing this. Signed-off-by: Eilís 'pidge' Ní Fhlannagáin <pidge@baylibre.com> --- .../selftest-users/creategroup1.bb | 32 ++++++++++++++++++ .../selftest-users/creategroup2.bb | 33 +++++++++++++++++++ .../selftest-users/useraddbadtask.bb | 20 +++++++++++ meta/classes/useradd.bbclass | 4 ++- .../lib/oeqa/selftest/cases/usergrouptests.py | 26 +++++++++++++++ 5 files changed, 114 insertions(+), 1 deletion(-) create mode 100644 meta-selftest/recipes-test/selftest-users/creategroup1.bb create mode 100644 meta-selftest/recipes-test/selftest-users/creategroup2.bb create mode 100644 meta-selftest/recipes-test/selftest-users/useraddbadtask.bb create mode 100644 meta/lib/oeqa/selftest/cases/usergrouptests.py