| Message ID | 20230816200931.2577181-8-charlie.johnston@ni.com |
|---|---|
| State | New |
| Headers | show |
| Series | Add new packagefeed recipe class | expand |
Hello, On 16/08/2023 15:08:37-0500, Charlie Johnston wrote: > To test the new packagefeed.bbclass, the testimage case for > testing on rpm images has been updated to use a newly added > packagefeed (packagefeed-core-rpmtest) instead > of manually moving the feeds from DEPLOY_DIR_RPM, removing > unwanted packages, and building the indexes manually. The > new logic creates the feed in DEPLOY_DIR_FEED_RPM via > building the packagefeed and links to it from the location > expected for the tests to run. > > The resulting feed is technically bigger than before but is > still smaller than creating a feed with all packages present > in DEPLOY_DIR_RPM without the manual removal step. > > These changes were tested against testimage runs of the > following images: > - core-image-minimal > - core-image-full-cmdline > - core-image-sato > - core-image-sato-sdk > > Signed-off-by: Charlie Johnston <charlie.johnston@ni.com> > --- > meta/classes-recipe/testimage.bbclass | 57 +++---------------- > .../packagefeeds/packagefeed-core-rpmtest.bb | 14 +++++ > 2 files changed, 22 insertions(+), 49 deletions(-) > create mode 100644 meta/recipes-core/packagefeeds/packagefeed-core-rpmtest.bb This causes: WARNING: packagefeed-core-rpmtest-1.0-r0 do_recipe_qa: QA Issue: Recipe packagefeed-core-rpmtest in /home/pokybuild/yocto-worker/a-full/build/meta/recipes-core/packagefeeds/packagefeed-core-rpmtest.bb does not contain a SUMMARY. Please add an entry. [missing-metadata] WARNING: packagefeed-core-rpmtest-1.0-r0 do_recipe_qa: QA Issue: Recipe packagefeed-core-rpmtest in /home/pokybuild/yocto-worker/a-full/build/meta/recipes-core/packagefeeds/packagefeed-core-rpmtest.bb does not have an assigned maintainer. Please add an entry into meta/conf/distro/include/maintainers.inc. [missing-maintainer]
On 8/18/23 03:40, Alexandre Belloni wrote: > [You don't often get email from alexandre.belloni@bootlin.com. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ] > > Hello, > > On 16/08/2023 15:08:37-0500, Charlie Johnston wrote: >> To test the new packagefeed.bbclass, the testimage case for >> testing on rpm images has been updated to use a newly added >> packagefeed (packagefeed-core-rpmtest) instead >> of manually moving the feeds from DEPLOY_DIR_RPM, removing >> unwanted packages, and building the indexes manually. The >> new logic creates the feed in DEPLOY_DIR_FEED_RPM via >> building the packagefeed and links to it from the location >> expected for the tests to run. >> >> The resulting feed is technically bigger than before but is >> still smaller than creating a feed with all packages present >> in DEPLOY_DIR_RPM without the manual removal step. >> >> These changes were tested against testimage runs of the >> following images: >> - core-image-minimal >> - core-image-full-cmdline >> - core-image-sato >> - core-image-sato-sdk >> >> Signed-off-by: Charlie Johnston <charlie.johnston@ni.com> >> --- >> meta/classes-recipe/testimage.bbclass | 57 +++---------------- >> .../packagefeeds/packagefeed-core-rpmtest.bb | 14 +++++ >> 2 files changed, 22 insertions(+), 49 deletions(-) >> create mode 100644 meta/recipes-core/packagefeeds/packagefeed-core-rpmtest.bb > > This causes: > > WARNING: packagefeed-core-rpmtest-1.0-r0 do_recipe_qa: QA Issue: Recipe packagefeed-core-rpmtest in /home/pokybuild/yocto-worker/a-full/build/meta/recipes-core/packagefeeds/packagefeed-core-rpmtest.bb does not contain a SUMMARY. Please add an entry. [missing-metadata] > WARNING: packagefeed-core-rpmtest-1.0-r0 do_recipe_qa: QA Issue: Recipe packagefeed-core-rpmtest in /home/pokybuild/yocto-worker/a-full/build/meta/recipes-core/packagefeeds/packagefeed-core-rpmtest.bb does not have an assigned maintainer. Please add an entry into meta/conf/distro/include/maintainers.inc. [missing-maintainer] > > > -- > Alexandre Belloni, co-owner and COO, Bootlin > Embedded Linux and Kernel engineering > https://bootlin.com Ah, thanks for catching that. Hadn't rebased since the do_recipe_qa task was added. Fixed and resubmitted. Thanks, Charlie Johnston
diff --git a/meta/classes-recipe/testimage.bbclass b/meta/classes-recipe/testimage.bbclass index e3068348ff..d862e6d98e 100644 --- a/meta/classes-recipe/testimage.bbclass +++ b/meta/classes-recipe/testimage.bbclass @@ -101,6 +101,7 @@ TESTIMAGEDEPENDS += "${@bb.utils.contains('IMAGE_PKGTYPE', 'rpm', 'dnf-native:do TESTIMAGEDEPENDS += "${@bb.utils.contains('IMAGE_PKGTYPE', 'rpm', 'createrepo-c-native:do_populate_sysroot', '', d)}" TESTIMAGEDEPENDS += "${@bb.utils.contains('IMAGE_PKGTYPE', 'ipk', 'opkg-utils-native:do_populate_sysroot package-index:do_package_index', '', d)}" TESTIMAGEDEPENDS += "${@bb.utils.contains('IMAGE_PKGTYPE', 'deb', 'apt-native:do_populate_sysroot package-index:do_package_index', '', d)}" +TESTIMAGEDEPENDS += "${@oe.utils.ifelse(d.getVar('IMAGE_PKGTYPE') == 'rpm' and ('dnf' in d.getVar('TEST_SUITES') or 'auto' in d.getVar('TEST_SUITES')), 'packagefeed-core-rpmtest:do_packagefeed', '')}" TESTIMAGELOCK = "${TMPDIR}/testimage.lock" TESTIMAGELOCK:qemuall = "" @@ -251,7 +252,7 @@ def testimage_main(d): if (d.getVar('IMAGE_PKGTYPE') == 'rpm' and ('dnf' in d.getVar('TEST_SUITES') or 'auto' in d.getVar('TEST_SUITES'))): - create_rpm_index(d) + copy_rpm_repos(d) logger = make_logger_bitbake_compatible(logging.getLogger("BitBake")) pn = d.getVar("PN") @@ -459,55 +460,13 @@ def get_runtime_paths(d): paths.append(path) return paths -def create_index(arg): - import subprocess +def copy_rpm_repos(d): + rpm_dir = os.path.join(d.getVar('DEPLOY_DIR_FEED'), 'rpm', 'packagefeed-core-rpmtest') + idx_path = os.path.join(d.getVar('WORKDIR'), 'oe-testimage-repo') - index_cmd = arg - try: - bb.note("Executing '%s' ..." % index_cmd) - result = subprocess.check_output(index_cmd, - stderr=subprocess.STDOUT, - shell=True) - result = result.decode('utf-8') - except subprocess.CalledProcessError as e: - return("Index creation command '%s' failed with return code " - '%d:\n%s' % (e.cmd, e.returncode, e.output.decode("utf-8"))) - if result: - bb.note(result) - return None - -def create_rpm_index(d): - import glob - # Index RPMs - rpm_createrepo = bb.utils.which(os.getenv('PATH'), "createrepo_c") - index_cmds = [] - archs = (d.getVar('ALL_MULTILIB_PACKAGE_ARCHS') or '').replace('-', '_') - - for arch in archs.split(): - rpm_dir = os.path.join(d.getVar('DEPLOY_DIR_RPM'), arch) - idx_path = os.path.join(d.getVar('WORKDIR'), 'oe-testimage-repo', arch) - - if not os.path.isdir(rpm_dir): - continue - - lockfilename = os.path.join(d.getVar('DEPLOY_DIR_RPM'), 'rpm.lock') - lf = bb.utils.lockfile(lockfilename, False) - oe.path.copyhardlinktree(rpm_dir, idx_path) - # Full indexes overload a 256MB image so reduce the number of rpms - # in the feed by filtering to specific packages needed by the tests. - package_list = glob.glob(idx_path + "*/*.rpm") - - for pkg in package_list: - if not os.path.basename(pkg).startswith(("dnf-test-", "busybox", "update-alternatives", "libc6", "musl")): - bb.utils.remove(pkg) - - bb.utils.unlockfile(lf) - cmd = '%s --update -q %s' % (rpm_createrepo, idx_path) - - # Create repodata - result = create_index(cmd) - if result: - bb.fatal('%s' % ('\n'.join(result))) + oe.path.copyhardlinktree(rpm_dir, idx_path) + # Test cases assume only arch directories and not a top-level repodata next to them. + oe.path.remove("%s/repodata" % idx_path) def package_extraction(d, test_suites): from oeqa.utils.package_manager import find_packages_to_extract diff --git a/meta/recipes-core/packagefeeds/packagefeed-core-rpmtest.bb b/meta/recipes-core/packagefeeds/packagefeed-core-rpmtest.bb new file mode 100644 index 0000000000..9cb0b933b9 --- /dev/null +++ b/meta/recipes-core/packagefeeds/packagefeed-core-rpmtest.bb @@ -0,0 +1,14 @@ +# +# Package feed containing the packages needed for rpm based testimages. +# + +inherit packagefeed + +RDEPENDS:${PN} += " \ + busybox \ + update-alternatives \ + glibc \ + dnf \ +" + +SKIP_RECIPE[packagefeed-core-rpmtest] ?= "${@bb.utils.contains('PACKAGE_CLASSES', 'package_rpm', '', 'is intended only for tests where PACKAGE_CLASSES includes package_rpm.', d)}"
To test the new packagefeed.bbclass, the testimage case for testing on rpm images has been updated to use a newly added packagefeed (packagefeed-core-rpmtest) instead of manually moving the feeds from DEPLOY_DIR_RPM, removing unwanted packages, and building the indexes manually. The new logic creates the feed in DEPLOY_DIR_FEED_RPM via building the packagefeed and links to it from the location expected for the tests to run. The resulting feed is technically bigger than before but is still smaller than creating a feed with all packages present in DEPLOY_DIR_RPM without the manual removal step. These changes were tested against testimage runs of the following images: - core-image-minimal - core-image-full-cmdline - core-image-sato - core-image-sato-sdk Signed-off-by: Charlie Johnston <charlie.johnston@ni.com> --- meta/classes-recipe/testimage.bbclass | 57 +++---------------- .../packagefeeds/packagefeed-core-rpmtest.bb | 14 +++++ 2 files changed, 22 insertions(+), 49 deletions(-) create mode 100644 meta/recipes-core/packagefeeds/packagefeed-core-rpmtest.bb