From patchwork Thu Mar 31 18:28:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 6118 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id EB1C1C433FE for ; Thu, 31 Mar 2022 18:29:19 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mx.groups.io with SMTP id smtpd.web12.700.1648751358534005533 for ; Thu, 31 Mar 2022 11:29:18 -0700 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: arm.com, ip: 217.140.110.172, mailfrom: ross.burton@arm.com) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 4FC15139F for ; Thu, 31 Mar 2022 11:29:17 -0700 (PDT) Received: from oss-tx204.lab.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 003D03F718 for ; Thu, 31 Mar 2022 11:29:16 -0700 (PDT) From: Ross Burton To: openembedded-core@lists.openembedded.org Subject: [PATCH 01/23] buildtools-tarball: include nativesdk-python3-pyyaml Date: Thu, 31 Mar 2022 19:28:53 +0100 Message-Id: <20220331182915.22128-1-ross.burton@arm.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 31 Mar 2022 18:29:19 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/163842 BitBake can optionally 'import yaml' if BB_LOGCONFIG specifies a yaml file. This is a 3rd party module, so that this works out of the box when buildtools is used -- either explicitly via buildtools-tarball or implicitly via eSDK -- we can add pyyaml to the buildtools. Signed-off-by: Ross Burton --- meta/recipes-core/meta/buildtools-tarball.bb | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/meta/recipes-core/meta/buildtools-tarball.bb b/meta/recipes-core/meta/buildtools-tarball.bb index fac91cfc3cf..f9f36e68c4b 100644 --- a/meta/recipes-core/meta/buildtools-tarball.bb +++ b/meta/recipes-core/meta/buildtools-tarball.bb @@ -7,13 +7,12 @@ TOOLCHAIN_TARGET_TASK ?= "" TOOLCHAIN_HOST_TASK ?= "\ nativesdk-sdk-provides-dummy \ - nativesdk-python3-core \ - nativesdk-python3-modules \ - nativesdk-python3-misc \ + nativesdk-python3 \ nativesdk-python3-git \ nativesdk-python3-jinja2 \ nativesdk-python3-testtools \ nativesdk-python3-subunit \ + nativesdk-python3-pyyaml \ nativesdk-ncurses-terminfo-base \ nativesdk-chrpath \ nativesdk-tar \ From patchwork Thu Mar 31 18:28:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 6117 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id EA78FC433EF for ; Thu, 31 Mar 2022 18:29:19 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mx.groups.io with SMTP id smtpd.web09.653.1648751358746723060 for ; Thu, 31 Mar 2022 11:29:19 -0700 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: arm.com, ip: 217.140.110.172, mailfrom: ross.burton@arm.com) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id D4E7813D5 for ; Thu, 31 Mar 2022 11:29:17 -0700 (PDT) Received: from oss-tx204.lab.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 86D313F718 for ; Thu, 31 Mar 2022 11:29:17 -0700 (PDT) From: Ross Burton To: openembedded-core@lists.openembedded.org Subject: [PATCH 02/23] image_types: hddimg and iso only work on x86 Date: Thu, 31 Mar 2022 19:28:54 +0100 Message-Id: <20220331182915.22128-2-ross.burton@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220331182915.22128-1-ross.burton@arm.com> References: <20220331182915.22128-1-ross.burton@arm.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 31 Mar 2022 18:29:19 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/163844 These image types use syslinux which is only available on x86, so only add them to IMAGE_TYPES on x86. Signed-off-by: Ross Burton --- meta/classes/image_types.bbclass | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/meta/classes/image_types.bbclass b/meta/classes/image_types.bbclass index f643ed3ce7f..960dab1a60e 100644 --- a/meta/classes/image_types.bbclass +++ b/meta/classes/image_types.bbclass @@ -264,8 +264,6 @@ IMAGE_TYPES = " \ ext3 ext3.gz \ ext4 ext4.gz \ btrfs \ - iso \ - hddimg \ squashfs squashfs-xz squashfs-lzo squashfs-lz4 squashfs-zst \ ubi ubifs multiubi \ tar tar.gz tar.bz2 tar.xz tar.lz4 tar.zst \ @@ -275,6 +273,9 @@ IMAGE_TYPES = " \ f2fs \ erofs erofs-lz4 erofs-lz4hc \ " +# These image types are x86 specific as they need syslinux +IMAGE_TYPES:append:x86 = " hddimg iso" +IMAGE_TYPES:append:x86-64 = " hddimg iso" # Compression is a special case of conversion. The old variable # names are still supported for backward-compatibility. When defining From patchwork Thu Mar 31 18:28:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 6119 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0B6C1C43217 for ; Thu, 31 Mar 2022 18:29:20 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mx.groups.io with SMTP id smtpd.web08.700.1648751358764181178 for ; Thu, 31 Mar 2022 11:29:19 -0700 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: arm.com, ip: 217.140.110.172, mailfrom: ross.burton@arm.com) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 671D41596 for ; Thu, 31 Mar 2022 11:29:18 -0700 (PDT) Received: from oss-tx204.lab.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 1926F3F718 for ; Thu, 31 Mar 2022 11:29:17 -0700 (PDT) From: Ross Burton To: openembedded-core@lists.openembedded.org Subject: [PATCH 03/23] oeqa/selftest/devtool: ensure Git username is set before upgrade tests Date: Thu, 31 Mar 2022 19:28:55 +0100 Message-Id: <20220331182915.22128-3-ross.burton@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220331182915.22128-1-ross.burton@arm.com> References: <20220331182915.22128-1-ross.burton@arm.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 31 Mar 2022 18:29:20 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/163843 The 'devtool upgrade' tests fail if Git doesn't know the user's name or email, so verify this before the tests start and skip if it is not. Signed-off-by: Ross Burton --- meta/lib/oeqa/selftest/cases/devtool.py | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/meta/lib/oeqa/selftest/cases/devtool.py b/meta/lib/oeqa/selftest/cases/devtool.py index e910672c314..9c69585a88c 100644 --- a/meta/lib/oeqa/selftest/cases/devtool.py +++ b/meta/lib/oeqa/selftest/cases/devtool.py @@ -1471,6 +1471,14 @@ class DevtoolExtractTests(DevtoolBase): class DevtoolUpgradeTests(DevtoolBase): + def setUp(self): + super().setUp() + try: + runCmd("git config --global user.name") + runCmd("git config --global user.email") + except: + self.skip("Git user.name and user.email must be set") + def test_devtool_upgrade(self): # Check preconditions self.assertTrue(not os.path.exists(self.workspacedir), 'This test cannot be run with a workspace directory under the build directory') From patchwork Thu Mar 31 18:28:56 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 6120 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id ED5F7C4332F for ; Thu, 31 Mar 2022 18:29:19 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mx.groups.io with SMTP id smtpd.web12.701.1648751359429613184 for ; Thu, 31 Mar 2022 11:29:19 -0700 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: arm.com, ip: 217.140.110.172, mailfrom: ross.burton@arm.com) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id EDB4215BF for ; Thu, 31 Mar 2022 11:29:18 -0700 (PDT) Received: from oss-tx204.lab.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 9FA503F718 for ; Thu, 31 Mar 2022 11:29:18 -0700 (PDT) From: Ross Burton To: openembedded-core@lists.openembedded.org Subject: [PATCH 04/23] oeqa/selftest/wic: use os.rename instead of bb.utils.rename Date: Thu, 31 Mar 2022 19:28:56 +0100 Message-Id: <20220331182915.22128-4-ross.burton@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220331182915.22128-1-ross.burton@arm.com> References: <20220331182915.22128-1-ross.burton@arm.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 31 Mar 2022 18:29:19 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/163845 bb.utils.rename() only exists to handle moves across filesystems. As these moves are within the same directory we can just use os.rename(). Signed-off-by: Ross Burton --- meta/lib/oeqa/selftest/cases/wic.py | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/meta/lib/oeqa/selftest/cases/wic.py b/meta/lib/oeqa/selftest/cases/wic.py index 6f3dc277439..673942fc9cc 100644 --- a/meta/lib/oeqa/selftest/cases/wic.py +++ b/meta/lib/oeqa/selftest/cases/wic.py @@ -1416,8 +1416,8 @@ class Wic2(WicTestCase): result = runCmd("%s/usr/sbin/sfdisk -F %s" % (sysroot, new_image_path)) self.assertTrue("0 B, 0 bytes, 0 sectors" in result.output) - bb.utils.rename(image_path, image_path + '.bak') - bb.utils.rename(new_image_path, image_path) + os.rename(image_path, image_path + '.bak') + os.rename(new_image_path, image_path) # Check if it boots in qemu with runqemu('core-image-minimal', ssh=False, runqemuparams='nographic') as qemu: @@ -1428,7 +1428,8 @@ class Wic2(WicTestCase): if os.path.exists(new_image_path): os.unlink(new_image_path) if os.path.exists(image_path + '.bak'): - bb.utils.rename(image_path + '.bak', image_path) + os.rename(image_path + '.bak', image_path) + def test_wic_ls_ext(self): """Test listing content of the ext partition using 'wic ls'""" From patchwork Thu Mar 31 18:28:57 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 6121 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id D4FA0C433F5 for ; Thu, 31 Mar 2022 18:29:20 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mx.groups.io with SMTP id smtpd.web11.663.1648751360113174708 for ; Thu, 31 Mar 2022 11:29:20 -0700 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: arm.com, ip: 217.140.110.172, mailfrom: ross.burton@arm.com) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 8045A139F for ; Thu, 31 Mar 2022 11:29:19 -0700 (PDT) Received: from oss-tx204.lab.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 3247C3F718 for ; Thu, 31 Mar 2022 11:29:19 -0700 (PDT) From: Ross Burton To: openembedded-core@lists.openembedded.org Subject: [PATCH 05/23] oeqa/selftest/wic: remove redundant asserts Date: Thu, 31 Mar 2022 19:28:57 +0100 Message-Id: <20220331182915.22128-5-ross.burton@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220331182915.22128-1-ross.burton@arm.com> References: <20220331182915.22128-1-ross.burton@arm.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 31 Mar 2022 18:29:20 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/163846 By default bitbake() will raise an assertion if it fails, so there's no need to wrap it in a further assert. Signed-off-by: Ross Burton --- meta/lib/oeqa/selftest/cases/wic.py | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/meta/lib/oeqa/selftest/cases/wic.py b/meta/lib/oeqa/selftest/cases/wic.py index 673942fc9cc..5d7f7bf6135 100644 --- a/meta/lib/oeqa/selftest/cases/wic.py +++ b/meta/lib/oeqa/selftest/cases/wic.py @@ -104,7 +104,7 @@ class WicTestCase(OESelftestTestCase): def _get_image_env_path(self, image): """Generate and obtain the path to .env""" if image not in WicTestCase.wicenv_cache: - self.assertEqual(0, bitbake('%s -c do_rootfs_wicenv' % image).status) + bitbake('%s -c do_rootfs_wicenv' % image) bb_vars = get_bb_vars(['STAGING_DIR', 'MACHINE'], image) stdir = bb_vars['STAGING_DIR'] machine = bb_vars['MACHINE'] @@ -694,7 +694,7 @@ part /etc --source rootfs --fstype=ext4 --change-directory=etc os.environ['PATH'] = get_bb_var("PATH", "wic-tools") # Get stock fstab from base-files recipe - self.assertEqual(0, bitbake('base-files -c do_install').status) + bitbake('base-files -c do_install') bf_fstab = os.path.join(get_bb_var('D', 'base-files'), 'etc/fstab') self.assertEqual(True, os.path.exists(bf_fstab)) bf_fstab_md5sum = runCmd('md5sum %s 2>/dev/null' % bf_fstab).output.split(" ")[0] @@ -828,7 +828,7 @@ class Wic2(WicTestCase): config = 'IMAGE_FSTYPES += "wic"\nWKS_FILE = "wic-image-minimal"\n'\ 'MACHINE_FEATURES:append = " efi"\n' self.append_config(config) - self.assertEqual(0, bitbake('wic-image-minimal').status) + bitbake('wic-image-minimal') self.remove_config(config) bb_vars = get_bb_vars(['DEPLOY_DIR_IMAGE', 'MACHINE']) @@ -848,7 +848,7 @@ class Wic2(WicTestCase): config = 'IMAGE_FSTYPES += "wic"\nWKS_FILE = "wic-image-minimal"\n'\ 'MACHINE_FEATURES:append = " efi"\n' self.append_config(config) - self.assertEqual(0, bitbake('wic-image-minimal').status) + bitbake('wic-image-minimal') self.remove_config(config) with runqemu('wic-image-minimal', ssh=False, runqemuparams='nographic') as qemu: @@ -867,7 +867,7 @@ class Wic2(WicTestCase): """Test core-image-minimal efi image under qemu""" config = 'IMAGE_FSTYPES = "wic"\nWKS_FILE = "mkefidisk.wks"\n' self.append_config(config) - self.assertEqual(0, bitbake('core-image-minimal ovmf').status) + bitbake('core-image-minimal ovmf') self.remove_config(config) with runqemu('core-image-minimal', ssh=False, @@ -1051,12 +1051,12 @@ class Wic2(WicTestCase): # build ext4 and then use it for a wic image config = 'IMAGE_FSTYPES = "ext4"\n' self.append_config(config) - self.assertEqual(0, bitbake('core-image-minimal').status) + bitbake('core-image-minimal') self.remove_config(config) config = 'IMAGE_FSTYPES = "wic"\nWKS_FILE = "test_rawcopy_plugin.wks.in"\n' self.append_config(config) - self.assertEqual(0, bitbake('core-image-minimal-mtdutils').status) + bitbake('core-image-minimal-mtdutils') self.remove_config(config) with runqemu('core-image-minimal-mtdutils', ssh=False, @@ -1096,7 +1096,7 @@ class Wic2(WicTestCase): """Test empty plugin""" config = 'IMAGE_FSTYPES = "wic"\nWKS_FILE = "test_empty_plugin.wks"\n' self.append_config(config) - self.assertEqual(0, bitbake('core-image-minimal').status) + bitbake('core-image-minimal') self.remove_config(config) bb_vars = get_bb_vars(['DEPLOY_DIR_IMAGE', 'MACHINE']) @@ -1117,7 +1117,7 @@ class Wic2(WicTestCase): """Test biosplusefi plugin in qemu""" config = 'IMAGE_FSTYPES = "wic"\nWKS_FILE = "test_biosplusefi_plugin.wks"\nMACHINE_FEATURES:append = " efi"\n' self.append_config(config) - self.assertEqual(0, bitbake('core-image-minimal').status) + bitbake('core-image-minimal') self.remove_config(config) with runqemu('core-image-minimal', ssh=False, @@ -1155,7 +1155,7 @@ class Wic2(WicTestCase): # The easiest way to work-around this issue is to make sure we already built an image here, hence the bitbake call config = 'IMAGE_FSTYPES = "wic"\nWKS_FILE = "test_biosplusefi_plugin.wks"\nMACHINE_FEATURES:append = " efi"\n' self.append_config(config) - self.assertEqual(0, bitbake('core-image-minimal').status) + bitbake('core-image-minimal') self.remove_config(config) img = 'core-image-minimal' @@ -1178,7 +1178,7 @@ class Wic2(WicTestCase): 'WKS_FILE = "test_efi_plugin.wks"\n'\ 'MACHINE_FEATURES:append = " efi"\n' self.append_config(config) - self.assertEqual(0, bitbake('core-image-minimal core-image-minimal-initramfs ovmf').status) + bitbake('core-image-minimal core-image-minimal-initramfs ovmf') self.remove_config(config) with runqemu('core-image-minimal', ssh=False, @@ -1382,7 +1382,7 @@ class Wic2(WicTestCase): # build an image config = 'IMAGE_FSTYPES = "wic"\nWKS_FILE = "directdisk.wks"\n' self.append_config(config) - self.assertEqual(0, bitbake('core-image-minimal').status) + bitbake('core-image-minimal') # get path to the image bb_vars = get_bb_vars(['DEPLOY_DIR_IMAGE', 'MACHINE']) From patchwork Thu Mar 31 18:28:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 6122 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id D4F3EC43219 for ; Thu, 31 Mar 2022 18:29:21 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mx.groups.io with SMTP id smtpd.web09.654.1648751360410980869 for ; Thu, 31 Mar 2022 11:29:20 -0700 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: arm.com, ip: 217.140.110.172, mailfrom: ross.burton@arm.com) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 12A2A13D5 for ; Thu, 31 Mar 2022 11:29:20 -0700 (PDT) Received: from oss-tx204.lab.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id B8B243F718 for ; Thu, 31 Mar 2022 11:29:19 -0700 (PDT) From: Ross Burton To: openembedded-core@lists.openembedded.org Subject: [PATCH 06/23] oeqa/selftest/wic: clean up only_for_arch decorator Date: Thu, 31 Mar 2022 19:28:58 +0100 Message-Id: <20220331182915.22128-6-ross.burton@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220331182915.22128-1-ross.burton@arm.com> References: <20220331182915.22128-1-ross.burton@arm.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 31 Mar 2022 18:29:21 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/163847 There's no need to pass a recipe name when determining the target architecture, there's no need to cap the size of the lru_cache as it will only have one entry, and __name__ is set by @wraps. Signed-off-by: Ross Burton --- meta/lib/oeqa/selftest/cases/wic.py | 14 +++++--------- 1 file changed, 5 insertions(+), 9 deletions(-) diff --git a/meta/lib/oeqa/selftest/cases/wic.py b/meta/lib/oeqa/selftest/cases/wic.py index 5d7f7bf6135..317b80ea27b 100644 --- a/meta/lib/oeqa/selftest/cases/wic.py +++ b/meta/lib/oeqa/selftest/cases/wic.py @@ -22,26 +22,22 @@ from oeqa.selftest.case import OESelftestTestCase from oeqa.utils.commands import runCmd, bitbake, get_bb_var, get_bb_vars, runqemu -@lru_cache(maxsize=32) -def get_host_arch(recipe): - """A cached call to get_bb_var('HOST_ARCH', )""" - return get_bb_var('HOST_ARCH', recipe) +@lru_cache() +def get_host_arch(): + return get_bb_var('HOST_ARCH') -def only_for_arch(archs, image='core-image-minimal'): +def only_for_arch(archs): """Decorator for wrapping test cases that can be run only for specific target architectures. A list of compatible architectures is passed in `archs`. - Current architecture will be determined by parsing bitbake output for - `image` recipe. """ def wrapper(func): @wraps(func) def wrapped_f(*args, **kwargs): - arch = get_host_arch(image) + arch = get_host_arch() if archs and arch not in archs: raise unittest.SkipTest("Testcase arch dependency not met: %s" % arch) return func(*args, **kwargs) - wrapped_f.__name__ = func.__name__ return wrapped_f return wrapper From patchwork Thu Mar 31 18:28:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 6123 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id D048BC433EF for ; Thu, 31 Mar 2022 18:29:21 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mx.groups.io with SMTP id smtpd.web10.680.1648751360894993401 for ; Thu, 31 Mar 2022 11:29:21 -0700 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: arm.com, ip: 217.140.110.172, mailfrom: ross.burton@arm.com) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 98F461596 for ; Thu, 31 Mar 2022 11:29:20 -0700 (PDT) Received: from oss-tx204.lab.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 4AEC73F718 for ; Thu, 31 Mar 2022 11:29:20 -0700 (PDT) From: Ross Burton To: openembedded-core@lists.openembedded.org Subject: [PATCH 07/23] oeqa/selftest/wic: don't hardcode kernel image type in test_wic_rm Date: Thu, 31 Mar 2022 19:28:59 +0100 Message-Id: <20220331182915.22128-7-ross.burton@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220331182915.22128-1-ross.burton@arm.com> References: <20220331182915.22128-1-ross.burton@arm.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 31 Mar 2022 18:29:21 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/163848 Don't assume bzImage, resepct KERNEL_IMAGETYPE. Signed-off-by: Ross Burton --- meta/lib/oeqa/selftest/cases/wic.py | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/meta/lib/oeqa/selftest/cases/wic.py b/meta/lib/oeqa/selftest/cases/wic.py index 317b80ea27b..eb376e1e2c5 100644 --- a/meta/lib/oeqa/selftest/cases/wic.py +++ b/meta/lib/oeqa/selftest/cases/wic.py @@ -1337,21 +1337,23 @@ class Wic2(WicTestCase): self.assertEqual(1, len(images)) sysroot = get_bb_var('RECIPE_SYSROOT_NATIVE', 'wic-tools') + # Not bulletproof but hopefully sufficient + kerneltype = get_bb_var('KERNEL_IMAGETYPE', 'virtual/kernel') # list directory content of the first partition result = runCmd("wic ls %s:1 -n %s" % (images[0], sysroot)) - self.assertIn('\nBZIMAGE ', result.output) + self.assertIn('\n%s ' % kerneltype.upper(), result.output) self.assertIn('\nEFI ', result.output) - # remove file - runCmd("wic rm %s:1/bzimage -n %s" % (images[0], sysroot)) + # remove file. EFI partitions are case-insensitive so exercise that too + runCmd("wic rm %s:1/%s -n %s" % (images[0], kerneltype.lower(), sysroot)) # remove directory runCmd("wic rm %s:1/efi -n %s" % (images[0], sysroot)) # check if they're removed result = runCmd("wic ls %s:1 -n %s" % (images[0], sysroot)) - self.assertNotIn('\nBZIMAGE ', result.output) + self.assertNotIn('\n%s ' % kerneltype.upper(), result.output) self.assertNotIn('\nEFI ', result.output) def test_mkfs_extraopts(self): From patchwork Thu Mar 31 18:29:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 6125 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id D3F28C4167B for ; Thu, 31 Mar 2022 18:29:22 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mx.groups.io with SMTP id smtpd.web11.663.1648751360113174708 for ; Thu, 31 Mar 2022 11:29:21 -0700 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: arm.com, ip: 217.140.110.172, mailfrom: ross.burton@arm.com) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 2B421139F for ; Thu, 31 Mar 2022 11:29:21 -0700 (PDT) Received: from oss-tx204.lab.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id D16843F718 for ; Thu, 31 Mar 2022 11:29:20 -0700 (PDT) From: Ross Burton To: openembedded-core@lists.openembedded.org Subject: [PATCH 08/23] oeqa/selftest/wic: add more arch-specific annotations Date: Thu, 31 Mar 2022 19:29:00 +0100 Message-Id: <20220331182915.22128-8-ross.burton@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220331182915.22128-1-ross.burton@arm.com> References: <20220331182915.22128-1-ross.burton@arm.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 31 Mar 2022 18:29:22 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/163849 Some tests which are marked as x86-specific will actually work on aarch64 (e.g. use EFI), whilst some other tests really are x86-specific (e.g. use syslinux). Signed-off-by: Ross Burton --- meta/lib/oeqa/selftest/cases/wic.py | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/meta/lib/oeqa/selftest/cases/wic.py b/meta/lib/oeqa/selftest/cases/wic.py index eb376e1e2c5..325fb7776a8 100644 --- a/meta/lib/oeqa/selftest/cases/wic.py +++ b/meta/lib/oeqa/selftest/cases/wic.py @@ -203,7 +203,7 @@ class Wic(WicTestCase): runCmd(cmd) self.assertEqual(1, len(glob(self.resultdir + "qemux86-directdisk-*direct"))) - @only_for_arch(['i586', 'i686', 'x86_64']) + @only_for_arch(['i586', 'i686', 'x86_64', 'aarch64']) def test_mkefidisk(self): """Test creation of mkefidisk image""" cmd = "wic create mkefidisk -e core-image-minimal -o %s" % self.resultdir @@ -221,7 +221,7 @@ class Wic(WicTestCase): runCmd(cmd) self.assertEqual(1, len(glob(self.resultdir + "directdisk-bootloader-config-*direct"))) - @only_for_arch(['i586', 'i686', 'x86_64']) + @only_for_arch(['i586', 'i686', 'x86_64', 'aarch64']) def test_systemd_bootdisk(self): """Test creation of systemd-bootdisk image""" config = 'MACHINE_FEATURES:append = " efi"\n' @@ -251,6 +251,7 @@ class Wic(WicTestCase): runCmd(cmd) self.assertEqual(1, len(glob(self.resultdir + "sdimage-bootpart-*direct"))) + # TODO this doesn't have to be x86-specific @only_for_arch(['i586', 'i686', 'x86_64']) def test_default_output_dir(self): """Test default output location""" @@ -355,6 +356,7 @@ class Wic(WicTestCase): "--outdir %s" % self.resultdir) self.assertEqual(1, len(glob(self.resultdir + "wictestdisk-*.direct"))) + # TODO this doesn't have to be x86-specific @only_for_arch(['i586', 'i686', 'x86_64']) def test_rootfs_indirect_recipes(self): """Test usage of rootfs plugin with rootfs recipes""" @@ -365,6 +367,7 @@ class Wic(WicTestCase): "--outdir %s" % self.resultdir) self.assertEqual(1, len(glob(self.resultdir + "directdisk-multi-rootfs*.direct"))) + # TODO this doesn't have to be x86-specific @only_for_arch(['i586', 'i686', 'x86_64']) def test_rootfs_artifacts(self): """Test usage of rootfs plugin with rootfs paths""" @@ -818,7 +821,7 @@ class Wic2(WicTestCase): self.resultdir)) self.assertEqual(1, len(glob(self.resultdir + "wictestdisk-*direct"))) - @only_for_arch(['i586', 'i686', 'x86_64']) + @only_for_arch(['i586', 'i686', 'x86_64', 'aarch64']) def test_wic_image_type(self): """Test building wic images by bitbake""" config = 'IMAGE_FSTYPES += "wic"\nWKS_FILE = "wic-image-minimal"\n'\ @@ -838,6 +841,7 @@ class Wic2(WicTestCase): self.assertTrue(os.path.islink(path)) self.assertTrue(os.path.isfile(os.path.realpath(path))) + # TODO this should work on aarch64 @only_for_arch(['i586', 'i686', 'x86_64']) def test_qemu(self): """Test wic-image-minimal under qemu""" @@ -1041,7 +1045,7 @@ class Wic2(WicTestCase): size = int(size[:-3]) self.assertGreaterEqual(size, 204800) - @only_for_arch(['i586', 'i686', 'x86_64']) + @only_for_arch(['i586', 'i686', 'x86_64', 'aarch64']) def test_rawcopy_plugin_qemu(self): """Test rawcopy plugin in qemu""" # build ext4 and then use it for a wic image @@ -1166,6 +1170,7 @@ class Wic2(WicTestCase): out = glob(self.resultdir + "%s-*.direct" % wksname) self.assertEqual(1, len(out)) + # TODO this test could also work on aarch64 @only_for_arch(['i586', 'i686', 'x86_64']) def test_efi_plugin_unified_kernel_image_qemu(self): """Test efi plugin's Unified Kernel Image feature in qemu""" @@ -1375,6 +1380,7 @@ class Wic2(WicTestCase): out = glob(self.resultdir + "%s-*direct" % wksname) self.assertEqual(1, len(out)) + @only_for_arch(['i586', 'i686', 'x86_64']) def test_expand_mbr_image(self): """Test wic write --expand command for mbr image""" # build an image From patchwork Thu Mar 31 18:29:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 6124 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id D3EEFC4321E for ; Thu, 31 Mar 2022 18:29:22 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mx.groups.io with SMTP id smtpd.web09.654.1648751360410980869 for ; Thu, 31 Mar 2022 11:29:22 -0700 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: arm.com, ip: 217.140.110.172, mailfrom: ross.burton@arm.com) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id B1CD213D5 for ; Thu, 31 Mar 2022 11:29:21 -0700 (PDT) Received: from oss-tx204.lab.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 639B33F718 for ; Thu, 31 Mar 2022 11:29:21 -0700 (PDT) From: Ross Burton To: openembedded-core@lists.openembedded.org Subject: [PATCH 09/23] oeqa/selftest/buildoptions: set PACKAGE_CLASSES in test_arch_work_dir_and_export_source Date: Thu, 31 Mar 2022 19:29:01 +0100 Message-Id: <20220331182915.22128-9-ross.burton@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220331182915.22128-1-ross.burton@arm.com> References: <20220331182915.22128-1-ross.burton@arm.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 31 Mar 2022 18:29:22 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/163850 test_arch_work_dir_and_export_source uses the archiver to generate SRPMS, so explicitly set PACKAGE_CLASSES to ensure that package_rpm is used. Signed-off-by: Ross Burton --- meta/lib/oeqa/selftest/cases/buildoptions.py | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/meta/lib/oeqa/selftest/cases/buildoptions.py b/meta/lib/oeqa/selftest/cases/buildoptions.py index bfe613b847a..135e88e1f06 100644 --- a/meta/lib/oeqa/selftest/cases/buildoptions.py +++ b/meta/lib/oeqa/selftest/cases/buildoptions.py @@ -177,7 +177,12 @@ class ArchiverTest(OESelftestTestCase): """ Test for archiving the work directory and exporting the source files. """ - self.write_config("INHERIT += \"archiver\"\nARCHIVER_MODE[src] = \"original\"\nARCHIVER_MODE[srpm] = \"1\"") + self.write_config(""" +INHERIT += "archiver" +PACKAGE_CLASSES = "package_rpm" +ARCHIVER_MODE[src] = "original" +ARCHIVER_MODE[srpm] = "1" +""") res = bitbake("xcursor-transparent-theme", ignore_status=True) self.assertEqual(res.status, 0, "\nCouldn't build xcursortransparenttheme.\nbitbake output %s" % res.output) deploy_dir_src = get_bb_var('DEPLOY_DIR_SRC') From patchwork Thu Mar 31 18:29:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 6126 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id D551BC433FE for ; Thu, 31 Mar 2022 18:29:23 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mx.groups.io with SMTP id smtpd.web08.701.1648751362549868474 for ; Thu, 31 Mar 2022 11:29:22 -0700 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: arm.com, ip: 217.140.110.172, mailfrom: ross.burton@arm.com) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 4402C139F for ; Thu, 31 Mar 2022 11:29:22 -0700 (PDT) Received: from oss-tx204.lab.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id EA2563F718 for ; Thu, 31 Mar 2022 11:29:21 -0700 (PDT) From: Ross Burton To: openembedded-core@lists.openembedded.org Subject: [PATCH 10/23] oeqa/runtime/decorator/package.py: remove use of strToSet Date: Thu, 31 Mar 2022 19:29:02 +0100 Message-Id: <20220331182915.22128-10-ross.burton@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220331182915.22128-1-ross.burton@arm.com> References: <20220331182915.22128-1-ross.burton@arm.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 31 Mar 2022 18:29:23 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/163851 There's no need to use a series of over-generalised functions to just wrap a string in a tuple. Signed-off-by: Ross Burton --- meta/lib/oeqa/runtime/decorator/package.py | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/meta/lib/oeqa/runtime/decorator/package.py b/meta/lib/oeqa/runtime/decorator/package.py index 2d7e174dbfd..8aba3f325bc 100644 --- a/meta/lib/oeqa/runtime/decorator/package.py +++ b/meta/lib/oeqa/runtime/decorator/package.py @@ -5,7 +5,6 @@ # from oeqa.core.decorator import OETestDecorator, registerDecorator -from oeqa.core.utils.misc import strToSet @registerDecorator class OEHasPackage(OETestDecorator): @@ -34,8 +33,12 @@ class OEHasPackage(OETestDecorator): def setUpDecorator(self): need_pkgs = set() unneed_pkgs = set() - pkgs = strToSet(self.need_pkgs) - for pkg in pkgs: + + # Turn literal strings into a list so we can just iterate over it + if isinstance(self.need_pkgs, str): + self.need_pkgs = [self.need_pkgs,] + + for pkg in self.need_pkgs: if pkg.startswith('!'): unneed_pkgs.add(pkg[1:]) else: From patchwork Thu Mar 31 18:29:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 6127 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id D0C3AC4332F for ; Thu, 31 Mar 2022 18:29:23 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mx.groups.io with SMTP id smtpd.web12.702.1648751363099344739 for ; Thu, 31 Mar 2022 11:29:23 -0700 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: arm.com, ip: 217.140.110.172, mailfrom: ross.burton@arm.com) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id CA75913D5 for ; Thu, 31 Mar 2022 11:29:22 -0700 (PDT) Received: from oss-tx204.lab.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 7C7793F718 for ; Thu, 31 Mar 2022 11:29:22 -0700 (PDT) From: Ross Burton To: openembedded-core@lists.openembedded.org Subject: [PATCH 11/23] oeqa/core/decorator: remove redundant code Date: Thu, 31 Mar 2022 19:29:03 +0100 Message-Id: <20220331182915.22128-11-ross.burton@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220331182915.22128-1-ross.burton@arm.com> References: <20220331182915.22128-1-ross.burton@arm.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 31 Mar 2022 18:29:23 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/163852 There's no need to wrap *tags in a potential list, as *tags will always be a tuple. Signed-off-by: Ross Burton --- meta/lib/oeqa/core/decorator/__init__.py | 11 +++-------- 1 file changed, 3 insertions(+), 8 deletions(-) diff --git a/meta/lib/oeqa/core/decorator/__init__.py b/meta/lib/oeqa/core/decorator/__init__.py index 1a82518ab6a..93efd30e1db 100644 --- a/meta/lib/oeqa/core/decorator/__init__.py +++ b/meta/lib/oeqa/core/decorator/__init__.py @@ -5,8 +5,7 @@ # from functools import wraps -from abc import abstractmethod, ABCMeta -from oeqa.core.utils.misc import strToList +from abc import ABCMeta decoratorClasses = set() @@ -65,15 +64,11 @@ class OETestDiscover(OETestDecorator): return registry['cases'] def OETestTag(*tags): - expandedtags = [] - for tag in tags: - expandedtags += strToList(tag) def decorator(item): if hasattr(item, "__oeqa_testtags"): # do not append, create a new list (to handle classes with inheritance) - item.__oeqa_testtags = list(item.__oeqa_testtags) + expandedtags + item.__oeqa_testtags = list(item.__oeqa_testtags) + list(tags) else: - item.__oeqa_testtags = expandedtags + item.__oeqa_testtags = tags return item return decorator - From patchwork Thu Mar 31 18:29:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 6128 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id D54F7C43217 for ; Thu, 31 Mar 2022 18:29:24 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mx.groups.io with SMTP id smtpd.web09.654.1648751360410980869 for ; Thu, 31 Mar 2022 11:29:23 -0700 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: arm.com, ip: 217.140.110.172, mailfrom: ross.burton@arm.com) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 5CD7D139F for ; Thu, 31 Mar 2022 11:29:23 -0700 (PDT) Received: from oss-tx204.lab.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 0ECDF3F718 for ; Thu, 31 Mar 2022 11:29:22 -0700 (PDT) From: Ross Burton To: openembedded-core@lists.openembedded.org Subject: [PATCH 12/23] testimage: inline updateTestData() Date: Thu, 31 Mar 2022 19:29:04 +0100 Message-Id: <20220331182915.22128-12-ross.burton@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220331182915.22128-1-ross.burton@arm.com> References: <20220331182915.22128-1-ross.burton@arm.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 31 Mar 2022 18:29:24 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/163853 updateTestData() is just a simple loop that is only used here, so just inline it. Signed-off-by: Ross Burton --- meta/classes/testimage.bbclass | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass index 898248992c8..8ffaeab2844 100644 --- a/meta/classes/testimage.bbclass +++ b/meta/classes/testimage.bbclass @@ -205,7 +205,6 @@ def testimage_main(d): import shutil from bb.utils import export_proxies - from oeqa.core.utils.misc import updateTestData from oeqa.runtime.context import OERuntimeTestContext from oeqa.runtime.context import OERuntimeTestContextExecutor from oeqa.core.target.qemu import supported_fstypes @@ -245,7 +244,8 @@ def testimage_main(d): # Some variables need to be updates (mostly paths) with the # ones of the current environment because some tests require them. - updateTestData(d, td, d.getVar('TESTIMAGE_UPDATE_VARS').split()) + for var in d.getVar('TESTIMAGE_UPDATE_VARS').split(): + td[var] = d.getVar(var) image_manifest = "%s.manifest" % image_name image_packages = OERuntimeTestContextExecutor.readPackagesManifest(image_manifest) From patchwork Thu Mar 31 18:29:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 6129 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id CF233C433F5 for ; Thu, 31 Mar 2022 18:29:24 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mx.groups.io with SMTP id smtpd.web11.665.1648751364200546164 for ; Thu, 31 Mar 2022 11:29:24 -0700 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: arm.com, ip: 217.140.110.172, mailfrom: ross.burton@arm.com) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E35B1139F for ; Thu, 31 Mar 2022 11:29:23 -0700 (PDT) Received: from oss-tx204.lab.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 954AA3F718 for ; Thu, 31 Mar 2022 11:29:23 -0700 (PDT) From: Ross Burton To: openembedded-core@lists.openembedded.org Subject: [PATCH 13/23] oeqa/core/utils/misc: remove redundant file Date: Thu, 31 Mar 2022 19:29:05 +0100 Message-Id: <20220331182915.22128-13-ross.burton@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220331182915.22128-1-ross.burton@arm.com> References: <20220331182915.22128-1-ross.burton@arm.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 31 Mar 2022 18:29:24 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/163854 This file dates back to 2016. Half of the functions have never been used, the rest are used in one place and have now been replaced. Signed-off-by: Ross Burton --- meta/lib/oeqa/core/utils/misc.py | 47 -------------------------------- 1 file changed, 47 deletions(-) delete mode 100644 meta/lib/oeqa/core/utils/misc.py diff --git a/meta/lib/oeqa/core/utils/misc.py b/meta/lib/oeqa/core/utils/misc.py deleted file mode 100644 index e1a59588eb6..00000000000 --- a/meta/lib/oeqa/core/utils/misc.py +++ /dev/null @@ -1,47 +0,0 @@ -# -# Copyright (C) 2016 Intel Corporation -# -# SPDX-License-Identifier: MIT -# - -def toList(obj, obj_type, obj_name="Object"): - if isinstance(obj, obj_type): - return [obj] - elif isinstance(obj, list): - return obj - else: - raise TypeError("%s must be %s or list" % (obj_name, obj_type)) - -def toSet(obj, obj_type, obj_name="Object"): - if isinstance(obj, obj_type): - return {obj} - elif isinstance(obj, list): - return set(obj) - elif isinstance(obj, set): - return obj - else: - raise TypeError("%s must be %s or set" % (obj_name, obj_type)) - -def strToList(obj, obj_name="Object"): - return toList(obj, str, obj_name) - -def strToSet(obj, obj_name="Object"): - return toSet(obj, str, obj_name) - -def intToList(obj, obj_name="Object"): - return toList(obj, int, obj_name) - -def dataStoteToDict(d, variables): - data = {} - - for v in variables: - data[v] = d.getVar(v) - - return data - -def updateTestData(d, td, variables): - """ - Updates variables with values of data store to test data. - """ - for var in variables: - td[var] = d.getVar(var) From patchwork Thu Mar 31 18:29:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 6131 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id D6603C4321E for ; Thu, 31 Mar 2022 18:29:25 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mx.groups.io with SMTP id smtpd.web09.658.1648751364862149994 for ; Thu, 31 Mar 2022 11:29:25 -0700 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: arm.com, ip: 217.140.110.172, mailfrom: ross.burton@arm.com) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 90E89139F for ; Thu, 31 Mar 2022 11:29:24 -0700 (PDT) Received: from oss-tx204.lab.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 27F913F718 for ; Thu, 31 Mar 2022 11:29:24 -0700 (PDT) From: Ross Burton To: openembedded-core@lists.openembedded.org Subject: [PATCH 14/23] oeqa/selftest: remove unused imports Date: Thu, 31 Mar 2022 19:29:06 +0100 Message-Id: <20220331182915.22128-14-ross.burton@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220331182915.22128-1-ross.burton@arm.com> References: <20220331182915.22128-1-ross.burton@arm.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 31 Mar 2022 18:29:25 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/163855 --- meta/lib/oeqa/selftest/cases/_sstatetests_noauto.py | 2 +- meta/lib/oeqa/selftest/cases/binutils.py | 5 +---- meta/lib/oeqa/selftest/cases/buildoptions.py | 2 +- meta/lib/oeqa/selftest/cases/distrodata.py | 3 --- meta/lib/oeqa/selftest/cases/eSDK.py | 2 +- meta/lib/oeqa/selftest/cases/fitimage.py | 3 +-- meta/lib/oeqa/selftest/cases/gcc.py | 2 +- meta/lib/oeqa/selftest/cases/glibc.py | 2 +- meta/lib/oeqa/selftest/cases/layerappend.py | 2 +- meta/lib/oeqa/selftest/cases/lic_checksum.py | 1 - meta/lib/oeqa/selftest/cases/manifest.py | 2 +- meta/lib/oeqa/selftest/cases/oescripts.py | 2 +- meta/lib/oeqa/selftest/cases/package.py | 1 - meta/lib/oeqa/selftest/cases/recipeutils.py | 6 +----- meta/lib/oeqa/selftest/cases/reproducible.py | 2 -- meta/lib/oeqa/selftest/cases/selftest.py | 1 - meta/lib/oeqa/selftest/cases/sstate.py | 5 +---- meta/lib/oeqa/selftest/cases/sstatetests.py | 3 +-- meta/lib/oeqa/selftest/cases/sysroot.py | 2 +- meta/lib/oeqa/selftest/cases/tinfoil.py | 1 - 20 files changed, 14 insertions(+), 35 deletions(-) diff --git a/meta/lib/oeqa/selftest/cases/_sstatetests_noauto.py b/meta/lib/oeqa/selftest/cases/_sstatetests_noauto.py index 7ac03f0cec6..bff6e7740c5 100644 --- a/meta/lib/oeqa/selftest/cases/_sstatetests_noauto.py +++ b/meta/lib/oeqa/selftest/cases/_sstatetests_noauto.py @@ -6,7 +6,7 @@ import os import shutil import oeqa.utils.ftools as ftools -from oeqa.utils.commands import runCmd, bitbake, get_bb_var, get_test_layer +from oeqa.utils.commands import runCmd, bitbake, get_bb_var from oeqa.selftest.cases.sstate import SStateBase diff --git a/meta/lib/oeqa/selftest/cases/binutils.py b/meta/lib/oeqa/selftest/cases/binutils.py index 821f52f5a88..3b0b44b3908 100644 --- a/meta/lib/oeqa/selftest/cases/binutils.py +++ b/meta/lib/oeqa/selftest/cases/binutils.py @@ -1,12 +1,9 @@ # SPDX-License-Identifier: MIT import os -import sys -import re -import logging from oeqa.core.decorator import OETestTag from oeqa.core.case import OEPTestResultTestCase from oeqa.selftest.case import OESelftestTestCase -from oeqa.utils.commands import bitbake, get_bb_var, get_bb_vars +from oeqa.utils.commands import bitbake, get_bb_vars def parse_values(content): for i in content: diff --git a/meta/lib/oeqa/selftest/cases/buildoptions.py b/meta/lib/oeqa/selftest/cases/buildoptions.py index 135e88e1f06..ad604d6ae26 100644 --- a/meta/lib/oeqa/selftest/cases/buildoptions.py +++ b/meta/lib/oeqa/selftest/cases/buildoptions.py @@ -9,7 +9,7 @@ import shutil import tempfile from oeqa.selftest.case import OESelftestTestCase from oeqa.selftest.cases.buildhistory import BuildhistoryBase -from oeqa.utils.commands import runCmd, bitbake, get_bb_var, get_bb_vars +from oeqa.utils.commands import bitbake, get_bb_var, get_bb_vars import oeqa.utils.ftools as ftools class ImageOptionsTests(OESelftestTestCase): diff --git a/meta/lib/oeqa/selftest/cases/distrodata.py b/meta/lib/oeqa/selftest/cases/distrodata.py index 03f31e9fcbf..b80d091c1c6 100644 --- a/meta/lib/oeqa/selftest/cases/distrodata.py +++ b/meta/lib/oeqa/selftest/cases/distrodata.py @@ -3,9 +3,6 @@ # from oeqa.selftest.case import OESelftestTestCase -from oeqa.utils.commands import runCmd, bitbake, get_bb_var, get_bb_vars -from oeqa.utils.decorators import testcase -from oeqa.utils.ftools import write_file import oe.recipeutils diff --git a/meta/lib/oeqa/selftest/cases/eSDK.py b/meta/lib/oeqa/selftest/cases/eSDK.py index f7279b32300..3ea0f663575 100644 --- a/meta/lib/oeqa/selftest/cases/eSDK.py +++ b/meta/lib/oeqa/selftest/cases/eSDK.py @@ -8,7 +8,7 @@ import os import glob import time from oeqa.selftest.case import OESelftestTestCase -from oeqa.utils.commands import runCmd, bitbake, get_bb_var, get_bb_vars +from oeqa.utils.commands import runCmd, bitbake, get_bb_vars class oeSDKExtSelfTest(OESelftestTestCase): """ diff --git a/meta/lib/oeqa/selftest/cases/fitimage.py b/meta/lib/oeqa/selftest/cases/fitimage.py index f6f6a8e7953..e6bfd1257ec 100644 --- a/meta/lib/oeqa/selftest/cases/fitimage.py +++ b/meta/lib/oeqa/selftest/cases/fitimage.py @@ -3,9 +3,8 @@ # from oeqa.selftest.case import OESelftestTestCase -from oeqa.utils.commands import runCmd, bitbake, get_bb_var, runqemu +from oeqa.utils.commands import runCmd, bitbake, get_bb_var import os -import json import re class FitImageTests(OESelftestTestCase): diff --git a/meta/lib/oeqa/selftest/cases/gcc.py b/meta/lib/oeqa/selftest/cases/gcc.py index 3efe15228f0..9308724ce5e 100644 --- a/meta/lib/oeqa/selftest/cases/gcc.py +++ b/meta/lib/oeqa/selftest/cases/gcc.py @@ -3,7 +3,7 @@ import os from oeqa.core.decorator import OETestTag from oeqa.core.case import OEPTestResultTestCase from oeqa.selftest.case import OESelftestTestCase -from oeqa.utils.commands import bitbake, get_bb_var, get_bb_vars, runqemu, Command +from oeqa.utils.commands import bitbake, get_bb_var, get_bb_vars, runqemu def parse_values(content): for i in content: diff --git a/meta/lib/oeqa/selftest/cases/glibc.py b/meta/lib/oeqa/selftest/cases/glibc.py index 6f96281ea55..4c149ab7023 100644 --- a/meta/lib/oeqa/selftest/cases/glibc.py +++ b/meta/lib/oeqa/selftest/cases/glibc.py @@ -4,7 +4,7 @@ import contextlib from oeqa.core.decorator import OETestTag from oeqa.core.case import OEPTestResultTestCase from oeqa.selftest.case import OESelftestTestCase -from oeqa.utils.commands import bitbake, get_bb_var, get_bb_vars, runqemu, Command +from oeqa.utils.commands import bitbake, get_bb_var, runqemu from oeqa.utils.nfs import unfs_server def parse_values(content): diff --git a/meta/lib/oeqa/selftest/cases/layerappend.py b/meta/lib/oeqa/selftest/cases/layerappend.py index dadc7c5d28c..8fb1e6c5304 100644 --- a/meta/lib/oeqa/selftest/cases/layerappend.py +++ b/meta/lib/oeqa/selftest/cases/layerappend.py @@ -5,7 +5,7 @@ import os from oeqa.selftest.case import OESelftestTestCase -from oeqa.utils.commands import runCmd, bitbake, get_bb_var +from oeqa.utils.commands import bitbake, get_bb_var import oeqa.utils.ftools as ftools class LayerAppendTests(OESelftestTestCase): diff --git a/meta/lib/oeqa/selftest/cases/lic_checksum.py b/meta/lib/oeqa/selftest/cases/lic_checksum.py index 91021ac3350..f8681ed755b 100644 --- a/meta/lib/oeqa/selftest/cases/lic_checksum.py +++ b/meta/lib/oeqa/selftest/cases/lic_checksum.py @@ -7,7 +7,6 @@ import tempfile from oeqa.selftest.case import OESelftestTestCase from oeqa.utils.commands import bitbake -from oeqa.utils import CommandError class LicenseTests(OESelftestTestCase): diff --git a/meta/lib/oeqa/selftest/cases/manifest.py b/meta/lib/oeqa/selftest/cases/manifest.py index 5d13f354685..0a04c13a856 100644 --- a/meta/lib/oeqa/selftest/cases/manifest.py +++ b/meta/lib/oeqa/selftest/cases/manifest.py @@ -5,7 +5,7 @@ import os from oeqa.selftest.case import OESelftestTestCase -from oeqa.utils.commands import get_bb_var, get_bb_vars, bitbake +from oeqa.utils.commands import get_bb_var, bitbake class ManifestEntry: '''A manifest item of a collection able to list missing packages''' diff --git a/meta/lib/oeqa/selftest/cases/oescripts.py b/meta/lib/oeqa/selftest/cases/oescripts.py index 91abf9654a5..b3261e512f5 100644 --- a/meta/lib/oeqa/selftest/cases/oescripts.py +++ b/meta/lib/oeqa/selftest/cases/oescripts.py @@ -8,7 +8,7 @@ import importlib import unittest from oeqa.selftest.case import OESelftestTestCase from oeqa.selftest.cases.buildhistory import BuildhistoryBase -from oeqa.utils.commands import Command, runCmd, bitbake, get_bb_var, get_test_layer +from oeqa.utils.commands import runCmd, bitbake, get_bb_var from oeqa.utils import CommandError class BuildhistoryDiffTests(BuildhistoryBase): diff --git a/meta/lib/oeqa/selftest/cases/package.py b/meta/lib/oeqa/selftest/cases/package.py index cebbb4f3f49..51d835259e4 100644 --- a/meta/lib/oeqa/selftest/cases/package.py +++ b/meta/lib/oeqa/selftest/cases/package.py @@ -4,7 +4,6 @@ from oeqa.selftest.case import OESelftestTestCase from oeqa.utils.commands import bitbake, get_bb_vars, get_bb_var, runqemu -import stat import subprocess, os import oe.path import re diff --git a/meta/lib/oeqa/selftest/cases/recipeutils.py b/meta/lib/oeqa/selftest/cases/recipeutils.py index f1dd63f65b4..74b2098ae8b 100644 --- a/meta/lib/oeqa/selftest/cases/recipeutils.py +++ b/meta/lib/oeqa/selftest/cases/recipeutils.py @@ -2,14 +2,10 @@ # SPDX-License-Identifier: MIT # -import os -import re -import time -import logging import bb.tinfoil from oeqa.selftest.case import OESelftestTestCase -from oeqa.utils.commands import runCmd, get_test_layer +from oeqa.utils.commands import get_test_layer def setUpModule(): diff --git a/meta/lib/oeqa/selftest/cases/reproducible.py b/meta/lib/oeqa/selftest/cases/reproducible.py index 7caf8c3e7d5..5042c11d8eb 100644 --- a/meta/lib/oeqa/selftest/cases/reproducible.py +++ b/meta/lib/oeqa/selftest/cases/reproducible.py @@ -9,8 +9,6 @@ import bb.utils import functools import multiprocessing import textwrap -import json -import unittest import tempfile import shutil import stat diff --git a/meta/lib/oeqa/selftest/cases/selftest.py b/meta/lib/oeqa/selftest/cases/selftest.py index af080dcf034..7268e259395 100644 --- a/meta/lib/oeqa/selftest/cases/selftest.py +++ b/meta/lib/oeqa/selftest/cases/selftest.py @@ -3,7 +3,6 @@ # import importlib -from oeqa.utils.commands import runCmd import oeqa.selftest from oeqa.selftest.case import OESelftestTestCase diff --git a/meta/lib/oeqa/selftest/cases/sstate.py b/meta/lib/oeqa/selftest/cases/sstate.py index 80ce9e353c0..176766331ad 100644 --- a/meta/lib/oeqa/selftest/cases/sstate.py +++ b/meta/lib/oeqa/selftest/cases/sstate.py @@ -3,14 +3,11 @@ # import datetime -import unittest import os import re -import shutil -import oeqa.utils.ftools as ftools from oeqa.selftest.case import OESelftestTestCase -from oeqa.utils.commands import runCmd, bitbake, get_bb_vars, get_test_layer +from oeqa.utils.commands import get_bb_vars class SStateBase(OESelftestTestCase): diff --git a/meta/lib/oeqa/selftest/cases/sstatetests.py b/meta/lib/oeqa/selftest/cases/sstatetests.py index 3038b400215..4a32af902fb 100644 --- a/meta/lib/oeqa/selftest/cases/sstatetests.py +++ b/meta/lib/oeqa/selftest/cases/sstatetests.py @@ -8,8 +8,7 @@ import glob import subprocess import tempfile -from oeqa.selftest.case import OESelftestTestCase -from oeqa.utils.commands import runCmd, bitbake, get_bb_var, get_test_layer, create_temp_layer +from oeqa.utils.commands import runCmd, bitbake, get_bb_var, create_temp_layer from oeqa.selftest.cases.sstate import SStateBase import oe diff --git a/meta/lib/oeqa/selftest/cases/sysroot.py b/meta/lib/oeqa/selftest/cases/sysroot.py index 79ab45235da..315d1a61c2c 100644 --- a/meta/lib/oeqa/selftest/cases/sysroot.py +++ b/meta/lib/oeqa/selftest/cases/sysroot.py @@ -5,7 +5,7 @@ import uuid from oeqa.selftest.case import OESelftestTestCase -from oeqa.utils.commands import bitbake +from oeqa.utils.commands import bitbake class SysrootTests(OESelftestTestCase): def test_sysroot_cleanup(self): diff --git a/meta/lib/oeqa/selftest/cases/tinfoil.py b/meta/lib/oeqa/selftest/cases/tinfoil.py index 6f26af23d54..c81d56d82b0 100644 --- a/meta/lib/oeqa/selftest/cases/tinfoil.py +++ b/meta/lib/oeqa/selftest/cases/tinfoil.py @@ -9,7 +9,6 @@ import logging import bb.tinfoil from oeqa.selftest.case import OESelftestTestCase -from oeqa.utils.commands import runCmd class TinfoilTests(OESelftestTestCase): """ Basic tests for the tinfoil API """ From patchwork Thu Mar 31 18:29:07 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 6130 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id CEFBCC43219 for ; Thu, 31 Mar 2022 18:29:25 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mx.groups.io with SMTP id smtpd.web09.659.1648751365453213846 for ; Thu, 31 Mar 2022 11:29:25 -0700 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: arm.com, ip: 217.140.110.172, mailfrom: ross.burton@arm.com) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 2317F13D5 for ; Thu, 31 Mar 2022 11:29:25 -0700 (PDT) Received: from oss-tx204.lab.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id C94773F718 for ; Thu, 31 Mar 2022 11:29:24 -0700 (PDT) From: Ross Burton To: openembedded-core@lists.openembedded.org Subject: [PATCH 15/23] oeqa/core/decorators/data: improve has_* logic Date: Thu, 31 Mar 2022 19:29:07 +0100 Message-Id: <20220331182915.22128-15-ross.burton@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220331182915.22128-1-ross.burton@arm.com> References: <20220331182915.22128-1-ross.burton@arm.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 31 Mar 2022 18:29:25 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/163856 has_feature() should be splitting the feature string into substrings and then looking for membership instead of looking for simple substrings. has_machine() should be using equality instead of substrings. Signed-off-by: Ross Burton --- meta/lib/oeqa/core/decorator/data.py | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/meta/lib/oeqa/core/decorator/data.py b/meta/lib/oeqa/core/decorator/data.py index bc4939e87c5..12197be246e 100644 --- a/meta/lib/oeqa/core/decorator/data.py +++ b/meta/lib/oeqa/core/decorator/data.py @@ -13,8 +13,8 @@ def has_feature(td, feature): Checks for feature in DISTRO_FEATURES or IMAGE_FEATURES. """ - if (feature in td.get('DISTRO_FEATURES', '') or - feature in td.get('IMAGE_FEATURES', '')): + if (feature in td.get('DISTRO_FEATURES', '').split() or + feature in td.get('IMAGE_FEATURES', '').split()): return True return False @@ -23,7 +23,7 @@ def has_machine(td, machine): Checks for MACHINE. """ - if (machine in td.get('MACHINE', '')): + if (machine == td.get('MACHINE', '')): return True return False From patchwork Thu Mar 31 18:29:08 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 6132 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id D55B3C433EF for ; Thu, 31 Mar 2022 18:29:26 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mx.groups.io with SMTP id smtpd.web08.703.1648751366006022797 for ; Thu, 31 Mar 2022 11:29:26 -0700 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: arm.com, ip: 217.140.110.172, mailfrom: ross.burton@arm.com) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A9BD3139F for ; Thu, 31 Mar 2022 11:29:25 -0700 (PDT) Received: from oss-tx204.lab.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 5BE313F718 for ; Thu, 31 Mar 2022 11:29:25 -0700 (PDT) From: Ross Burton To: openembedded-core@lists.openembedded.org Subject: [PATCH 16/23] oeqa/selftest: tag tests that use runqemu Date: Thu, 31 Mar 2022 19:29:08 +0100 Message-Id: <20220331182915.22128-16-ross.burton@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220331182915.22128-1-ross.burton@arm.com> References: <20220331182915.22128-1-ross.burton@arm.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 31 Mar 2022 18:29:26 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/163857 There may be environments or machines which don't have working runqemu, so tag all of the tests which use runqemu() so that they can be skipped. Signed-off-by: Ross Burton --- meta/lib/oeqa/selftest/cases/devtool.py | 3 ++- meta/lib/oeqa/selftest/cases/gcc.py | 7 +++++++ meta/lib/oeqa/selftest/cases/glibc.py | 1 + meta/lib/oeqa/selftest/cases/imagefeatures.py | 3 +++ meta/lib/oeqa/selftest/cases/overlayfs.py | 6 +++++- meta/lib/oeqa/selftest/cases/runqemu.py | 2 ++ meta/lib/oeqa/selftest/cases/runtime_test.py | 9 +++++---- meta/lib/oeqa/selftest/cases/wic.py | 7 +++++++ 8 files changed, 32 insertions(+), 6 deletions(-) diff --git a/meta/lib/oeqa/selftest/cases/devtool.py b/meta/lib/oeqa/selftest/cases/devtool.py index 9c69585a88c..ba5dca0359e 100644 --- a/meta/lib/oeqa/selftest/cases/devtool.py +++ b/meta/lib/oeqa/selftest/cases/devtool.py @@ -9,10 +9,10 @@ import tempfile import glob import fnmatch -import oeqa.utils.ftools as ftools from oeqa.selftest.case import OESelftestTestCase from oeqa.utils.commands import runCmd, bitbake, get_bb_var, create_temp_layer from oeqa.utils.commands import get_bb_vars, runqemu, get_test_layer +from oeqa.core.decorator import OETestTag oldmetapath = None @@ -1351,6 +1351,7 @@ class DevtoolExtractTests(DevtoolBase): matches2 = glob.glob(stampprefix2 + '*') self.assertFalse(matches2, 'Stamp files exist for recipe %s that should have been cleaned' % testrecipe2) + @OETestTag("runqemu") def test_devtool_deploy_target(self): # NOTE: Whilst this test would seemingly be better placed as a runtime test, # unfortunately the runtime tests run under bitbake and you can't run diff --git a/meta/lib/oeqa/selftest/cases/gcc.py b/meta/lib/oeqa/selftest/cases/gcc.py index 9308724ce5e..b9ea03ae62d 100644 --- a/meta/lib/oeqa/selftest/cases/gcc.py +++ b/meta/lib/oeqa/selftest/cases/gcc.py @@ -114,37 +114,44 @@ class GccLibItmSelfTest(GccSelfTestBase): self.run_check("libitm") @OETestTag("toolchain-system") +@OETestTag("runqemu") class GccCrossSelfTestSystemEmulated(GccSelfTestBase): def test_cross_gcc(self): self.run_check_emulated("gcc") @OETestTag("toolchain-system") +@OETestTag("runqemu") class GxxCrossSelfTestSystemEmulated(GccSelfTestBase): def test_cross_gxx(self): self.run_check_emulated("g++") @OETestTag("toolchain-system") +@OETestTag("runqemu") class GccLibAtomicSelfTestSystemEmulated(GccSelfTestBase): def test_libatomic(self): self.run_check_emulated("libatomic") @OETestTag("toolchain-system") +@OETestTag("runqemu") class GccLibGompSelfTestSystemEmulated(GccSelfTestBase): def test_libgomp(self): self.run_check_emulated("libgomp") @OETestTag("toolchain-system") +@OETestTag("runqemu") class GccLibStdCxxSelfTestSystemEmulated(GccSelfTestBase): def test_libstdcxx(self): self.run_check_emulated("libstdc++-v3") @OETestTag("toolchain-system") +@OETestTag("runqemu") class GccLibSspSelfTestSystemEmulated(GccSelfTestBase): def test_libssp(self): self.check_skip("libssp") self.run_check_emulated("libssp") @OETestTag("toolchain-system") +@OETestTag("runqemu") class GccLibItmSelfTestSystemEmulated(GccSelfTestBase): def test_libitm(self): self.check_skip("libitm") diff --git a/meta/lib/oeqa/selftest/cases/glibc.py b/meta/lib/oeqa/selftest/cases/glibc.py index 4c149ab7023..6fc98e9cb47 100644 --- a/meta/lib/oeqa/selftest/cases/glibc.py +++ b/meta/lib/oeqa/selftest/cases/glibc.py @@ -83,6 +83,7 @@ class GlibcSelfTest(GlibcSelfTestBase): self.run_check() @OETestTag("toolchain-system") +@OETestTag("runqemu") class GlibcSelfTestSystemEmulated(GlibcSelfTestBase): def test_glibc(self): self.run_check_emulated() diff --git a/meta/lib/oeqa/selftest/cases/imagefeatures.py b/meta/lib/oeqa/selftest/cases/imagefeatures.py index d36d45c5516..6b94ace4eba 100644 --- a/meta/lib/oeqa/selftest/cases/imagefeatures.py +++ b/meta/lib/oeqa/selftest/cases/imagefeatures.py @@ -3,6 +3,7 @@ # from oeqa.selftest.case import OESelftestTestCase +from oeqa.core.decorator import OETestTag from oeqa.utils.commands import runCmd, bitbake, get_bb_var, runqemu from oeqa.utils.sshcontrol import SSHControl import glob @@ -14,6 +15,7 @@ class ImageFeatures(OESelftestTestCase): test_user = 'tester' root_user = 'root' + @OETestTag("runqemu") def test_non_root_user_can_connect_via_ssh_without_password(self): """ Summary: Check if non root user can connect via ssh without password @@ -39,6 +41,7 @@ class ImageFeatures(OESelftestTestCase): status, output = ssh.run("true") self.assertEqual(status, 0, 'ssh to user %s failed with %s' % (user, output)) + @OETestTag("runqemu") def test_all_users_can_connect_via_ssh_without_password(self): """ Summary: Check if all users can connect via ssh without password diff --git a/meta/lib/oeqa/selftest/cases/overlayfs.py b/meta/lib/oeqa/selftest/cases/overlayfs.py index 56ae48ce64b..472746a64f4 100644 --- a/meta/lib/oeqa/selftest/cases/overlayfs.py +++ b/meta/lib/oeqa/selftest/cases/overlayfs.py @@ -3,7 +3,8 @@ # from oeqa.selftest.case import OESelftestTestCase -from oeqa.utils.commands import runCmd, bitbake, get_bb_var, runqemu +from oeqa.utils.commands import bitbake, runqemu +from oeqa.core.decorator import OETestTag def getline_qemu(out, line): for l in out.split('\n'): @@ -185,6 +186,7 @@ EOT line = getline_qemu(output, "upperdir=/mnt/overlay/upper/usr/share/another-overlay-mount") self.assertTrue(line and line.startswith("overlay"), msg=output) + @OETestTag("runqemu") def test_correct_image_fstab(self): """ Summary: Check that we can create an image when all parameters are @@ -203,6 +205,7 @@ EOT self._test_correct_image('base-files', base_files_append) + @OETestTag("runqemu") def test_correct_image_unit(self): """ Summary: Check that we can create an image when all parameters are @@ -238,6 +241,7 @@ EOT self._test_correct_image('systemd-machine-units', systemd_machine_unit_append) +@OETestTag("runqemu") class OverlayFSEtcRunTimeTests(OESelftestTestCase): """overlayfs-etc class tests""" diff --git a/meta/lib/oeqa/selftest/cases/runqemu.py b/meta/lib/oeqa/selftest/cases/runqemu.py index da22f77b276..c1d277a095a 100644 --- a/meta/lib/oeqa/selftest/cases/runqemu.py +++ b/meta/lib/oeqa/selftest/cases/runqemu.py @@ -12,6 +12,7 @@ from oeqa.core.decorator import OETestTag from oeqa.selftest.case import OESelftestTestCase from oeqa.utils.commands import bitbake, runqemu, get_bb_var, runCmd +@OETestTag("runqemu") class RunqemuTests(OESelftestTestCase): """Runqemu test class""" @@ -149,6 +150,7 @@ SYSLINUX_TIMEOUT = "10" # bootup various filesystem types, including live image(iso and hddimg) # where live image was not supported on all qemu architecture. @OETestTag("machine") +@OETestTag("runqemu") class QemuTest(OESelftestTestCase): @classmethod diff --git a/meta/lib/oeqa/selftest/cases/runtime_test.py b/meta/lib/oeqa/selftest/cases/runtime_test.py index 642f0eb6375..7b7371b6e07 100644 --- a/meta/lib/oeqa/selftest/cases/runtime_test.py +++ b/meta/lib/oeqa/selftest/cases/runtime_test.py @@ -4,16 +4,15 @@ from oeqa.selftest.case import OESelftestTestCase from oeqa.utils.commands import runCmd, bitbake, get_bb_var, get_bb_vars, runqemu -from oeqa.utils.sshcontrol import SSHControl +from oeqa.core.decorator import OETestTag import os -import re import tempfile -import shutil import oe.lsb from oeqa.core.decorator.data import skipIfNotQemu class TestExport(OESelftestTestCase): + @OETestTag("runqemu") def test_testexport_basic(self): """ Summary: Check basic testexport functionality with only ping test enabled. @@ -106,6 +105,7 @@ class TestExport(OESelftestTestCase): self.assertEqual(0, result.status, "Couldn't run tar from SDK") +@OETestTag("runqemu") class TestImage(OESelftestTestCase): def test_testimage_install(self): @@ -240,6 +240,7 @@ class TestImage(OESelftestTestCase): bitbake('core-image-minimal') bitbake('-c testimage core-image-minimal') +@OETestTag("runqemu") class Postinst(OESelftestTestCase): def init_manager_loop(self, init_manager): @@ -357,6 +358,7 @@ class Postinst(OESelftestTestCase): self.assertFalse(os.path.isfile(os.path.join(hosttestdir, "rootfs-after-failure")), "rootfs-after-failure file was created") +@OETestTag("runqemu") class SystemTap(OESelftestTestCase): """ Summary: The purpose of this test case is to verify native crosstap @@ -433,4 +435,3 @@ IMAGE_INSTALL:append = " systemtap-runtime" cmd = "crosstap -r root@192.168.7.2 -s %s/process/ syscalls_by_pid.stp" % systemtap_examples result = runCmd(cmd) self.assertEqual(0, result.status, 'crosstap syscalls_by_pid returned a non 0 status:%s' % result.output) - diff --git a/meta/lib/oeqa/selftest/cases/wic.py b/meta/lib/oeqa/selftest/cases/wic.py index 325fb7776a8..2eb80ac1941 100644 --- a/meta/lib/oeqa/selftest/cases/wic.py +++ b/meta/lib/oeqa/selftest/cases/wic.py @@ -19,6 +19,7 @@ from functools import wraps, lru_cache from tempfile import NamedTemporaryFile from oeqa.selftest.case import OESelftestTestCase +from oeqa.core.decorator import OETestTag from oeqa.utils.commands import runCmd, bitbake, get_bb_var, get_bb_vars, runqemu @@ -843,6 +844,7 @@ class Wic2(WicTestCase): # TODO this should work on aarch64 @only_for_arch(['i586', 'i686', 'x86_64']) + @OETestTag("runqemu") def test_qemu(self): """Test wic-image-minimal under qemu""" config = 'IMAGE_FSTYPES += "wic"\nWKS_FILE = "wic-image-minimal"\n'\ @@ -863,6 +865,7 @@ class Wic2(WicTestCase): self.assertEqual(output, 'UUID=2c71ef06-a81d-4735-9d3a-379b69c6bdba\t/media\text4\tdefaults\t0\t0') @only_for_arch(['i586', 'i686', 'x86_64']) + @OETestTag("runqemu") def test_qemu_efi(self): """Test core-image-minimal efi image under qemu""" config = 'IMAGE_FSTYPES = "wic"\nWKS_FILE = "mkefidisk.wks"\n' @@ -1046,6 +1049,7 @@ class Wic2(WicTestCase): self.assertGreaterEqual(size, 204800) @only_for_arch(['i586', 'i686', 'x86_64', 'aarch64']) + @OETestTag("runqemu") def test_rawcopy_plugin_qemu(self): """Test rawcopy plugin in qemu""" # build ext4 and then use it for a wic image @@ -1113,6 +1117,7 @@ class Wic2(WicTestCase): self.assertEqual('1', result.output) @only_for_arch(['i586', 'i686', 'x86_64']) + @OETestTag("runqemu") def test_biosplusefi_plugin_qemu(self): """Test biosplusefi plugin in qemu""" config = 'IMAGE_FSTYPES = "wic"\nWKS_FILE = "test_biosplusefi_plugin.wks"\nMACHINE_FEATURES:append = " efi"\n' @@ -1172,6 +1177,7 @@ class Wic2(WicTestCase): # TODO this test could also work on aarch64 @only_for_arch(['i586', 'i686', 'x86_64']) + @OETestTag("runqemu") def test_efi_plugin_unified_kernel_image_qemu(self): """Test efi plugin's Unified Kernel Image feature in qemu""" config = 'IMAGE_FSTYPES = "wic"\n'\ @@ -1381,6 +1387,7 @@ class Wic2(WicTestCase): self.assertEqual(1, len(out)) @only_for_arch(['i586', 'i686', 'x86_64']) + @OETestTag("runqemu") def test_expand_mbr_image(self): """Test wic write --expand command for mbr image""" # build an image From patchwork Thu Mar 31 18:29:09 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 6134 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id D5AB3C4332F for ; Thu, 31 Mar 2022 18:29:27 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mx.groups.io with SMTP id smtpd.web08.705.1648751366521100656 for ; Thu, 31 Mar 2022 11:29:26 -0700 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: arm.com, ip: 217.140.110.172, mailfrom: ross.burton@arm.com) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 3DBEA13D5 for ; Thu, 31 Mar 2022 11:29:26 -0700 (PDT) Received: from oss-tx204.lab.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id E286A3F718 for ; Thu, 31 Mar 2022 11:29:25 -0700 (PDT) From: Ross Burton To: openembedded-core@lists.openembedded.org Subject: [PATCH 17/23] oeqa: rationalise skipifqemu decorators Date: Thu, 31 Mar 2022 19:29:09 +0100 Message-Id: <20220331182915.22128-17-ross.burton@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220331182915.22128-1-ross.burton@arm.com> References: <20220331182915.22128-1-ross.burton@arm.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 31 Mar 2022 18:29:27 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/163858 Signed-off-by: Ross Burton --- meta/lib/oeqa/core/decorator/data.py | 44 ++++--------------- meta/lib/oeqa/runtime/cases/boot.py | 2 +- .../oeqa/runtime/cases/ethernet_ip_connman.py | 3 +- meta/lib/oeqa/runtime/cases/ltp_stress.py | 3 +- meta/lib/oeqa/runtime/cases/storage.py | 16 +++---- meta/lib/oeqa/runtime/cases/suspend.py | 2 +- meta/lib/oeqa/runtime/cases/usb_hid.py | 2 +- meta/lib/oeqa/selftest/cases/runtime_test.py | 4 +- 8 files changed, 25 insertions(+), 51 deletions(-) diff --git a/meta/lib/oeqa/core/decorator/data.py b/meta/lib/oeqa/core/decorator/data.py index 12197be246e..3ce10e54999 100644 --- a/meta/lib/oeqa/core/decorator/data.py +++ b/meta/lib/oeqa/core/decorator/data.py @@ -27,17 +27,6 @@ def has_machine(td, machine): return True return False -def is_qemu(td, qemu): - """ - Checks if MACHINE is qemu. - """ - - machine = td.get('MACHINE', '') - if (qemu in td.get('MACHINE', '') or - machine.startswith('qemu')): - return True - return False - @registerDecorator class skipIfDataVar(OETestDecorator): """ @@ -189,34 +178,19 @@ class skipIfMachine(OETestDecorator): @registerDecorator class skipIfNotQemu(OETestDecorator): """ - Skip test based on MACHINE. - - value must be a qemu MACHINE or it will skip the test - with msg as the reason. + Skip test if MACHINE is not qemu* """ - - attrs = ('value', 'msg') - def setUpDecorator(self): - msg = ('Checking if %s is not this MACHINE' % self.value) - self.logger.debug(msg) - if not is_qemu(self.case.td, self.value): - self.case.skipTest(self.msg) + self.logger.debug("Checking if not qemu MACHINE") + if not self.case.td.get('MACHINE', '').startswith('qemu'): + self.case.skipTest('Test only runs on qemu machines') @registerDecorator class skipIfQemu(OETestDecorator): """ - Skip test based on Qemu Machine. - - value must not be a qemu machine or it will skip the test - with msg as the reason. - """ - - attrs = ('value', 'msg') - + Skip test if MACHINE is qemu* + """ def setUpDecorator(self): - msg = ('Checking if %s is this MACHINE' % self.value) - self.logger.debug(msg) - if is_qemu(self.case.td, self.value): - self.case.skipTest(self.msg) - + self.logger.debug("Checking if qemu MACHINE") + if self.case.td.get('MACHINE', '').startswith('qemu'): + self.case.skipTest('Test only runs on real hardware') diff --git a/meta/lib/oeqa/runtime/cases/boot.py b/meta/lib/oeqa/runtime/cases/boot.py index 2142f400a05..e1ad88a1744 100644 --- a/meta/lib/oeqa/runtime/cases/boot.py +++ b/meta/lib/oeqa/runtime/cases/boot.py @@ -13,7 +13,7 @@ from oeqa.core.decorator.data import skipIfQemu class BootTest(OERuntimeTestCase): @OETimeout(120) - @skipIfQemu('qemuall', 'Test only runs on real hardware') + @skipIfQemu() @OETestDepends(['ssh.SSHTest.test_ssh']) def test_reboot(self): output = '' diff --git a/meta/lib/oeqa/runtime/cases/ethernet_ip_connman.py b/meta/lib/oeqa/runtime/cases/ethernet_ip_connman.py index e0106128389..b93ee299415 100644 --- a/meta/lib/oeqa/runtime/cases/ethernet_ip_connman.py +++ b/meta/lib/oeqa/runtime/cases/ethernet_ip_connman.py @@ -11,7 +11,7 @@ class Ethernet_Test(OERuntimeTestCase): x = '.'.join(x) return x - @skipIfQemu('qemuall', 'Test only runs on real hardware') + @skipIfQemu() @OETestDepends(['ssh.SSHTest.test_ssh']) def test_set_virtual_ip(self): (status, output) = self.target.run("ifconfig eth0 | grep 'inet ' | awk '{print $2}'") @@ -22,6 +22,7 @@ class Ethernet_Test(OERuntimeTestCase): (status, output) = self.target.run("ifconfig eth0:1 %s netmask 255.255.255.0 && sleep 2 && ping -c 5 %s && ifconfig eth0:1 down" % (virtual_ip,virtual_ip)) self.assertEqual(status, 0, msg='Failed to create virtual ip address, output: %s' % output) + @skipIfQemu() @OETestDepends(['ethernet_ip_connman.Ethernet_Test.test_set_virtual_ip']) def test_get_ip_from_dhcp(self): (status, output) = self.target.run("connmanctl services | grep -E '*AO Wired|*AR Wired' | awk '{print $3}'") diff --git a/meta/lib/oeqa/runtime/cases/ltp_stress.py b/meta/lib/oeqa/runtime/cases/ltp_stress.py index 2445ffbc939..ce6f4bf59d3 100644 --- a/meta/lib/oeqa/runtime/cases/ltp_stress.py +++ b/meta/lib/oeqa/runtime/cases/ltp_stress.py @@ -89,8 +89,7 @@ class LtpStressTest(LtpStressBase): # LTP stress runtime tests # - @skipIfQemu('qemuall', 'Test only runs on real hardware') - + @skipIfQemu() @OETestDepends(['ssh.SSHTest.test_ssh']) @OEHasPackage(["ltp"]) def test_ltp_stress(self): diff --git a/meta/lib/oeqa/runtime/cases/storage.py b/meta/lib/oeqa/runtime/cases/storage.py index 166d26b252b..972ef8210c0 100644 --- a/meta/lib/oeqa/runtime/cases/storage.py +++ b/meta/lib/oeqa/runtime/cases/storage.py @@ -91,24 +91,24 @@ class UsbTest(StorageBase): self.test_file = "usb.tst" self.test_dir = os.path.join(self.mount_point, "oeqa") - @skipIfQemu('qemuall', 'Test only runs on real hardware') + @skipIfQemu() @OETestDepends(['ssh.SSHTest.test_ssh']) def test_usb_mount(self): self.storage_umount(2) self.storage_mount(5) - @skipIfQemu('qemuall', 'Test only runs on real hardware') + @skipIfQemu() @OETestDepends(['storage.UsbTest.test_usb_mount']) def test_usb_basic_operations(self): self.storage_basic() - @skipIfQemu('qemuall', 'Test only runs on real hardware') + @skipIfQemu() @OETestDepends(['storage.UsbTest.test_usb_basic_operations']) def test_usb_basic_rw(self): self.storage_write() self.storage_read() - @skipIfQemu('qemuall', 'Test only runs on real hardware') + @skipIfQemu() @OETestDepends(['storage.UsbTest.test_usb_mount']) def test_usb_umount(self): self.storage_umount(2) @@ -126,24 +126,24 @@ class MMCTest(StorageBase): self.test_file = "mmc.tst" self.test_dir = os.path.join(self.mount_point, "oeqa") - @skipIfQemu('qemuall', 'Test only runs on real hardware') + @skipIfQemu() @OETestDepends(['ssh.SSHTest.test_ssh']) def test_mmc_mount(self): self.storage_umount(2) self.storage_mount() - @skipIfQemu('qemuall', 'Test only runs on real hardware') + @skipIfQemu() @OETestDepends(['storage.MMCTest.test_mmc_mount']) def test_mmc_basic_operations(self): self.storage_basic() - @skipIfQemu('qemuall', 'Test only runs on real hardware') + @skipIfQemu() @OETestDepends(['storage.MMCTest.test_mmc_basic_operations']) def test_mmc_basic_rw(self): self.storage_write() self.storage_read() - @skipIfQemu('qemuall', 'Test only runs on real hardware') + @skipIfQemu() @OETestDepends(['storage.MMCTest.test_mmc_mount']) def test_mmc_umount(self): self.storage_umount(2) diff --git a/meta/lib/oeqa/runtime/cases/suspend.py b/meta/lib/oeqa/runtime/cases/suspend.py index 67b6f7e56fa..0382d48f458 100644 --- a/meta/lib/oeqa/runtime/cases/suspend.py +++ b/meta/lib/oeqa/runtime/cases/suspend.py @@ -23,7 +23,7 @@ class Suspend_Test(OERuntimeTestCase): (status, output) = self.target.run('sudo rtcwake -m mem -s 10') self.assertEqual(status, 0, msg = 'Failed to suspends your system to RAM, output : %s' % output) - @skipIfQemu('qemuall', 'Test only runs on real hardware') + @skipIfQemu() @OETestDepends(['ssh.SSHTest.test_ssh']) def test_suspend(self): self.test_date() diff --git a/meta/lib/oeqa/runtime/cases/usb_hid.py b/meta/lib/oeqa/runtime/cases/usb_hid.py index 3c292cf661c..8743174370d 100644 --- a/meta/lib/oeqa/runtime/cases/usb_hid.py +++ b/meta/lib/oeqa/runtime/cases/usb_hid.py @@ -14,7 +14,7 @@ class USB_HID_Test(OERuntimeTestCase): return self.assertEqual(status, 0, msg = 'Failed to suspends your system to RAM, output : %s' % output) @OEHasPackage(['xdotool']) - @skipIfQemu('qemuall', 'Test only runs on real hardware') + @skipIfQemu() @OETestDepends(['ssh.SSHTest.test_ssh']) def test_USB_Hid_input(self): self.keyboard_mouse_simulation() diff --git a/meta/lib/oeqa/selftest/cases/runtime_test.py b/meta/lib/oeqa/selftest/cases/runtime_test.py index 7b7371b6e07..2ad89490fcf 100644 --- a/meta/lib/oeqa/selftest/cases/runtime_test.py +++ b/meta/lib/oeqa/selftest/cases/runtime_test.py @@ -281,7 +281,7 @@ class Postinst(OESelftestTestCase): - @skipIfNotQemu('qemuall', 'Test only runs in qemu') + @skipIfNotQemu() def test_postinst_rootfs_and_boot_sysvinit(self): """ Summary: The purpose of this test case is to verify Post-installation @@ -302,7 +302,7 @@ class Postinst(OESelftestTestCase): self.init_manager_loop("sysvinit") - @skipIfNotQemu('qemuall', 'Test only runs in qemu') + @skipIfNotQemu() def test_postinst_rootfs_and_boot_systemd(self): """ Summary: The purpose of this test case is to verify Post-installation From patchwork Thu Mar 31 18:29:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 6133 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id D16BCC433F5 for ; Thu, 31 Mar 2022 18:29:27 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mx.groups.io with SMTP id smtpd.web08.703.1648751366006022797 for ; Thu, 31 Mar 2022 11:29:27 -0700 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: arm.com, ip: 217.140.110.172, mailfrom: ross.burton@arm.com) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C3B211596 for ; Thu, 31 Mar 2022 11:29:26 -0700 (PDT) Received: from oss-tx204.lab.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 74C923F718 for ; Thu, 31 Mar 2022 11:29:26 -0700 (PDT) From: Ross Burton To: openembedded-core@lists.openembedded.org Subject: [PATCH 18/23] oeqa/selftest/oescripts: refactor skipping logic Date: Thu, 31 Mar 2022 19:29:10 +0100 Message-Id: <20220331182915.22128-18-ross.burton@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220331182915.22128-1-ross.burton@arm.com> References: <20220331182915.22128-1-ross.burton@arm.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 31 Mar 2022 18:29:27 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/163859 OEScriptTests currently skips if cairo isn't present, and does a build of core-image-minimal. This is only required for the OEPybootchartguyTests tests, so move that logic there so that the OEListPackageconfigTests run even if cairo isn't available. This leaves OEScriptTests as a simple class containing the scripts_dir assignment, which can then be reused by other tests to avoid code duplication. Signed-off-by: Ross Burton --- meta/lib/oeqa/selftest/cases/oescripts.py | 14 +++++--------- 1 file changed, 5 insertions(+), 9 deletions(-) diff --git a/meta/lib/oeqa/selftest/cases/oescripts.py b/meta/lib/oeqa/selftest/cases/oescripts.py index b3261e512f5..cd687816c83 100644 --- a/meta/lib/oeqa/selftest/cases/oescripts.py +++ b/meta/lib/oeqa/selftest/cases/oescripts.py @@ -34,21 +34,19 @@ class BuildhistoryDiffTests(BuildhistoryBase): if expected_endlines: self.fail('Missing expected line endings:\n %s' % '\n '.join(expected_endlines)) -@unittest.skipUnless(importlib.util.find_spec("cairo"), "Python cairo module is not present") class OEScriptTests(OESelftestTestCase): + scripts_dir = os.path.join(get_bb_var('COREBASE'), 'scripts') + +@unittest.skipUnless(importlib.util.find_spec("cairo"), "Python cairo module is not present") +class OEPybootchartguyTests(OEScriptTests): @classmethod def setUpClass(cls): super(OEScriptTests, cls).setUpClass() - import cairo bitbake("core-image-minimal -c rootfs -f") cls.tmpdir = get_bb_var('TMPDIR') cls.buildstats = cls.tmpdir + "/buildstats/" + sorted(os.listdir(cls.tmpdir + "/buildstats"))[-1] - scripts_dir = os.path.join(get_bb_var('COREBASE'), 'scripts') - -class OEPybootchartguyTests(OEScriptTests): - def test_pybootchartguy_help(self): runCmd('%s/pybootchartgui/pybootchartgui.py --help' % self.scripts_dir) @@ -65,9 +63,7 @@ class OEPybootchartguyTests(OEScriptTests): self.assertTrue(os.path.exists(self.tmpdir + "/charts.pdf")) -class OEGitproxyTests(OESelftestTestCase): - - scripts_dir = os.path.join(get_bb_var('COREBASE'), 'scripts') +class OEGitproxyTests(OEScriptTests): def test_oegitproxy_help(self): try: From patchwork Thu Mar 31 18:29:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 6135 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id D4627C4321E for ; Thu, 31 Mar 2022 18:29:28 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mx.groups.io with SMTP id smtpd.web12.705.1648751367615235700 for ; Thu, 31 Mar 2022 11:29:27 -0700 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: arm.com, ip: 217.140.110.172, mailfrom: ross.burton@arm.com) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 5506E139F for ; Thu, 31 Mar 2022 11:29:27 -0700 (PDT) Received: from oss-tx204.lab.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 071163F885 for ; Thu, 31 Mar 2022 11:29:26 -0700 (PDT) From: Ross Burton To: openembedded-core@lists.openembedded.org Subject: [PATCH 19/23] oeqa/selftest/wic: cleanup WicTestCase.setUpLocal Date: Thu, 31 Mar 2022 19:29:11 +0100 Message-Id: <20220331182915.22128-19-ross.burton@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220331182915.22128-1-ross.burton@arm.com> References: <20220331182915.22128-1-ross.burton@arm.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 31 Mar 2022 18:29:28 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/163860 Use os.path.join to construct paths, and invoke bitbake once instead of three times. Signed-off-by: Ross Burton --- meta/lib/oeqa/selftest/cases/wic.py | 12 ++++-------- 1 file changed, 4 insertions(+), 8 deletions(-) diff --git a/meta/lib/oeqa/selftest/cases/wic.py b/meta/lib/oeqa/selftest/cases/wic.py index 2eb80ac1941..6da70b8c287 100644 --- a/meta/lib/oeqa/selftest/cases/wic.py +++ b/meta/lib/oeqa/selftest/cases/wic.py @@ -75,22 +75,18 @@ class WicTestCase(OESelftestTestCase): def setUpLocal(self): """This code is executed before each test method.""" - self.resultdir = self.builddir + "/wic-tmp/" + self.resultdir = os.path.join(self.builddir, "wic-tmp") super(WicTestCase, self).setUpLocal() # Do this here instead of in setUpClass as the base setUp does some # clean up which can result in the native tools built earlier in # setUpClass being unavailable. if not WicTestCase.image_is_ready: - if get_bb_var('USE_NLS') == 'yes': - bitbake('wic-tools') - else: - self.skipTest('wic-tools cannot be built due its (intltool|gettext)-native dependency and NLS disable') + if get_bb_var('USE_NLS') != 'yes': + self.skipTest('wic-tools needs USE_NLS=yes') - bitbake('core-image-minimal') - bitbake('core-image-minimal-mtdutils') + bitbake('wic-tools core-image-minimal core-image-minimal-mtdutils') WicTestCase.image_is_ready = True - rmtree(self.resultdir, ignore_errors=True) def tearDownLocal(self): From patchwork Thu Mar 31 18:29:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 6136 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id D45FFC43219 for ; Thu, 31 Mar 2022 18:29:28 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mx.groups.io with SMTP id smtpd.web08.709.1648751368182288285 for ; Thu, 31 Mar 2022 11:29:28 -0700 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: arm.com, ip: 217.140.110.172, mailfrom: ross.burton@arm.com) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id DBC1E13D5 for ; Thu, 31 Mar 2022 11:29:27 -0700 (PDT) Received: from oss-tx204.lab.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 8DC923F718 for ; Thu, 31 Mar 2022 11:29:27 -0700 (PDT) From: Ross Burton To: openembedded-core@lists.openembedded.org Subject: [PATCH 20/23] oeqa/selftest/wic: rearrange tests Date: Thu, 31 Mar 2022 19:29:12 +0100 Message-Id: <20220331182915.22128-20-ross.burton@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220331182915.22128-1-ross.burton@arm.com> References: <20220331182915.22128-1-ross.burton@arm.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 31 Mar 2022 18:29:28 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/163861 Split the tests into further classes: one which exercises the CLI and doesn't need to build images at all, and another which is just the tests that manipulate existing images. Signed-off-by: Ross Burton --- meta/lib/oeqa/selftest/cases/wic.py | 154 ++++++++++++++-------------- 1 file changed, 77 insertions(+), 77 deletions(-) diff --git a/meta/lib/oeqa/selftest/cases/wic.py b/meta/lib/oeqa/selftest/cases/wic.py index 6da70b8c287..c2496486bc2 100644 --- a/meta/lib/oeqa/selftest/cases/wic.py +++ b/meta/lib/oeqa/selftest/cases/wic.py @@ -104,8 +104,7 @@ class WicTestCase(OESelftestTestCase): WicTestCase.wicenv_cache[image] = os.path.join(stdir, machine, 'imgdata') return WicTestCase.wicenv_cache[image] -class Wic(WicTestCase): - +class CLITests(OESelftestTestCase): def test_version(self): """Test wic --version""" runCmd('wic --version') @@ -166,6 +165,7 @@ class Wic(WicTestCase): """Test wic without command""" self.assertEqual(1, runCmd('wic', ignore_status=True).status) +class Wic(WicTestCase): def test_build_image_name(self): """Test wic create wictestdisk --image-name=core-image-minimal""" cmd = "wic create wictestdisk --image-name=core-image-minimal -o %s" % self.resultdir @@ -1271,6 +1271,81 @@ class Wic2(WicTestCase): self.assertEqual(dest_stat.st_blocks, 8) os.unlink(dest) + def test_mkfs_extraopts(self): + """Test wks option --mkfs-extraopts for empty and not empty partitions""" + img = 'core-image-minimal' + with NamedTemporaryFile("w", suffix=".wks") as wks: + wks.writelines( + ['part ext2 --fstype ext2 --source rootfs --mkfs-extraopts "-D -F -i 8192"\n', + "part btrfs --fstype btrfs --source rootfs --size 40M --mkfs-extraopts='--quiet'\n", + 'part squash --fstype squashfs --source rootfs --mkfs-extraopts "-no-sparse -b 4096"\n', + 'part emptyvfat --fstype vfat --size 1M --mkfs-extraopts "-S 1024 -s 64"\n', + 'part emptymsdos --fstype msdos --size 1M --mkfs-extraopts "-S 1024 -s 64"\n', + 'part emptyext2 --fstype ext2 --size 1M --mkfs-extraopts "-D -F -i 8192"\n', + 'part emptybtrfs --fstype btrfs --size 100M --mkfs-extraopts "--mixed -K"\n']) + wks.flush() + cmd = "wic create %s -e %s -o %s" % (wks.name, img, self.resultdir) + runCmd(cmd) + wksname = os.path.splitext(os.path.basename(wks.name))[0] + out = glob(self.resultdir + "%s-*direct" % wksname) + self.assertEqual(1, len(out)) + + @only_for_arch(['i586', 'i686', 'x86_64']) + @OETestTag("runqemu") + def test_expand_mbr_image(self): + """Test wic write --expand command for mbr image""" + # build an image + config = 'IMAGE_FSTYPES = "wic"\nWKS_FILE = "directdisk.wks"\n' + self.append_config(config) + bitbake('core-image-minimal') + + # get path to the image + bb_vars = get_bb_vars(['DEPLOY_DIR_IMAGE', 'MACHINE']) + deploy_dir = bb_vars['DEPLOY_DIR_IMAGE'] + machine = bb_vars['MACHINE'] + image_path = os.path.join(deploy_dir, 'core-image-minimal-%s.wic' % machine) + + self.remove_config(config) + + try: + # expand image to 1G + new_image_path = None + with NamedTemporaryFile(mode='wb', suffix='.wic.exp', + dir=deploy_dir, delete=False) as sparse: + sparse.truncate(1024 ** 3) + new_image_path = sparse.name + + sysroot = get_bb_var('RECIPE_SYSROOT_NATIVE', 'wic-tools') + cmd = "wic write -n %s --expand 1:0 %s %s" % (sysroot, image_path, new_image_path) + runCmd(cmd) + + # check if partitions are expanded + orig = runCmd("wic ls %s -n %s" % (image_path, sysroot)) + exp = runCmd("wic ls %s -n %s" % (new_image_path, sysroot)) + orig_sizes = [int(line.split()[3]) for line in orig.output.split('\n')[1:]] + exp_sizes = [int(line.split()[3]) for line in exp.output.split('\n')[1:]] + self.assertEqual(orig_sizes[0], exp_sizes[0]) # first partition is not resized + self.assertTrue(orig_sizes[1] < exp_sizes[1]) + + # Check if all free space is partitioned + result = runCmd("%s/usr/sbin/sfdisk -F %s" % (sysroot, new_image_path)) + self.assertTrue("0 B, 0 bytes, 0 sectors" in result.output) + + os.rename(image_path, image_path + '.bak') + os.rename(new_image_path, image_path) + + # Check if it boots in qemu + with runqemu('core-image-minimal', ssh=False, runqemuparams='nographic') as qemu: + cmd = "ls /etc/" + status, output = qemu.run_serial('true') + self.assertEqual(1, status, 'Failed to run command "%s": %s' % (cmd, output)) + finally: + if os.path.exists(new_image_path): + os.unlink(new_image_path) + if os.path.exists(image_path + '.bak'): + os.rename(image_path + '.bak', image_path) + +class ModifyTests(WicTestCase): def test_wic_ls(self): """Test listing image content using 'wic ls'""" runCmd("wic create wictestdisk " @@ -1363,81 +1438,6 @@ class Wic2(WicTestCase): self.assertNotIn('\n%s ' % kerneltype.upper(), result.output) self.assertNotIn('\nEFI ', result.output) - def test_mkfs_extraopts(self): - """Test wks option --mkfs-extraopts for empty and not empty partitions""" - img = 'core-image-minimal' - with NamedTemporaryFile("w", suffix=".wks") as wks: - wks.writelines( - ['part ext2 --fstype ext2 --source rootfs --mkfs-extraopts "-D -F -i 8192"\n', - "part btrfs --fstype btrfs --source rootfs --size 40M --mkfs-extraopts='--quiet'\n", - 'part squash --fstype squashfs --source rootfs --mkfs-extraopts "-no-sparse -b 4096"\n', - 'part emptyvfat --fstype vfat --size 1M --mkfs-extraopts "-S 1024 -s 64"\n', - 'part emptymsdos --fstype msdos --size 1M --mkfs-extraopts "-S 1024 -s 64"\n', - 'part emptyext2 --fstype ext2 --size 1M --mkfs-extraopts "-D -F -i 8192"\n', - 'part emptybtrfs --fstype btrfs --size 100M --mkfs-extraopts "--mixed -K"\n']) - wks.flush() - cmd = "wic create %s -e %s -o %s" % (wks.name, img, self.resultdir) - runCmd(cmd) - wksname = os.path.splitext(os.path.basename(wks.name))[0] - out = glob(self.resultdir + "%s-*direct" % wksname) - self.assertEqual(1, len(out)) - - @only_for_arch(['i586', 'i686', 'x86_64']) - @OETestTag("runqemu") - def test_expand_mbr_image(self): - """Test wic write --expand command for mbr image""" - # build an image - config = 'IMAGE_FSTYPES = "wic"\nWKS_FILE = "directdisk.wks"\n' - self.append_config(config) - bitbake('core-image-minimal') - - # get path to the image - bb_vars = get_bb_vars(['DEPLOY_DIR_IMAGE', 'MACHINE']) - deploy_dir = bb_vars['DEPLOY_DIR_IMAGE'] - machine = bb_vars['MACHINE'] - image_path = os.path.join(deploy_dir, 'core-image-minimal-%s.wic' % machine) - - self.remove_config(config) - - try: - # expand image to 1G - new_image_path = None - with NamedTemporaryFile(mode='wb', suffix='.wic.exp', - dir=deploy_dir, delete=False) as sparse: - sparse.truncate(1024 ** 3) - new_image_path = sparse.name - - sysroot = get_bb_var('RECIPE_SYSROOT_NATIVE', 'wic-tools') - cmd = "wic write -n %s --expand 1:0 %s %s" % (sysroot, image_path, new_image_path) - runCmd(cmd) - - # check if partitions are expanded - orig = runCmd("wic ls %s -n %s" % (image_path, sysroot)) - exp = runCmd("wic ls %s -n %s" % (new_image_path, sysroot)) - orig_sizes = [int(line.split()[3]) for line in orig.output.split('\n')[1:]] - exp_sizes = [int(line.split()[3]) for line in exp.output.split('\n')[1:]] - self.assertEqual(orig_sizes[0], exp_sizes[0]) # first partition is not resized - self.assertTrue(orig_sizes[1] < exp_sizes[1]) - - # Check if all free space is partitioned - result = runCmd("%s/usr/sbin/sfdisk -F %s" % (sysroot, new_image_path)) - self.assertTrue("0 B, 0 bytes, 0 sectors" in result.output) - - os.rename(image_path, image_path + '.bak') - os.rename(new_image_path, image_path) - - # Check if it boots in qemu - with runqemu('core-image-minimal', ssh=False, runqemuparams='nographic') as qemu: - cmd = "ls /etc/" - status, output = qemu.run_serial('true') - self.assertEqual(1, status, 'Failed to run command "%s": %s' % (cmd, output)) - finally: - if os.path.exists(new_image_path): - os.unlink(new_image_path) - if os.path.exists(image_path + '.bak'): - os.rename(image_path + '.bak', image_path) - - def test_wic_ls_ext(self): """Test listing content of the ext partition using 'wic ls'""" runCmd("wic create wictestdisk " From patchwork Thu Mar 31 18:29:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 6139 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id EB50AC433FE for ; Thu, 31 Mar 2022 18:29:29 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mx.groups.io with SMTP id smtpd.web09.661.1648751368872686118 for ; Thu, 31 Mar 2022 11:29:29 -0700 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: arm.com, ip: 217.140.110.172, mailfrom: ross.burton@arm.com) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 89729139F for ; Thu, 31 Mar 2022 11:29:28 -0700 (PDT) Received: from oss-tx204.lab.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 20CDB3F718 for ; Thu, 31 Mar 2022 11:29:28 -0700 (PDT) From: Ross Burton To: openembedded-core@lists.openembedded.org Subject: [PATCH 21/23] oeqa/selftest/wic: use os.path.join to join paths Date: Thu, 31 Mar 2022 19:29:13 +0100 Message-Id: <20220331182915.22128-21-ross.burton@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220331182915.22128-1-ross.burton@arm.com> References: <20220331182915.22128-1-ross.burton@arm.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 31 Mar 2022 18:29:29 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/163862 Instead of using string concatenation, use os.path.join. Signed-off-by: Ross Burton --- meta/lib/oeqa/selftest/cases/wic.py | 92 ++++++++++++++--------------- 1 file changed, 46 insertions(+), 46 deletions(-) diff --git a/meta/lib/oeqa/selftest/cases/wic.py b/meta/lib/oeqa/selftest/cases/wic.py index c2496486bc2..5bdd9ae87e3 100644 --- a/meta/lib/oeqa/selftest/cases/wic.py +++ b/meta/lib/oeqa/selftest/cases/wic.py @@ -170,14 +170,14 @@ class Wic(WicTestCase): """Test wic create wictestdisk --image-name=core-image-minimal""" cmd = "wic create wictestdisk --image-name=core-image-minimal -o %s" % self.resultdir runCmd(cmd) - self.assertEqual(1, len(glob(self.resultdir + "wictestdisk-*.direct"))) + self.assertEqual(1, len(glob(os.path.join (self.resultdir, "wictestdisk-*.direct")))) @only_for_arch(['i586', 'i686', 'x86_64']) def test_gpt_image(self): """Test creation of core-image-minimal with gpt table and UUID boot""" cmd = "wic create directdisk-gpt --image-name core-image-minimal -o %s" % self.resultdir runCmd(cmd) - self.assertEqual(1, len(glob(self.resultdir + "directdisk-*.direct"))) + self.assertEqual(1, len(glob(os.path.join(self.resultdir, "directdisk-*.direct")))) @only_for_arch(['i586', 'i686', 'x86_64']) def test_iso_image(self): @@ -190,22 +190,22 @@ class Wic(WicTestCase): self.remove_config(config) cmd = "wic create mkhybridiso --image-name core-image-minimal -o %s" % self.resultdir runCmd(cmd) - self.assertEqual(1, len(glob(self.resultdir + "HYBRID_ISO_IMG-*.direct"))) - self.assertEqual(1, len(glob(self.resultdir + "HYBRID_ISO_IMG-*.iso"))) + self.assertEqual(1, len(glob(os.path.join(self.resultdir, "HYBRID_ISO_IMG-*.direct")))) + self.assertEqual(1, len(glob(os.path.join (self.resultdir, "HYBRID_ISO_IMG-*.iso")))) @only_for_arch(['i586', 'i686', 'x86_64']) def test_qemux86_directdisk(self): """Test creation of qemux-86-directdisk image""" cmd = "wic create qemux86-directdisk -e core-image-minimal -o %s" % self.resultdir runCmd(cmd) - self.assertEqual(1, len(glob(self.resultdir + "qemux86-directdisk-*direct"))) + self.assertEqual(1, len(glob(os.path.join(self.resultdir, "qemux86-directdisk-*direct")))) @only_for_arch(['i586', 'i686', 'x86_64', 'aarch64']) def test_mkefidisk(self): """Test creation of mkefidisk image""" cmd = "wic create mkefidisk -e core-image-minimal -o %s" % self.resultdir runCmd(cmd) - self.assertEqual(1, len(glob(self.resultdir + "mkefidisk-*direct"))) + self.assertEqual(1, len(glob(os.path.join(self.resultdir, "mkefidisk-*direct")))) @only_for_arch(['i586', 'i686', 'x86_64']) def test_bootloader_config(self): @@ -216,7 +216,7 @@ class Wic(WicTestCase): self.remove_config(config) cmd = "wic create directdisk-bootloader-config -e core-image-minimal -o %s" % self.resultdir runCmd(cmd) - self.assertEqual(1, len(glob(self.resultdir + "directdisk-bootloader-config-*direct"))) + self.assertEqual(1, len(glob(os.path.join(self.resultdir, "directdisk-bootloader-config-*direct")))) @only_for_arch(['i586', 'i686', 'x86_64', 'aarch64']) def test_systemd_bootdisk(self): @@ -227,7 +227,7 @@ class Wic(WicTestCase): self.remove_config(config) cmd = "wic create systemd-bootdisk -e core-image-minimal -o %s" % self.resultdir runCmd(cmd) - self.assertEqual(1, len(glob(self.resultdir + "systemd-bootdisk-*direct"))) + self.assertEqual(1, len(glob(os.path.join(self.resultdir, "systemd-bootdisk-*direct")))) def test_efi_bootpart(self): """Test creation of efi-bootpart image""" @@ -236,7 +236,7 @@ class Wic(WicTestCase): self.append_config('IMAGE_EFI_BOOT_FILES = "%s;kernel"\n' % kimgtype) runCmd(cmd) sysroot = get_bb_var('RECIPE_SYSROOT_NATIVE', 'wic-tools') - images = glob(self.resultdir + "mkefidisk-*.direct") + images = glob(os.path.join(self.resultdir, "mkefidisk-*.direct")) result = runCmd("wic ls %s:1/ -n %s" % (images[0], sysroot)) self.assertIn("kernel",result.output) @@ -246,7 +246,7 @@ class Wic(WicTestCase): kimgtype = get_bb_var('KERNEL_IMAGETYPE', 'core-image-minimal') self.write_config('IMAGE_BOOT_FILES = "%s"\n' % kimgtype) runCmd(cmd) - self.assertEqual(1, len(glob(self.resultdir + "sdimage-bootpart-*direct"))) + self.assertEqual(1, len(glob(os.path.join(self.resultdir, "sdimage-bootpart-*direct")))) # TODO this doesn't have to be x86-specific @only_for_arch(['i586', 'i686', 'x86_64']) @@ -277,28 +277,28 @@ class Wic(WicTestCase): "-n %(recipe_sysroot_native)s " "-r %(image_rootfs)s " "-o %(resultdir)s" % bbvars) - self.assertEqual(1, len(glob(self.resultdir + "directdisk-*.direct"))) + self.assertEqual(1, len(glob(os.path.join(self.resultdir, "directdisk-*.direct")))) def test_compress_gzip(self): """Test compressing an image with gzip""" runCmd("wic create wictestdisk " "--image-name core-image-minimal " "-c gzip -o %s" % self.resultdir) - self.assertEqual(1, len(glob(self.resultdir + "wictestdisk-*.direct.gz"))) + self.assertEqual(1, len(glob(os.path.join(self.resultdir, "wictestdisk-*.direct.gz")))) def test_compress_bzip2(self): """Test compressing an image with bzip2""" runCmd("wic create wictestdisk " "--image-name=core-image-minimal " "-c bzip2 -o %s" % self.resultdir) - self.assertEqual(1, len(glob(self.resultdir + "wictestdisk-*.direct.bz2"))) + self.assertEqual(1, len(glob(os.path.join(self.resultdir, "wictestdisk-*.direct.bz2")))) def test_compress_xz(self): """Test compressing an image with xz""" runCmd("wic create wictestdisk " "--image-name=core-image-minimal " "--compress-with=xz -o %s" % self.resultdir) - self.assertEqual(1, len(glob(self.resultdir + "wictestdisk-*.direct.xz"))) + self.assertEqual(1, len(glob(os.path.join(self.resultdir, "wictestdisk-*.direct.xz")))) def test_wrong_compressor(self): """Test how wic breaks if wrong compressor is provided""" @@ -312,23 +312,23 @@ class Wic(WicTestCase): runCmd("wic create wictestdisk " "--image-name=core-image-minimal " "-D -o %s" % self.resultdir) - self.assertEqual(1, len(glob(self.resultdir + "wictestdisk-*.direct"))) - self.assertEqual(1, len(glob(self.resultdir + "tmp.wic*"))) + self.assertEqual(1, len(glob(os.path.join(self.resultdir, "wictestdisk-*.direct")))) + self.assertEqual(1, len(glob(os.path.join(self.resultdir, "tmp.wic*")))) def test_debug_long(self): """Test --debug option""" runCmd("wic create wictestdisk " "--image-name=core-image-minimal " "--debug -o %s" % self.resultdir) - self.assertEqual(1, len(glob(self.resultdir + "wictestdisk-*.direct"))) - self.assertEqual(1, len(glob(self.resultdir + "tmp.wic*"))) + self.assertEqual(1, len(glob(os.path.join(self.resultdir, "wictestdisk-*.direct")))) + self.assertEqual(1, len(glob(os.path.join(self.resultdir, "tmp.wic*")))) def test_skip_build_check_short(self): """Test -s option""" runCmd("wic create wictestdisk " "--image-name=core-image-minimal " "-s -o %s" % self.resultdir) - self.assertEqual(1, len(glob(self.resultdir + "wictestdisk-*.direct"))) + self.assertEqual(1, len(glob(os.path.join(self.resultdir, "wictestdisk-*.direct")))) def test_skip_build_check_long(self): """Test --skip-build-check option""" @@ -336,14 +336,14 @@ class Wic(WicTestCase): "--image-name=core-image-minimal " "--skip-build-check " "--outdir %s" % self.resultdir) - self.assertEqual(1, len(glob(self.resultdir + "wictestdisk-*.direct"))) + self.assertEqual(1, len(glob(os.path.join(self.resultdir, "wictestdisk-*.direct")))) def test_build_rootfs_short(self): """Test -f option""" runCmd("wic create wictestdisk " "--image-name=core-image-minimal " "-f -o %s" % self.resultdir) - self.assertEqual(1, len(glob(self.resultdir + "wictestdisk-*.direct"))) + self.assertEqual(1, len(glob(os.path.join(self.resultdir, "wictestdisk-*.direct")))) def test_build_rootfs_long(self): """Test --build-rootfs option""" @@ -351,7 +351,7 @@ class Wic(WicTestCase): "--image-name=core-image-minimal " "--build-rootfs " "--outdir %s" % self.resultdir) - self.assertEqual(1, len(glob(self.resultdir + "wictestdisk-*.direct"))) + self.assertEqual(1, len(glob(os.path.join(self.resultdir, "wictestdisk-*.direct")))) # TODO this doesn't have to be x86-specific @only_for_arch(['i586', 'i686', 'x86_64']) @@ -362,7 +362,7 @@ class Wic(WicTestCase): "--rootfs rootfs1=core-image-minimal " "--rootfs rootfs2=core-image-minimal " "--outdir %s" % self.resultdir) - self.assertEqual(1, len(glob(self.resultdir + "directdisk-multi-rootfs*.direct"))) + self.assertEqual(1, len(glob(os.path.join(self.resultdir, "directdisk-multi-rootfs*.direct")))) # TODO this doesn't have to be x86-specific @only_for_arch(['i586', 'i686', 'x86_64']) @@ -382,7 +382,7 @@ class Wic(WicTestCase): "--rootfs-dir rootfs1=%(image_rootfs)s " "--rootfs-dir rootfs2=%(image_rootfs)s " "--outdir %(resultdir)s" % bbvars) - self.assertEqual(1, len(glob(self.resultdir + "%(wks)s-*.direct" % bbvars))) + self.assertEqual(1, len(glob(os.path.join(self.resultdir, "%(wks)s-*.direct" % bbvars)))) def test_exclude_path(self): """Test --exclude-path wks option.""" @@ -403,7 +403,7 @@ part /etc --source rootfs --ondisk mmcblk0 --fstype=ext4 --exclude-path bin/ --r % (wks_file, self.resultdir)) os.remove(wks_file) - wicout = glob(self.resultdir + "%s-*direct" % 'temp') + wicout = glob(os.path.join(self.resultdir, "%s-*direct" % 'temp')) self.assertEqual(1, len(wicout)) wicimg = wicout[0] @@ -691,7 +691,7 @@ part /etc --source rootfs --fstype=ext4 --change-directory=etc # Get stock fstab from base-files recipe bitbake('base-files -c do_install') - bf_fstab = os.path.join(get_bb_var('D', 'base-files'), 'etc/fstab') + bf_fstab = os.path.join(get_bb_var('D', 'base-files'), 'etc', 'fstab') self.assertEqual(True, os.path.exists(bf_fstab)) bf_fstab_md5sum = runCmd('md5sum %s 2>/dev/null' % bf_fstab).output.split(" ")[0] @@ -746,7 +746,7 @@ part /etc --source rootfs --fstype=ext4 --change-directory=etc runCmd("wic create wictestdisk " "--image-name core-image-minimal " "--extra-space %i -o %s" % (extraspace ,self.resultdir)) - wicout = glob(self.resultdir + "wictestdisk-*.direct") + wicout = glob(os.path.join(self.resultdir, "wictestdisk-*.direct")) self.assertEqual(1, len(wicout)) size = os.path.getsize(wicout[0]) self.assertTrue(size > extraspace) @@ -757,15 +757,15 @@ class Wic2(WicTestCase): """Test generation of .bmap file -m option""" cmd = "wic create wictestdisk -e core-image-minimal -m -o %s" % self.resultdir runCmd(cmd) - self.assertEqual(1, len(glob(self.resultdir + "wictestdisk-*direct"))) - self.assertEqual(1, len(glob(self.resultdir + "wictestdisk-*direct.bmap"))) + self.assertEqual(1, len(glob(os.path.join(self.resultdir, "wictestdisk-*direct")))) + self.assertEqual(1, len(glob(os.path.join(self.resultdir, "wictestdisk-*direct.bmap")))) def test_bmap_long(self): """Test generation of .bmap file --bmap option""" cmd = "wic create wictestdisk -e core-image-minimal --bmap -o %s" % self.resultdir runCmd(cmd) - self.assertEqual(1, len(glob(self.resultdir + "wictestdisk-*direct"))) - self.assertEqual(1, len(glob(self.resultdir + "wictestdisk-*direct.bmap"))) + self.assertEqual(1, len(glob(os.path.join(self.resultdir, "wictestdisk-*direct")))) + self.assertEqual(1, len(glob(os.path.join(self.resultdir, "wictestdisk-*direct.bmap")))) def test_image_env(self): """Test generation of .env files.""" @@ -801,7 +801,7 @@ class Wic2(WicTestCase): "--image-name=%s -v %s -n %s -o %s" % (image, imgenvdir, native_sysroot, self.resultdir)) - self.assertEqual(1, len(glob(self.resultdir + "wictestdisk-*direct"))) + self.assertEqual(1, len(glob(os.path.join(self.resultdir, "wictestdisk-*direct")))) def test_image_vars_dir_long(self): """Test image vars directory selection --vars option""" @@ -816,7 +816,7 @@ class Wic2(WicTestCase): "--outdir %s" % (image, imgenvdir, native_sysroot, self.resultdir)) - self.assertEqual(1, len(glob(self.resultdir + "wictestdisk-*direct"))) + self.assertEqual(1, len(glob(os.path.join(self.resultdir, "wictestdisk-*direct")))) @only_for_arch(['i586', 'i686', 'x86_64', 'aarch64']) def test_wic_image_type(self): @@ -899,7 +899,7 @@ class Wic2(WicTestCase): wksname = os.path.splitext(os.path.basename(wkspath))[0] - wicout = glob(self.resultdir + "%s-*direct" % wksname) + wicout = glob(os.path.join(self.resultdir, "%s-*direct" % wksname)) if not wicout: return (p, None) @@ -1078,7 +1078,7 @@ class Wic2(WicTestCase): cmd = "wic create %s -e %s -o %s" % (wks.name, img, self.resultdir) runCmd(cmd) wksname = os.path.splitext(os.path.basename(wks.name))[0] - out = glob(self.resultdir + "%s-*direct" % wksname) + out = glob(os.path.join(self.resultdir, "%s-*direct" % wksname)) self.assertEqual(1, len(out)) def test_rawcopy_plugin(self): @@ -1168,7 +1168,7 @@ class Wic2(WicTestCase): cmd = "wic create %s -e %s -o %s" % (wks.name, img, self.resultdir) runCmd(cmd) wksname = os.path.splitext(os.path.basename(wks.name))[0] - out = glob(self.resultdir + "%s-*.direct" % wksname) + out = glob(os.path.join(self.resultdir, "%s-*.direct" % wksname)) self.assertEqual(1, len(out)) # TODO this test could also work on aarch64 @@ -1218,7 +1218,7 @@ class Wic2(WicTestCase): cmd = "wic create %s -e %s -o %s" % (wks.name, img, self.resultdir) runCmd(cmd) wksname = os.path.splitext(os.path.basename(wks.name))[0] - out = glob(self.resultdir + "%s-*direct" % wksname) + out = glob(os.path.join(self.resultdir, "%s-*direct" % wksname)) self.assertEqual(1, len(out)) def test_kickstart_parser(self): @@ -1230,7 +1230,7 @@ class Wic2(WicTestCase): cmd = "wic create %s -e core-image-minimal -o %s" % (wks.name, self.resultdir) runCmd(cmd) wksname = os.path.splitext(os.path.basename(wks.name))[0] - out = glob(self.resultdir + "%s-*direct" % wksname) + out = glob(os.path.join(self.resultdir, "%s-*direct" % wksname)) self.assertEqual(1, len(out)) def test_image_bootpart_globbed(self): @@ -1241,7 +1241,7 @@ class Wic2(WicTestCase): self.append_config(config) runCmd(cmd) self.remove_config(config) - self.assertEqual(1, len(glob(self.resultdir + "sdimage-bootpart-*direct"))) + self.assertEqual(1, len(glob(os.path.join(self.resultdir, "sdimage-bootpart-*direct")))) def test_sparse_copy(self): """Test sparse_copy with FIEMAP and SEEK_HOLE filemap APIs""" @@ -1287,7 +1287,7 @@ class Wic2(WicTestCase): cmd = "wic create %s -e %s -o %s" % (wks.name, img, self.resultdir) runCmd(cmd) wksname = os.path.splitext(os.path.basename(wks.name))[0] - out = glob(self.resultdir + "%s-*direct" % wksname) + out = glob(os.path.join(self.resultdir, "%s-*direct" % wksname)) self.assertEqual(1, len(out)) @only_for_arch(['i586', 'i686', 'x86_64']) @@ -1351,7 +1351,7 @@ class ModifyTests(WicTestCase): runCmd("wic create wictestdisk " "--image-name=core-image-minimal " "-D -o %s" % self.resultdir) - images = glob(self.resultdir + "wictestdisk-*.direct") + images = glob(os.path.join(self.resultdir, "wictestdisk-*.direct")) self.assertEqual(1, len(images)) sysroot = get_bb_var('RECIPE_SYSROOT_NATIVE', 'wic-tools') @@ -1369,7 +1369,7 @@ class ModifyTests(WicTestCase): runCmd("wic create wictestdisk " "--image-name=core-image-minimal " "-D -o %s" % self.resultdir) - images = glob(self.resultdir + "wictestdisk-*.direct") + images = glob(os.path.join(self.resultdir, "wictestdisk-*.direct")) self.assertEqual(1, len(images)) sysroot = get_bb_var('RECIPE_SYSROOT_NATIVE', 'wic-tools') @@ -1415,7 +1415,7 @@ class ModifyTests(WicTestCase): runCmd("wic create mkefidisk " "--image-name=core-image-minimal " "-D -o %s" % self.resultdir) - images = glob(self.resultdir + "mkefidisk-*.direct") + images = glob(os.path.join(self.resultdir, "mkefidisk-*.direct")) self.assertEqual(1, len(images)) sysroot = get_bb_var('RECIPE_SYSROOT_NATIVE', 'wic-tools') @@ -1443,7 +1443,7 @@ class ModifyTests(WicTestCase): runCmd("wic create wictestdisk " "--image-name=core-image-minimal " "-D -o %s" % self.resultdir) - images = glob(self.resultdir + "wictestdisk-*.direct") + images = glob(os.path.join(self.resultdir, "wictestdisk-*.direct")) self.assertEqual(1, len(images)) sysroot = get_bb_var('RECIPE_SYSROOT_NATIVE', 'wic-tools') @@ -1458,7 +1458,7 @@ class ModifyTests(WicTestCase): runCmd("wic create wictestdisk " "--image-name=core-image-minimal " "-D -o %s" % self.resultdir) - images = glob(self.resultdir + "wictestdisk-*.direct") + images = glob(os.path.join(self.resultdir, "wictestdisk-*.direct")) self.assertEqual(1, len(images)) sysroot = get_bb_var('RECIPE_SYSROOT_NATIVE', 'wic-tools') @@ -1494,7 +1494,7 @@ class ModifyTests(WicTestCase): runCmd("wic create mkefidisk " "--image-name=core-image-minimal " "-D -o %s" % self.resultdir) - images = glob(self.resultdir + "mkefidisk-*.direct") + images = glob(os.path.join(self.resultdir, "mkefidisk-*.direct")) self.assertEqual(1, len(images)) sysroot = get_bb_var('RECIPE_SYSROOT_NATIVE', 'wic-tools') From patchwork Thu Mar 31 18:29:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 6137 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id D6662C4332F for ; Thu, 31 Mar 2022 18:29:30 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mx.groups.io with SMTP id smtpd.web12.708.1648751369496433477 for ; Thu, 31 Mar 2022 11:29:29 -0700 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: arm.com, ip: 217.140.110.172, mailfrom: ross.burton@arm.com) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 1BA5B13D5 for ; Thu, 31 Mar 2022 11:29:29 -0700 (PDT) Received: from oss-tx204.lab.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id C1E883F718 for ; Thu, 31 Mar 2022 11:29:28 -0700 (PDT) From: Ross Burton To: openembedded-core@lists.openembedded.org Subject: [PATCH 22/23] oeqa/selftest/wic: use self.td instead of get_bb_var to save on bitbake calls Date: Thu, 31 Mar 2022 19:29:14 +0100 Message-Id: <20220331182915.22128-22-ross.burton@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220331182915.22128-1-ross.burton@arm.com> References: <20220331182915.22128-1-ross.burton@arm.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 31 Mar 2022 18:29:30 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/163863 When a test case starts, self.td is populated with all the variables in the data store. Typically this can be used instead of get_bb_var(), which saves a bitbake call per variable lookup. The only catch is that in parallel runs the build directory is moved after td is populated, so paths in the build directory are wrong: these still need to be fetched in the test. Signed-off-by: Ross Burton --- meta/lib/oeqa/selftest/cases/wic.py | 28 ++++++++++++---------------- 1 file changed, 12 insertions(+), 16 deletions(-) diff --git a/meta/lib/oeqa/selftest/cases/wic.py b/meta/lib/oeqa/selftest/cases/wic.py index 5bdd9ae87e3..de74c07a039 100644 --- a/meta/lib/oeqa/selftest/cases/wic.py +++ b/meta/lib/oeqa/selftest/cases/wic.py @@ -82,7 +82,7 @@ class WicTestCase(OESelftestTestCase): # clean up which can result in the native tools built earlier in # setUpClass being unavailable. if not WicTestCase.image_is_ready: - if get_bb_var('USE_NLS') != 'yes': + if self.td['USE_NLS'] != 'yes': self.skipTest('wic-tools needs USE_NLS=yes') bitbake('wic-tools core-image-minimal core-image-minimal-mtdutils') @@ -98,9 +98,8 @@ class WicTestCase(OESelftestTestCase): """Generate and obtain the path to .env""" if image not in WicTestCase.wicenv_cache: bitbake('%s -c do_rootfs_wicenv' % image) - bb_vars = get_bb_vars(['STAGING_DIR', 'MACHINE'], image) - stdir = bb_vars['STAGING_DIR'] - machine = bb_vars['MACHINE'] + stdir = get_bb_var('STAGING_DIR', image) + machine = self.td["MACHINE"] WicTestCase.wicenv_cache[image] = os.path.join(stdir, machine, 'imgdata') return WicTestCase.wicenv_cache[image] @@ -827,9 +826,8 @@ class Wic2(WicTestCase): bitbake('wic-image-minimal') self.remove_config(config) - bb_vars = get_bb_vars(['DEPLOY_DIR_IMAGE', 'MACHINE']) - deploy_dir = bb_vars['DEPLOY_DIR_IMAGE'] - machine = bb_vars['MACHINE'] + deploy_dir = get_bb_var('DEPLOY_DIR_IMAGE') + machine = self.td['MACHINE'] prefix = os.path.join(deploy_dir, 'wic-image-minimal-%s.' % machine) # check if we have result image and manifests symlinks # pointing to existing files @@ -1069,7 +1067,7 @@ class Wic2(WicTestCase): def _rawcopy_plugin(self, fstype): """Test rawcopy plugin""" img = 'core-image-minimal' - machine = get_bb_var('MACHINE', img) + machine = self.td["MACHINE"] params = ',unpack' if fstype.endswith('.gz') else '' with NamedTemporaryFile("w", suffix=".wks") as wks: wks.write('part / --source rawcopy --sourceparams="file=%s-%s.%s%s"\n'\ @@ -1098,12 +1096,11 @@ class Wic2(WicTestCase): self.append_config(config) bitbake('core-image-minimal') self.remove_config(config) + deploy_dir = get_bb_var('DEPLOY_DIR_IMAGE') + machine = self.td['MACHINE'] - bb_vars = get_bb_vars(['DEPLOY_DIR_IMAGE', 'MACHINE']) - deploy_dir = bb_vars['DEPLOY_DIR_IMAGE'] - machine = bb_vars['MACHINE'] image_path = os.path.join(deploy_dir, 'core-image-minimal-%s.wic' % machine) - self.assertEqual(True, os.path.exists(image_path)) + self.assertTrue(os.path.exists(image_path)) sysroot = get_bb_var('RECIPE_SYSROOT_NATIVE', 'wic-tools') @@ -1245,7 +1242,7 @@ class Wic2(WicTestCase): def test_sparse_copy(self): """Test sparse_copy with FIEMAP and SEEK_HOLE filemap APIs""" - libpath = os.path.join(get_bb_var('COREBASE'), 'scripts', 'lib', 'wic') + libpath = os.path.join(self.td['COREBASE'], 'scripts', 'lib', 'wic') sys.path.insert(0, libpath) from filemap import FilemapFiemap, FilemapSeek, sparse_copy, ErrorNotSupp with NamedTemporaryFile("w", suffix=".wic-sparse") as sparse: @@ -1300,9 +1297,8 @@ class Wic2(WicTestCase): bitbake('core-image-minimal') # get path to the image - bb_vars = get_bb_vars(['DEPLOY_DIR_IMAGE', 'MACHINE']) - deploy_dir = bb_vars['DEPLOY_DIR_IMAGE'] - machine = bb_vars['MACHINE'] + deploy_dir = get_bb_var('DEPLOY_DIR_IMAGE') + machine = self.td['MACHINE'] image_path = os.path.join(deploy_dir, 'core-image-minimal-%s.wic' % machine) self.remove_config(config) From patchwork Thu Mar 31 18:29:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 6138 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id D6642C433EF for ; Thu, 31 Mar 2022 18:29:30 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mx.groups.io with SMTP id smtpd.web11.667.1648751370055526314 for ; Thu, 31 Mar 2022 11:29:30 -0700 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: arm.com, ip: 217.140.110.172, mailfrom: ross.burton@arm.com) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A23AA139F for ; Thu, 31 Mar 2022 11:29:29 -0700 (PDT) Received: from oss-tx204.lab.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 542013F718 for ; Thu, 31 Mar 2022 11:29:29 -0700 (PDT) From: Ross Burton To: openembedded-core@lists.openembedded.org Subject: [PATCH 23/23] oeqa/selftest: generalise test_devtool_virtual_kernel_modify Date: Thu, 31 Mar 2022 19:29:15 +0100 Message-Id: <20220331182915.22128-23-ross.burton@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220331182915.22128-1-ross.burton@arm.com> References: <20220331182915.22128-1-ross.burton@arm.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Thu, 31 Mar 2022 18:29:30 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/163864 Generalise this test so that it works on more than qemux86-64: - Don't edit a file in arch/x86 to cause a rebuild, instead use init/ - Look for the edits in the build tree, as the deployed kernel could be of any type (zImage/bzImage/etc) and edits may be in the compressed part. Also remove redundant checks on the result of runCmd(), as this will raise AssertionError exceptions itself so the explicit asserts will never trigger. Signed-off-by: Ross Burton --- meta/lib/oeqa/selftest/cases/devtool.py | 38 +++++++++++-------------- 1 file changed, 17 insertions(+), 21 deletions(-) diff --git a/meta/lib/oeqa/selftest/cases/devtool.py b/meta/lib/oeqa/selftest/cases/devtool.py index ba5dca0359e..3eea2b1a0ea 100644 --- a/meta/lib/oeqa/selftest/cases/devtool.py +++ b/meta/lib/oeqa/selftest/cases/devtool.py @@ -1875,8 +1875,9 @@ class DevtoolUpgradeTests(DevtoolBase): Expected: devtool modify is able to checkout the source of the kernel and modification to the source and configurations are reflected when building the kernel. - """ - kernel_provider = get_bb_var('PREFERRED_PROVIDER_virtual/kernel') + """ + kernel_provider = self.td['PREFERRED_PROVIDER_virtual/kernel'] + # Clean up the environment bitbake('%s -c clean' % kernel_provider) tempdir = tempfile.mkdtemp(prefix='devtoolqa') @@ -1903,33 +1904,28 @@ class DevtoolUpgradeTests(DevtoolBase): self.assertExists(os.path.join(tempdir, 'Makefile'), 'Extracted source could not be found') #Step 4.2 configfile = os.path.join(tempdir,'.config') - diff = runCmd('diff %s %s' % (tmpconfig, configfile)) - self.assertEqual(0,diff.status,'Kernel .config file is not the same using bitbake and devtool') + runCmd('diff %s %s' % (tmpconfig, configfile)) + #Step 4.3 #NOTE: virtual/kernel is mapped to kernel_provider - result = runCmd('devtool build %s' % kernel_provider) - self.assertEqual(0,result.status,'Cannot build kernel using `devtool build`') + runCmd('devtool build %s' % kernel_provider) kernelfile = os.path.join(get_bb_var('KBUILD_OUTPUT', kernel_provider), 'vmlinux') self.assertExists(kernelfile, 'Kernel was not build correctly') #Modify the kernel source - modfile = os.path.join(tempdir,'arch/x86/boot/header.S') - modstring = "Use a boot loader. Devtool testing." - modapplied = runCmd("sed -i 's/Use a boot loader./%s/' %s" % (modstring, modfile)) - self.assertEqual(0,modapplied.status,'Modification to %s on kernel source failed' % modfile) + modfile = os.path.join(tempdir, 'init/version.c') + runCmd("sed -i 's/Linux/LiNuX/g' %s" % (modfile)) + #Modify the configuration - codeconfigfile = os.path.join(tempdir,'.config.new') + codeconfigfile = os.path.join(tempdir, '.config.new') modconfopt = "CONFIG_SG_POOL=n" - modconf = runCmd("sed -i 's/CONFIG_SG_POOL=y/%s/' %s" % (modconfopt, codeconfigfile)) - self.assertEqual(0,modconf.status,'Modification to %s failed' % codeconfigfile) + runCmd("sed -i 's/CONFIG_SG_POOL=y/%s/' %s" % (modconfopt, codeconfigfile)) + #Build again kernel with devtool - rebuild = runCmd('devtool build %s' % kernel_provider) - self.assertEqual(0,rebuild.status,'Fail to build kernel after modification of source and config') + runCmd('devtool build %s' % kernel_provider) + #Step 4.4 - bzimagename = 'bzImage-' + get_bb_var('KERNEL_VERSION_NAME', kernel_provider) - bzimagefile = os.path.join(get_bb_var('D', kernel_provider),'boot', bzimagename) - checkmodcode = runCmd("grep '%s' %s" % (modstring, bzimagefile)) - self.assertEqual(0,checkmodcode.status,'Modification on kernel source failed') + runCmd("grep '%s' %s" % ('LiNuX', kernelfile)) + #Step 4.5 - checkmodconfg = runCmd("grep %s %s" % (modconfopt, codeconfigfile)) - self.assertEqual(0,checkmodconfg.status,'Modification to configuration file failed') + runCmd("grep %s %s" % (modconfopt, codeconfigfile))