From patchwork Fri Jun 16 12:36:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Hoyes X-Patchwork-Id: 25838 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id D5F44EB64D7 for ; Fri, 16 Jun 2023 12:38:22 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mx.groups.io with SMTP id smtpd.web11.7353.1686919101542541866 for ; Fri, 16 Jun 2023 05:38:22 -0700 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: arm.com, ip: 217.140.110.172, mailfrom: peter.hoyes@arm.com) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 2E67E1FB; Fri, 16 Jun 2023 05:39:05 -0700 (PDT) Received: from e125920.arm.com (unknown [10.57.86.49]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 843003F71E; Fri, 16 Jun 2023 05:38:20 -0700 (PDT) From: Peter Hoyes To: bitbake-devel@lists.openembedded.org Cc: Peter Hoyes Subject: [PATCH 1/5] bitbake: tests/fetch: Mark TestTimeout as not a test suite Date: Fri, 16 Jun 2023 13:36:47 +0100 Message-Id: <20230616123651.915234-2-peter.hoyes@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230616123651.915234-1-peter.hoyes@arm.com> References: <20230616123651.915234-1-peter.hoyes@arm.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Fri, 16 Jun 2023 12:38:22 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/14839 From: Peter Hoyes pytest test collection attempts to collect all classes containing "Test". TestTimeout is not a test class so add the __test__ = False attribute to indicate this. Signed-off-by: Peter Hoyes --- bitbake/lib/bb/tests/fetch.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/bitbake/lib/bb/tests/fetch.py b/bitbake/lib/bb/tests/fetch.py index d230120271..ba7be048a8 100644 --- a/bitbake/lib/bb/tests/fetch.py +++ b/bitbake/lib/bb/tests/fetch.py @@ -24,7 +24,8 @@ def skipIfNoNetwork(): return lambda f: f class TestTimeout(Exception): - pass + # Indicate to pytest that this is not a test suite + __test__ = False class Timeout(): From patchwork Fri Jun 16 12:36:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Hoyes X-Patchwork-Id: 25840 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id B24D7EB64D7 for ; Fri, 16 Jun 2023 12:38:32 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mx.groups.io with SMTP id smtpd.web11.7354.1686919103594966248 for ; Fri, 16 Jun 2023 05:38:23 -0700 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: arm.com, ip: 217.140.110.172, mailfrom: peter.hoyes@arm.com) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 37AFE1FB; Fri, 16 Jun 2023 05:39:07 -0700 (PDT) Received: from e125920.arm.com (unknown [10.57.86.49]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 76E823F71E; Fri, 16 Jun 2023 05:38:22 -0700 (PDT) From: Peter Hoyes To: bitbake-devel@lists.openembedded.org Cc: Peter Hoyes Subject: [PATCH 2/5] bitbake: tests/fetch: Rename assertRaisesRegexp to assertRaisesRegex Date: Fri, 16 Jun 2023 13:36:48 +0100 Message-Id: <20230616123651.915234-3-peter.hoyes@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230616123651.915234-1-peter.hoyes@arm.com> References: <20230616123651.915234-1-peter.hoyes@arm.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Fri, 16 Jun 2023 12:38:32 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/14840 From: Peter Hoyes TestCase.assertRaisesRegexp was renamed to assertRaisesRegex in Python 3.2, so rename to fix a warning during test execution. Signed-off-by: Peter Hoyes --- bitbake/lib/bb/tests/fetch.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/bitbake/lib/bb/tests/fetch.py b/bitbake/lib/bb/tests/fetch.py index ba7be048a8..871c174e82 100644 --- a/bitbake/lib/bb/tests/fetch.py +++ b/bitbake/lib/bb/tests/fetch.py @@ -2486,7 +2486,7 @@ class CrateTest(FetcherTest): uris = self.d.getVar('SRC_URI').split() fetcher = bb.fetch2.Fetch(uris, self.d) - with self.assertRaisesRegexp(bb.fetch2.FetchError, "Fetcher failure for URL"): + with self.assertRaisesRegex(bb.fetch2.FetchError, "Fetcher failure for URL"): fetcher.download() class NPMTest(FetcherTest): From patchwork Fri Jun 16 12:36:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Hoyes X-Patchwork-Id: 25842 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id BAA8CC3DA40 for ; Fri, 16 Jun 2023 12:38:32 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mx.groups.io with SMTP id smtpd.web11.7355.1686919104968426625 for ; Fri, 16 Jun 2023 05:38:25 -0700 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: arm.com, ip: 217.140.110.172, mailfrom: peter.hoyes@arm.com) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 9D6821FB; Fri, 16 Jun 2023 05:39:08 -0700 (PDT) Received: from e125920.arm.com (unknown [10.57.86.49]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 043E03F71E; Fri, 16 Jun 2023 05:38:23 -0700 (PDT) From: Peter Hoyes To: bitbake-devel@lists.openembedded.org Cc: Peter Hoyes Subject: [PATCH 3/5] bitbake: tests/fetch: Set git config if not already set Date: Fri, 16 Jun 2023 13:36:49 +0100 Message-Id: <20230616123651.915234-4-peter.hoyes@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230616123651.915234-1-peter.hoyes@arm.com> References: <20230616123651.915234-1-peter.hoyes@arm.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Fri, 16 Jun 2023 12:38:32 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/14841 From: Peter Hoyes git config returns an error code if user.email or user.name are not set, so bb.process.run raises an ExecutionError, failing the test. Improve the logic by catching the ExectionError and using this to set a default value. Change a direct invocation of 'git init' to use self.git_init so that it follows the code path above. Remove the related comment from the README now that git sets up the user details automatically. Signed-off-by: Peter Hoyes --- bitbake/README | 3 +-- bitbake/lib/bb/tests/fetch.py | 12 +++++++++--- 2 files changed, 10 insertions(+), 5 deletions(-) diff --git a/bitbake/README b/bitbake/README index a5f5c1b64f..bdb1e81dc1 100644 --- a/bitbake/README +++ b/bitbake/README @@ -45,8 +45,7 @@ it has so many corner cases. The datastore has many tests too. Testing with the recommended before submitting patches, particularly to the fetcher and datastore. We also appreciate new test cases and may require them for more obscure issues. -To run the tests "zstd" and "git" must be installed. Git must be correctly configured, in -particular the user.email and user.name values must be set. +To run the tests "zstd" and "git" must be installed. The assumption is made that this testsuite is run from an initialized OpenEmbedded build environment (i.e. `source oe-init-build-env` is used). If this is not the case, run the diff --git a/bitbake/lib/bb/tests/fetch.py b/bitbake/lib/bb/tests/fetch.py index 871c174e82..8ca7e6c155 100644 --- a/bitbake/lib/bb/tests/fetch.py +++ b/bitbake/lib/bb/tests/fetch.py @@ -429,9 +429,15 @@ class FetcherTest(unittest.TestCase): # a common setup is to use other default # branch than master. self.git(['checkout', '-b', 'master'], cwd=cwd) - if not self.git(['config', 'user.email'], cwd=cwd): + + try: + self.git(['config', 'user.email'], cwd=cwd) + except bb.process.ExecutionError: self.git(['config', 'user.email', 'you@example.com'], cwd=cwd) - if not self.git(['config', 'user.name'], cwd=cwd): + + try: + self.git(['config', 'user.name'], cwd=cwd) + except bb.process.ExecutionError: self.git(['config', 'user.name', 'Your Name'], cwd=cwd) class MirrorUriTest(FetcherTest): @@ -3038,7 +3044,7 @@ class FetchPremirroronlyLocalTest(FetcherTest): self.mirrorname = "git2_git.fake.repo.bitbake.tar.gz" recipeurl = "git:/git.fake.repo/bitbake" os.makedirs(self.gitdir) - self.git("init", self.gitdir) + self.git_init(cwd=self.gitdir) for i in range(0): self.git_new_commit() bb.process.run('tar -czvf {} .'.format(os.path.join(self.mirrordir, self.mirrorname)), cwd = self.gitdir) From patchwork Fri Jun 16 12:36:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Hoyes X-Patchwork-Id: 25839 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id BA56CEB64D8 for ; Fri, 16 Jun 2023 12:38:32 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mx.groups.io with SMTP id smtpd.web10.7325.1686919106507514709 for ; Fri, 16 Jun 2023 05:38:26 -0700 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: arm.com, ip: 217.140.110.172, mailfrom: peter.hoyes@arm.com) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 2C53B1FB; Fri, 16 Jun 2023 05:39:10 -0700 (PDT) Received: from e125920.arm.com (unknown [10.57.86.49]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 76BAE3F71E; Fri, 16 Jun 2023 05:38:25 -0700 (PDT) From: Peter Hoyes To: bitbake-devel@lists.openembedded.org Cc: Peter Hoyes Subject: [PATCH 4/5] bitbake: tests: Use assertLogs to test logging output Date: Fri, 16 Jun 2023 13:36:50 +0100 Message-Id: <20230616123651.915234-5-peter.hoyes@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230616123651.915234-1-peter.hoyes@arm.com> References: <20230616123651.915234-1-peter.hoyes@arm.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Fri, 16 Jun 2023 12:38:32 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/14842 From: Peter Hoyes By default, pytest captures all stdout and exposes it using its built-in fixtures (capsys, caplog etc), so stdout does not support getvalue(). To support running tests using both unittest and pytest, use assertLogs to capture logging and assert on the log output instead. Signed-off-by: Peter Hoyes --- bitbake/lib/bb/tests/fetch.py | 6 +++--- bitbake/lib/bb/tests/parse.py | 16 +++++++++------- 2 files changed, 12 insertions(+), 10 deletions(-) diff --git a/bitbake/lib/bb/tests/fetch.py b/bitbake/lib/bb/tests/fetch.py index 8ca7e6c155..20593764fd 100644 --- a/bitbake/lib/bb/tests/fetch.py +++ b/bitbake/lib/bb/tests/fetch.py @@ -3183,7 +3183,7 @@ class FetchPremirroronlyBrokenTarball(FetcherTest): import sys self.d.setVar("SRCREV", "0"*40) fetcher = bb.fetch.Fetch([self.recipe_url], self.d) - with self.assertRaises(bb.fetch2.FetchError): + with self.assertRaises(bb.fetch2.FetchError), self.assertLogs() as logs: fetcher.download() - stdout = sys.stdout.getvalue() - self.assertFalse(" not a git repository (or any parent up to mount point /)" in stdout) + output = "".join(logs.output) + self.assertFalse(" not a git repository (or any parent up to mount point /)" in output) diff --git a/bitbake/lib/bb/tests/parse.py b/bitbake/lib/bb/tests/parse.py index a3165d95bd..304bbbe222 100644 --- a/bitbake/lib/bb/tests/parse.py +++ b/bitbake/lib/bb/tests/parse.py @@ -186,14 +186,16 @@ deltask ${EMPTYVAR} """ def test_parse_addtask_deltask(self): import sys - f = self.parsehelper(self.addtask_deltask) - d = bb.parse.handle(f.name, self.d)[''] - stdout = sys.stdout.getvalue() - self.assertTrue("addtask contained multiple 'before' keywords" in stdout) - self.assertTrue("addtask contained multiple 'after' keywords" in stdout) - self.assertTrue('addtask ignored: " do_patch"' in stdout) - #self.assertTrue('dependent task do_foo for do_patch does not exist' in stdout) + with self.assertLogs() as logs: + f = self.parsehelper(self.addtask_deltask) + d = bb.parse.handle(f.name, self.d)[''] + + output = "".join(logs.output) + self.assertTrue("addtask contained multiple 'before' keywords" in output) + self.assertTrue("addtask contained multiple 'after' keywords" in output) + self.assertTrue('addtask ignored: " do_patch"' in output) + #self.assertTrue('dependent task do_foo for do_patch does not exist' in output) broken_multiline_comment = """ # First line of comment \\ From patchwork Fri Jun 16 12:36:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Hoyes X-Patchwork-Id: 25841 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id BE029EB64DB for ; Fri, 16 Jun 2023 12:38:32 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mx.groups.io with SMTP id smtpd.web11.7357.1686919109470562100 for ; Fri, 16 Jun 2023 05:38:29 -0700 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: arm.com, ip: 217.140.110.172, mailfrom: peter.hoyes@arm.com) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 1042A1FB; Fri, 16 Jun 2023 05:39:13 -0700 (PDT) Received: from e125920.arm.com (unknown [10.57.86.49]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 88C123F71E; Fri, 16 Jun 2023 05:38:28 -0700 (PDT) From: Peter Hoyes To: bitbake-devel@lists.openembedded.org Cc: Peter Hoyes Subject: [PATCH 5/5] bitbake: Bootstrap pytest for self-tests Date: Fri, 16 Jun 2023 13:36:51 +0100 Message-Id: <20230616123651.915234-6-peter.hoyes@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230616123651.915234-1-peter.hoyes@arm.com> References: <20230616123651.915234-1-peter.hoyes@arm.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Fri, 16 Jun 2023 12:38:32 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/14843 From: Peter Hoyes To support running the bitbake self-tests using pytest, add a pytest.ini This enables these tests to be run using the following command from the bitbake directory. pytest For now, testpaths only includes the bitbake selftests. Add a note to the README. Signed-off-by: Peter Hoyes --- bitbake/README | 5 +++++ bitbake/lib/bb/tests/conftest.py | 8 ++++++++ bitbake/pytest.ini | 7 +++++++ 3 files changed, 20 insertions(+) create mode 100644 bitbake/lib/bb/tests/conftest.py create mode 100644 bitbake/pytest.ini diff --git a/bitbake/README b/bitbake/README index bdb1e81dc1..78610e65f1 100644 --- a/bitbake/README +++ b/bitbake/README @@ -53,3 +53,8 @@ testsuite as follows: export PATH=$(pwd)/bin:$PATH bin/bitbake-selftest + +The testsuite can alternatively be executed using pytest, e.g. obtained from PyPI (in this +case, the PATH is configured automatically): + + pytest diff --git a/bitbake/lib/bb/tests/conftest.py b/bitbake/lib/bb/tests/conftest.py new file mode 100644 index 0000000000..59a8c6753b --- /dev/null +++ b/bitbake/lib/bb/tests/conftest.py @@ -0,0 +1,8 @@ +import pathlib +import pytest + +BB_BIN_PATH = pathlib.Path(__file__).parents[3].resolve() / 'bin' + +@pytest.fixture(autouse=True) +def bitbake_path(monkeypatch): + monkeypatch.setenv('PATH', str(BB_BIN_PATH), prepend=':') diff --git a/bitbake/pytest.ini b/bitbake/pytest.ini new file mode 100644 index 0000000000..3927e49021 --- /dev/null +++ b/bitbake/pytest.ini @@ -0,0 +1,7 @@ +[pytest] +addopts = --import-mode=importlib +minversion = 7.3 +junit_family = xunit1 +python_files = tests/*.py +pythonpath = lib +testpaths = lib/bb