From patchwork Sat Jan 29 02:29:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Kjellerstedt X-Patchwork-Id: 3084 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id C4C4AC433F5 for ; Sat, 29 Jan 2022 02:29:44 +0000 (UTC) Received: from smtp1.axis.com (smtp1.axis.com [195.60.68.17]) by mx.groups.io with SMTP id smtpd.web09.1520.1643423382741682585 for ; Fri, 28 Jan 2022 18:29:43 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@axis.com header.s=axis-central1 header.b=cWtzl+M9; spf=pass (domain: axis.com, ip: 195.60.68.17, mailfrom: peter.kjellerstedt@axis.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=axis.com; q=dns/txt; s=axis-central1; t=1643423383; x=1674959383; h=from:to:subject:date:message-id:mime-version: content-transfer-encoding; bh=ifav73bE8eM6sRyEyNBdHSpsk5MccARjqUD2JH2VpGQ=; b=cWtzl+M9SMx9jfD/MxgcKogKHxuSuZvp/EqCeKk+yK1OevBD5vGt2IDr u/AL/FgbqmXv8FeCMwqpiBMn5NmNrvlOgLf5uoISqOU232QXD1dbqTZeP wGnbjXQs/f1XqzCSg6MHDQKsWYmRE7HGHSqC7f3pHL4ws8Ptmo5PrusYI bjHhsRTvp8y7h8gi6BNPGNvNql9TUy/cSZI6ewk0ezXYygZH6UCCz2Rh5 iyqKEn2qm0Yq0Jk2ud5LBzCYh33nIv7SAOi6wOWwQXki+wjjSgcDO+izY 54w3xhlNi6SQiwr3XmNUDgL1pSxYi4VvI19l5FdZZoQkxITO967T8dLBH Q==; From: Peter Kjellerstedt To: Subject: [PATCH 1/9] bitbake-user-manual: Remove unnecessary \n from a PREMIRRORS example Date: Sat, 29 Jan 2022 03:29:31 +0100 Message-ID: <20220129022939.19329-1-pkj@axis.com> X-Mailer: git-send-email 2.21.3 MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sat, 29 Jan 2022 02:29:44 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/13294 Since commit 044fb04d (fetch2: Allow whitespace only mirror entries) there is no need to separate the entries in PREMIRRORS with "\n". Signed-off-by: Peter Kjellerstedt --- .../bitbake-user-manual-ref-variables.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/bitbake/doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst b/bitbake/doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst index 1bb55fc501..9d1828c3ba 100644 --- a/bitbake/doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst +++ b/bitbake/doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst @@ -1179,10 +1179,10 @@ overview of their function and contents. your configuration:: PREMIRRORS:prepend = "\ - git://.*/.* http://downloads.yoctoproject.org/mirror/sources/ \n \ - ftp://.*/.* http://downloads.yoctoproject.org/mirror/sources/ \n \ - http://.*/.* http://downloads.yoctoproject.org/mirror/sources/ \n \ - https://.*/.* http://downloads.yoctoproject.org/mirror/sources/ \n" + git://.*/.* http://downloads.yoctoproject.org/mirror/sources/ \ + ftp://.*/.* http://downloads.yoctoproject.org/mirror/sources/ \ + http://.*/.* http://downloads.yoctoproject.org/mirror/sources/ \ + https://.*/.* http://downloads.yoctoproject.org/mirror/sources/" These changes cause the build system to intercept Git, FTP, HTTP, and HTTPS requests and direct them to the ``http://`` sources mirror. You can From patchwork Sat Jan 29 02:29:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Kjellerstedt X-Patchwork-Id: 3088 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8F3FEC4332F for ; Sat, 29 Jan 2022 02:29:45 +0000 (UTC) Received: from smtp1.axis.com (smtp1.axis.com [195.60.68.17]) by mx.groups.io with SMTP id smtpd.web09.1520.1643423382741682585 for ; Fri, 28 Jan 2022 18:29:44 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@axis.com header.s=axis-central1 header.b=Mzln/oVu; spf=pass (domain: axis.com, ip: 195.60.68.17, mailfrom: peter.kjellerstedt@axis.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=axis.com; q=dns/txt; s=axis-central1; t=1643423384; x=1674959384; h=from:to:subject:date:message-id:in-reply-to:references: mime-version:content-transfer-encoding; bh=f6Zr2hWk2Qm3s/eTypHBMWuPbLA15/lmHQkFvdVLdA4=; b=Mzln/oVuIvntmWuqlpR4gCjaFbtTgF8ba13CzTGkzfwxp97yVTEfTzfJ L9dFfozvkrxTimU+4JwvjZMukSLllJIsSL7EbYxGjWhhPrTwn2G995km4 RLiJfDgLmn8VScmPopUVIXfato+zZWrBYr/BoiCHqsUQ5zmGvJwiWZNLs LlsOYJcwBm4EzbAnMMCzmS9PADD7k98+t4YozPqTVDrNtR5f4gPnRvC3B YTBzlZ85y2RCLzrXXA9Gb3E/QdDdaCuxrYXkzx/F0LGpQaMiz5prddcpg 40fbrirQ3g7NY6cG/P60E1z35biylJ7PVpBpe8q5FPJ4OaRSi6Y448gjB Q==; From: Peter Kjellerstedt To: Subject: [PATCH 2/9] tests/fetch: Remove unnecessary \n from mirror variables Date: Sat, 29 Jan 2022 03:29:32 +0100 Message-ID: <20220129022939.19329-2-pkj@axis.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20220129022939.19329-1-pkj@axis.com> References: <20220129022939.19329-1-pkj@axis.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sat, 29 Jan 2022 02:29:45 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/13296 Since commit 044fb04d (fetch2: Allow whitespace only mirror entries) there is no need to separate the entries in mirror variables with "\n". Signed-off-by: Peter Kjellerstedt --- bitbake/lib/bb/tests/fetch.py | 46 +++++++++++++++++------------------ 1 file changed, 23 insertions(+), 23 deletions(-) diff --git a/bitbake/lib/bb/tests/fetch.py b/bitbake/lib/bb/tests/fetch.py index ec7d83c959..e1d79bf964 100644 --- a/bitbake/lib/bb/tests/fetch.py +++ b/bitbake/lib/bb/tests/fetch.py @@ -442,10 +442,10 @@ class MirrorUriTest(FetcherTest): #("file://sstate-xyz.tgz", "file://.*/.*", "file:///somewhere/1234/sstate-cache") : "file:///somewhere/1234/sstate-cache/sstate-xyz.tgz", } - mirrorvar = "http://.*/.* file:///somepath/downloads/ \n" \ - "git://someserver.org/bitbake git://git.openembedded.org/bitbake \n" \ - "https://.*/.* file:///someotherpath/downloads/ \n" \ - "http://.*/.* file:///someotherpath/downloads/ \n" + mirrorvar = "http://.*/.* file:///somepath/downloads/ " \ + "git://someserver.org/bitbake git://git.openembedded.org/bitbake " \ + "https://.*/.* file:///someotherpath/downloads/ " \ + "http://.*/.* file:///someotherpath/downloads/" def test_urireplace(self): for k, v in self.replaceuris.items(): @@ -470,8 +470,8 @@ class MirrorUriTest(FetcherTest): def test_mirror_of_mirror(self): # Test if mirror of a mirror works - mirrorvar = self.mirrorvar + " http://.*/.* http://otherdownloads.yoctoproject.org/downloads/ \n" - mirrorvar = mirrorvar + " http://otherdownloads.yoctoproject.org/.* http://downloads2.yoctoproject.org/downloads/ \n" + mirrorvar = self.mirrorvar + " http://.*/.* http://otherdownloads.yoctoproject.org/downloads/" + mirrorvar = mirrorvar + " http://otherdownloads.yoctoproject.org/.* http://downloads2.yoctoproject.org/downloads/" fetcher = bb.fetch.FetchData("http://downloads.yoctoproject.org/releases/bitbake/bitbake-1.0.tar.gz", self.d) mirrors = bb.fetch2.mirror_from_string(mirrorvar) uris, uds = bb.fetch2.build_mirroruris(fetcher, mirrors, self.d) @@ -480,8 +480,8 @@ class MirrorUriTest(FetcherTest): 'http://otherdownloads.yoctoproject.org/downloads/bitbake-1.0.tar.gz', 'http://downloads2.yoctoproject.org/downloads/bitbake-1.0.tar.gz']) - recmirrorvar = "https://.*/[^/]* http://AAAA/A/A/A/ \n" \ - "https://.*/[^/]* https://BBBB/B/B/B/ \n" + recmirrorvar = "https://.*/[^/]* http://AAAA/A/A/A/ " \ + "https://.*/[^/]* https://BBBB/B/B/B/" def test_recursive(self): fetcher = bb.fetch.FetchData("https://downloads.yoctoproject.org/releases/bitbake/bitbake-1.0.tar.gz", self.d) @@ -503,7 +503,7 @@ class GitDownloadDirectoryNamingTest(FetcherTest): self.d.setVar('SRCREV', '82ea737a0b42a8b53e11c9cde141e9e9c0bd8c40') def setup_mirror_rewrite(self): - self.d.setVar("PREMIRRORS", self.recipe_url + " " + self.mirror_url + " \n") + self.d.setVar("PREMIRRORS", self.recipe_url + " " + self.mirror_url) @skipIfNoNetwork() def test_that_directory_is_named_after_recipe_url_when_no_mirroring_is_used(self): @@ -552,7 +552,7 @@ class TarballNamingTest(FetcherTest): self.d.setVar('SRCREV', '82ea737a0b42a8b53e11c9cde141e9e9c0bd8c40') def setup_mirror_rewrite(self): - self.d.setVar("PREMIRRORS", self.recipe_url + " " + self.mirror_url + " \n") + self.d.setVar("PREMIRRORS", self.recipe_url + " " + self.mirror_url) @skipIfNoNetwork() def test_that_the_recipe_tarball_is_created_when_no_mirroring_is_used(self): @@ -587,7 +587,7 @@ class GitShallowTarballNamingTest(FetcherTest): self.d.setVar('SRCREV', '82ea737a0b42a8b53e11c9cde141e9e9c0bd8c40') def setup_mirror_rewrite(self): - self.d.setVar("PREMIRRORS", self.recipe_url + " " + self.mirror_url + " \n") + self.d.setVar("PREMIRRORS", self.recipe_url + " " + self.mirror_url) @skipIfNoNetwork() def test_that_the_tarball_is_named_after_recipe_url_when_no_mirroring_is_used(self): @@ -867,14 +867,14 @@ class FetcherNetworkTest(FetcherTest): @skipIfNoNetwork() def test_fetch_mirror_of_mirror(self): - self.d.setVar("MIRRORS", "http://.*/.* http://invalid2.yoctoproject.org/ \n http://invalid2.yoctoproject.org/.* https://downloads.yoctoproject.org/releases/bitbake") + self.d.setVar("MIRRORS", "http://.*/.* http://invalid2.yoctoproject.org/ http://invalid2.yoctoproject.org/.* https://downloads.yoctoproject.org/releases/bitbake") fetcher = bb.fetch.Fetch(["http://invalid.yoctoproject.org/releases/bitbake/bitbake-1.0.tar.gz"], self.d) fetcher.download() self.assertEqual(os.path.getsize(self.dldir + "/bitbake-1.0.tar.gz"), 57749) @skipIfNoNetwork() def test_fetch_file_mirror_of_mirror(self): - self.d.setVar("MIRRORS", "http://.*/.* file:///some1where/ \n file:///some1where/.* file://some2where/ \n file://some2where/.* https://downloads.yoctoproject.org/releases/bitbake") + self.d.setVar("MIRRORS", "http://.*/.* file:///some1where/ file:///some1where/.* file://some2where/ file://some2where/.* https://downloads.yoctoproject.org/releases/bitbake") fetcher = bb.fetch.Fetch(["http://invalid.yoctoproject.org/releases/bitbake/bitbake-1.0.tar.gz"], self.d) os.mkdir(self.dldir + "/some2where") fetcher.download() @@ -981,13 +981,13 @@ class FetcherNetworkTest(FetcherTest): def test_gitfetch_finds_local_tarball_for_mirrored_url_when_previous_downloaded_by_the_recipe_url(self): recipeurl = "git://git.openembedded.org/bitbake" mirrorurl = "git://someserver.org/bitbake" - self.d.setVar("PREMIRRORS", "git://someserver.org/bitbake git://git.openembedded.org/bitbake \n") + self.d.setVar("PREMIRRORS", "git://someserver.org/bitbake git://git.openembedded.org/bitbake") self.gitfetcher(recipeurl, mirrorurl) @skipIfNoNetwork() def test_gitfetch_finds_local_tarball_when_previous_downloaded_from_a_premirror(self): recipeurl = "git://someserver.org/bitbake" - self.d.setVar("PREMIRRORS", "git://someserver.org/bitbake git://git.openembedded.org/bitbake \n") + self.d.setVar("PREMIRRORS", "git://someserver.org/bitbake git://git.openembedded.org/bitbake") self.gitfetcher(recipeurl, recipeurl) @skipIfNoNetwork() @@ -997,7 +997,7 @@ class FetcherNetworkTest(FetcherTest): self.sourcedir = self.unpackdir.replace("unpacked", "sourcemirror.git") os.chdir(self.tempdir) bb.process.run("git clone %s %s 2> /dev/null" % (realurl, self.sourcedir), shell=True) - self.d.setVar("PREMIRRORS", "%s git://%s;protocol=file \n" % (recipeurl, self.sourcedir)) + self.d.setVar("PREMIRRORS", "%s git://%s;protocol=file" % (recipeurl, self.sourcedir)) self.gitfetcher(recipeurl, recipeurl) @skipIfNoNetwork() @@ -1852,7 +1852,7 @@ class GitShallowTest(FetcherTest): # Set up the mirror mirrordir = os.path.join(self.tempdir, 'mirror') bb.utils.rename(self.dldir, mirrordir) - self.d.setVar('PREMIRRORS', 'gitsm://.*/.* file://%s/\n' % mirrordir) + self.d.setVar('PREMIRRORS', 'gitsm://.*/.* file://%s/' % mirrordir) # Fetch from the mirror bb.utils.remove(self.dldir, recurse=True) @@ -1967,7 +1967,7 @@ class GitShallowTest(FetcherTest): # Set up the mirror mirrordir = os.path.join(self.tempdir, 'mirror') bb.utils.mkdirhier(mirrordir) - self.d.setVar('PREMIRRORS', 'git://.*/.* file://%s/\n' % mirrordir) + self.d.setVar('PREMIRRORS', 'git://.*/.* file://%s/' % mirrordir) bb.utils.rename(os.path.join(self.dldir, mirrortarball), os.path.join(mirrordir, mirrortarball)) @@ -2349,7 +2349,7 @@ class NPMTest(FetcherTest): mirrordir = os.path.join(self.tempdir, 'mirror') bb.utils.mkdirhier(mirrordir) os.replace(ud.localpath, os.path.join(mirrordir, pkgname)) - self.d.setVar('PREMIRRORS', 'https?$://.*/.* file://%s/\n' % mirrordir) + self.d.setVar('PREMIRRORS', 'https?$://.*/.* file://%s/' % mirrordir) self.d.setVar('BB_FETCH_PREMIRRORONLY', '1') # Fetch again self.assertFalse(os.path.exists(ud.localpath)) @@ -2370,7 +2370,7 @@ class NPMTest(FetcherTest): bb.utils.mkdirhier(mirrordir) mirrorfilename = os.path.join(mirrordir, os.path.basename(ud.localpath)) os.replace(ud.localpath, mirrorfilename) - self.d.setVar('PREMIRRORS', 'https?$://.*/.* file://%s\n' % mirrorfilename) + self.d.setVar('PREMIRRORS', 'https?$://.*/.* file://%s' % mirrorfilename) self.d.setVar('BB_FETCH_PREMIRRORONLY', '1') # Fetch again self.assertFalse(os.path.exists(ud.localpath)) @@ -2390,7 +2390,7 @@ class NPMTest(FetcherTest): mirrordir = os.path.join(self.tempdir, 'mirror') bb.utils.mkdirhier(mirrordir) os.replace(ud.localpath, os.path.join(mirrordir, os.path.basename(ud.localpath))) - self.d.setVar('MIRRORS', 'https?$://.*/.* file://%s/\n' % mirrordir) + self.d.setVar('MIRRORS', 'https?$://.*/.* file://%s/' % mirrordir) # Update the resolved url to an invalid url with open(ud.resolvefile, 'r') as f: url = f.read() @@ -2706,7 +2706,7 @@ class NPMTest(FetcherTest): mirrordir = os.path.join(self.tempdir, 'mirror') bb.utils.mkdirhier(mirrordir) os.replace(ud.localpath, os.path.join(mirrordir, os.path.basename(ud.localpath))) - self.d.setVar('PREMIRRORS', 'https?$://.*/.* file://%s/\n' % mirrordir) + self.d.setVar('PREMIRRORS', 'https?$://.*/.* file://%s/' % mirrordir) self.d.setVar('BB_FETCH_PREMIRRORONLY', '1') # Fetch again self.assertFalse(os.path.exists(ud.localpath)) @@ -2735,7 +2735,7 @@ class NPMTest(FetcherTest): mirrordir = os.path.join(self.tempdir, 'mirror') bb.utils.mkdirhier(mirrordir) os.replace(ud.localpath, os.path.join(mirrordir, os.path.basename(ud.localpath))) - self.d.setVar('MIRRORS', 'https?$://.*/.* file://%s/\n' % mirrordir) + self.d.setVar('MIRRORS', 'https?$://.*/.* file://%s/' % mirrordir) # Fetch again with invalid url self.assertFalse(os.path.exists(ud.localpath)) swfile = self.create_shrinkwrap_file({ From patchwork Sat Jan 29 02:29:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Kjellerstedt X-Patchwork-Id: 3083 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id C5808C433FE for ; Sat, 29 Jan 2022 02:29:44 +0000 (UTC) Received: from smtp2.axis.com (smtp2.axis.com [195.60.68.18]) by mx.groups.io with SMTP id smtpd.web11.1501.1643423382962218181 for ; Fri, 28 Jan 2022 18:29:43 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@axis.com header.s=axis-central1 header.b=EgXR1rza; spf=pass (domain: axis.com, ip: 195.60.68.18, mailfrom: peter.kjellerstedt@axis.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=axis.com; q=dns/txt; s=axis-central1; t=1643423383; x=1674959383; h=from:to:subject:date:message-id:in-reply-to:references: mime-version:content-transfer-encoding; bh=lRDcb+2+5/hx8BqoxxfXEj3PXCvUFuPfhcfqxsm8bQw=; b=EgXR1rza8nQLrV2vUlU7UbyYXK+c9c1/wTlRD7oS3/jN0kE9uVk/UHKf YiLvTWU3DavxauO3hnnr8C9b1r+GpHH5P5OJKEjrccYbOklCkPgCpttuN QpHVq3yFxKiwz7lDvGAmBcz3SqHaE0JHn7K9/Jcw5mmotm1LrTIVgWw0y o4oS2HzVKHBMe7q3wsRj9GnqqrW90ytEGgmDGSAwuuGDRS3eNUpOIgruH N3VqmRCJqFH3LvIzaNP9iG8+6whB7qaTWVuZCHcK8/r3EeJZWoCdqUhnj 94+FV0Afm6tBBwMCbMF8jJPO52TQ6whGo59UQdpyrMAfGuv6qDsM37i7W g==; From: Peter Kjellerstedt To: Subject: [PATCH 3/9] tests/fetch: Skip the crate tests if network tests are disabled Date: Sat, 29 Jan 2022 03:29:33 +0100 Message-ID: <20220129022939.19329-3-pkj@axis.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20220129022939.19329-1-pkj@axis.com> References: <20220129022939.19329-1-pkj@axis.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sat, 29 Jan 2022 02:29:44 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/13295 Both test_crate_url and test_crate_url_multi require Internet access. Signed-off-by: Peter Kjellerstedt --- bitbake/lib/bb/tests/fetch.py | 2 ++ 1 file changed, 2 insertions(+) diff --git a/bitbake/lib/bb/tests/fetch.py b/bitbake/lib/bb/tests/fetch.py index e1d79bf964..c654662c02 100644 --- a/bitbake/lib/bb/tests/fetch.py +++ b/bitbake/lib/bb/tests/fetch.py @@ -2261,6 +2261,7 @@ class GitURLWithSpacesTest(FetcherTest): self.assertEqual(ud.fullmirror, os.path.join(self.dldir, "git2_" + ref['gitsrcname'] + '.tar.gz')) class CrateTest(FetcherTest): + @skipIfNoNetwork() def test_crate_url(self): uri = "crate://crates.io/glob/0.2.11" @@ -2277,6 +2278,7 @@ class CrateTest(FetcherTest): self.assertTrue(os.path.exists(self.tempdir + "/cargo_home/bitbake/glob-0.2.11/.cargo-checksum.json")) self.assertTrue(os.path.exists(self.tempdir + "/cargo_home/bitbake/glob-0.2.11/src/lib.rs")) + @skipIfNoNetwork() def test_crate_url_multi(self): uri = "crate://crates.io/glob/0.2.11 crate://crates.io/time/0.1.35" From patchwork Sat Jan 29 02:29:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Kjellerstedt X-Patchwork-Id: 3086 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8FA90C43217 for ; Sat, 29 Jan 2022 02:29:45 +0000 (UTC) Received: from smtp2.axis.com (smtp2.axis.com [195.60.68.18]) by mx.groups.io with SMTP id smtpd.web11.1501.1643423382962218181 for ; Fri, 28 Jan 2022 18:29:44 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@axis.com header.s=axis-central1 header.b=SQkIBaFT; spf=pass (domain: axis.com, ip: 195.60.68.18, mailfrom: peter.kjellerstedt@axis.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=axis.com; q=dns/txt; s=axis-central1; t=1643423384; x=1674959384; h=from:to:subject:date:message-id:in-reply-to:references: mime-version:content-transfer-encoding; bh=VyjO7n14+956zHjTUI0lo0DNdKDZTBAjN+z1x4yuNBs=; b=SQkIBaFTKClOPIgjl2dstvTzZsfkHO5dA432w9B1Xy5WHloAcy12qt/9 VBEJ6A3LePTgmdjvtwl0qDUaAl0sY19ddjrH+6qESv3qIxoSZiN3yvV+Q vuwqFEWf8N5TBuYemjwl9piWP9mZJlGNCUmxqvQZhad6okWB0LD4/g0kW 3AhP/2bVdrWXVLUAk0Oe37eJFnMAZvYc6n95Ars5t1TWc/G2FnL7LrlFI Ugm8wPKWMOpzx6S/HxCo8sDWXJjwN606+AoHbeJufs63nAPFZxfX2Qp+Y hENHrJCbCvcXU6Thk+3CZqtNBy05NUJSykTL77SKFrnKVd73Q0rsEEpgq A==; From: Peter Kjellerstedt To: Subject: [PATCH 4/9] tests/fetch: Improve the verbose messages for skipped tests Date: Sat, 29 Jan 2022 03:29:34 +0100 Message-ID: <20220129022939.19329-4-pkj@axis.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20220129022939.19329-1-pkj@axis.com> References: <20220129022939.19329-1-pkj@axis.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sat, 29 Jan 2022 02:29:45 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/13297 This changes: test_foo .. skipped 'Network tests being skipped' test_foo .. skipped 'npm not installed, tests being skipped' to: test_foo .. skipped 'network test' test_foo .. skipped 'npm not installed' to avoid double "skipped" in the output. Signed-off-by: Peter Kjellerstedt --- bitbake/lib/bb/tests/fetch.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/bitbake/lib/bb/tests/fetch.py b/bitbake/lib/bb/tests/fetch.py index c654662c02..5cccdf6ef4 100644 --- a/bitbake/lib/bb/tests/fetch.py +++ b/bitbake/lib/bb/tests/fetch.py @@ -18,7 +18,7 @@ from bb.tests.support.httpserver import HTTPService def skipIfNoNetwork(): if os.environ.get("BB_SKIP_NETTESTS") == "yes": - return unittest.skip("Network tests being skipped") + return unittest.skip("network test") return lambda f: f class URITest(unittest.TestCase): @@ -2301,7 +2301,7 @@ class NPMTest(FetcherTest): def skipIfNoNpm(): import shutil if not shutil.which('npm'): - return unittest.skip('npm not installed, tests being skipped') + return unittest.skip('npm not installed') return lambda f: f @skipIfNoNpm() From patchwork Sat Jan 29 02:29:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Kjellerstedt X-Patchwork-Id: 3090 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 95BDAC4167E for ; Sat, 29 Jan 2022 02:29:46 +0000 (UTC) Received: from smtp1.axis.com (smtp1.axis.com [195.60.68.17]) by mx.groups.io with SMTP id smtpd.web09.1520.1643423382741682585 for ; Fri, 28 Jan 2022 18:29:45 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@axis.com header.s=axis-central1 header.b=e7Us/qLC; spf=pass (domain: axis.com, ip: 195.60.68.17, mailfrom: peter.kjellerstedt@axis.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=axis.com; q=dns/txt; s=axis-central1; t=1643423385; x=1674959385; h=from:to:subject:date:message-id:in-reply-to:references: mime-version:content-transfer-encoding; bh=RthV/CXPjdf6ciBHxVla9AOATvzEB+Fs8JxB4C83DBQ=; b=e7Us/qLCrcY8i81dAVsOHsQ1qWPO3hL91qMEea0ccdkSHxQmI1WvvgZQ LTYt8J5KZEEsEWROyMdzXnkvoO6Qv9rYlDBymkSeOhZwyz3LALMXLlOXD hwGgA8OlW3pBkS6EikcuoX0oWo/oMZoBTLgOWmhtBkXjpo9Ud9t+bfY7A eLYZMudJHqC5Z4YabKj2cXAzWsp0x8yzHPGLkdZ+HBeRhgjwJIwwL9gVI K8FuFaOhONtfZZqN63ZnFMiT/blcf5wsD9NAh9XePTkGCjQJ1o7ysexKi 1XFt9r15IhRE3cpOLgMwNprfwEDpPu6R6yJIXDy6mTcZDLiepZ+pSXpTa Q==; From: Peter Kjellerstedt To: Subject: [PATCH 5/9] tests/fetch: Unify how git commands are run Date: Sat, 29 Jan 2022 03:29:35 +0100 Message-ID: <20220129022939.19329-5-pkj@axis.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20220129022939.19329-1-pkj@axis.com> References: <20220129022939.19329-1-pkj@axis.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sat, 29 Jan 2022 02:29:46 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/13298 This unifies the three separate implementations of git() from GitMakeShallowTest, GitShallowTest and GitLfsTest into one implementation in FetcherTest. It also makes use of this in FetcherLocalTest and FetcherNetworkTest. Signed-off-by: Peter Kjellerstedt --- bitbake/lib/bb/tests/fetch.py | 73 +++++++++++++---------------------- 1 file changed, 27 insertions(+), 46 deletions(-) diff --git a/bitbake/lib/bb/tests/fetch.py b/bitbake/lib/bb/tests/fetch.py index 5cccdf6ef4..b1ff2c005f 100644 --- a/bitbake/lib/bb/tests/fetch.py +++ b/bitbake/lib/bb/tests/fetch.py @@ -393,6 +393,15 @@ class FetcherTest(unittest.TestCase): bb.process.run('chmod u+rw -R %s' % self.tempdir) bb.utils.prunedir(self.tempdir) + def git(self, cmd, cwd=None): + if isinstance(cmd, str): + cmd = 'git ' + cmd + else: + cmd = ['git'] + cmd + if cwd is None: + cwd = self.gitdir + return bb.process.run(cmd, cwd=cwd)[0] + class MirrorUriTest(FetcherTest): replaceuris = { @@ -699,28 +708,24 @@ class FetcherLocalTest(FetcherTest): # Create dummy local Git repo src_dir = tempfile.mkdtemp(dir=self.tempdir, prefix='gitfetch_localusehead_') - src_dir = os.path.abspath(src_dir) - bb.process.run("git init", cwd=src_dir) - bb.process.run("git config user.email 'you@example.com'", cwd=src_dir) - bb.process.run("git config user.name 'Your Name'", cwd=src_dir) - bb.process.run("git commit --allow-empty -m'Dummy commit'", - cwd=src_dir) + self.gitdir = os.path.abspath(src_dir) + self.git('init') + self.git('config user.email "you@example.com"') + self.git('config user.name "Your Name"') + self.git(['commit', '--allow-empty', '-m', 'Dummy commit']) # Use other branch than master - bb.process.run("git checkout -b my-devel", cwd=src_dir) - bb.process.run("git commit --allow-empty -m'Dummy commit 2'", - cwd=src_dir) - stdout = bb.process.run("git rev-parse HEAD", cwd=src_dir) - orig_rev = stdout[0].strip() + self.git(['checkout', '-b', 'my-devel']) + self.git(['commit', '--allow-empty', '-m', 'Dummy commit 2']) + orig_rev = self.git(['rev-parse', 'HEAD']).strip() # Fetch and check revision self.d.setVar("SRCREV", "AUTOINC") - url = "git://" + src_dir + ";protocol=file;" + suffix + url = "git://" + self.gitdir + ";protocol=file;" + suffix fetcher = bb.fetch.Fetch([url], self.d) fetcher.download() fetcher.unpack(self.unpackdir) - stdout = bb.process.run("git rev-parse HEAD", - cwd=os.path.join(self.unpackdir, 'git')) - unpack_rev = stdout[0].strip() + unpack_rev = self.git(['rev-parse', 'HEAD'], + cwd=os.path.join(self.unpackdir, 'git')).strip() self.assertEqual(orig_rev, unpack_rev) def test_local_gitfetch_usehead(self): @@ -920,7 +925,8 @@ class FetcherNetworkTest(FetcherTest): def gitfetcher(self, url1, url2): def checkrevision(self, fetcher): fetcher.unpack(self.unpackdir) - revision = bb.process.run("git rev-parse HEAD", shell=True, cwd=self.unpackdir + "/git")[0].strip() + revision = self.git(['rev-parse', 'HEAD'], + cwd=os.path.join(self.unpackdir, 'git')).strip() self.assertEqual(revision, "270a05b0b4ba0959fe0624d2a4885d7b70426da5") self.d.setVar("BB_GENERATE_MIRROR_TARBALLS", "1") @@ -996,7 +1002,7 @@ class FetcherNetworkTest(FetcherTest): recipeurl = "git://someserver.org/bitbake" self.sourcedir = self.unpackdir.replace("unpacked", "sourcemirror.git") os.chdir(self.tempdir) - bb.process.run("git clone %s %s 2> /dev/null" % (realurl, self.sourcedir), shell=True) + self.git(['clone', realurl, self.sourcedir], cwd=self.tempdir) self.d.setVar("PREMIRRORS", "%s git://%s;protocol=file" % (recipeurl, self.sourcedir)) self.gitfetcher(recipeurl, recipeurl) @@ -1437,9 +1443,9 @@ class GitMakeShallowTest(FetcherTest): FetcherTest.setUp(self) self.gitdir = os.path.join(self.tempdir, 'gitshallow') bb.utils.mkdirhier(self.gitdir) - bb.process.run('git init', cwd=self.gitdir) - bb.process.run('git config user.email "you@example.com"', cwd=self.gitdir) - bb.process.run('git config user.name "Your Name"', cwd=self.gitdir) + self.git('init') + self.git('config user.email "you@example.com"') + self.git('config user.name "Your Name"') def assertRefs(self, expected_refs): actual_refs = self.git(['for-each-ref', '--format=%(refname)']).splitlines() @@ -1453,13 +1459,6 @@ class GitMakeShallowTest(FetcherTest): actual_count = len(revs.splitlines()) self.assertEqual(expected_count, actual_count, msg='Object count `%d` is not the expected `%d`' % (actual_count, expected_count)) - def git(self, cmd): - if isinstance(cmd, str): - cmd = 'git ' + cmd - else: - cmd = ['git'] + cmd - return bb.process.run(cmd, cwd=self.gitdir)[0] - def make_shallow(self, args=None): if args is None: args = ['HEAD'] @@ -1595,15 +1594,6 @@ class GitShallowTest(FetcherTest): actual_count = len(revs.splitlines()) self.assertEqual(expected_count, actual_count, msg='Object count `%d` is not the expected `%d`' % (actual_count, expected_count)) - def git(self, cmd, cwd=None): - if isinstance(cmd, str): - cmd = 'git ' + cmd - else: - cmd = ['git'] + cmd - if cwd is None: - cwd = self.gitdir - return bb.process.run(cmd, cwd=cwd)[0] - def add_empty_file(self, path, cwd=None, msg=None): if msg is None: msg = path @@ -2140,7 +2130,7 @@ class GitLfsTest(FetcherTest): self.gitdir = os.path.join(self.tempdir, 'git') self.srcdir = os.path.join(self.tempdir, 'gitsource') - + self.d.setVar('WORKDIR', self.tempdir) self.d.setVar('S', self.gitdir) self.d.delVar('PREMIRRORS') @@ -2158,15 +2148,6 @@ class GitLfsTest(FetcherTest): self.git(['add', '.gitattributes'], cwd=self.srcdir) self.git(['commit', '-m', "attributes", '.gitattributes'], cwd=self.srcdir) - def git(self, cmd, cwd=None): - if isinstance(cmd, str): - cmd = 'git ' + cmd - else: - cmd = ['git'] + cmd - if cwd is None: - cwd = self.gitdir - return bb.process.run(cmd, cwd=cwd)[0] - def fetch(self, uri=None, download=True): uris = self.d.getVar('SRC_URI').split() uri = uris[0] From patchwork Sat Jan 29 02:29:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Kjellerstedt X-Patchwork-Id: 3085 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 952C7C4321E for ; Sat, 29 Jan 2022 02:29:46 +0000 (UTC) Received: from smtp1.axis.com (smtp1.axis.com [195.60.68.17]) by mx.groups.io with SMTP id smtpd.web10.1536.1643423385092428832 for ; Fri, 28 Jan 2022 18:29:45 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@axis.com header.s=axis-central1 header.b=OTAcS9I4; spf=pass (domain: axis.com, ip: 195.60.68.17, mailfrom: peter.kjellerstedt@axis.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=axis.com; q=dns/txt; s=axis-central1; t=1643423385; x=1674959385; h=from:to:subject:date:message-id:in-reply-to:references: mime-version:content-transfer-encoding; bh=bWLB4sQJDd3F9buonTBUSguvYD928WUqKjVeyXsE4UQ=; b=OTAcS9I4PU8ifYMFrQ+knjgS8qSJx+UhbFWVjGVVyek68j6i2Z7rgcEm IFrpJudK0TvWbDjYSS6j44xlOKexUhq0+cJFNbYhlZ7sBJc0hL0jJy9dd geMwSh1SVSbtWNkbByuOrMuNTWlhTYdkpVtQvTWq8lM7pvEmZjl+qk9gu SFXIHGcPEhw5A6L6qiQJNeKG4OjchfMDbVj4yYRBnUMxXxeyb1wIZQsib FHeNLvFxn7D9a8CCxHTJc6BgvZ+CUDA/PkW31XrNmxx8J+bMpVjiySX/F tKTV3WjDDw27zixVbuVykQr0QJpirulrbukQEyWROSkyd4EfljkvH/sW1 g==; From: Peter Kjellerstedt To: Subject: [PATCH 6/9] tests/fetch: Only set the Git user name/email if they are not already set Date: Sat, 29 Jan 2022 03:29:36 +0100 Message-ID: <20220129022939.19329-6-pkj@axis.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20220129022939.19329-1-pkj@axis.com> References: <20220129022939.19329-1-pkj@axis.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sat, 29 Jan 2022 02:29:46 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/13299 If they are set, assume they are correct. Signed-off-by: Peter Kjellerstedt --- bitbake/lib/bb/tests/fetch.py | 31 +++++++++++++------------------ 1 file changed, 13 insertions(+), 18 deletions(-) diff --git a/bitbake/lib/bb/tests/fetch.py b/bitbake/lib/bb/tests/fetch.py index b1ff2c005f..16a1b6f342 100644 --- a/bitbake/lib/bb/tests/fetch.py +++ b/bitbake/lib/bb/tests/fetch.py @@ -402,6 +402,13 @@ class FetcherTest(unittest.TestCase): cwd = self.gitdir return bb.process.run(cmd, cwd=cwd)[0] + def git_init(self, cwd=None): + self.git('init', cwd=cwd) + if not self.git(['config', 'user.email'], cwd=cwd): + self.git(['config', 'user.email', 'you@example.com'], cwd=cwd) + if not self.git(['config', 'user.name'], cwd=cwd): + self.git(['config', 'user.name', 'Your Name'], cwd=cwd) + class MirrorUriTest(FetcherTest): replaceuris = { @@ -709,9 +716,7 @@ class FetcherLocalTest(FetcherTest): src_dir = tempfile.mkdtemp(dir=self.tempdir, prefix='gitfetch_localusehead_') self.gitdir = os.path.abspath(src_dir) - self.git('init') - self.git('config user.email "you@example.com"') - self.git('config user.name "Your Name"') + self.git_init() self.git(['commit', '--allow-empty', '-m', 'Dummy commit']) # Use other branch than master self.git(['checkout', '-b', 'my-devel']) @@ -1443,9 +1448,7 @@ class GitMakeShallowTest(FetcherTest): FetcherTest.setUp(self) self.gitdir = os.path.join(self.tempdir, 'gitshallow') bb.utils.mkdirhier(self.gitdir) - self.git('init') - self.git('config user.email "you@example.com"') - self.git('config user.name "Your Name"') + self.git_init() def assertRefs(self, expected_refs): actual_refs = self.git(['for-each-ref', '--format=%(refname)']).splitlines() @@ -1561,9 +1564,7 @@ class GitShallowTest(FetcherTest): self.srcdir = os.path.join(self.tempdir, 'gitsource') bb.utils.mkdirhier(self.srcdir) - self.git('init', cwd=self.srcdir) - self.git('config user.email "you@example.com"', cwd=self.srcdir) - self.git('config user.name "Your Name"', cwd=self.srcdir) + self.git_init(cwd=self.srcdir) self.d.setVar('WORKDIR', self.tempdir) self.d.setVar('S', self.gitdir) self.d.delVar('PREMIRRORS') @@ -1788,9 +1789,7 @@ class GitShallowTest(FetcherTest): smdir = os.path.join(self.tempdir, 'gitsubmodule') bb.utils.mkdirhier(smdir) - self.git('init', cwd=smdir) - self.git('config user.email "you@example.com"', cwd=smdir) - self.git('config user.name "Your Name"', cwd=smdir) + self.git_init(cwd=smdir) # Make this look like it was cloned from a remote... self.git('config --add remote.origin.url "%s"' % smdir, cwd=smdir) self.git('config --add remote.origin.fetch "+refs/heads/*:refs/remotes/origin/*"', cwd=smdir) @@ -1820,9 +1819,7 @@ class GitShallowTest(FetcherTest): smdir = os.path.join(self.tempdir, 'gitsubmodule') bb.utils.mkdirhier(smdir) - self.git('init', cwd=smdir) - self.git('config user.email "you@example.com"', cwd=smdir) - self.git('config user.name "Your Name"', cwd=smdir) + self.git_init(cwd=smdir) # Make this look like it was cloned from a remote... self.git('config --add remote.origin.url "%s"' % smdir, cwd=smdir) self.git('config --add remote.origin.fetch "+refs/heads/*:refs/remotes/origin/*"', cwd=smdir) @@ -2140,9 +2137,7 @@ class GitLfsTest(FetcherTest): self.d.setVar('AUTOREV', '${@bb.fetch2.get_autorev(d)}') bb.utils.mkdirhier(self.srcdir) - self.git('init', cwd=self.srcdir) - self.git('config user.email "you@example.com"', cwd=self.srcdir) - self.git('config user.name "Your Name"', cwd=self.srcdir) + self.git_init(cwd=self.srcdir) with open(os.path.join(self.srcdir, '.gitattributes'), 'wt') as attrs: attrs.write('*.mp3 filter=lfs -text') self.git(['add', '.gitattributes'], cwd=self.srcdir) From patchwork Sat Jan 29 02:29:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Kjellerstedt X-Patchwork-Id: 3087 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 89E51C43219 for ; Sat, 29 Jan 2022 02:29:46 +0000 (UTC) Received: from smtp2.axis.com (smtp2.axis.com [195.60.68.18]) by mx.groups.io with SMTP id smtpd.web11.1501.1643423382962218181 for ; Fri, 28 Jan 2022 18:29:45 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@axis.com header.s=axis-central1 header.b=HiVtvZIl; spf=pass (domain: axis.com, ip: 195.60.68.18, mailfrom: peter.kjellerstedt@axis.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=axis.com; q=dns/txt; s=axis-central1; t=1643423385; x=1674959385; h=from:to:subject:date:message-id:in-reply-to:references: mime-version:content-transfer-encoding; bh=oKoTKm4QNLq+di132Xq4KnfrRHHyvtvw8Cd+B9ydoSM=; b=HiVtvZIllVfd/Jdm5OUMhNzW09a2jdCtPx48RkhwwkiDS49EoOuVcx2D KEq8RqrDT2U+cU38V+dZjFBI7FYw2dGX2C0+UUrre6EKs7knIF2rj1TJx LsZLHYpC2o7zcepZMlL6wditEpR13C/72LEUOoInTv3jGywWUuhjqRZs7 yw7mRR7nJnnAoz0Z3PjqI+BDco2eJwUQanik/ApAYgOH88yAWvvv+lPgR Absa4KGjnaTsyCHUN0j476wnMJP9kBsnA6To0eGzAPgWSWQNSltbGHOmD lhGycCtxzAzLHYXumhg8p/7F4vVh9p2GwOP3BgGnnEdXWE9aHf9qD85HV Q==; From: Peter Kjellerstedt To: Subject: [PATCH 7/9] tests/fetch: Make test_npm_premirrors work with the current fetcher Date: Sat, 29 Jan 2022 03:29:37 +0100 Message-ID: <20220129022939.19329-7-pkj@axis.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20220129022939.19329-1-pkj@axis.com> References: <20220129022939.19329-1-pkj@axis.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sat, 29 Jan 2022 02:29:46 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/13300 There are two totally opposite use cases for how a premirror is expected to behave in combination with specifying a downloadfilename= parameter in the SRC_URI. On the one hand there is the expectation that it works like any other mirror, which means the premirror is expected to contain a file with the original name specified in the SRC_URI. On the other hand there is the expectation that one can use the artefacts downloaded by bitbake in ${DL_DIR} as a premirror, in which case it is expected to contain a file with the name from the downloadfilename= parameter. The latter case has been how downloaded files have been handled until commit 8a3ff9f3 (fetch2: fix premirror URI when downloadfilename defined), where the fetcher was changed to store files as per the first case. This is also when the test_npm_premirrors test case was modified in commit 5ba191a0 (tests/fetch: add and fix npm tests) to expect the first case. The above change was later reverted in commit 96c30007 (fetch2: fix downloadfilename issue with premirror). However the test_npm_premirrors test case was not updated to match, and has been failing ever since. This has probably gone unnoticed because the npm related test cases require that npm is installed on the host. This commit updates test_npm_premirrors to expect that premirrors use the filenames specified by downloadfilename= as this matches the current fetcher implementation and also is the most likely use case for premirrors. It also tries to mimic how one typically might setup the premirror directory by simply copying the download directory. Signed-off-by: Peter Kjellerstedt --- bitbake/lib/bb/tests/fetch.py | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-) diff --git a/bitbake/lib/bb/tests/fetch.py b/bitbake/lib/bb/tests/fetch.py index 16a1b6f342..9fb6378fdc 100644 --- a/bitbake/lib/bb/tests/fetch.py +++ b/bitbake/lib/bb/tests/fetch.py @@ -2322,15 +2322,24 @@ class NPMTest(FetcherTest): ud = fetcher.ud[fetcher.urls[0]] fetcher.download() self.assertTrue(os.path.exists(ud.localpath)) - # Setup the mirror - pkgname = os.path.basename(ud.proxy.urls[0].split(';')[0]) + + # Setup the mirror by renaming the download directory mirrordir = os.path.join(self.tempdir, 'mirror') - bb.utils.mkdirhier(mirrordir) - os.replace(ud.localpath, os.path.join(mirrordir, pkgname)) - self.d.setVar('PREMIRRORS', 'https?$://.*/.* file://%s/' % mirrordir) + bb.utils.rename(self.dldir, mirrordir) + os.mkdir(self.dldir) + + # Configure the premirror to be used + self.d.setVar('PREMIRRORS', 'https?$://.*/.* file://%s/npm2' % mirrordir) self.d.setVar('BB_FETCH_PREMIRRORONLY', '1') + # Fetch again self.assertFalse(os.path.exists(ud.localpath)) + # The npm fetcher doesn't handle that the .resolved file disappears + # while the fetcher object exists, which it does when we rename the + # download directory to "mirror" above. Thus we need a new fetcher to go + # with the now empty download directory. + fetcher = bb.fetch.Fetch([url], self.d) + ud = fetcher.ud[fetcher.urls[0]] fetcher.download() self.assertTrue(os.path.exists(ud.localpath)) From patchwork Sat Jan 29 02:29:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Kjellerstedt X-Patchwork-Id: 3091 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 95A1FC4167D for ; Sat, 29 Jan 2022 02:29:46 +0000 (UTC) Received: from smtp2.axis.com (smtp2.axis.com [195.60.68.18]) by mx.groups.io with SMTP id smtpd.web12.1486.1643423385313383936 for ; Fri, 28 Jan 2022 18:29:45 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@axis.com header.s=axis-central1 header.b=D21xAtwj; spf=pass (domain: axis.com, ip: 195.60.68.18, mailfrom: peter.kjellerstedt@axis.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=axis.com; q=dns/txt; s=axis-central1; t=1643423385; x=1674959385; h=from:to:subject:date:message-id:in-reply-to:references: mime-version:content-transfer-encoding; bh=1uoTPw+HChDwsAD73ihE9SNecrlYvBybpxHXVQXZsCw=; b=D21xAtwj8y3EYlCwMojD8DmYdkVKqAQcYiddo63nxl137CA0NFzbILKY dOga5Qda0YCW4wgRXOIUQ3CQpmGvtLfUUQ3XKkvKZTZjbVtLX90/KiHp3 kJpByTWXCJbRwtKzYVotlxwTwmOU+S59xmvwHoB1cPPb/1kQhcRj6WBYF QbJTQLbFU2XFpyYZtv7eyEPbe/CAOgqKHdcSWe9nEdr7k4DuNtFLHopy2 8gcQJG+htJQC/HyP2/dWevyPeJB0zWKDQEsdienhaFAZ1B4Jpx4swbZA/ METXDGlET+4H4z5IsfENUWljNNWlC8jBIwNS+8efnI2g69KpsxZzug070 A==; From: Peter Kjellerstedt To: Subject: [PATCH 8/9] fetch2: Correct handling of replacing the basename in URIs Date: Sat, 29 Jan 2022 03:29:38 +0100 Message-ID: <20220129022939.19329-8-pkj@axis.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20220129022939.19329-1-pkj@axis.com> References: <20220129022939.19329-1-pkj@axis.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sat, 29 Jan 2022 02:29:46 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/13301 The solution implementated in commit 96c30007 (fetch2: fix downloadfilename issue with premirror) missed two corner cases. The first is if the basename of the original URI also appears somewhere else in the replacement URI, in which case it would also be replaced. The second is if the basename of the original URI partially matches the basename of the replacement URI, in which case the inital part of the basename from the replacement URI would be left behind. The second case caused test_npm_premirrors_with_specified_filename to fail. The solution is to prefix the basename with a slash when matching to avoid partial matches, and only replace the basename at the end of the URI. This also adds two test cases that test for these problems. Before this they would give the following errors: - ['file:///mirror/example/1.0.0/some-example-1.0.0.tgz;downloadfilename=some-example-1.0.0.tgz'] + ['file:///mirror/some-example-1.0.0.tgz/1.0.0/some-example-1.0.0.tgz;downloadfilename=some-example-1.0.0.tgz'] ? +++++ ++++++++++ - ['file:///mirror/some-example-1.0.0.tgz;downloadfilename=some-example-1.0.0.tgz'] + ['file:///mirror/some-some-example-1.0.0.tgz;downloadfilename=some-example-1.0.0.tgz'] ? +++++ Signed-off-by: Peter Kjellerstedt --- bitbake/lib/bb/fetch2/__init__.py | 11 +++++---- bitbake/lib/bb/tests/fetch.py | 39 +++++++++++++++++-------------- 2 files changed, 28 insertions(+), 22 deletions(-) diff --git a/bitbake/lib/bb/fetch2/__init__.py b/bitbake/lib/bb/fetch2/__init__.py index d37174185a..28a3e54c7f 100644 --- a/bitbake/lib/bb/fetch2/__init__.py +++ b/bitbake/lib/bb/fetch2/__init__.py @@ -473,10 +473,13 @@ def uri_replace(ud, uri_find, uri_replace, replacements, d, mirrortarball=None): basename = os.path.basename(ud.localpath) if basename: uri_basename = os.path.basename(uri_decoded[loc]) - if uri_basename and basename != uri_basename and result_decoded[loc].endswith(uri_basename): - result_decoded[loc] = result_decoded[loc].replace(uri_basename, basename) - elif not result_decoded[loc].endswith(basename): - result_decoded[loc] = os.path.join(result_decoded[loc], basename) + # Prefix with a slash as a sentinel in case + # result_decoded[loc] does not contain one. + path = "/" + result_decoded[loc] + if uri_basename and basename != uri_basename and path.endswith("/" + uri_basename): + result_decoded[loc] = path[1:-len(uri_basename)] + basename + elif not path.endswith("/" + basename): + result_decoded[loc] = os.path.join(path[1:], basename) else: return None result = encodeurl(result_decoded) diff --git a/bitbake/lib/bb/tests/fetch.py b/bitbake/lib/bb/tests/fetch.py index 9fb6378fdc..8b13e178e6 100644 --- a/bitbake/lib/bb/tests/fetch.py +++ b/bitbake/lib/bb/tests/fetch.py @@ -412,34 +412,34 @@ class FetcherTest(unittest.TestCase): class MirrorUriTest(FetcherTest): replaceuris = { - ("git://git.invalid.infradead.org/mtd-utils.git;tag=1234567890123456789012345678901234567890", "git://.*/.*", "http://somewhere.org/somedir/") + ("git://git.invalid.infradead.org/mtd-utils.git;tag=1234567890123456789012345678901234567890", "git://.*/.*", "http://somewhere.org/somedir/") : "http://somewhere.org/somedir/git2_git.invalid.infradead.org.mtd-utils.git.tar.gz", - ("git://git.invalid.infradead.org/mtd-utils.git;tag=1234567890123456789012345678901234567890", "git://.*/([^/]+/)*([^/]*)", "git://somewhere.org/somedir/\\2;protocol=http") - : "git://somewhere.org/somedir/mtd-utils.git;tag=1234567890123456789012345678901234567890;protocol=http", - ("git://git.invalid.infradead.org/foo/mtd-utils.git;tag=1234567890123456789012345678901234567890", "git://.*/([^/]+/)*([^/]*)", "git://somewhere.org/somedir/\\2;protocol=http") - : "git://somewhere.org/somedir/mtd-utils.git;tag=1234567890123456789012345678901234567890;protocol=http", - ("git://git.invalid.infradead.org/foo/mtd-utils.git;tag=1234567890123456789012345678901234567890", "git://.*/([^/]+/)*([^/]*)", "git://somewhere.org/\\2;protocol=http") - : "git://somewhere.org/mtd-utils.git;tag=1234567890123456789012345678901234567890;protocol=http", + ("git://git.invalid.infradead.org/mtd-utils.git;tag=1234567890123456789012345678901234567890", "git://.*/([^/]+/)*([^/]*)", "git://somewhere.org/somedir/\\2;protocol=http") + : "git://somewhere.org/somedir/mtd-utils.git;tag=1234567890123456789012345678901234567890;protocol=http", + ("git://git.invalid.infradead.org/foo/mtd-utils.git;tag=1234567890123456789012345678901234567890", "git://.*/([^/]+/)*([^/]*)", "git://somewhere.org/somedir/\\2;protocol=http") + : "git://somewhere.org/somedir/mtd-utils.git;tag=1234567890123456789012345678901234567890;protocol=http", + ("git://git.invalid.infradead.org/foo/mtd-utils.git;tag=1234567890123456789012345678901234567890", "git://.*/([^/]+/)*([^/]*)", "git://somewhere.org/\\2;protocol=http") + : "git://somewhere.org/mtd-utils.git;tag=1234567890123456789012345678901234567890;protocol=http", ("git://someserver.org/bitbake;tag=1234567890123456789012345678901234567890", "git://someserver.org/bitbake", "git://git.openembedded.org/bitbake") : "git://git.openembedded.org/bitbake;tag=1234567890123456789012345678901234567890", - ("file://sstate-xyz.tgz", "file://.*", "file:///somewhere/1234/sstate-cache") + ("file://sstate-xyz.tgz", "file://.*", "file:///somewhere/1234/sstate-cache") : "file:///somewhere/1234/sstate-cache/sstate-xyz.tgz", - ("file://sstate-xyz.tgz", "file://.*", "file:///somewhere/1234/sstate-cache/") + ("file://sstate-xyz.tgz", "file://.*", "file:///somewhere/1234/sstate-cache/") : "file:///somewhere/1234/sstate-cache/sstate-xyz.tgz", - ("http://somewhere.org/somedir1/somedir2/somefile_1.2.3.tar.gz", "http://.*/.*", "http://somewhere2.org/somedir3") + ("http://somewhere.org/somedir1/somedir2/somefile_1.2.3.tar.gz", "http://.*/.*", "http://somewhere2.org/somedir3") : "http://somewhere2.org/somedir3/somefile_1.2.3.tar.gz", - ("http://somewhere.org/somedir1/somefile_1.2.3.tar.gz", "http://somewhere.org/somedir1/somefile_1.2.3.tar.gz", "http://somewhere2.org/somedir3/somefile_1.2.3.tar.gz") + ("http://somewhere.org/somedir1/somefile_1.2.3.tar.gz", "http://somewhere.org/somedir1/somefile_1.2.3.tar.gz", "http://somewhere2.org/somedir3/somefile_1.2.3.tar.gz") : "http://somewhere2.org/somedir3/somefile_1.2.3.tar.gz", ("http://www.apache.org/dist/subversion/subversion-1.7.1.tar.bz2", "http://www.apache.org/dist", "http://archive.apache.org/dist") : "http://archive.apache.org/dist/subversion/subversion-1.7.1.tar.bz2", ("http://www.apache.org/dist/subversion/subversion-1.7.1.tar.bz2", "http://.*/.*", "file:///somepath/downloads/") : "file:///somepath/downloads/subversion-1.7.1.tar.bz2", - ("git://git.invalid.infradead.org/mtd-utils.git;tag=1234567890123456789012345678901234567890", "git://.*/.*", "git://somewhere.org/somedir/BASENAME;protocol=http") - : "git://somewhere.org/somedir/mtd-utils.git;tag=1234567890123456789012345678901234567890;protocol=http", - ("git://git.invalid.infradead.org/foo/mtd-utils.git;tag=1234567890123456789012345678901234567890", "git://.*/.*", "git://somewhere.org/somedir/BASENAME;protocol=http") - : "git://somewhere.org/somedir/mtd-utils.git;tag=1234567890123456789012345678901234567890;protocol=http", - ("git://git.invalid.infradead.org/foo/mtd-utils.git;tag=1234567890123456789012345678901234567890", "git://.*/.*", "git://somewhere.org/somedir/MIRRORNAME;protocol=http") - : "git://somewhere.org/somedir/git.invalid.infradead.org.foo.mtd-utils.git;tag=1234567890123456789012345678901234567890;protocol=http", + ("git://git.invalid.infradead.org/mtd-utils.git;tag=1234567890123456789012345678901234567890", "git://.*/.*", "git://somewhere.org/somedir/BASENAME;protocol=http") + : "git://somewhere.org/somedir/mtd-utils.git;tag=1234567890123456789012345678901234567890;protocol=http", + ("git://git.invalid.infradead.org/foo/mtd-utils.git;tag=1234567890123456789012345678901234567890", "git://.*/.*", "git://somewhere.org/somedir/BASENAME;protocol=http") + : "git://somewhere.org/somedir/mtd-utils.git;tag=1234567890123456789012345678901234567890;protocol=http", + ("git://git.invalid.infradead.org/foo/mtd-utils.git;tag=1234567890123456789012345678901234567890", "git://.*/.*", "git://somewhere.org/somedir/MIRRORNAME;protocol=http") + : "git://somewhere.org/somedir/git.invalid.infradead.org.foo.mtd-utils.git;tag=1234567890123456789012345678901234567890;protocol=http", ("http://somewhere.org/somedir1/somedir2/somefile_1.2.3.tar.gz", "http://.*/.*", "http://somewhere2.org") : "http://somewhere2.org/somefile_1.2.3.tar.gz", ("http://somewhere.org/somedir1/somedir2/somefile_1.2.3.tar.gz", "http://.*/.*", "http://somewhere2.org/") @@ -450,8 +450,11 @@ class MirrorUriTest(FetcherTest): : "git://user2@git.openembedded.org/bitbake;tag=1234567890123456789012345678901234567890;branch=master;protocol=http", ("git://someserver.org/bitbake;tag=1234567890123456789012345678901234567890;protocol=git;branch=master", "git://someserver.org/bitbake", "git://someotherserver.org/bitbake;protocol=https") : "git://someotherserver.org/bitbake;tag=1234567890123456789012345678901234567890;protocol=https;branch=master", - ("gitsm://git.qemu.org/git/seabios.git/;protocol=https;name=roms/seabios;subpath=roms/seabios;bareclone=1;nobranch=1;rev=1234567890123456789012345678901234567890", "gitsm://.*/.*", "http://petalinux.xilinx.com/sswreleases/rel-v${XILINX_VER_MAIN}/downloads") : "http://petalinux.xilinx.com/sswreleases/rel-v%24%7BXILINX_VER_MAIN%7D/downloads/git2_git.qemu.org.git.seabios.git..tar.gz", + ("https://somewhere.org/example/1.0.0/example;downloadfilename=some-example-1.0.0.tgz", "https://.*/.*", "file:///mirror/PATH") + : "file:///mirror/example/1.0.0/some-example-1.0.0.tgz;downloadfilename=some-example-1.0.0.tgz", + ("https://somewhere.org/example-1.0.0.tgz;downloadfilename=some-example-1.0.0.tgz", "https://.*/.*", "file:///mirror/some-example-1.0.0.tgz") + : "file:///mirror/some-example-1.0.0.tgz;downloadfilename=some-example-1.0.0.tgz", #Renaming files doesn't work #("http://somewhere.org/somedir1/somefile_1.2.3.tar.gz", "http://somewhere.org/somedir1/somefile_1.2.3.tar.gz", "http://somewhere2.org/somedir3/somefile_2.3.4.tar.gz") : "http://somewhere2.org/somedir3/somefile_2.3.4.tar.gz" From patchwork Sat Jan 29 02:29:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Kjellerstedt X-Patchwork-Id: 3089 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8B7DDC433FE for ; Sat, 29 Jan 2022 02:29:47 +0000 (UTC) Received: from smtp1.axis.com (smtp1.axis.com [195.60.68.17]) by mx.groups.io with SMTP id smtpd.web09.1520.1643423382741682585 for ; Fri, 28 Jan 2022 18:29:46 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@axis.com header.s=axis-central1 header.b=FXwBuGVY; spf=pass (domain: axis.com, ip: 195.60.68.17, mailfrom: peter.kjellerstedt@axis.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=axis.com; q=dns/txt; s=axis-central1; t=1643423386; x=1674959386; h=from:to:subject:date:message-id:in-reply-to:references: mime-version:content-transfer-encoding; bh=vE4M3bGPDw8yGo3rX4lR3FA0kxkDhcdEOXi/Z0S6+Kw=; b=FXwBuGVY3SkoEVYMSlSZF+vp3+QD7+Mq3vsfadXXUiSkPG+fGLAWRBR/ yLjql4Gk7LmuQ7UFM1Pqtm6EHk5AyOuhcR7SUBU7IM4JKLLbc+zVSsWma viE+20Y+6LJFeeAXVnstSbZLFWvzQQ6d6gKlgjncoRi/75664gv0ErvJ/ 8vA5u9dm5YxeMwzNkz62YyhSbAoB5jqA1BfP1gHFBvpIchjwjCvRAehTv V4IbxEwuVtY2Rg3MEWCBXcS456mm87CD11OXfy37DoxudKFp/rHI8y/VE RCdSoAarJ7lXCYOS4Iw9aRcpenwdGcLwG9VOJnfcY8SkYrKBnnEu+nbvr A==; From: Peter Kjellerstedt To: Subject: [PATCH 9/9] fetch2: npm: Put all downloaded files in the npm2 directory Date: Sat, 29 Jan 2022 03:29:39 +0100 Message-ID: <20220129022939.19329-9-pkj@axis.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: <20220129022939.19329-1-pkj@axis.com> References: <20220129022939.19329-1-pkj@axis.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Sat, 29 Jan 2022 02:29:47 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/13302 Previously npm files that specify downloadfilename= in the SRC_URI would be downloaded to the root of ${DL_DIR} rather than in the ${DL_DIR}/npm2 directory where all other npm files are downloaded. This should make it simpler when setting up and configuring a premirror with the downloaded npm packages. Signed-off-by: Peter Kjellerstedt --- bitbake/lib/bb/fetch2/npm.py | 10 +++++++--- bitbake/lib/bb/fetch2/npmsw.py | 2 +- bitbake/lib/bb/tests/fetch.py | 2 +- 3 files changed, 9 insertions(+), 5 deletions(-) diff --git a/bitbake/lib/bb/fetch2/npm.py b/bitbake/lib/bb/fetch2/npm.py index b3a3a444ee..8f7c10ac9b 100644 --- a/bitbake/lib/bb/fetch2/npm.py +++ b/bitbake/lib/bb/fetch2/npm.py @@ -52,9 +52,13 @@ def npm_filename(package, version): """Get the filename of a npm package""" return npm_package(package) + "-" + version + ".tgz" -def npm_localfile(package, version): +def npm_localfile(package, version=None): """Get the local filename of a npm package""" - return os.path.join("npm2", npm_filename(package, version)) + if version is not None: + filename = npm_filename(package, version) + else: + filename = package + return os.path.join("npm2", filename) def npm_integrity(integrity): """ @@ -157,7 +161,7 @@ class Npm(FetchMethod): # Using the 'downloadfilename' parameter as local filename # or the npm package name. if "downloadfilename" in ud.parm: - ud.localfile = d.expand(ud.parm["downloadfilename"]) + ud.localfile = npm_localfile(d.expand(ud.parm["downloadfilename"])) else: ud.localfile = npm_localfile(ud.package, ud.version) diff --git a/bitbake/lib/bb/fetch2/npmsw.py b/bitbake/lib/bb/fetch2/npmsw.py index 879ba5de0f..a8c4d3528f 100644 --- a/bitbake/lib/bb/fetch2/npmsw.py +++ b/bitbake/lib/bb/fetch2/npmsw.py @@ -117,7 +117,7 @@ class NpmShrinkWrap(FetchMethod): # Handle http tarball sources elif version.startswith("http") and integrity: - localfile = os.path.join("npm2", os.path.basename(version)) + localfile = npm_localfile(os.path.basename(version)) uri = URI(version) uri.params["downloadfilename"] = localfile diff --git a/bitbake/lib/bb/tests/fetch.py b/bitbake/lib/bb/tests/fetch.py index 8b13e178e6..3fd8fac229 100644 --- a/bitbake/lib/bb/tests/fetch.py +++ b/bitbake/lib/bb/tests/fetch.py @@ -2399,7 +2399,7 @@ class NPMTest(FetcherTest): url = 'npm://registry.npmjs.org;package=@savoirfairelinux/node-server-example;version=1.0.0;destsuffix=foo/bar;downloadfilename=foo-bar.tgz' fetcher = bb.fetch.Fetch([url], self.d) fetcher.download() - self.assertTrue(os.path.exists(os.path.join(self.dldir, 'foo-bar.tgz'))) + self.assertTrue(os.path.exists(os.path.join(self.dldir, 'npm2', 'foo-bar.tgz'))) fetcher.unpack(self.unpackdir) unpackdir = os.path.join(self.unpackdir, 'foo', 'bar') self.assertTrue(os.path.exists(os.path.join(unpackdir, 'package.json')))