From patchwork Tue Jan 17 15:36:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Zhukov X-Patchwork-Id: 18259 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 66AFCC6379F for ; Tue, 17 Jan 2023 15:37:05 +0000 (UTC) Received: from out5-smtp.messagingengine.com (out5-smtp.messagingengine.com [66.111.4.29]) by mx.groups.io with SMTP id smtpd.web11.199752.1673969823677355135 for ; Tue, 17 Jan 2023 07:37:04 -0800 Authentication-Results: mx.groups.io; dkim=fail reason="signature has expired" header.i=@zhukoff.net header.s=fm3 header.b=xv7Uq/tg; spf=pass (domain: zhukoff.net, ip: 66.111.4.29, mailfrom: pavel@zhukoff.net) Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.nyi.internal (Postfix) with ESMTP id EAAA95C017C; Tue, 17 Jan 2023 10:37:02 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute4.internal (MEProxy); Tue, 17 Jan 2023 10:37:02 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=zhukoff.net; h= cc:cc:content-transfer-encoding:date:date:from:from:in-reply-to :message-id:mime-version:reply-to:sender:subject:subject:to:to; s=fm3; t=1673969822; x=1674056222; bh=8AUyGqJVAAgAXPney4S4NulbB psUlxzFaVRpA3YSIU8=; b=xv7Uq/tg3G1hNeXCshKwto1OeqqyON/i4gxn7ViPk PDDGCvLTPOC/R4QaeuXyup6hZNqHhY8l6W4sve9MnB6Ck8C0aO1hM11A4iRWeU4v PolgjZbqtSNFLISdj8oTpJbsEqLYKMpksS33HdOej+wPj1eUf6ZEpU5MLd7dOMTg /zvPb5a5M5+FY3LATDjJH4Z706yn0MfZViETAxIJp5YNHzMguXe0LCDvmzSuY46k JZOhVWt5pEOjUEc+VFxS2kk5LuJZzz5svcnGLW1CH1RhUygF9gQqmWtjH3uvFXYs bK1ULKyKepGntGWKvyFpj27JHtansM26iVot9KStgwXGw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding:date:date :feedback-id:feedback-id:from:from:in-reply-to:message-id :mime-version:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t= 1673969822; x=1674056222; bh=8AUyGqJVAAgAXPney4S4NulbBpsUlxzFaVR pA3YSIU8=; b=VfCxiziggHvPtCLQiDkBY9Cik2wMCoRi4VCNkUecpSWuE+J4N1w jquxd+Ad1/TvqnNyX46AukPmsHWAnkqj0126gfg2Vmw1dCOtP7LMXYVbZlIsVsar S3F023ferT5DXaB1TuXx+nWcOY4e+3AwOcJUx4oTWXF7xJLkFefTDM4BKQ7P0Hwb tBx1/hnBRaB0KhqNkaNZlZuPy2z9Dr2/76KIfx98HmwToBpcx15BP5Rh+WyNrNWu zEDytwFRkbyiWCyUqAGOyPvl23Go6GKSn6fYmTrtmFMt9OkUqkytFSY26dqV7PVB wAxK+D8s30pb2hMFdtr1vjAMhYKKRVR7QRQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvhedruddtiedgjeehucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefhvfevufffkffoggfgsedtkeertd ertddtnecuhfhrohhmpefrrghvvghlucgkhhhukhhovhcuoehprghvvghlseiihhhukhho fhhfrdhnvghtqeenucggtffrrghtthgvrhhnpeevfeejkeejudegledtkeejhfettdefje euffehieekvddugfdtffffledvvdefhfenucevlhhushhtvghrufhiiigvpedtnecurfgr rhgrmhepmhgrihhlfhhrohhmpehprghvvghlseiihhhukhhofhhfrdhnvght X-ME-Proxy: Feedback-ID: ib94946c9:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 17 Jan 2023 10:37:02 -0500 (EST) From: Pavel Zhukov To: bitbake-devel@lists.openembedded.org Cc: Pavel Zhukov Subject: [PATCH 1/2] fetch2: kill parameters in uri_find_decoded Date: Tue, 17 Jan 2023 16:36:42 +0100 Message-Id: <20230117153643.21716-1-pavel@zhukoff.net> X-Mailer: git-send-email 2.39.0 MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 17 Jan 2023 15:37:05 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/14318 Fixes: File "/bitbake/./lib/bb/fetch2/__init__.py", line 446, in uri_replace if uri_decoded[loc][k] != uri_find_decoded[loc][k]: KeyError: 'module' Signed-off-by: Pavel Zhukov --- bitbake/lib/bb/fetch2/__init__.py | 1 + 1 file changed, 1 insertion(+) diff --git a/bitbake/lib/bb/fetch2/__init__.py b/bitbake/lib/bb/fetch2/__init__.py index 893ec6f6bd..b1cd6b25c2 100644 --- a/bitbake/lib/bb/fetch2/__init__.py +++ b/bitbake/lib/bb/fetch2/__init__.py @@ -469,6 +469,7 @@ def uri_replace(ud, uri_find, uri_replace, replacements, d, mirrortarball=None): basename = os.path.basename(mirrortarball) # Kill parameters, they make no sense for mirror tarballs uri_decoded[5] = {} + uri_find_decoded[5] = {} elif ud.localpath and ud.method.supports_checksum(ud): basename = os.path.basename(ud.localpath) if basename: From patchwork Tue Jan 17 15:36:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Zhukov X-Patchwork-Id: 18260 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 65424C3DA78 for ; Tue, 17 Jan 2023 15:37:05 +0000 (UTC) Received: from out5-smtp.messagingengine.com (out5-smtp.messagingengine.com [66.111.4.29]) by mx.groups.io with SMTP id smtpd.web11.199754.1673969824656473711 for ; Tue, 17 Jan 2023 07:37:04 -0800 Authentication-Results: mx.groups.io; dkim=fail reason="signature has expired" header.i=@zhukoff.net header.s=fm3 header.b=rlkOgD36; spf=pass (domain: zhukoff.net, ip: 66.111.4.29, mailfrom: pavel@zhukoff.net) Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailout.nyi.internal (Postfix) with ESMTP id 027F25C0111; Tue, 17 Jan 2023 10:37:04 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute2.internal (MEProxy); Tue, 17 Jan 2023 10:37:04 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=zhukoff.net; h= cc:cc:content-transfer-encoding:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:sender :subject:subject:to:to; s=fm3; t=1673969823; x=1674056223; bh=dG EikX6zmYU0CmysaEjjrWQJCcnN1DMaFRrCtrfDw1s=; b=rlkOgD36wlvTpjo0zf tWLAODffASgVY/6cJpNE+EvQ1IvKvCNwVON9Z18M6x7QfpKKdgKkqeF9QoQ9EHhW t5Uk1bbbOKGRRFl2gRwsBA03NhIgBYowtkG5RzMMF91NnPwCfaBoYHEDPndbbXE3 V8izlXWKZMQlVgeIXMk2UpMjeLuCnaRXO+1XxCEKQLujzntQgXpG84sf7VxvIqQy tKKktuNm310ud4v2UxXXTUEI/f72PiRy86+QJZEgdmpY36Ee8DohWBK5fN5rOMjn jgoDWGSXqt83tAZkuWOhifnsALAbkPxjpbbkAfY0/SRDmgKlIPG9ou9/ojLVLU2U ZAhA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; t=1673969823; x=1674056223; bh=dGEikX6zmYU0C mysaEjjrWQJCcnN1DMaFRrCtrfDw1s=; b=b0MnjvgxzetgDt2NgkbgyeJ114AgD jE/9avWP38lIDKLm0t/acwFBDIbju3DU4oWiBxBXfMYfiCy6836ELUP/AmHN8mbr Xa0+4UirVmvpAg5nskkXK2f7LTQOydGzclnyGsUV9LWI1LFLp7OjxDRe7PYoEo+B z98JRWAywQFKYKfXu/iXgyfWrzAHTBDTNsxc4mymPPeVbORGKyAYFuUDRFz8nB/d HraFXp82c5RQH0WFkdzEWc+9bFCSHjDCOuoqJ0iJJPqWmLJ99i8XOGOffqoUcvwE knj6qWEphFY4kGeQh+qWmsr8KOT1/P6OdlKWvshrVtMBEpe9eiL8A+JAw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvhedruddtiedgjeehucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefhvfevufffkffojghfggfgsedtke ertdertddtnecuhfhrohhmpefrrghvvghlucgkhhhukhhovhcuoehprghvvghlseiihhhu khhofhhfrdhnvghtqeenucggtffrrghtthgvrhhnpeeukeeuueegudfhheetgeekudekje efkeeugefhledtueelgeeijeetffejvefhfeenucffohhmrghinhepihhmfhhrvggvugho mhdrohhrghenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhroh hmpehprghvvghlseiihhhukhhofhhfrdhnvght X-ME-Proxy: Feedback-ID: ib94946c9:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 17 Jan 2023 10:37:03 -0500 (EST) From: Pavel Zhukov To: bitbake-devel@lists.openembedded.org Cc: Pavel Zhukov Subject: [PATCH 2/2] fetch2/tests: Add test for Mercurial Date: Tue, 17 Jan 2023 16:36:43 +0100 Message-Id: <20230117153643.21716-2-pavel@zhukoff.net> X-Mailer: git-send-email 2.39.0 In-Reply-To: <20230117153643.21716-1-pavel@zhukoff.net> References: <20230117153643.21716-1-pavel@zhukoff.net> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 17 Jan 2023 15:37:05 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/14319 Basic test for mercurial functionality including PREMIRRORONLY with disabled network Signed-off-by: Pavel Zhukov --- bitbake/lib/bb/tests/fetch.py | 44 +++++++++++++++++++++++++++++++++++ 1 file changed, 44 insertions(+) diff --git a/bitbake/lib/bb/tests/fetch.py b/bitbake/lib/bb/tests/fetch.py index ad3d4dea7d..2e5cd14205 100644 --- a/bitbake/lib/bb/tests/fetch.py +++ b/bitbake/lib/bb/tests/fetch.py @@ -2936,6 +2936,50 @@ class FetchPremirroronlyNetworkTest(FetcherTest): with self.assertRaises(bb.fetch2.NetworkAccess): fetcher.download() +class FetchPremirroronlyMercurialTest(FetcherTest): + """ Test for premirrors with mercurial repos + the test covers also basic hg:// clone (see fetch_and_create_tarball + """ + def skipIfNoHg(): + import shutil + if not shutil.which('hg'): + return unittest.skip('Mercurial not installed') + return lambda f: f + + def setUp(self): + super(FetchPremirroronlyMercurialTest, self).setUp() + self.mirrordir = os.path.join(self.tempdir, "mirrors") + os.mkdir(self.mirrordir) + self.reponame = "libgnt" + self.clonedir = os.path.join(self.tempdir, "hg") + self.recipe_url = "hg://keep.imfreedom.org/libgnt;module=libgnt" + self.d.setVar("SRCREV", "53e8b422faaf") + self.mirrorname = "hg_libgnt_keep.imfreedom.org_.libgnt.tar.gz" + + def fetch_and_create_tarball(self): + """ + Ask bitbake to download repo and prepare mirror tarball for us + """ + self.d.setVar("BB_GENERATE_MIRROR_TARBALLS", "1") + fetcher = bb.fetch.Fetch([self.recipe_url], self.d) + fetcher.download() + mirrorfile = os.path.join(self.d.getVar("DL_DIR"), self.mirrorname) + self.assertTrue(os.path.exists(mirrorfile), "Mirror tarball {} has not been created".format(mirrorfile)) + ## moving tarball to mirror directory + os.rename(mirrorfile, os.path.join(self.mirrordir, self.mirrorname)) + self.d.setVar("BB_GENERATE_MIRROR_TARBALLS", "0") + + + @skipIfNoNetwork() + @skipIfNoHg() + def test_premirror_mercurial(self): + self.fetch_and_create_tarball() + self.d.setVar("PREMIRRORS", self.recipe_url + " " + "file://{}".format(self.mirrordir) + " \n") + self.d.setVar("BB_FETCH_PREMIRRORONLY", "1") + self.d.setVar("BB_NO_NETWORK", "1") + fetcher = bb.fetch.Fetch([self.recipe_url], self.d) + fetcher.download() + class FetchPremirroronlyBrokenTarball(FetcherTest): def setUp(self):