From patchwork Tue Oct 10 09:30:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alexis_Lothor=C3=A9?= X-Patchwork-Id: 31896 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id E395BCD6913 for ; Tue, 10 Oct 2023 09:29:26 +0000 (UTC) Received: from relay4-d.mail.gandi.net (relay4-d.mail.gandi.net [217.70.183.196]) by mx.groups.io with SMTP id smtpd.web10.87199.1696930157395357652 for ; Tue, 10 Oct 2023 02:29:17 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@bootlin.com header.s=gm1 header.b=YjfiA+i7; spf=pass (domain: bootlin.com, ip: 217.70.183.196, mailfrom: alexis.lothore@bootlin.com) Received: by mail.gandi.net (Postfix) with ESMTPSA id C16CFE0007; Tue, 10 Oct 2023 09:29:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bootlin.com; s=gm1; t=1696930155; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=JgroQvfISW2d8VN1NpoW7tH1e1wiVSw7wgJb4WtnDLs=; b=YjfiA+i7SxtC7cpFuY9JKzPPgBUwmlO7X0dhmn41KAVfATMQEy3beuDqAAuuj87wpqoDtj fYsOP+0FW3DhP45LKCh8yvVZ5HGfRgaW2i7hVc8ajiVeHJ2+wsjs7i/mE44PueLXwVqxCR JIg+pXbFzGUqcI09HErDwNzm0C8ZxY0cSbrj/mkixDfLSAPVs5PBZNslrjantyM+cIdq+2 0Efm2gJphrG3Gldg78o/CxXvjEs0geA7/3uDVv3eYyCjY8S+sc3ULut4wOSn9F/NMXyt1i ap69B0zv/0xEDoSGSzYG6wn1JT808c27j931fpbzu3txxoLxMT6pBr44q8nCmw== From: =?utf-8?q?Alexis_Lothor=C3=A9?= To: Cc: Thomas Petazzoni , Alexandre Belloni Subject: [OE-Core][PATCH 1/2] oeqa/utils/gitarchive: fix tag pattern searching Date: Tue, 10 Oct 2023 11:30:12 +0200 Message-ID: <20231010093013.16922-2-alexis.lothore@bootlin.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231010093013.16922-1-alexis.lothore@bootlin.com> References: <20231010093013.16922-1-alexis.lothore@bootlin.com> MIME-Version: 1.0 X-GND-Sasl: alexis.lothore@bootlin.com List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 10 Oct 2023 09:29:26 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/188877 From: Alexis Lothoré Whenever we ask gitarchive to search for tags, we can provide it with a pattern (containing glob patterns). However, when searching for example for tags matching branch master-next, it can find more tags which does not correspond exactly to branch master-next (e.g. abelloni/master-next tags will match). Prevent those additional tags from being fetched by gitarchive by using a more specific pattern: prefix user-provided pattern with "refs/tags" Signed-off-by: Alexis Lothoré --- meta/lib/oeqa/utils/gitarchive.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/meta/lib/oeqa/utils/gitarchive.py b/meta/lib/oeqa/utils/gitarchive.py index f9c152681db7..2fe48cdcac7f 100644 --- a/meta/lib/oeqa/utils/gitarchive.py +++ b/meta/lib/oeqa/utils/gitarchive.py @@ -113,7 +113,7 @@ def get_tags(repo, log, pattern=None, url=None): # First try to fetch tags from repository configured remote cmd.append('origin') if pattern: - cmd.append(pattern) + cmd.append("refs/tags/"+pattern) try: tags_refs = repo.run_cmd(cmd) tags = ["".join(d.split()[1].split('/', 2)[2:]) for d in tags_refs.splitlines()] From patchwork Tue Oct 10 09:30:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alexis_Lothor=C3=A9?= X-Patchwork-Id: 31897 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 041B6CD6911 for ; Tue, 10 Oct 2023 09:29:27 +0000 (UTC) Received: from relay4-d.mail.gandi.net (relay4-d.mail.gandi.net [217.70.183.196]) by mx.groups.io with SMTP id smtpd.web11.86889.1696930157154986674 for ; Tue, 10 Oct 2023 02:29:17 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@bootlin.com header.s=gm1 header.b=p7fjMwEq; spf=pass (domain: bootlin.com, ip: 217.70.183.196, mailfrom: alexis.lothore@bootlin.com) Received: by mail.gandi.net (Postfix) with ESMTPSA id 594B7E0009; Tue, 10 Oct 2023 09:29:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bootlin.com; s=gm1; t=1696930155; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=HkeArefO9HsbyIwz4RbGcfN9YOSfzHpqnWlmyFc58P4=; b=p7fjMwEqRQTi9YXJkw+EH1Im1X8+iCU+bwxvNO+D0E4IqrCXIkiBphLLRV692Bgey5NADM Y25UBRujbPYph90XHR0bOIbcNslqVUgWhTFzGUpxLYaS82Hwr52x4Sax8lUXTRzMbgJH0M dW8AwO3CrOVjHOSHy3pn1WmCydDu3HZ9FdpCRj0Be2syrkGa/dPyJHROo2cZtvAM0GP53A 7eyU7bFieRQ0+hdIvTmGI4E0+xLWeWM6GwuBC9FYCy6KTHmOWHaQ8HF4KkWmrLRRvQVbbk Iu/XutTwDAgaNzWqv/bW7iZ2qFKUPAI2bthtORQKXnD7PbR2fPWnjXSq1JPE8g== From: =?utf-8?q?Alexis_Lothor=C3=A9?= To: Cc: Thomas Petazzoni , Alexandre Belloni Subject: [OE-Core][PATCH 2/2] oeqa/utils/gitarchive: ensure tag matches regex before getting its fields Date: Tue, 10 Oct 2023 11:30:13 +0200 Message-ID: <20231010093013.16922-3-alexis.lothore@bootlin.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20231010093013.16922-1-alexis.lothore@bootlin.com> References: <20231010093013.16922-1-alexis.lothore@bootlin.com> MIME-Version: 1.0 X-GND-Sasl: alexis.lothore@bootlin.com List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 10 Oct 2023 09:29:27 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/188878 From: Alexis Lothoré Whenever we ask gitarchive to retrieve test results for specific revisions, we first do a "large" search in get_tags, which uses glob patterns with git ls-remote, and then we filter received tags with a regex to parse the tags fields. Currently gitarchive assumes that all tags returned by get_tags will match the regex. This assumption is wrong (for example searching "master-next" in get_tags may return some tags like "abelloni/master-next), and leads then to exception when we try to retrieve tags fields: Traceback (most recent call last): File "/home/pokybuild/yocto-worker/a-full/build/scripts/resulttool", line 78, in sys.exit(main()) ^^^^^^ File "/home/pokybuild/yocto-worker/a-full/build/scripts/resulttool", line 72, in main ret = args.func(args, logger) ^^^^^^^^^^^^^^^^^^^^^^^ File "/home/pokybuild/yocto-worker/a-full/build/scripts/lib/resulttool/regression.py", line 315, in regression_git revs2 = gitarchive.get_test_revs(logger, repo, tag_name, branch=args.branch2) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/pokybuild/yocto-worker/a-full/build/meta/lib/oeqa/utils/gitarchive.py", line 246, in get_test_revs fields, runs = get_test_runs(log, repo, tag_name, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/pokybuild/yocto-worker/a-full/build/meta/lib/oeqa/utils/gitarchive.py", line 238, in get_test_runs groups = m.groupdict() Fix this exception by merely skipping those additionals tags which won't match the regex Signed-off-by: Alexis Lothoré --- meta/lib/oeqa/utils/gitarchive.py | 2 ++ 1 file changed, 2 insertions(+) diff --git a/meta/lib/oeqa/utils/gitarchive.py b/meta/lib/oeqa/utils/gitarchive.py index 2fe48cdcac7f..10cb267dfa92 100644 --- a/meta/lib/oeqa/utils/gitarchive.py +++ b/meta/lib/oeqa/utils/gitarchive.py @@ -235,6 +235,8 @@ def get_test_runs(log, repo, tag_name, **kwargs): revs = [] for tag in tags: m = tag_re.match(tag) + if not m: + continue groups = m.groupdict() revs.append([groups[f] for f in undef_fields] + [tag])