From patchwork Wed Sep 20 22:30:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Steve Sakoman X-Patchwork-Id: 30861 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id EAA72CD492E for ; Wed, 20 Sep 2023 22:31:33 +0000 (UTC) Received: from mail-ot1-f51.google.com (mail-ot1-f51.google.com [209.85.210.51]) by mx.groups.io with SMTP id smtpd.web10.3151.1695249087824822039 for ; Wed, 20 Sep 2023 15:31:27 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@sakoman-com.20230601.gappssmtp.com header.s=20230601 header.b=qDT3bHx4; spf=softfail (domain: sakoman.com, ip: 209.85.210.51, mailfrom: steve@sakoman.com) Received: by mail-ot1-f51.google.com with SMTP id 46e09a7af769-6beff322a97so240120a34.3 for ; Wed, 20 Sep 2023 15:31:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sakoman-com.20230601.gappssmtp.com; s=20230601; t=1695249087; x=1695853887; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=cCdwuK9lv358AIi7n/5sgh4CqQmdvTqp6GgKr5UrQpg=; b=qDT3bHx4Oy9K9X7FkYHBQX/0e8ZDHn3tT2AtM1ZUfn1CqGU88l9fn6Myv0HkoAOSlP 0tz4QtsC3ryGkUbLnt1mAbZcrSbTwsdxLXRY62OhN4HY7wiFgJvLBwdEc/gYQL+vAkRZ PlqJ07pI6GVwtXx69Pe9xE1RuXFqnb/xPij/9YfjcYm083HUMtNuxp3mqnqL1q5KOxX2 thHCqfWelhyTWzFet0Z/gnUw+fC8/HUj4sE2gR82B1/quPIpWbRdBYKsStWfl30xnWzC /qNdDgcOdi2VAsDX0oID6Qj84UfgJr05NaHUOXsZlgXFMN8RarLThfHqfrqZptayaV/7 hJVg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695249087; x=1695853887; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=cCdwuK9lv358AIi7n/5sgh4CqQmdvTqp6GgKr5UrQpg=; b=vp6VGAlFkSKQnOhplVA532wjiS55hD7RUF7wh15+spubvOWwtE7kxxUHbQ4EALkpgF Om81ditOsaMUx8BXIWjPLhkhbY1PcHAZk3E6eWD6vYZeeJY0VN+K4BBYXCuVq3gGxnyD FQ5j8OA33hdUNc2O02sY23OcNI+nWGMOjDoonANp9pR2E5ZQkOwyq0Zn5kierU6fW0j8 16ii4nD3LZqBUUh7NadOJ00u/1yLSFfnlnkMmYXktXI5ljLx0bLhwz3Iw8EwG4WRIiem OMg4aTNijvC4OFH81UM2ZbP1/eVPalqYRDP3WGSdgkKCajBzoQaPZAhCEnW4ZC74plPh ol9A== X-Gm-Message-State: AOJu0YzvIQ0DPmeX4+KKbkwDv/ufAAXipTYBqtshQdlk5zDeoCKvWu1I z2OzqYbnXEB4edOyPe3YksSYzgGnx8fNRuVNhDs= X-Google-Smtp-Source: AGHT+IFSYYcwzmH9mPGgctBc7hKZg/Tia8aBdNh7E4lSis4Sjocp+1glAj3hgcryRiEmryaLi5e17g== X-Received: by 2002:a05:6358:8a3:b0:143:4e8d:4446 with SMTP id m35-20020a05635808a300b001434e8d4446mr5022395rwj.2.1695249086686; Wed, 20 Sep 2023 15:31:26 -0700 (PDT) Received: from hexa.lan (dhcp-72-234-106-30.hawaiiantel.net. [72.234.106.30]) by smtp.gmail.com with ESMTPSA id a13-20020a63704d000000b00578f1a71a91sm11535pgn.79.2023.09.20.15.31.25 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 Sep 2023 15:31:26 -0700 (PDT) From: Steve Sakoman To: openembedded-core@lists.openembedded.org Subject: [OE-core][kirkstone 14/20] oeqa/utils/gitarchive: fix tag computation when creating archive Date: Wed, 20 Sep 2023 12:30:51 -1000 Message-Id: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 20 Sep 2023 22:31:33 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/187962 From: Alexis Lothoré Sporadic errors have been observed in autobuilder when trying to store new tests results: error: failed to push some refs to 'push.yoctoproject.org:yocto-testresults' hint: Updates were rejected because the tag already exists in the remote. The new tag name is generated by gitarchive based on known tags from the repository (learnt with git tag). In autobuilder case, this repository is a shallow clone, so git tag only returns most recent tags, which mean we could miss some older tags which exist in remote but not locally. In this case, gitarchive will likely create a tag which already exists in remote, and so will fail to push Fix this tag duplication by using git ls-remote to learn about existing tags instead of git tag. Two places which wrongly read only local tags has been identified in gitarchive: expand_tag_strings and get_test_runs Fixes [YOCTO #15140] Signed-off-by: Alexis Lothoré Signed-off-by: Richard Purdie (cherry picked from commit 5a0a7da85a3acfd4a20a07478eabefdab60f313a) Signed-off-by: Steve Sakoman --- meta/lib/oeqa/utils/gitarchive.py | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/meta/lib/oeqa/utils/gitarchive.py b/meta/lib/oeqa/utils/gitarchive.py index 6e8040eb5c..73beafecb5 100644 --- a/meta/lib/oeqa/utils/gitarchive.py +++ b/meta/lib/oeqa/utils/gitarchive.py @@ -116,7 +116,8 @@ def expand_tag_strings(repo, name_pattern, msg_subj_pattern, msg_body_pattern, tag_re = tag_re.format(tag_number='(?P[0-9]{1,5})') keyws['tag_number'] = 0 - for existing_tag in repo.run_cmd('tag').splitlines(): + tags_refs = repo.run_cmd(['ls-remote', '--refs', '--tags', '-q']) + for existing_tag in ["".join(d.split()[1].split('/', 2)[2:]) for d in tags_refs.splitlines()]: match = re.match(tag_re, existing_tag) if match and int(match.group('tag_number')) >= keyws['tag_number']: @@ -181,7 +182,8 @@ def get_test_runs(log, repo, tag_name, **kwargs): # Get a list of all matching tags tag_pattern = tag_name.format(**str_fields) - tags = repo.run_cmd(['tag', '-l', tag_pattern]).splitlines() + revs = repo.run_cmd(['ls-remote', '--refs', '--tags', 'origin', '-q', tag_pattern]).splitlines() + tags = ["".join(d.split()[1].split('/', 2)[2:]) for d in revs] log.debug("Found %d tags matching pattern '%s'", len(tags), tag_pattern) # Parse undefined fields from tag names