From patchwork Wed Jan 26 19:12:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 2976 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 96F95C63682 for ; Wed, 26 Jan 2022 19:12:58 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mx.groups.io with SMTP id smtpd.web09.17362.1643224377210780441 for ; Wed, 26 Jan 2022 11:12:57 -0800 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: arm.com, ip: 217.140.110.172, mailfrom: ross.burton@arm.com) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 8E0D01FB for ; Wed, 26 Jan 2022 11:12:55 -0800 (PST) Received: from oss-tx204.lab.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 3858C3F7D8 for ; Wed, 26 Jan 2022 11:12:55 -0800 (PST) From: Ross Burton To: meta-arm@lists.yoctoproject.org Subject: [PATCH 1/3] scripts/machine-summary: improve layer_path Date: Wed, 26 Jan 2022 19:12:51 +0000 Message-Id: <20220126191253.1844130-1-ross.burton@arm.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 26 Jan 2022 19:12:58 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/meta-arm/message/2916 As this can get called lots but the data doesn't change, cache the responses. Also return a pathlib.Path, as this is 2022. Signed-off-by: Ross Burton --- scripts/machine-summary.py | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/scripts/machine-summary.py b/scripts/machine-summary.py index 4888a007..c07c2d15 100755 --- a/scripts/machine-summary.py +++ b/scripts/machine-summary.py @@ -30,16 +30,23 @@ def safe_patches(patches): return False return True -def layer_path(layername, d): +def layer_path(layername: str, d) -> pathlib.Path: """ Return the path to the specified layer, or None if the layer isn't present. """ - import re + if not hasattr(layer_path, "cache"): + # Don't use functools.lru_cache as we don't want d changing to invalidate the cache + layer_path.cache = {} + + if layername in layer_path.cache: + return layer_path.cache[layername] + bbpath = d.getVar("BBPATH").split(":") pattern = d.getVar('BBFILE_PATTERN_' + layername) for path in reversed(sorted(bbpath)): if re.match(pattern, path + "/"): - return path + layer_path.cache[layername] = pathlib.Path(path) + return layer_path.cache[layername] return None def extract_patch_info(src_uri, d): From patchwork Wed Jan 26 19:12:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 2977 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id A6979C2BA4C for ; Wed, 26 Jan 2022 19:12:58 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mx.groups.io with SMTP id smtpd.web12.17224.1643224377375971628 for ; Wed, 26 Jan 2022 11:12:57 -0800 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: arm.com, ip: 217.140.110.172, mailfrom: ross.burton@arm.com) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 1F6EA1063 for ; Wed, 26 Jan 2022 11:12:56 -0800 (PST) Received: from oss-tx204.lab.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id BF0A53F7D8 for ; Wed, 26 Jan 2022 11:12:55 -0800 (PST) From: Ross Burton To: meta-arm@lists.yoctoproject.org Subject: [PATCH 2/3] scripts/machine-summary: link patches to cgit Date: Wed, 26 Jan 2022 19:12:52 +0000 Message-Id: <20220126191253.1844130-2-ross.burton@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220126191253.1844130-1-ross.burton@arm.com> References: <20220126191253.1844130-1-ross.burton@arm.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 26 Jan 2022 19:12:58 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/meta-arm/message/2915 Add links from the patch names to the patches in cgit for ease of inspection. Signed-off-by: Ross Burton --- scripts/machine-summary.py | 20 +++++++++++++++++--- scripts/report-details.html.jinja | 7 +++++-- 2 files changed, 22 insertions(+), 5 deletions(-) diff --git a/scripts/machine-summary.py b/scripts/machine-summary.py index c07c2d15..47d8ff34 100755 --- a/scripts/machine-summary.py +++ b/scripts/machine-summary.py @@ -49,6 +49,19 @@ def layer_path(layername: str, d) -> pathlib.Path: return layer_path.cache[layername] return None +def get_url_for_patch(layer: str, localpath: pathlib.Path, d) -> str: + relative = localpath.relative_to(layer_path(layer, d)) + + # TODO: use layerindexlib + # TODO: assumes default branch + if layer == "core": + return f"https://git.openembedded.org/openembedded-core/tree/meta/{relative}" + elif layer in ("meta-arm", "meta-arm-bsp", "arm-toolchain", "meta-atp", "meta-gem5"): + return f"https://git.yoctoproject.org/meta-arm/tree/{layer}/{relative}" + else: + print(f"WARNING: Don't know web URL for layer {layer}", file=sys.stderr) + return None + def extract_patch_info(src_uri, d): """ Parse the specified patch entry from a SRC_URI and return (base name, layer name, status) tuple @@ -56,9 +69,10 @@ def extract_patch_info(src_uri, d): import bb.fetch, bb.utils info = {} - localpath = bb.fetch.decodeurl(src_uri)[2] - info["name"] = os.path.basename(localpath) - info["layer"] = bb.utils.get_file_layer(localpath, d) + localpath = pathlib.Path(bb.fetch.decodeurl(src_uri)[2]) + info["name"] = localpath.name + info["layer"] = bb.utils.get_file_layer(str(localpath), d) + info["url"] = get_url_for_patch(info["layer"], localpath, d) status = "Unknown" with open(localpath, errors="ignore") as f: diff --git a/scripts/report-details.html.jinja b/scripts/report-details.html.jinja index 19ede510..a656c261 100644 --- a/scripts/report-details.html.jinja +++ b/scripts/report-details.html.jinja @@ -47,8 +47,11 @@ {% for pinfo in data.patches %} - - {{ pinfo.name }} + + {% if pinfo.url %}{% endif %} + {{ pinfo.name }} + {% if pinfo.url %}{% endif %} + {{ pinfo.layer }} {{ make_patch_tag(pinfo.status)}} From patchwork Wed Jan 26 19:12:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 2975 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 96299C28CF5 for ; Wed, 26 Jan 2022 19:12:58 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mx.groups.io with SMTP id smtpd.web11.17854.1643224377378615561 for ; Wed, 26 Jan 2022 11:12:57 -0800 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: arm.com, ip: 217.140.110.172, mailfrom: ross.burton@arm.com) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A5CBE11B3 for ; Wed, 26 Jan 2022 11:12:56 -0800 (PST) Received: from oss-tx204.lab.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 519343F7D8 for ; Wed, 26 Jan 2022 11:12:56 -0800 (PST) From: Ross Burton To: meta-arm@lists.yoctoproject.org Subject: [PATCH 3/3] CI: enable memory-resident bitbake Date: Wed, 26 Jan 2022 19:12:53 +0000 Message-Id: <20220126191253.1844130-3-ross.burton@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220126191253.1844130-1-ross.burton@arm.com> References: <20220126191253.1844130-1-ross.burton@arm.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 26 Jan 2022 19:12:58 -0000 X-Groupsio-URL: https://lists.yoctoproject.org/g/meta-arm/message/2917 Enable memory-resident bitbake by setting the server timeout to 60s. For most builds this doesn't have an impact as bitbake is ran once, but this reduces the time for the pending-upgrades report from 10 minutes to 3 minutes, as it doesn't need to constantly restart bitbake. Signed-off-by: Ross Burton --- ci/base.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/ci/base.yml b/ci/base.yml index 0af0b62d..795bd11f 100644 --- a/ci/base.yml +++ b/ci/base.yml @@ -26,6 +26,7 @@ env: local_conf_header: base: | + BB_SERVER_TIMEOUT = "60" CONF_VERSION = "2" PACKAGE_CLASSES = "package_ipk" LICENSE_FLAGS_WHITELIST += "armcompiler Arm-FVP-EULA"