From patchwork Tue Feb 14 16:46:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 19548 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 98D31C05027 for ; Tue, 14 Feb 2023 16:46:25 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mx.groups.io with SMTP id smtpd.web11.12280.1676393181240225761 for ; Tue, 14 Feb 2023 08:46:21 -0800 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: arm.com, ip: 217.140.110.172, mailfrom: ross.burton@arm.com) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 77AE01042; Tue, 14 Feb 2023 08:47:03 -0800 (PST) Received: from oss-tx204.lab.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 5F38E3F881; Tue, 14 Feb 2023 08:46:20 -0800 (PST) From: Ross Burton To: openembedded-core@lists.openembedded.org Cc: nd@arm.com Subject: [PATCH 1/2] lib/buildstats: handle tasks that never finished Date: Tue, 14 Feb 2023 16:46:14 +0000 Message-Id: <20230214164615.2793394-1-ross.burton@arm.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 14 Feb 2023 16:46:25 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/177158 If a task is aborted the buildstats file isn't complete, so calculate when the build finished and use that as a end time. Signed-off-by: Ross Burton --- scripts/lib/buildstats.py | 33 +++++++++++++++++++++++++++------ 1 file changed, 27 insertions(+), 6 deletions(-) diff --git a/scripts/lib/buildstats.py b/scripts/lib/buildstats.py index 3b76286ba57..fa94c655399 100644 --- a/scripts/lib/buildstats.py +++ b/scripts/lib/buildstats.py @@ -79,8 +79,8 @@ class BSTask(dict): return self['rusage']['ru_oublock'] @classmethod - def from_file(cls, buildstat_file): - """Read buildstat text file""" + def from_file(cls, buildstat_file, fallback_end=0): + """Read buildstat text file. fallback_end is an optional end time for tasks that are not recorded as finishing.""" bs_task = cls() log.debug("Reading task buildstats from %s", buildstat_file) end_time = None @@ -108,7 +108,10 @@ class BSTask(dict): bs_task[ru_type][ru_key] = val elif key == 'Status': bs_task['status'] = val - if end_time is not None and start_time is not None: + # If the task didn't finish, fill in the fallback end time if specified + if start_time and not end_time and fallback_end: + end_time = fallback_end + if start_time and end_time: bs_task['elapsed_time'] = end_time - start_time else: raise BSError("{} looks like a invalid buildstats file".format(buildstat_file)) @@ -226,15 +229,33 @@ class BuildStats(dict): epoch = match.group('epoch') return name, epoch, version, revision + @staticmethod + def parse_top_build_stats(path): + """ + Parse the top-level build_stats file for build-wide start and duration. + """ + with open(path) as fobj: + for line in fobj.readlines(): + key, val = line.split(':', 1) + val = val.strip() + if key == 'Build Started': + start = float(val) + elif key == "Elapsed time": + elapsed = float(val.split()[0]) + return start, elapsed + @classmethod def from_dir(cls, path): """Load buildstats from a buildstats directory""" - if not os.path.isfile(os.path.join(path, 'build_stats')): + top_stats = os.path.join(path, 'build_stats') + if not os.path.isfile(top_stats): raise BSError("{} does not look like a buildstats directory".format(path)) log.debug("Reading buildstats directory %s", path) - buildstats = cls() + build_started, build_elapsed = buildstats.parse_top_build_stats(top_stats) + build_end = build_started + build_elapsed + subdirs = os.listdir(path) for dirname in subdirs: recipe_dir = os.path.join(path, dirname) @@ -244,7 +265,7 @@ class BuildStats(dict): bsrecipe = BSRecipe(name, epoch, version, revision) for task in os.listdir(recipe_dir): bsrecipe.tasks[task] = BSTask.from_file( - os.path.join(recipe_dir, task)) + os.path.join(recipe_dir, task), build_end) if name in buildstats: raise BSError("Cannot handle multiple versions of the same " "package ({})".format(name)) From patchwork Tue Feb 14 16:46:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ross Burton X-Patchwork-Id: 19547 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9AF7AC6379F for ; Tue, 14 Feb 2023 16:46:25 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mx.groups.io with SMTP id smtpd.web11.12281.1676393182006836978 for ; Tue, 14 Feb 2023 08:46:22 -0800 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: arm.com, ip: 217.140.110.172, mailfrom: ross.burton@arm.com) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 2C818150C; Tue, 14 Feb 2023 08:47:04 -0800 (PST) Received: from oss-tx204.lab.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 0C5EC3F881; Tue, 14 Feb 2023 08:46:20 -0800 (PST) From: Ross Burton To: openembedded-core@lists.openembedded.org Cc: nd@arm.com Subject: [PATCH 2/2] cml1: remove redundand addtask Date: Tue, 14 Feb 2023 16:46:15 +0000 Message-Id: <20230214164615.2793394-2-ross.burton@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230214164615.2793394-1-ross.burton@arm.com> References: <20230214164615.2793394-1-ross.burton@arm.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 14 Feb 2023 16:46:25 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/177159 The configure task is added by base.bbclass, no need to do it again. Signed-off-by: Ross Burton --- meta/classes-recipe/cml1.bbclass | 1 - 1 file changed, 1 deletion(-) diff --git a/meta/classes-recipe/cml1.bbclass b/meta/classes-recipe/cml1.bbclass index b79091383de..a09a042c3f1 100644 --- a/meta/classes-recipe/cml1.bbclass +++ b/meta/classes-recipe/cml1.bbclass @@ -21,7 +21,6 @@ cml1_do_configure() { } EXPORT_FUNCTIONS do_configure -addtask configure after do_unpack do_patch before do_compile inherit terminal