From patchwork Mon Sep 25 08:04:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Stephan X-Patchwork-Id: 31083 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 91695CE7A81 for ; Mon, 25 Sep 2023 08:05:09 +0000 (UTC) Received: from mail-lf1-f51.google.com (mail-lf1-f51.google.com [209.85.167.51]) by mx.groups.io with SMTP id smtpd.web11.56862.1695629099436579098 for ; Mon, 25 Sep 2023 01:04:59 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@baylibre-com.20230601.gappssmtp.com header.s=20230601 header.b=Zl8hbU6J; spf=pass (domain: baylibre.com, ip: 209.85.167.51, mailfrom: jstephan@baylibre.com) Received: by mail-lf1-f51.google.com with SMTP id 2adb3069b0e04-50435a9f800so7766382e87.2 for ; Mon, 25 Sep 2023 01:04:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=baylibre-com.20230601.gappssmtp.com; s=20230601; t=1695629097; x=1696233897; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=G3C+hcvu4p1bTarZp6dpkHzO9zosb4NF0Oh21d4H9t4=; b=Zl8hbU6Jijib/iq43vF2kvAkgskHnlFNPySI4KoFg3BoBaXMlPHfuCbXg5JYijFbYN EzJ30ZkfliK6LXZLvSx6lukUo5CvQD2V+KGH5Evw1CZGornK1ZnH5Srj2wxZywMT/diu v1YrtYCxgoORuo4+YzY6+e2C0pn9ADeL5pVndp2DbjiVYKPxUbfWtM2TNPQvVVXQEid0 Nuv7p6TZfYEFu3Ptz73S5e0tqHLhUaQSnh0xB8l68CDGAcqzpX+NG+v9QHCdakvION4B Bebvc/QtymwXB2D1N2ev8cdhAh+MDkgfUuyIL1Xs7tjUvW4ky4mICmTKNFuC4RQAlqqp a29Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695629097; x=1696233897; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=G3C+hcvu4p1bTarZp6dpkHzO9zosb4NF0Oh21d4H9t4=; b=OTuARqdL70tGDHwKaleYsckX9ki6Ta85oaAas2NUn6YaMPJlEG1miYA3/xGZIVJzKE TH6h9kw34m7Y8VidnhggXfqXQzp+19DJ1gyfjfstawiI2ZKgezo3S59I2QZDMSIxjPpX 1VVEkrNydq35ihx2l3hPfKKz51duPkyZqaNYv8qxeSabpOvE5GaKTIF7tClQC5OTYzHj 3OKJXErKrF9QuFGHCf0jKSBFNFzzlVJRRI+KmWaDQ2aRZzuNyoyEIesrDbyTofSIKIFs QIj0ddtBN64WkIXFtfNcro9BMLdypf9qgNAr1fXVcB9kH4OvnX7i4nBtjPumgrUWdSYA rQjQ== X-Gm-Message-State: AOJu0YxP0QlrKATWNkrHRrrNHU6opuxvvEoIaG6QIPuNrKaanqMaeR6m vEukMGLLPQfEsETj+nPMNulW/qGt4E59Y3a+QRg= X-Google-Smtp-Source: AGHT+IF1YdKOCUVshNicGcwgz3P8A2BxqAiISvIdxJYWC9FiE4IrH7wRkBRlQMK0eef9K/jiIAkvSg== X-Received: by 2002:ac2:5b9c:0:b0:503:2877:67d3 with SMTP id o28-20020ac25b9c000000b00503287767d3mr4299201lfn.67.1695629097289; Mon, 25 Sep 2023 01:04:57 -0700 (PDT) Received: from localhost.localdomain ([2a01:e0a:55f:21e0:9e19:4376:dea6:dbfa]) by smtp.gmail.com with ESMTPSA id q12-20020a05600c040c00b0040586360a36sm3196950wmb.17.2023.09.25.01.04.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Sep 2023 01:04:57 -0700 (PDT) From: Julien Stephan To: openembedded-core@lists.openembedded.org Cc: Julien Stephan Subject: [PATCH v5 1/5] bitbake.conf: include bblock.conf Date: Mon, 25 Sep 2023 10:04:48 +0200 Message-ID: <20230925080452.803540-2-jstephan@baylibre.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230925080452.803540-1-jstephan@baylibre.com> References: <20230925080452.803540-1-jstephan@baylibre.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Mon, 25 Sep 2023 08:05:09 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/188175 include conf/bblock.conf. This file is generated by the bblock tool. It locks some package tasks by fixing their signatures. See bblock -h for more details Signed-off-by: Julien Stephan --- meta/conf/bitbake.conf | 1 + 1 file changed, 1 insertion(+) diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf index 67577c8fe7e..6a8deee4f72 100644 --- a/meta/conf/bitbake.conf +++ b/meta/conf/bitbake.conf @@ -829,6 +829,7 @@ include conf/documentation.conf include conf/licenses.conf require conf/sanity.conf require conf/cve-check-map.conf +include conf/bblock.conf ################################################################## # Weak variables (usually to retain backwards compatibility) From patchwork Mon Sep 25 08:04:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Stephan X-Patchwork-Id: 31084 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 94312CE7A8C for ; Mon, 25 Sep 2023 08:05:09 +0000 (UTC) Received: from mail-wm1-f48.google.com (mail-wm1-f48.google.com [209.85.128.48]) by mx.groups.io with SMTP id smtpd.web11.56861.1695629099432561881 for ; Mon, 25 Sep 2023 01:05:00 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@baylibre-com.20230601.gappssmtp.com header.s=20230601 header.b=BSxWOrPs; spf=pass (domain: baylibre.com, ip: 209.85.128.48, mailfrom: jstephan@baylibre.com) Received: by mail-wm1-f48.google.com with SMTP id 5b1f17b1804b1-405524e6769so20433575e9.1 for ; Mon, 25 Sep 2023 01:04:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=baylibre-com.20230601.gappssmtp.com; s=20230601; t=1695629098; x=1696233898; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=PKDGjOP4KLL8O//oIraloqqh4cprsxG1ID2VXIAZE04=; b=BSxWOrPs2yV8eqcofZoTv+jgCxVM7KvMMnrJOXV8o6tgW2ypyM9+Ou6DjCDUyQKOzL 04poaO6r9qP9oSUFgEdnlYrqoxEK3aZL0uvH7k8TsXRxvZlRiLGsVGFs8rdS+yhQNTKQ VVv9zxo5JXLfz2NUfqiSx4PpWQj4+FZ5y0cVMLl9JT52p4X1QMZKKybJ08vOyCTJ2iSk G6wDQ7xgvFm8TXa8KzGgP/xZBcUDtEQNS7JfrhliIREZNSNoGTG82SP8hhilS+Nihta5 55N9sWv21FdL7cl51Jpnj5wR0ATgJt1tRLKgDdasRNBv6uczlNHLLpP/+tmvIDocWnAc AiEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695629098; x=1696233898; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=PKDGjOP4KLL8O//oIraloqqh4cprsxG1ID2VXIAZE04=; b=WyX7Oubg5tokrAyCQpDsd/6vKd3pfhJ3PUXgDJ54xbQoYyw0wUSbJlgLnl7IcFM93V C9ORWUlzhyIOiAJWwxWejBuMEgvNuxKmYo/Ht2RFDMsPz8rjKodHDxFVYRyt4BELMKYZ h3GrJ3gBQj9SLCYMxAffSiBEKc786xkFPZ3yaQZ2Upwp/AnPSk8GP6eEpe4zZmMf5H0w caZciOoGo5hbgk08Mfh3CKIm377cz4+I4OLS7YBs13fYOZmnmEvzCCSMq+HFF1jgdKPc gQ8pFqTpdd9Fmif9oFuQVvqIX4oHOc2XANW4Ua6kbUX6FKWcDCcyZVnD/3wI6bYAsC3Q bcYQ== X-Gm-Message-State: AOJu0Yxeovy3LaLMEqweGCHGtUpyNPd5NhT93SYwVsEdUAgqWzudqO7B dgI9AMXAmEE2+zyM2Md29rtHh7JReQHx9ZTAojw= X-Google-Smtp-Source: AGHT+IGpbsM12tTg4jnQbM+ciB7zF0vyFxLDJy/wIC3k3+zejfTKIC76VpBgjeDaCbR3TXaWLXVWbA== X-Received: by 2002:a05:600c:310f:b0:405:358c:ba74 with SMTP id g15-20020a05600c310f00b00405358cba74mr6920252wmo.0.1695629097820; Mon, 25 Sep 2023 01:04:57 -0700 (PDT) Received: from localhost.localdomain ([2a01:e0a:55f:21e0:9e19:4376:dea6:dbfa]) by smtp.gmail.com with ESMTPSA id q12-20020a05600c040c00b0040586360a36sm3196950wmb.17.2023.09.25.01.04.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Sep 2023 01:04:57 -0700 (PDT) From: Julien Stephan To: openembedded-core@lists.openembedded.org Cc: Julien Stephan Subject: [PATCH v5 2/5] bitbake: cooker: add a new function to retrieve task signatures Date: Mon, 25 Sep 2023 10:04:49 +0200 Message-ID: <20230925080452.803540-3-jstephan@baylibre.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230925080452.803540-1-jstephan@baylibre.com> References: <20230925080452.803540-1-jstephan@baylibre.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Mon, 25 Sep 2023 08:05:09 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/188176 adding a new command in cooker to compute and get task signatures this commit also add the associated command and event needed to get the signatures using tinfoil Signed-off-by: Julien Stephan --- bitbake/lib/bb/command.py | 6 ++++++ bitbake/lib/bb/cooker.py | 31 +++++++++++++++++++++++++++++++ bitbake/lib/bb/event.py | 8 ++++++++ 3 files changed, 45 insertions(+) diff --git a/bitbake/lib/bb/command.py b/bitbake/lib/bb/command.py index 8663eed9331..f2ee5871616 100644 --- a/bitbake/lib/bb/command.py +++ b/bitbake/lib/bb/command.py @@ -781,3 +781,9 @@ class CommandsAsync: bb.event.fire(bb.event.FindSigInfoResult(res), command.cooker.databuilder.mcdata[mc]) command.finishAsyncCommand() findSigInfo.needcache = False + + def getTaskSignatures(self, command, params): + res = command.cooker.getTaskSignatures(params[0], params[1]) + bb.event.fire(bb.event.GetTaskSignatureResult(res), command.cooker.data) + command.finishAsyncCommand() + getTaskSignatures.needcache = True diff --git a/bitbake/lib/bb/cooker.py b/bitbake/lib/bb/cooker.py index 87aa71bb657..599c7ddaa28 100644 --- a/bitbake/lib/bb/cooker.py +++ b/bitbake/lib/bb/cooker.py @@ -1442,6 +1442,37 @@ class BBCooker: self.idleCallBackRegister(buildFileIdle, rq) + def getTaskSignatures(self, target, tasks): + sig = [] + getAllTaskSignatures = False + + if not tasks: + tasks = ["do_build"] + getAllTaskSignatures = True + + for task in tasks: + taskdata, runlist = self.buildTaskData(target, task, self.configuration.halt) + rq = bb.runqueue.RunQueue(self, self.data, self.recipecaches, taskdata, runlist) + rq.rqdata.prepare() + + for l in runlist: + mc, pn, taskname, fn = l + + taskdep = rq.rqdata.dataCaches[mc].task_deps[fn] + for t in taskdep['tasks']: + if t in taskdep['nostamp'] or "setscene" in t: + continue + tid = bb.runqueue.build_tid(mc, fn, t) + + if t in task or getAllTaskSignatures: + try: + rq.rqdata.prepare_task_hash(tid) + sig.append([pn, t, rq.rqdata.get_task_unihash(tid)]) + except KeyError: + sig.append(self.getTaskSignatures(target, [t])[0]) + + return sig + def buildTargets(self, targets, task): """ Attempt to build the targets specified diff --git a/bitbake/lib/bb/event.py b/bitbake/lib/bb/event.py index 0d0e0a68aac..f8acacd80d1 100644 --- a/bitbake/lib/bb/event.py +++ b/bitbake/lib/bb/event.py @@ -857,6 +857,14 @@ class FindSigInfoResult(Event): Event.__init__(self) self.result = result +class GetTaskSignatureResult(Event): + """ + Event to return results from GetTaskSignatures command + """ + def __init__(self, sig): + Event.__init__(self) + self.sig = sig + class ParseError(Event): """ Event to indicate parse failed From patchwork Mon Sep 25 08:04:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Stephan X-Patchwork-Id: 31087 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9CA24CE7A95 for ; Mon, 25 Sep 2023 08:05:09 +0000 (UTC) Received: from mail-wm1-f45.google.com (mail-wm1-f45.google.com [209.85.128.45]) by mx.groups.io with SMTP id smtpd.web11.56864.1695629100135755391 for ; Mon, 25 Sep 2023 01:05:00 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@baylibre-com.20230601.gappssmtp.com header.s=20230601 header.b=OeUQRjmo; spf=pass (domain: baylibre.com, ip: 209.85.128.45, mailfrom: jstephan@baylibre.com) Received: by mail-wm1-f45.google.com with SMTP id 5b1f17b1804b1-405361bb93bso55225665e9.3 for ; Mon, 25 Sep 2023 01:04:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=baylibre-com.20230601.gappssmtp.com; s=20230601; t=1695629098; x=1696233898; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=BHTP2xUV0AoiWxyMWH18f9u1yEcIKbYCSeqDXUokSwM=; b=OeUQRjmoIP2DVAycRRMf++H2Gdn3je/nqpD0G0UyaETzHb+DU3df7QC2DOD7tIImXs VxHzUcQ27M2cpV6j4YhwreH9zVN9XbdmSzt31EN7Ia5tyeT6BTKXPl9gzTn45dZu8mUp 8zLopa4pQae/duSvLectKUfeGtcAMBx4VmSL9nCCX0fi2sRudmXA0sl/Jbr/t0vs1l23 nx4TQbv71vrqffnUWnrObr2I4PfFlH8Oe4ViJtTGMB1hiB6HNs/VRqoCa5fJSR78UCZY x5n7MTgWRrNqzkd0WpQHTW1DLyMUlvc5PBotxOZp3Yg56zbIIgi1oBaxdPvaduKRwnQe V2Jw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695629098; x=1696233898; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BHTP2xUV0AoiWxyMWH18f9u1yEcIKbYCSeqDXUokSwM=; b=RJE5N6qAfpL6FDtVlJpznLGD5V79oanWm69VN6QppR1iL5OQRnbkRQGaBwAVmY0SDC 18Pph6z2tsiL7V5QUuY8BTwMf5yBNVfOFQEPUNb0xJzJqvinEu0F+GogNX30eKNzQrX8 GqyWSHiHjjDYLHoSkg4mvY+I345EgNsY4MCwg8OIxw4wtbwrDGiyPa6HwMIlTK9EO26N wZ/VSfaZyDis3ZYlTZdH6r1kLpRM5EnKanQkxLq8RDYOroKblAd/b9wBghXe9hS/puIe itKfwe84ju656IxtYlb05tge580cXox91/p5FOom50dGQCUiU+bEFFxPAv3uhLiW7Q5O me1w== X-Gm-Message-State: AOJu0YwVptTNvvbUTyE2rqJpkxQrk6aYZGrLqXuLjPkIqH007bU7ugfJ 6dYHYt2A5ZgguzPPNsD0ni48e4Z41ambbTmkhEU= X-Google-Smtp-Source: AGHT+IHYyMnxmHhtwVukKGm6cWuC517ULfu6E2wMQUUNXECGJQ2xLFR9pDT3Gl/QPQkqf59R23JqwA== X-Received: by 2002:a05:600c:861b:b0:405:95c3:e79d with SMTP id ha27-20020a05600c861b00b0040595c3e79dmr379387wmb.40.1695629098415; Mon, 25 Sep 2023 01:04:58 -0700 (PDT) Received: from localhost.localdomain ([2a01:e0a:55f:21e0:9e19:4376:dea6:dbfa]) by smtp.gmail.com with ESMTPSA id q12-20020a05600c040c00b0040586360a36sm3196950wmb.17.2023.09.25.01.04.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Sep 2023 01:04:58 -0700 (PDT) From: Julien Stephan To: openembedded-core@lists.openembedded.org Cc: Julien Stephan Subject: [PATCH v5 3/5] sstatesig: add a new info level for SIGGEN_LOCKEDSIGS_TASKSIG_CHECK Date: Mon, 25 Sep 2023 10:04:50 +0200 Message-ID: <20230925080452.803540-4-jstephan@baylibre.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230925080452.803540-1-jstephan@baylibre.com> References: <20230925080452.803540-1-jstephan@baylibre.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Mon, 25 Sep 2023 08:05:09 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/188177 as of now, SIGGEN_LOCKEDSIGS_TASKSIG_CHECK can take 2 values: "warn" and "error", displaying respectively a warning or a fatal error message only when a task is locked and the task signature is different from the locked one. The "info" level is introduced to add a "note" message to remind the user that a recipe is locked even if the signature is equivalent to the locked one. The "warn" and "error" level display the warn/error message for each task having a mismatch of the signature. Doing this with the "info" level would result in very verbose output if there are several tasks locked, so the info level will only print once the list of recipes that have locked signature. Signed-off-by: Julien Stephan --- meta/lib/oe/sstatesig.py | 18 +++++++++++++++++- 1 file changed, 17 insertions(+), 1 deletion(-) diff --git a/meta/lib/oe/sstatesig.py b/meta/lib/oe/sstatesig.py index 633a0fd4502..5bf1697e727 100644 --- a/meta/lib/oe/sstatesig.py +++ b/meta/lib/oe/sstatesig.py @@ -104,6 +104,8 @@ class SignatureGeneratorOEBasicHashMixIn(object): self.lockedhashfn = {} self.machine = data.getVar("MACHINE") self.mismatch_msgs = [] + self.mismatch_number = 0 + self.lockedsigs_msgs = "" self.unlockedrecipes = (data.getVar("SIGGEN_UNLOCKED_RECIPES") or "").split() self.unlockedrecipes = { k: "" for k in self.unlockedrecipes } @@ -187,6 +189,7 @@ class SignatureGeneratorOEBasicHashMixIn(object): #bb.warn("Using %s %s %s" % (recipename, task, h)) if h != h_locked and h_locked != unihash: + self.mismatch_number += 1 self.mismatch_msgs.append('The %s:%s sig is computed to be %s, but the sig is locked to %s in %s' % (recipename, task, h, h_locked, var)) @@ -267,6 +270,15 @@ class SignatureGeneratorOEBasicHashMixIn(object): warn_msgs = [] error_msgs = [] sstate_missing_msgs = [] + info_msgs = None + + if self.lockedsigs: + if len(self.lockedsigs) > 10: + self.lockedsigs_msgs = "There are %s recipes with locked tasks (%s task(s) have non matching signature)" % (len(self.lockedsigs), self.mismatch_number) + else: + self.lockedsigs_msgs = "The following recipes have locked tasks:" + for pn in self.lockedsigs: + self.lockedsigs_msgs += " %s" % (pn) for tid in sq_data['hash']: if tid not in found: @@ -279,7 +291,9 @@ class SignatureGeneratorOEBasicHashMixIn(object): % (pn, taskname, sq_data['hash'][tid])) checklevel = d.getVar("SIGGEN_LOCKEDSIGS_TASKSIG_CHECK") - if checklevel == 'warn': + if checklevel == 'info': + info_msgs = self.lockedsigs_msgs + if checklevel == 'warn' or checklevel == 'info': warn_msgs += self.mismatch_msgs elif checklevel == 'error': error_msgs += self.mismatch_msgs @@ -290,6 +304,8 @@ class SignatureGeneratorOEBasicHashMixIn(object): elif checklevel == 'error': error_msgs += sstate_missing_msgs + if info_msgs: + bb.note(info_msgs) if warn_msgs: bb.warn("\n".join(warn_msgs)) if error_msgs: From patchwork Mon Sep 25 08:04:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Stephan X-Patchwork-Id: 31086 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id AB4FCCE7A89 for ; Mon, 25 Sep 2023 08:05:09 +0000 (UTC) Received: from mail-wm1-f42.google.com (mail-wm1-f42.google.com [209.85.128.42]) by mx.groups.io with SMTP id smtpd.web10.56258.1695629100921640653 for ; Mon, 25 Sep 2023 01:05:01 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@baylibre-com.20230601.gappssmtp.com header.s=20230601 header.b=2M5RT3qp; spf=pass (domain: baylibre.com, ip: 209.85.128.42, mailfrom: jstephan@baylibre.com) Received: by mail-wm1-f42.google.com with SMTP id 5b1f17b1804b1-3ff1c397405so66120115e9.3 for ; Mon, 25 Sep 2023 01:05:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=baylibre-com.20230601.gappssmtp.com; s=20230601; t=1695629099; x=1696233899; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=bgepOdc7k1McPs079y0Ee/esHUoC5+fmU7aVGlyMQAk=; b=2M5RT3qpknURrETna7r5xOxEY6e53wC/fQqHZ0SPWvXMY05Hgzs/zSZb4YjkzxhPd2 rGpd+RRSetyakIXz6qj2tc2Vi8H6saZGGSlCVrTbHGyPvDj5WKnthW+8ZDjnsuitMRF9 ilDw8K2ErkQKtgn7tE84hGBO4+6jMIpmFGY4kY+dlpkAoTMLPYeVsL9QJeZ3aqbO13TY UUYv8SzANYByGzoJ77CfFbjhYeZuUdtDckMKozl0i6yxZAHSdrtLgMAe8M/ENRvNlEdj NLKZxoeCSK9HtxHhsMpgtjc50IsFpGrTw9cR4MFdqcgHmJWUiaLIAzl8Uha4JthasjuJ gfUw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695629099; x=1696233899; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=bgepOdc7k1McPs079y0Ee/esHUoC5+fmU7aVGlyMQAk=; b=UoOvhfvUuKPDeLMHO+V/9cOyxVqUcWV+CH1Y9Ri35utS4zT7NIxRWwgUCLJxBW2atw 4MMDfeCU7h1Uq5hmdDaj4BnTsg42XtCxIeZxoPAxgO8HlkSNnAKpLssbRaRc4QRMhsa8 uUdi+XlCfdIqjgJwWuW08j1wfomXoI0021r4SceXAMSIQeXnQspm/O2SOAJtYXUMBSuU Bkg0rTRhvep1clE4g6BeZwu8CjplLYCg1+64zTp86vllKRwitrcwrIHTpZx0oEbzw06O 7S35h4/TUoze0I5RVQaMmM5ND1ijwpFFWSZrm0F1gTxuXvDQXbXKB5sBT9l5iZK542S6 rbZg== X-Gm-Message-State: AOJu0YyNnXVATHCQoSmgjOvL9IjDBbfX7+JgctYHh/RS7rcPY/CMul3R 4y6E6d3o9pnx/ejBXlpHkzpoz01Sq8FbdttzNoI= X-Google-Smtp-Source: AGHT+IEfG7bpcnk37Ad1UkNPIGwhG8MLyCFORNYS/icOQieGC5oGyaoK/Sv+Yz8MPj26VLnopVNvkw== X-Received: by 2002:a05:600c:2a43:b0:403:b64:ad0a with SMTP id x3-20020a05600c2a4300b004030b64ad0amr4336749wme.26.1695629099087; Mon, 25 Sep 2023 01:04:59 -0700 (PDT) Received: from localhost.localdomain ([2a01:e0a:55f:21e0:9e19:4376:dea6:dbfa]) by smtp.gmail.com with ESMTPSA id q12-20020a05600c040c00b0040586360a36sm3196950wmb.17.2023.09.25.01.04.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Sep 2023 01:04:58 -0700 (PDT) From: Julien Stephan To: openembedded-core@lists.openembedded.org Cc: Julien Stephan Subject: [PATCH v5 4/5] scripts/bblock: add a script to lock/unlock recipes Date: Mon, 25 Sep 2023 10:04:51 +0200 Message-ID: <20230925080452.803540-5-jstephan@baylibre.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230925080452.803540-1-jstephan@baylibre.com> References: <20230925080452.803540-1-jstephan@baylibre.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Mon, 25 Sep 2023 08:05:09 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/188178 bblock script allows to lock/unlock recipes to latest task signatures. The idea is to prevent some recipes to be rebuilt during development. For example when working on rust recipe, one may not want rust-native to be rebuilt. This tool can be used, with proper environment set up, using the following command: bblock See help for more details if a 's task signature change, this task will not be built again and sstate cache will be used. [YOCTO #13425] Signed-off-by: Julien Stephan --- scripts/bblock | 184 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 184 insertions(+) create mode 100755 scripts/bblock diff --git a/scripts/bblock b/scripts/bblock new file mode 100755 index 00000000000..0082059af81 --- /dev/null +++ b/scripts/bblock @@ -0,0 +1,184 @@ +#!/usr/bin/env python3 +# bblock +# lock/unlock task to latest signature +# +# Copyright (c) 2023 BayLibre, SAS +# Author: Julien Stepahn +# +# SPDX-License-Identifier: GPL-2.0-only +# + +import os +import sys +import logging + +scripts_path = os.path.dirname(os.path.realpath(__file__)) +lib_path = scripts_path + "/lib" +sys.path = sys.path + [lib_path] + +import scriptpath + +scriptpath.add_bitbake_lib_path() + +import bb.tinfoil +import bb.msg + +import argparse_oe + +myname = os.path.basename(sys.argv[0]) +logger = bb.msg.logger_create(myname) + + +def getTaskSignatures(tinfoil, pn, tasks): + tinfoil.set_event_mask( + [ + "bb.event.GetTaskSignatureResult", + "logging.LogRecord", + "bb.command.CommandCompleted", + "bb.command.CommandFailed", + ] + ) + ret = tinfoil.run_command("getTaskSignatures", pn, tasks) + if ret: + while True: + event = tinfoil.wait_event(1) + if event: + if isinstance(event, bb.command.CommandCompleted): + break + elif isinstance(event, bb.command.CommandFailed): + logger.error(str(event)) + sys.exit(2) + elif isinstance(event, bb.event.GetTaskSignatureResult): + sig = event.sig + elif isinstance(event, logging.LogRecord): + logger.handle(event) + else: + logger.error("No result returned from getTaskSignatures command") + sys.exit(2) + return sig + + +def parseRecipe(tinfoil, recipe): + try: + tinfoil.parse_recipes() + d = tinfoil.parse_recipe(recipe) + except Exception: + logger.error("Failed to get recipe info for: %s" % recipe) + sys.exit(1) + return d + + +def bblockDump(lockfile): + try: + with open(lockfile, "r") as lockfile: + for line in lockfile: + print(line.strip()) + except IOError: + return 1 + return 0 + + +def bblockReset(lockfile, pns, package_archs, tasks): + if not pns: + logger.info("Unlocking all recipes") + try: + os.remove(lockfile) + except FileNotFoundError: + pass + else: + logger.info("Unlocking {pns}".format(pns=pns)) + tmp_lockfile = lockfile + ".tmp" + with open(lockfile, "r") as infile, open(tmp_lockfile, "w") as outfile: + for line in infile: + if not ( + any(element in line for element in pns) + and any(element in line for element in package_archs.split()) + ): + outfile.write(line) + else: + if tasks and not any(element in line for element in tasks): + outfile.write(line) + os.remove(lockfile) + os.rename(tmp_lockfile, lockfile) + + +def main(): + parser = argparse_oe.ArgumentParser(description="Lock and unlock a recipe") + parser.add_argument("pn", nargs="*", help="Space separated list of recipe to lock") + parser.add_argument( + "-t", + "--tasks", + help="Comma separated list of tasks", + type=lambda s: [ + task if task.startswith("do_") else "do_" + task for task in s.split(",") + ], + ) + parser.add_argument( + "-r", + "--reset", + action="store_true", + help="Unlock pn recipes, or all recipes if pn is empty", + ) + parser.add_argument( + "-d", + "--dump", + action="store_true", + help="Dump generated bblock.conf file", + ) + + global_args, unparsed_args = parser.parse_known_args() + + with bb.tinfoil.Tinfoil() as tinfoil: + tinfoil.prepare(config_only=True) + + package_archs = tinfoil.config_data.getVar("PACKAGE_ARCHS") + builddir = tinfoil.config_data.getVar("TOPDIR") + lockfile = "{builddir}/conf/bblock.conf".format(builddir=builddir) + + if global_args.dump: + bblockDump(lockfile) + return 0 + + if global_args.reset: + bblockReset(lockfile, global_args.pn, package_archs, global_args.tasks) + return 0 + + with open(lockfile, "a") as lockfile: + s = "" + if lockfile.tell() == 0: + s = "# Generated by bblock\n" + s += 'SIGGEN_LOCKEDSIGS_TASKSIG_CHECK = "info"\n' + s += 'SIGGEN_LOCKEDSIGS_TYPES += "${PACKAGE_ARCHS}"\n' + s += "\n" + + for pn in global_args.pn: + d = parseRecipe(tinfoil, pn) + package_arch = d.getVar("PACKAGE_ARCH") + siggen_locked_sigs_package_arch = d.getVar( + "SIGGEN_LOCKEDSIGS_{package_arch}".format(package_arch=package_arch) + ) + sigs = getTaskSignatures(tinfoil, [pn], global_args.tasks) + for sig in sigs: + new_entry = "{pn}:{taskname}:{sig}".format( + pn=sig[0], taskname=sig[1], sig=sig[2] + ) + if ( + siggen_locked_sigs_package_arch + and not new_entry in siggen_locked_sigs_package_arch + ) or not siggen_locked_sigs_package_arch: + s += 'SIGGEN_LOCKEDSIGS_{package_arch} += "{new_entry}"\n'.format( + package_arch=package_arch, new_entry=new_entry + ) + lockfile.write(s) + return 0 + + +if __name__ == "__main__": + try: + ret = main() + except Exception: + ret = 1 + import traceback + + traceback.print_exc() + sys.exit(ret) From patchwork Mon Sep 25 08:04:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Stephan X-Patchwork-Id: 31085 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id ACFC4CE7A97 for ; Mon, 25 Sep 2023 08:05:09 +0000 (UTC) Received: from mail-wr1-f41.google.com (mail-wr1-f41.google.com [209.85.221.41]) by mx.groups.io with SMTP id smtpd.web11.56865.1695629101551783433 for ; Mon, 25 Sep 2023 01:05:01 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@baylibre-com.20230601.gappssmtp.com header.s=20230601 header.b=PrI71RHC; spf=pass (domain: baylibre.com, ip: 209.85.221.41, mailfrom: jstephan@baylibre.com) Received: by mail-wr1-f41.google.com with SMTP id ffacd0b85a97d-323168869daso3230987f8f.2 for ; Mon, 25 Sep 2023 01:05:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=baylibre-com.20230601.gappssmtp.com; s=20230601; t=1695629100; x=1696233900; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=MDGl7H9sTXMwHZMNo4kEhpWMHlVdQNts6Kvuv2MRZgQ=; b=PrI71RHCN4Pke1HyPVdRltxe8ua9feSiUhs2D1mn3J3JQJ59KHaJQnUCkfoGyiy/Vo LnPw1QYo3fhyBbGi58TBTLQQqN28SUVD8zHQ1Lnrgmy//jbmgfKcnXwgml2OD810ZKJf GcaMlP7vdzg/oL4zucSEY+EvjyoVRgwGLoGXI+yCeGKuOq5pTigvaHAePXbSayUEMfhl ljb+qAmGe6vWWFzoNQeWZ49zEm8pfEUYCvNxvjO3kZKYqCJWyUZkhz0lDUbe5PzBQeM7 AjfjbblvdKhxeEtss8BcQlULc31Q1eFCcOoB2NshCfiFf6PjUeUuQWrHs56Dd8TbtgJN vAKg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695629100; x=1696233900; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MDGl7H9sTXMwHZMNo4kEhpWMHlVdQNts6Kvuv2MRZgQ=; b=EETAALc64qnp/Q8LhDSWosTlIVD6L7zvFHBpkq8trYAbTaakt0lvdpEkPxDQua/Zf5 OyncyutjAOJEeOGYWNl3XodiRHmiFrzHv4d5BqLDWTqWMr/oYoMoZMMUWT9vZ2uy3MvG +9RwJQWySbsx0JG3IyI16hRMUfZ4O4vetPEjBXlRt9cthu1AkAv0vyrFp6NsK8k5xvM7 Z/55AJnwpeSH5rrHe8uUrtxVOsHQHzi1zDzV3kcNyYFF37280gVneDUGvbekqA1sfeZA IVZvw/dngeOVQ7mAcuPnpSKX/sbWWXP8sDr1M3KqPLft6umzhiLbHtz5zpOR0+9iWP8H eB3w== X-Gm-Message-State: AOJu0YzE33dom0KUivbZPVAN/WwnNqr7fxy8cdRnmqBVF9Sr7It74XLB tVzB/RTH+EAvZy4dx0mcSCVbpmYwSHzpWKUmUcE= X-Google-Smtp-Source: AGHT+IFEMBb8yEOnYlp9QoHNlsHNQ7fCpObnW9w20ZX0AoG7/sZdfqXysVb1ubEKvrlq/9fVhYsb9Q== X-Received: by 2002:a5d:4d83:0:b0:31a:d4d0:6e98 with SMTP id b3-20020a5d4d83000000b0031ad4d06e98mr4894978wru.8.1695629099873; Mon, 25 Sep 2023 01:04:59 -0700 (PDT) Received: from localhost.localdomain ([2a01:e0a:55f:21e0:9e19:4376:dea6:dbfa]) by smtp.gmail.com with ESMTPSA id q12-20020a05600c040c00b0040586360a36sm3196950wmb.17.2023.09.25.01.04.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Sep 2023 01:04:59 -0700 (PDT) From: Julien Stephan To: openembedded-core@lists.openembedded.org Cc: Julien Stephan Subject: [PATCH v5 5/5] oeqa/selftest/bblock: add self test for bblock tool Date: Mon, 25 Sep 2023 10:04:52 +0200 Message-ID: <20230925080452.803540-6-jstephan@baylibre.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230925080452.803540-1-jstephan@baylibre.com> References: <20230925080452.803540-1-jstephan@baylibre.com> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Mon, 25 Sep 2023 08:05:09 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/188179 it implements various combination of locking single/multiple recipe(s)/task(s) it also tests that locked sig are architecture dependant Signed-off-by: Julien Stephan --- meta/lib/oeqa/selftest/cases/bblock.py | 201 +++++++++++++++++++++++++ 1 file changed, 201 insertions(+) create mode 100644 meta/lib/oeqa/selftest/cases/bblock.py diff --git a/meta/lib/oeqa/selftest/cases/bblock.py b/meta/lib/oeqa/selftest/cases/bblock.py new file mode 100644 index 00000000000..07143c59676 --- /dev/null +++ b/meta/lib/oeqa/selftest/cases/bblock.py @@ -0,0 +1,201 @@ +# +# Copyright (c) 2023 BayLibre, SAS +# Author: Julien Stepahn +# +# SPDX-License-Identifier: GPL-2.0-only +# + +import os +import re +import bb.tinfoil + +import oeqa.utils.ftools as ftools +from oeqa.utils.commands import runCmd, get_bb_var, get_bb_vars, bitbake + +from oeqa.selftest.case import OESelftestTestCase + + +class BBLock(OESelftestTestCase): + @classmethod + def setUpClass(cls): + super(BBLock, cls).setUpClass() + cls.lockfile = cls.builddir + "/conf/bblock.conf" + + def unlock_recipes(self, recipes=None, tasks=None): + cmd = "bblock -r " + if recipes: + cmd += " ".join(recipes) + if tasks: + cmd += " -t " + ",".join(tasks) + result = runCmd(cmd) + + if recipes: + # ensure all signatures are removed from lockfile + contents = ftools.read_file(self.lockfile) + for recipe in recipes: + for task in tasks: + find_in_contents = re.search( + 'SIGGEN_LOCKEDSIGS_.+\s\+=\s"%s:%s:.*"' % (recipe, task), + contents, + ) + self.assertFalse( + find_in_contents, + msg="%s:%s should not be present into bblock.conf anymore" + % (recipe, task), + ) + self.assertExists(self.lockfile) + else: + self.assertNotExists(self.lockfile) + + def lock_recipes(self, recipes, tasks=None): + cmd = "bblock " + " ".join(recipes) + if tasks: + cmd += " -t " + ",".join(tasks) + + result = runCmd(cmd) + + self.assertExists(self.lockfile) + + # ensure all signatures are added to lockfile + contents = ftools.read_file(self.lockfile) + for recipe in recipes: + if tasks: + for task in tasks: + find_in_contents = re.search( + 'SIGGEN_LOCKEDSIGS_.+\s\+=\s"%s:%s:.*"' % (recipe, task), + contents, + ) + self.assertTrue( + find_in_contents, + msg="%s:%s was not added into bblock.conf. bblock output: %s" + % (recipe, task, result.output), + ) + + def modify_tasks(self, recipes, tasks): + task_append = "" + for recipe in recipes: + bb_vars = get_bb_vars(["PV"], recipe) + recipe_pv = bb_vars["PV"] + recipe_append_file = recipe + "_" + recipe_pv + ".bbappend" + + os.mkdir(os.path.join(self.testlayer_path, "recipes-test", recipe)) + recipe_append_path = os.path.join( + self.testlayer_path, "recipes-test", recipe, recipe_append_file + ) + + for task in tasks: + task_append += "%s:append() {\n#modify task hash \n}\n" % task + ftools.write_file(recipe_append_path, task_append) + self.add_command_to_tearDown( + "rm -rf %s" % os.path.join(self.testlayer_path, "recipes-test", recipe) + ) + + def test_lock_single_recipe_single_task(self): + recipes = ["quilt"] + tasks = ["do_compile"] + self._run_test(recipes, tasks) + + def test_lock_single_recipe_multiple_tasks(self): + recipes = ["quilt"] + tasks = ["do_compile", "do_install"] + self._run_test(recipes, tasks) + + def test_lock_single_recipe_all_tasks(self): + recipes = ["quilt"] + self._run_test(recipes, None) + + def test_lock_multiple_recipe_single_task(self): + recipes = ["quilt", "bc"] + tasks = ["do_compile"] + self._run_test(recipes, tasks) + + def test_lock_architecture_specific(self): + # unlock all recipes and ensure no bblock.conf file exist + self.unlock_recipes() + + recipes = ["quilt"] + tasks = ["do_compile"] + + # lock quilt's do_compile task for another machine + if self.td["MACHINE"] == "qemux86-64": + machine = "qemuarm" + else: + machine = "qemux86-64" + + self.write_config('MACHINE = "%s"\n' % machine) + + self.lock_recipes(recipes, tasks) + + self.write_config('MACHINE = "%s"\n' % self.td["MACHINE"]) + # modify quilt's do_compile task + self.modify_tasks(recipes, tasks) + + # build quilt using the default machine + # No Note/Warning should be emitted since sig is locked for another machine + # (quilt package is architecture dependant) + info_message = "NOTE: The following recipes have locked tasks: " + recipes[0] + warn_message = "The %s:%s sig is computed to be" % (recipes[0], tasks[0]) + result = bitbake(recipes[0] + " -n") + self.assertNotIn(info_message, result.output) + self.assertNotIn(warn_message, result.output) + + # unlock all recipes + self.unlock_recipes() + + def _run_test(self, recipes, tasks=None): + # unlock all recipes and ensure no bblock.conf file exist + self.unlock_recipes() + + # lock tasks for recipes + result = self.lock_recipes(recipes, tasks) + + if not tasks: + tasks = [] + result = bitbake("-c listtasks " + recipes[0]) + with bb.tinfoil.Tinfoil() as tinfoil: + tinfoil.prepare(config_only=False, quiet=2) + d = tinfoil.parse_recipe(recipes[0]) + + for line in result.output.splitlines(): + if line.startswith("do_"): + task = line.split()[0] + if "setscene" in task: + continue + if d.getVarFlag(task, "nostamp"): + continue + tasks.append(task) + + # build recipes. At this stage we should have a Note about recipes + # having locked task's sig, but no warning since sig still match + info_message = "NOTE: The following recipes have locked tasks: " + " ".join( + recipes + ) + for recipe in recipes: + result = bitbake(recipe + " -n") + self.assertIn(info_message, result.output) + for task in tasks: + warn_message = "The %s:%s sig is computed to be" % (recipe, task) + self.assertNotIn(warn_message, result.output) + + # modify all tasks that are locked to trigger a sig change then build the recipes + # at this stage we should have a Note as before, but also a Warning for all + # locked tasks indicating the sig mismatch + self.modify_tasks(recipes, tasks) + for recipe in recipes: + result = bitbake(recipe + " -n") + self.assertIn(info_message, result.output) + for task in tasks: + warn_message = "The %s:%s sig is computed to be" % (recipe, task) + self.assertIn(warn_message, result.output) + + # unlock all tasks and rebuild, no more Note/Warning should remain + self.unlock_recipes(recipes, tasks) + for recipe in recipes: + result = bitbake(recipe + " -n") + self.assertNotIn(info_message, result.output) + for task in tasks: + warn_message = "The %s:%s sig is computed to be" % (recipe, task) + self.assertNotIn(warn_message, result.output) + + # unlock all recipes + self.unlock_recipes()