From patchwork Tue Aug 1 14:38:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Purdie X-Patchwork-Id: 28259 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id B0BF1C41513 for ; Tue, 1 Aug 2023 14:38:20 +0000 (UTC) Received: from mail-wr1-f41.google.com (mail-wr1-f41.google.com [209.85.221.41]) by mx.groups.io with SMTP id smtpd.web10.14468.1690900696555260344 for ; Tue, 01 Aug 2023 07:38:16 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@linuxfoundation.org header.s=google header.b=OT6/CLF8; spf=pass (domain: linuxfoundation.org, ip: 209.85.221.41, mailfrom: richard.purdie@linuxfoundation.org) Received: by mail-wr1-f41.google.com with SMTP id ffacd0b85a97d-3178fa77b27so4096260f8f.2 for ; Tue, 01 Aug 2023 07:38:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linuxfoundation.org; s=google; t=1690900695; x=1691505495; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=6L4Ma5eNbezaBLNUb0LVCNsX9V3RtJTGEQhqHrzzU8Q=; b=OT6/CLF8c4EigcbbY6nMLGLIlcBmtEpHsdgK9IAlE43JYapno3PfitWAoIg237cqkV zbLgB9OlJ/jXo+lSBtDTpu1cL769+T747vv0F+dmyue5lj+7DkUj0hgNTbE2OdcLxECS cF0AzGSW8H4IyDFo6s9nzRFZA2xuWllLHO9gM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690900695; x=1691505495; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=6L4Ma5eNbezaBLNUb0LVCNsX9V3RtJTGEQhqHrzzU8Q=; b=JXMet5vIhhdzUHCeHrgwPi0sXto9csvxP8OY3+NwNeuxDbMJM54EHr8xXXXuIG/B+F zHaX0Zo1zGvnuoWfJGAMAXY0RK2pzRC9nmRi81jbtYMgg+R6izA/j8n8SMEW++W2lQxH wJeRw0fx2ESFVIbpKM2NpfmgwIybyzFEMxiNKjmKaAL0EXFEYGZHhy+9gu9ni3vfJpkD uGfepMYWQhJrWGpBF0+Amr66F1dg5dmUuqx9LTpmhaqWfjjPL+bKHvym9hdQlXItnRHd 5Bkz36W4CIiq6cFNIrHjQ5TqZ/lSq1rN9JAMZTYhj+dd8NhjwGBkUAYttS7E/rIUZtKE n+Zw== X-Gm-Message-State: ABy/qLbP9rvhZ8g79MLhfqfNHOmYIgvg9K8mt1pl9r98/3T1Ziz9IISG Eek24XKC++lsfk+CDZjD0xRQYQsN0SMJ5uiKFaM= X-Google-Smtp-Source: APBJJlHWljiX8fwotRPx9GWY7lFneFT0XaE8zrG1+JCzF8jr4WiUiaGUjCGDTWb9qNBSP7JEXUjpLA== X-Received: by 2002:adf:e386:0:b0:313:f45f:74a1 with SMTP id e6-20020adfe386000000b00313f45f74a1mr2591303wrm.51.1690900694590; Tue, 01 Aug 2023 07:38:14 -0700 (PDT) Received: from max.int.rpsys.net ([2001:8b0:aba:5f3c:e3b7:1714:595d:8417]) by smtp.gmail.com with ESMTPSA id d14-20020adff2ce000000b003176053506fsm12572887wrp.99.2023.08.01.07.38.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 01 Aug 2023 07:38:14 -0700 (PDT) From: Richard Purdie To: bitbake-devel@lists.openembedded.org Cc: Christopher Larson Subject: [PATCH 1/6] tests.data: add test for inline python calling a def'd function Date: Tue, 1 Aug 2023 15:38:08 +0100 Message-Id: <20230801143813.2206785-1-richard.purdie@linuxfoundation.org> X-Mailer: git-send-email 2.39.2 MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 01 Aug 2023 14:38:20 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/14900 From: Christopher Larson This is a test for an issue seen long ago, to avoid regressions, where a reference to a def'd function in the metadata would return the string value from the metadata rather than the function in inline python. Signed-off-by: Christopher Larson --- lib/bb/tests/data.py | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/lib/bb/tests/data.py b/lib/bb/tests/data.py index 8c043b709d..251130b857 100644 --- a/lib/bb/tests/data.py +++ b/lib/bb/tests/data.py @@ -77,6 +77,15 @@ class DataExpansions(unittest.TestCase): val = self.d.expand("${@d.getVar('foo') + ' ${bar}'}") self.assertEqual(str(val), "value_of_foo value_of_bar") + def test_python_snippet_function_reference(self): + self.d.setVar("TESTVAL", "testvalue") + self.d.setVar("testfunc", 'd.getVar("TESTVAL")') + self.d.setVarFlag("testfunc", "func", "1") + context = bb.utils.get_context() + context["testfunc"] = lambda d: d.getVar("TESTVAL") + val = self.d.expand("${@testfunc(d)}") + self.assertEqual(str(val), "testvalue") + def test_python_unexpanded(self): self.d.setVar("bar", "${unsetvar}") val = self.d.expand("${@d.getVar('foo') + ' ${bar}'}") From patchwork Tue Aug 1 14:38:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Purdie X-Patchwork-Id: 28261 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id BBA35C04A94 for ; Tue, 1 Aug 2023 14:38:20 +0000 (UTC) Received: from mail-wm1-f41.google.com (mail-wm1-f41.google.com [209.85.128.41]) by mx.groups.io with SMTP id smtpd.web10.14469.1690900697034593919 for ; Tue, 01 Aug 2023 07:38:17 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@linuxfoundation.org header.s=google header.b=Tqhjpmvr; spf=pass (domain: linuxfoundation.org, ip: 209.85.128.41, mailfrom: richard.purdie@linuxfoundation.org) Received: by mail-wm1-f41.google.com with SMTP id 5b1f17b1804b1-3fbef8ad9bbso62457825e9.0 for ; Tue, 01 Aug 2023 07:38:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linuxfoundation.org; s=google; t=1690900695; x=1691505495; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=+d0d6BT2DMb3OUYRdQrWTKOA4OBY2Q2pM2WyA5N52ao=; b=Tqhjpmvrz9mXH8R9M7lkrs9I/6ME4updYJGptxHE+RUnTjnkNgzhu+LOpIv67GVoEc /AYxoOeshYlmwX+2edCEYDdvhZ2mURTn/uc3dVwPlIordWPVcjpIpO3ZD5H0CQ+A1X8A /XgX5KDzJupVaTk3cRuQ5hdRq6eJETi19axKc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690900695; x=1691505495; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=+d0d6BT2DMb3OUYRdQrWTKOA4OBY2Q2pM2WyA5N52ao=; b=TmN6MaJdQaN6jiRwMjRhU3tKfhMEna9uSTBPxNyQcaJEMbyymq57OxGMSThEchaM7/ 5GfJ1NMGKNzYTdNfCrqtxtRD6L/MjR+17DSs4M9an9Rc8zerCvu/TJDCfB+yNOs9Gs2y bhbE2XOP2OBQKQypSRJa3f2ofVrPGODKuQ1muovNycvoelysV/XlDz0ru9ofaLEKG4ET j1DlnekNnCJXQmVirlGuEH6rda33kbMq2izYsDS/hit095IulNDZvXDi7bX3W1zjq0px kc4cE+DPcNsIMKbWXe+RffHEigNfAHa+LbJP/hKndCZDKVbG5lfsuk2Qr7bHMT83KZXv tz2A== X-Gm-Message-State: ABy/qLYvpDj0GHKxlm65p9xL2kXxxdpamamDBNQZ+RnFGkO8z3VZstdS VJbUGnZCfM0Gylse8d4ogkadcOgbO/6EOOkD+wU= X-Google-Smtp-Source: APBJJlFKk84xV4wXR4q9OZJ5pAlI4854CKtf7VuKgHIYzW5A3qS833jUqWMMRz0pVEzHGVSLTlZLNg== X-Received: by 2002:a5d:6611:0:b0:314:1fdc:796d with SMTP id n17-20020a5d6611000000b003141fdc796dmr2136318wru.70.1690900695370; Tue, 01 Aug 2023 07:38:15 -0700 (PDT) Received: from max.int.rpsys.net ([2001:8b0:aba:5f3c:e3b7:1714:595d:8417]) by smtp.gmail.com with ESMTPSA id d14-20020adff2ce000000b003176053506fsm12572887wrp.99.2023.08.01.07.38.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 01 Aug 2023 07:38:14 -0700 (PDT) From: Richard Purdie To: bitbake-devel@lists.openembedded.org Cc: Christopher Larson Subject: [PATCH 2/6] tests.codeparser: add test for exec of builtin from inline python Date: Tue, 1 Aug 2023 15:38:09 +0100 Message-Id: <20230801143813.2206785-2-richard.purdie@linuxfoundation.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230801143813.2206785-1-richard.purdie@linuxfoundation.org> References: <20230801143813.2206785-1-richard.purdie@linuxfoundation.org> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 01 Aug 2023 14:38:20 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/14901 From: Christopher Larson This ensures that any change to the presence of builtins in inline python execs will be noticed. Signed-off-by: Christopher Larson --- lib/bb/tests/codeparser.py | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/lib/bb/tests/codeparser.py b/lib/bb/tests/codeparser.py index 7f5d59ca74..a64c614b0b 100644 --- a/lib/bb/tests/codeparser.py +++ b/lib/bb/tests/codeparser.py @@ -44,6 +44,7 @@ class VariableReferenceTest(ReferenceTest): def parseExpression(self, exp): parsedvar = self.d.expandWithRefs(exp, None) self.references = parsedvar.references + self.execs = parsedvar.execs def test_simple_reference(self): self.setEmptyVars(["FOO"]) @@ -61,6 +62,11 @@ class VariableReferenceTest(ReferenceTest): self.parseExpression("${@d.getVar('BAR') + 'foo'}") self.assertReferences(set(["BAR"])) + def test_python_exec_reference(self): + self.parseExpression("${@eval('3 * 5')}") + self.assertReferences(set()) + self.assertExecs(set(["eval"])) + class ShellReferenceTest(ReferenceTest): def parseExpression(self, exp): From patchwork Tue Aug 1 14:38:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Purdie X-Patchwork-Id: 28263 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id BBA64C04FDF for ; Tue, 1 Aug 2023 14:38:20 +0000 (UTC) Received: from mail-wr1-f43.google.com (mail-wr1-f43.google.com [209.85.221.43]) by mx.groups.io with SMTP id smtpd.web10.14471.1690900698233122147 for ; Tue, 01 Aug 2023 07:38:18 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@linuxfoundation.org header.s=google header.b=OgJPzBpv; spf=pass (domain: linuxfoundation.org, ip: 209.85.221.43, mailfrom: richard.purdie@linuxfoundation.org) Received: by mail-wr1-f43.google.com with SMTP id ffacd0b85a97d-3177163aa97so5918226f8f.0 for ; Tue, 01 Aug 2023 07:38:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linuxfoundation.org; s=google; t=1690900696; x=1691505496; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=5xr6xKJOGFMNKRN/POjNx+b7Lcy2saM9y0FysfgZW34=; b=OgJPzBpv3ijzb/x7wLFotPRdhyE/XNNGXBS7lhvFtp+QuiOh5uSJdkKvSgoaBRgAXN UqPrlWjWsRMw1Vpu/9UkkiQwjqzk5/daXjkEbgkNvSZueKr8z2xvp7js4zBuK955dp7B VStwyOxL40Js55xXNV642cFJSPQh+p1n6FCzA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690900696; x=1691505496; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=5xr6xKJOGFMNKRN/POjNx+b7Lcy2saM9y0FysfgZW34=; b=BhCwLq5g605zkg7lZrx9YOvXjYeV24TnCFIEHjWSFl22BOtHGa8qawyYuhS8av4sMX Iw/8apCeSJxO2XzgDQOaE4lyKjm8SqLVWSZZhnUFadJ4wN0H0JU3U4/K4HTRd5sO9HoG TnPnUY8jOicDsoTzDTPiYPpHgNUh7/w1tgqWLhH80w3CSnFtQbg9VTMW7PKYOupHa/xo RwyQ/fu1ot4ngRtDdAGe+1eg8MgHzb3EWjx/HIvio3AWiMUS+x9uhGHyGMZcn4UQVTOU R0A0IVNfAYL+luUadvBJvBv8nVPYf+u3CsD/o1JEc7GJokurOqVhAL1sg5jVh2x3HatK votA== X-Gm-Message-State: ABy/qLYGWD4+ErJmVfnjXrHnNaLFiibiJZSDpGCuhOkHdfMghVbMeM6L b63wBkCHozvq/8qALK5vC0U52ru091C61Udyt8o= X-Google-Smtp-Source: APBJJlEHPvR8/riw7NlYRCg5VtR1ri5kHL1yq3klSDeAHu2tAs9al14T2Py7MwoWwE9TQg1KxDUEeg== X-Received: by 2002:adf:e60e:0:b0:317:58e4:e941 with SMTP id p14-20020adfe60e000000b0031758e4e941mr2488828wrm.33.1690900696406; Tue, 01 Aug 2023 07:38:16 -0700 (PDT) Received: from max.int.rpsys.net ([2001:8b0:aba:5f3c:e3b7:1714:595d:8417]) by smtp.gmail.com with ESMTPSA id d14-20020adff2ce000000b003176053506fsm12572887wrp.99.2023.08.01.07.38.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 01 Aug 2023 07:38:15 -0700 (PDT) From: Richard Purdie To: bitbake-devel@lists.openembedded.org Cc: Christopher Larson Subject: [PATCH 3/6] data_smart: check for python builtins directly for context lookup Date: Tue, 1 Aug 2023 15:38:10 +0100 Message-Id: <20230801143813.2206785-3-richard.purdie@linuxfoundation.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230801143813.2206785-1-richard.purdie@linuxfoundation.org> References: <20230801143813.2206785-1-richard.purdie@linuxfoundation.org> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 01 Aug 2023 14:38:20 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/14902 From: Christopher Larson This avoids the need to hardcode a list of python builtins. This also slightly changes behavior, in a case like `${@eval("3")}`, this will ensure we always call the builtin, even if the metadata has an 'eval' variable defined. Signed-off-by: Christopher Larson --- lib/bb/data_smart.py | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/lib/bb/data_smart.py b/lib/bb/data_smart.py index c597dbade8..fe0bacd13b 100644 --- a/lib/bb/data_smart.py +++ b/lib/bb/data_smart.py @@ -16,7 +16,10 @@ BitBake build tools. # # Based on functions from the base bb module, Copyright 2003 Holger Schurig -import copy, re, sys, traceback +import builtins +import copy +import re +import sys from collections.abc import MutableMapping import logging import hashlib @@ -150,17 +153,18 @@ class VariableParse: value = utils.better_eval(codeobj, DataContext(self.d), {'d' : self.d}) return str(value) - class DataContext(dict): + excluded = set([i for i in dir(builtins) if not i.startswith('_')] + ['bb', 'os', 'oe']) + def __init__(self, metadata, **kwargs): self.metadata = metadata dict.__init__(self, **kwargs) self['d'] = metadata def __missing__(self, key): - # Skip commonly accessed invalid variables - if key in ['bb', 'oe', 'int', 'bool', 'time', 'str', 'os']: + if key in self.excluded: raise KeyError(key) + value = self.metadata.getVar(key) if value is None or self.metadata.getVarFlag(key, 'func', False): raise KeyError(key) From patchwork Tue Aug 1 14:38:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Purdie X-Patchwork-Id: 28260 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id B0BCCC001DF for ; Tue, 1 Aug 2023 14:38:20 +0000 (UTC) Received: from mail-wm1-f52.google.com (mail-wm1-f52.google.com [209.85.128.52]) by mx.groups.io with SMTP id smtpd.web10.14472.1690900699074839337 for ; Tue, 01 Aug 2023 07:38:19 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@linuxfoundation.org header.s=google header.b=eH6SYr8q; spf=pass (domain: linuxfoundation.org, ip: 209.85.128.52, mailfrom: richard.purdie@linuxfoundation.org) Received: by mail-wm1-f52.google.com with SMTP id 5b1f17b1804b1-3fbea14700bso53895895e9.3 for ; Tue, 01 Aug 2023 07:38:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linuxfoundation.org; s=google; t=1690900697; x=1691505497; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=lhmHqDgb3DfaCTKvBsZiA1byhpAWcvHXDbDS37NaWdc=; b=eH6SYr8qtjZHRyKvFOVGgX22kYFkU/Vnrch7Ela03l8M8QsYxsEC5xn9Gp4N7OqUVa 86PzG7PfYWeSU3ziB93yDZh8OvAlr4RZmH7sFwZ0qh6Y9zQ2ePOZhATMcWV1WjzJ1NMB 5xHQFpLrAOKzBjGNpOR6dpykbCbpG5l4llBtw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690900697; x=1691505497; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=lhmHqDgb3DfaCTKvBsZiA1byhpAWcvHXDbDS37NaWdc=; b=eqAHOPuhEZUHUmHgVOWeoObtFhrzwKkoLyB31j59iL13KFHDpd0C5kndjRT1FTiWbH u76PiwcOT1gPe+qtuTHBg/dM86YmvLfpoR+SdLnCe+Pr4sI41WJN1cJbhyzNwdzXU60Y Pz39UdrIrZuueSTv20cRWNVdz1ncu1cCO2m2JNNAux7fuHDtDzHz9rGthIOzFKkRE6EL XFmfgtDUYmuFg8LV8P9ywiaUhf0Ko2Z6MN8AL6I4U76TI6BO/TTStcL+GCslvHA2oKsd mBAG4rmCm0M4CpR3JlydRqgSy24fj6WQhZPfQOtPL9V2IWVqGXhphZS5K7CoYA+ZzHx9 t2mQ== X-Gm-Message-State: ABy/qLauHtwbsFNYDL3+UfaWyBU1NMkMSljnZyHGbRHLpl4s1g9ZXAdw uxhrh7oA0ffnEWPTAxZcGC2vZt0daECP5IoZRL0= X-Google-Smtp-Source: APBJJlE7AQQXAi8o0CNQstIC4olYokb+tAJwECEyOSKyGI23tS0MSVSheB4zTWEnLTXvZf9QgA6zTw== X-Received: by 2002:adf:e848:0:b0:30a:f030:8637 with SMTP id d8-20020adfe848000000b0030af0308637mr2487362wrn.22.1690900697334; Tue, 01 Aug 2023 07:38:17 -0700 (PDT) Received: from max.int.rpsys.net ([2001:8b0:aba:5f3c:e3b7:1714:595d:8417]) by smtp.gmail.com with ESMTPSA id d14-20020adff2ce000000b003176053506fsm12572887wrp.99.2023.08.01.07.38.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 01 Aug 2023 07:38:16 -0700 (PDT) From: Richard Purdie To: bitbake-devel@lists.openembedded.org Cc: Christopher Larson Subject: [PATCH 4/6] tests.data: add test for builtin preferred over metadata value Date: Tue, 1 Aug 2023 15:38:11 +0100 Message-Id: <20230801143813.2206785-4-richard.purdie@linuxfoundation.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230801143813.2206785-1-richard.purdie@linuxfoundation.org> References: <20230801143813.2206785-1-richard.purdie@linuxfoundation.org> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 01 Aug 2023 14:38:20 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/14903 From: Christopher Larson This test makes sure that '${@eval()}' calls the eval builtin, even if an 'eval' variable is defined in the metadata. Signed-off-by: Christopher Larson --- lib/bb/tests/data.py | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/lib/bb/tests/data.py b/lib/bb/tests/data.py index 251130b857..b7676859a6 100644 --- a/lib/bb/tests/data.py +++ b/lib/bb/tests/data.py @@ -86,6 +86,10 @@ class DataExpansions(unittest.TestCase): val = self.d.expand("${@testfunc(d)}") self.assertEqual(str(val), "testvalue") + def test_python_snippet_builtin_metadata(self): + self.d.setVar("eval", "INVALID") + self.d.expand("${@eval('3')}") + def test_python_unexpanded(self): self.d.setVar("bar", "${unsetvar}") val = self.d.expand("${@d.getVar('foo') + ' ${bar}'}") From patchwork Tue Aug 1 14:38:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Purdie X-Patchwork-Id: 28262 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id B19BEC001E0 for ; Tue, 1 Aug 2023 14:38:20 +0000 (UTC) Received: from mail-wr1-f45.google.com (mail-wr1-f45.google.com [209.85.221.45]) by mx.groups.io with SMTP id smtpd.web10.14473.1690900699936748722 for ; Tue, 01 Aug 2023 07:38:20 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@linuxfoundation.org header.s=google header.b=FY+Bbyuj; spf=pass (domain: linuxfoundation.org, ip: 209.85.221.45, mailfrom: richard.purdie@linuxfoundation.org) Received: by mail-wr1-f45.google.com with SMTP id ffacd0b85a97d-3178dd771ceso4251749f8f.2 for ; Tue, 01 Aug 2023 07:38:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linuxfoundation.org; s=google; t=1690900698; x=1691505498; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=fxYYj5ym56bdbxp81EuLmR9zectCa+Ak2JaUT5vcx7M=; b=FY+BbyujsDULpIGZu1gTFi0uolNps3h7han+VdYK79ifXbEDJugCx92ng6G0B36OaG LnvYyzYTTA1S18FkD0q8G5NweeIPLKTrRQdbJ8jUAMoep3HGgIsf8OY2m3TD4nho/Gs0 NESZmUyAhn79G6l+23QnWqA9glgQmc1UNXUuY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690900698; x=1691505498; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=fxYYj5ym56bdbxp81EuLmR9zectCa+Ak2JaUT5vcx7M=; b=Ewyw6AUeu+yCwfNbFuYdRNSK+HH78IpXwYfnDV6PGVk4hZ0RFFavuqNEOhMrAjBEe6 WtaYANqAhXbpBXq4c72McvKjqLgy6T5dX74B2thaYPkqnh+m4X4XAxefiiSr0Q6tmRqx eKNdnfkzj6oy7AsUFA5dgDUxr9LqWcR/2KAY4Fs+N+sSG/pbe9RGzKs1TGubCxWss2WY oAFRnn8JYbhJFyXGZNZonIA2KIly/lKTGoGHSAiL38mYXLY3Z7eouEwYLDZB1XtNsMHv ZEidIJPVDyea+wF9hyqePSZzExtixw7HnH0eT4ZOINA4LhUEn4IaEw8UjlbBTFDekcs5 mB8Q== X-Gm-Message-State: ABy/qLZ38wSUKuGsCOkAV0WA+BetTHux1jHHd+WX/kXn+HByDvk+D8xr uoB8ls+/CXsZCQwhY/cUJIhCQ8g6hfgLZAtKVBs= X-Google-Smtp-Source: APBJJlHMhxVstV/QsoN9PRtn2FQQ/0BB7RKiVb1Qp8s0w7qoqk7xyjoyUR6w/71b7FdA5XPeLKZpwA== X-Received: by 2002:adf:e509:0:b0:314:3344:72f6 with SMTP id j9-20020adfe509000000b00314334472f6mr2585304wrm.32.1690900698284; Tue, 01 Aug 2023 07:38:18 -0700 (PDT) Received: from max.int.rpsys.net ([2001:8b0:aba:5f3c:e3b7:1714:595d:8417]) by smtp.gmail.com with ESMTPSA id d14-20020adff2ce000000b003176053506fsm12572887wrp.99.2023.08.01.07.38.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 01 Aug 2023 07:38:17 -0700 (PDT) From: Richard Purdie To: bitbake-devel@lists.openembedded.org Cc: Christopher Larson Subject: [PATCH 5/6] data_smart: directly check for methodpool functions in context lookup Date: Tue, 1 Aug 2023 15:38:12 +0100 Message-Id: <20230801143813.2206785-5-richard.purdie@linuxfoundation.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230801143813.2206785-1-richard.purdie@linuxfoundation.org> References: <20230801143813.2206785-1-richard.purdie@linuxfoundation.org> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 01 Aug 2023 14:38:20 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/14904 From: Christopher Larson We previously checked for the existence of the 'func' flag to determine if we should avoid looking up in the metadata. This was done to ensure the user gets the function for 'def' python functions rather than their string contents. We can sidestep the metadata lookup and check our function context directly, instead. Signed-off-by: Christopher Larson --- lib/bb/data_smart.py | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/lib/bb/data_smart.py b/lib/bb/data_smart.py index fe0bacd13b..0128a5bb17 100644 --- a/lib/bb/data_smart.py +++ b/lib/bb/data_smart.py @@ -154,19 +154,20 @@ class VariableParse: return str(value) class DataContext(dict): - excluded = set([i for i in dir(builtins) if not i.startswith('_')] + ['bb', 'os', 'oe']) + excluded = set([i for i in dir(builtins) if not i.startswith('_')] + ['oe']) def __init__(self, metadata, **kwargs): self.metadata = metadata dict.__init__(self, **kwargs) self['d'] = metadata + self.context = set(bb.utils.get_context()) def __missing__(self, key): - if key in self.excluded: + if key in self.excluded or key in self.context: raise KeyError(key) value = self.metadata.getVar(key) - if value is None or self.metadata.getVarFlag(key, 'func', False): + if value is None: raise KeyError(key) else: return value From patchwork Tue Aug 1 14:38:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Purdie X-Patchwork-Id: 28264 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 954B4C001DF for ; Tue, 1 Aug 2023 14:38:30 +0000 (UTC) Received: from mail-wr1-f46.google.com (mail-wr1-f46.google.com [209.85.221.46]) by mx.groups.io with SMTP id smtpd.web10.14474.1690900701057741913 for ; Tue, 01 Aug 2023 07:38:21 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@linuxfoundation.org header.s=google header.b=XJ17oXDx; spf=pass (domain: linuxfoundation.org, ip: 209.85.221.46, mailfrom: richard.purdie@linuxfoundation.org) Received: by mail-wr1-f46.google.com with SMTP id ffacd0b85a97d-307d20548adso5139030f8f.0 for ; Tue, 01 Aug 2023 07:38:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linuxfoundation.org; s=google; t=1690900699; x=1691505499; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9zQ4+h7HYsDcWko95jgu8wuevjqvDxJSY6eiOwUPJkE=; b=XJ17oXDxDt7VJY5/KTTl9XeAHFh2edw1a8rqavjg29/bnijpiizlEruEpxiMpW9Pt2 X2Bfkx8JvI7gvdRGxTi8CN1QEQmGrtB4OzApe9BJQ8VKKzbD7PQ+MVfBI2R8ZMUmsYtz dIQZefe8tXAfKGXOm5sH/kgoaLrj9tecMJw/Y= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690900699; x=1691505499; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9zQ4+h7HYsDcWko95jgu8wuevjqvDxJSY6eiOwUPJkE=; b=ep019tqgKFpcslTrPYzjxQbycVdqbC9HZmg+ANGR18JrPyoVoPmvuCSPiuSKBDZ+IF 9BWJoN+70q/0N3krbsJQfdcCjYgbnN5RWTgDstBx9tETK2HLFS+YdJIbH1Dk+LR/7DLg G2CmZ1orU7YAG7VUEM9urHGihhKONlTlgYf7pQQ7pBiPy4CKhHXT+R/p6hEniB6xMyBI emZWAu4NAvbM0gaxHP9X6BBXa1JMzCBYTnWTJlpJyiUzQ94q9mBLUJUfLq6hgGTMF/dZ 5FZ8uyn3BDK/DjumxtrQPlTBRniNEX26Z0srPUq0qzV5Riq1IGGeO9MoTabn6xDSoyUO jxaA== X-Gm-Message-State: ABy/qLaBeHElZChg7D+L5HZlaI2piOR3lsxr7gMF0+VrtgkVY+HnP1id ylT6SMzg0GAUERyTFY0DTNqzP2EZGx0jxQGqopo= X-Google-Smtp-Source: APBJJlEBP17WZoBmJ4XHvT0NQ7K31MiaxSz6/+7oQrQ9wKEu2NdZqy9wq/ts4WV6ygCaJ2H8LCHAWA== X-Received: by 2002:adf:cd11:0:b0:317:597b:9f92 with SMTP id w17-20020adfcd11000000b00317597b9f92mr2471541wrm.57.1690900699327; Tue, 01 Aug 2023 07:38:19 -0700 (PDT) Received: from max.int.rpsys.net ([2001:8b0:aba:5f3c:e3b7:1714:595d:8417]) by smtp.gmail.com with ESMTPSA id d14-20020adff2ce000000b003176053506fsm12572887wrp.99.2023.08.01.07.38.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 01 Aug 2023 07:38:18 -0700 (PDT) From: Richard Purdie To: bitbake-devel@lists.openembedded.org Cc: Christopher Larson Subject: [PATCH 6/6] bb.tests.data: don't require the func flag for context functions Date: Tue, 1 Aug 2023 15:38:13 +0100 Message-Id: <20230801143813.2206785-6-richard.purdie@linuxfoundation.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230801143813.2206785-1-richard.purdie@linuxfoundation.org> References: <20230801143813.2206785-1-richard.purdie@linuxfoundation.org> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Tue, 01 Aug 2023 14:38:30 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/14905 From: Christopher Larson Update test_python_snippet_function_reference to not require the 'func' flag, now that we know the real function will be returned for context functions without the flag. Signed-off-by: Christopher Larson --- lib/bb/tests/data.py | 1 - 1 file changed, 1 deletion(-) diff --git a/lib/bb/tests/data.py b/lib/bb/tests/data.py index b7676859a6..98e430ce2a 100644 --- a/lib/bb/tests/data.py +++ b/lib/bb/tests/data.py @@ -80,7 +80,6 @@ class DataExpansions(unittest.TestCase): def test_python_snippet_function_reference(self): self.d.setVar("TESTVAL", "testvalue") self.d.setVar("testfunc", 'd.getVar("TESTVAL")') - self.d.setVarFlag("testfunc", "func", "1") context = bb.utils.get_context() context["testfunc"] = lambda d: d.getVar("TESTVAL") val = self.d.expand("${@testfunc(d)}")