From patchwork Wed Aug 10 13:43:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Purdie X-Patchwork-Id: 11231 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id BC231C3F6B0 for ; Wed, 10 Aug 2022 13:43:45 +0000 (UTC) Received: from mail-wr1-f41.google.com (mail-wr1-f41.google.com [209.85.221.41]) by mx.groups.io with SMTP id smtpd.web11.5774.1660139021453022420 for ; Wed, 10 Aug 2022 06:43:41 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@linuxfoundation.org header.s=google header.b=BSLWnSPw; spf=pass (domain: linuxfoundation.org, ip: 209.85.221.41, mailfrom: richard.purdie@linuxfoundation.org) Received: by mail-wr1-f41.google.com with SMTP id z16so17744173wrh.12 for ; Wed, 10 Aug 2022 06:43:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linuxfoundation.org; s=google; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:from:to:cc; bh=mffHTrJYgT6xdIgFpLXUDTYaf5/8FiHkdJ1q/0R4gAY=; b=BSLWnSPwxLYyiYQoaVjUcBwchDow8JQ1hJPZQPwBy8CTkKu//Q9RHvFXy7cgxqkIrv HSJnHum8b40Lt/mKGtqroZk0igi6Kc0lK5vqi5GVnW/8YO00mtKkQRkViEes650L+u1f 1D+hnwLBfflhPbhjEkGOKvRBHOueqdoAou+HM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:x-gm-message-state:from:to:cc; bh=mffHTrJYgT6xdIgFpLXUDTYaf5/8FiHkdJ1q/0R4gAY=; b=IHGWtZzHfLeVobikL2Anm7jFpg9i70c5JYjapRtA7KgE9PC7/i95TRdeGV/Q5PDZvj RfVg05CX1x2kyZGYItPzQWfgLvWtVJmdRiP9ts0NwB+NJmVnEZMjFQEW7i19JGLYLuYP mhVuHWsraW3Yyo/5SQj2CaXBEZKH1osMkFwCo2M5EI1Ardgm5J8CCHIPvr99i1KpoL85 pYWoEitGIyzeH+KRDFh0o5Jb9sGlkhp6wA0SIgO0fGt/IfAl3pvWWoDU6cwRmg81EH4z i2KIpyu1rdH3lBjKG63f8DoiP/QZIMu5GIGc9IHKJKblsn6PXOQhrK5EyWYCev/9XH2J dLSg== X-Gm-Message-State: ACgBeo0bSA1LrAZCagw8VELwJyIGCYSTwBx+loTefNwyQnRax5p4e6tU CxOqL3LYqQpho+PtB+83CbsU/xh5nUPqDA== X-Google-Smtp-Source: AA6agR7ITJkrdalQhXa6cWImgtLuZ50rEuCPccznyY6orBZHIVAn7489mBQUKkggwOJpO6NzNxnN2Q== X-Received: by 2002:a5d:59a8:0:b0:220:7076:dce0 with SMTP id p8-20020a5d59a8000000b002207076dce0mr17932363wrr.570.1660139019338; Wed, 10 Aug 2022 06:43:39 -0700 (PDT) Received: from max.int.rpsys.net ([2001:8b0:aba:5f3c:54f2:80ed:9b2b:a32b]) by smtp.gmail.com with ESMTPSA id f8-20020a05600c154800b003a540fef440sm2917768wmg.1.2022.08.10.06.43.38 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Aug 2022 06:43:38 -0700 (PDT) From: Richard Purdie To: bitbake-devel@lists.openembedded.org Subject: [PATCH 1/3] BBHandler: Allow earlier exit for classes not found Date: Wed, 10 Aug 2022 14:43:35 +0100 Message-Id: <20220810134337.420149-1-richard.purdie@linuxfoundation.org> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 10 Aug 2022 13:43:45 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/13880 Rather than relying on later code to error if the class isn't found, exit earlier and more clearly from a code perspective. Signed-off-by: Richard Purdie --- lib/bb/parse/parse_py/BBHandler.py | 3 +++ 1 file changed, 3 insertions(+) diff --git a/lib/bb/parse/parse_py/BBHandler.py b/lib/bb/parse/parse_py/BBHandler.py index 68415735fd..532a4e08f1 100644 --- a/lib/bb/parse/parse_py/BBHandler.py +++ b/lib/bb/parse/parse_py/BBHandler.py @@ -56,6 +56,9 @@ def inherit(files, fn, lineno, d): if abs_fn: file = abs_fn + if not os.path.exists(file): + raise ParseError("Could not inherit file %s" % (file), fn, lineno) + if not file in __inherit_cache: logger.debug("Inheriting %s (from %s:%d)" % (file, fn, lineno)) __inherit_cache.append( file ) From patchwork Wed Aug 10 13:43:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Purdie X-Patchwork-Id: 11233 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8608EC00140 for ; Wed, 10 Aug 2022 13:43:55 +0000 (UTC) Received: from mail-wm1-f44.google.com (mail-wm1-f44.google.com [209.85.128.44]) by mx.groups.io with SMTP id smtpd.web10.5758.1660139022440384019 for ; Wed, 10 Aug 2022 06:43:42 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@linuxfoundation.org header.s=google header.b=Z/cVKm3r; spf=pass (domain: linuxfoundation.org, ip: 209.85.128.44, mailfrom: richard.purdie@linuxfoundation.org) Received: by mail-wm1-f44.google.com with SMTP id az6-20020a05600c600600b003a530cebbe3so1022685wmb.0 for ; Wed, 10 Aug 2022 06:43:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linuxfoundation.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc; bh=OmJ9ghvtoiiKVUhO31w6bEM1jqcGy+qMdiO958bL8yU=; b=Z/cVKm3rejtW/mmf2hVPlCkOT/QLtHKkuxF7KKI11JVzQZkSNUz3fCbKWje2948KzR Kc3ZgxRdOtYklvjXWPpdyvOS8oVbtRk4dGpvIW89XfplNms6EcFExxQ3eOSeU+lMtqAs vVM0BJOY7nFE5pGbZwr6IwyQOhFlUWXCuW904= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc; bh=OmJ9ghvtoiiKVUhO31w6bEM1jqcGy+qMdiO958bL8yU=; b=V5DgLE6P+wLhjXZbjeUCJJHp2m2Z1suYOR4hev24+eLPqlybjEx77nifeAH7+Z21uw SQuqB/1xd0yZr4cB+LD16AAfcti2xPWErt3pc1RVtIxFnYD4Jkyeaw77j6LuMRg+Lq0l jQUGzo5fFJSfqgthcaP499JgXUAPvsZgzNRGQIfVk4L/b88xs2cK/8hS3O4RlCoPU/+6 6l7YsBtVWRGU4NmEShvuz3qSVyoD8w8IyIt9SX6cc73riy3Qg5FtVZh189wcy6mJhHR6 52mf52YSqXZXITRx6hALhwdRWpGgD2o2JKKm9ItPFukbglWLpf0q7xM9J79jPy9K/1XD AL8Q== X-Gm-Message-State: ACgBeo0V5NPLn9EmoUCzu0jdsB2/eQyWbvrLn9fVeMNgKreZh00VNh0n f9vDJuvFNkTPumpQkBiKfKZml/5JvzZrxA== X-Google-Smtp-Source: AA6agR5cG4jiKB9THAh1/fc7o9rZ5Jt97v6QnEX7hZCca0PgE/YTbGDz3g1CWo418agLqUGwBKHRIw== X-Received: by 2002:a05:600c:3d93:b0:3a3:3a93:fb16 with SMTP id bi19-20020a05600c3d9300b003a33a93fb16mr2647627wmb.190.1660139020343; Wed, 10 Aug 2022 06:43:40 -0700 (PDT) Received: from max.int.rpsys.net ([2001:8b0:aba:5f3c:54f2:80ed:9b2b:a32b]) by smtp.gmail.com with ESMTPSA id f8-20020a05600c154800b003a540fef440sm2917768wmg.1.2022.08.10.06.43.39 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Aug 2022 06:43:39 -0700 (PDT) From: Richard Purdie To: bitbake-devel@lists.openembedded.org Subject: [PATCH 2/3] BBHandler: Make inherit calls more directly Date: Wed, 10 Aug 2022 14:43:36 +0100 Message-Id: <20220810134337.420149-2-richard.purdie@linuxfoundation.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220810134337.420149-1-richard.purdie@linuxfoundation.org> References: <20220810134337.420149-1-richard.purdie@linuxfoundation.org> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 10 Aug 2022 13:43:55 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/13881 Rather than recursing into the conf handler code, simply call into the parse code directly when inheriting files as we've already resolved the paths and don't need anything the other codepath brings. This makes the codepath clearer at the expense of some slight duplication. Signed-off-by: Richard Purdie --- lib/bb/parse/parse_py/BBHandler.py | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/lib/bb/parse/parse_py/BBHandler.py b/lib/bb/parse/parse_py/BBHandler.py index 532a4e08f1..1189114341 100644 --- a/lib/bb/parse/parse_py/BBHandler.py +++ b/lib/bb/parse/parse_py/BBHandler.py @@ -63,7 +63,10 @@ def inherit(files, fn, lineno, d): logger.debug("Inheriting %s (from %s:%d)" % (file, fn, lineno)) __inherit_cache.append( file ) d.setVar('__inherit_cache', __inherit_cache) - include(fn, file, lineno, d, "inherit") + try: + bb.parse.handle(file, d, True) + except (IOError, OSError) as exc: + raise ParseError("Could not inherit file %s: %s" % (fn, exc.strerror), fn, lineno) __inherit_cache = d.getVar('__inherit_cache', False) or [] def get_statements(filename, absolute_filename, base_name): From patchwork Wed Aug 10 13:43:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Purdie X-Patchwork-Id: 11232 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id BA880C00140 for ; Wed, 10 Aug 2022 13:43:45 +0000 (UTC) Received: from mail-wm1-f45.google.com (mail-wm1-f45.google.com [209.85.128.45]) by mx.groups.io with SMTP id smtpd.web11.5775.1660139023504136489 for ; Wed, 10 Aug 2022 06:43:43 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@linuxfoundation.org header.s=google header.b=CqUWmtea; spf=pass (domain: linuxfoundation.org, ip: 209.85.128.45, mailfrom: richard.purdie@linuxfoundation.org) Received: by mail-wm1-f45.google.com with SMTP id ay12so5004636wmb.1 for ; Wed, 10 Aug 2022 06:43:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linuxfoundation.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc; bh=v0UY2WKrjXoHCGp3G7AZCm4u3PovMkmNFKUD9ciWXLg=; b=CqUWmteamYhJki2xZ7b2QXYxsYgZplhl1AlvBXKdVARpgG5Klpc//UsxLu2UzrWucr D3RAFgx94L2p35KrS8OEeWD8gwaeiaO9J2SpF7H4EGd+hiHaOoUslSOV0Y5gwG2v4P0G yaKXEWL4WzSIGBNKnBFOyv41g98HvjGeoYNjw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc; bh=v0UY2WKrjXoHCGp3G7AZCm4u3PovMkmNFKUD9ciWXLg=; b=y0AWikY+V7RJqaC6SKkT6GxB3CR9NqkCL8C3r19LVmeI/4/g6NsKEWCTP3b1GhuqBX mnz7Fe7DGLpr2Yuz6oFQVfxoQXpKe5JPbjjBOmwLNChbFxiAUORifusvHXB//EX12Jhu 9m/I9DqzrtdYrv6Jl056xwDYYiBD37n2SIOniU8CgAXA4E1XMij8XcXuN4srL3TEpbt4 uG85g4LpYHRvxWRnO7TSKQ3sgwmGhChoPHdDlbXEd5OrooSwbasLp9MdtsIjPPi7RUGO iGb6XQjAXdNvnNJzEVuiPkSBSzCbShiRgecqp8vh4FFUetkGWhNp8sJirpCjaEFjia0w m/Hg== X-Gm-Message-State: ACgBeo0XgoIclshFqLOvNPr5Z+5nXBrI5jtQN0GqJP9mHU8ZgZAM8WN/ sfSdBxWC7m76951p2jkrX3Opp5qZfAzVew== X-Google-Smtp-Source: AA6agR5KGXcHWIzdKX3dJKKPyUXpskmr1ci9umVZQapjZ+uWbR5U4JLm/Lkd+d4hzhjPiTIl5ZgnhQ== X-Received: by 2002:a7b:ca58:0:b0:3a5:3c1c:6d71 with SMTP id m24-20020a7bca58000000b003a53c1c6d71mr2592613wml.118.1660139021760; Wed, 10 Aug 2022 06:43:41 -0700 (PDT) Received: from max.int.rpsys.net ([2001:8b0:aba:5f3c:54f2:80ed:9b2b:a32b]) by smtp.gmail.com with ESMTPSA id f8-20020a05600c154800b003a540fef440sm2917768wmg.1.2022.08.10.06.43.40 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Aug 2022 06:43:40 -0700 (PDT) From: Richard Purdie To: bitbake-devel@lists.openembedded.org Subject: [PATCH 3/3] BBHandler/cooker: Implement recipe and global classes Date: Wed, 10 Aug 2022 14:43:37 +0100 Message-Id: <20220810134337.420149-3-richard.purdie@linuxfoundation.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220810134337.420149-1-richard.purdie@linuxfoundation.org> References: <20220810134337.420149-1-richard.purdie@linuxfoundation.org> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 10 Aug 2022 13:43:45 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/13882 We have some confusion for users since some classes are meant to work in the configuration space (or "globally") and some are meant to be selected by recipes individually. The cleanest way I could find to clarify this is to create "classes-global" and "classes-recipe" directories which contain the approproate classes and have bitbake switch scope between them at the appropriate point during parsing. The existing "classes" directory is always searched as a fallback. Once a class is moved to a specific directory, it will no longer be found in the incorrect context. A good example from OE is that INHERIT += "testimage" will no longer work but IMAGE_CLASSES += "testimage" will, which makes the global scope cleaner by only including it where it is useful and intended to be used (images). Signed-off-by: Richard Purdie --- lib/bb/cooker.py | 1 + lib/bb/cookerdata.py | 2 ++ lib/bb/parse/parse_py/BBHandler.py | 29 ++++++++++++++++++----------- 3 files changed, 21 insertions(+), 11 deletions(-) diff --git a/lib/bb/cooker.py b/lib/bb/cooker.py index 2adf4d297d..1b6ee3032c 100644 --- a/lib/bb/cooker.py +++ b/lib/bb/cooker.py @@ -402,6 +402,7 @@ class BBCooker: for mc in self.databuilder.mcdata.values(): mc.renameVar("__depends", "__base_depends") self.add_filewatch(mc.getVar("__base_depends", False), self.configwatcher) + mc.setVar("__bbclasstype", "recipe") self.baseconfig_valid = True self.parsecache_valid = False diff --git a/lib/bb/cookerdata.py b/lib/bb/cookerdata.py index d54ac932e5..b3a171509b 100644 --- a/lib/bb/cookerdata.py +++ b/lib/bb/cookerdata.py @@ -254,6 +254,7 @@ class CookerDataBuilder(object): filtered_keys = bb.utils.approved_variables() bb.data.inheritFromOS(self.basedata, self.savedenv, filtered_keys) self.basedata.setVar("BB_ORIGENV", self.savedenv) + self.basedata.setVar("__bbclasstype", "global") if worker: self.basedata.setVar("BB_WORKERCONTEXT", "1") @@ -448,6 +449,7 @@ class CookerDataBuilder(object): # Handle any INHERITs and inherit the base class bbclasses = ["base"] + (data.getVar('INHERIT') or "").split() + bb.warn(str(bbclasses)) for bbclass in bbclasses: data = _inherit(bbclass, data) diff --git a/lib/bb/parse/parse_py/BBHandler.py b/lib/bb/parse/parse_py/BBHandler.py index 1189114341..18e6868387 100644 --- a/lib/bb/parse/parse_py/BBHandler.py +++ b/lib/bb/parse/parse_py/BBHandler.py @@ -44,17 +44,24 @@ def inherit(files, fn, lineno, d): __inherit_cache = d.getVar('__inherit_cache', False) or [] files = d.expand(files).split() for file in files: - if not os.path.isabs(file) and not file.endswith(".bbclass"): - file = os.path.join('classes', '%s.bbclass' % file) - - if not os.path.isabs(file): - bbpath = d.getVar("BBPATH") - abs_fn, attempts = bb.utils.which(bbpath, file, history=True) - for af in attempts: - if af != abs_fn: - bb.parse.mark_dependency(d, af) - if abs_fn: - file = abs_fn + classtype = d.getVar("__bbclasstype", False) + origfile = file + for t in ["classes-" + classtype, "classes"]: + file = origfile + if not os.path.isabs(file) and not file.endswith(".bbclass"): + file = os.path.join(t, '%s.bbclass' % file) + + if not os.path.isabs(file): + bbpath = d.getVar("BBPATH") + abs_fn, attempts = bb.utils.which(bbpath, file, history=True) + for af in attempts: + if af != abs_fn: + bb.parse.mark_dependency(d, af) + if abs_fn: + file = abs_fn + + if os.path.exists(file): + break if not os.path.exists(file): raise ParseError("Could not inherit file %s" % (file), fn, lineno)