Message ID | 20221127213640.539995-1-richard.purdie@linuxfoundation.org |
---|---|
State | New |
Headers | show |
Series | [RFC] base: Switch to use addpylib directive | expand |
Hi, I get the below backtrace with this patch on master-next. The same patchset works fine on debian-11. Are there any specific versions of packages that need to be installed as dependency ? Tested on host os: ubuntu-18.04/ubuntu-20.04 tuxbake@b8b6586bfb2c:/source/source/poky/build$ bitbake -e ERROR: Error parsing configuration files Traceback (most recent call last): File "/source/source/poky/bitbake/lib/bb/cookerdata.py", line 187, in parse_config_file(fn='/source/source/poky/meta/conf/layer.conf', data=<bb.data_smart.DataSmart object at 0x7f020867e250>, include=True): def parse_config_file(fn, data, include=True): > return bb.parse.handle(fn, data, include) File "/source/source/poky/bitbake/lib/bb/parse/__init__.py", line 107, in handle(fn='/source/source/poky/meta/conf/layer.conf', data=<bb.data_smart.DataSmart object at 0x7f020867e250>, include=True): with data.inchistory.include(fn): > return h['handle'](fn, data, include) raise ParseError("not a BitBake file", fn) File "/source/source/poky/bitbake/lib/bb/parse/parse_py/ConfHandler.py", line 152, in handle(fn='/source/source/poky/meta/conf/layer.conf', data=<bb.data_smart.DataSmart object at 0x7f020867e250>, include=True): data.setVar('FILE', abs_fn) > statements.eval(data) if oldfile: File "/source/source/poky/bitbake/lib/bb/parse/ast.py", line 21, in StatementGroup.eval(data=<bb.data_smart.DataSmart object at 0x7f020867e250>): for statement in self: > statement.eval(data) File "/source/source/poky/bitbake/lib/bb/parse/ast.py", line 324, in PyLibNode.eval(data=<bb.data_smart.DataSmart object at 0x7f020867e250>): bb.codeparser.modulecode_deps[entry][2] |= bb.codeparser.modulecode_deps[dup][2] > bb.codeparser.modulecode_deps[entry][3] |= bb.codeparser.modulecode_deps[dup][3] TypeError: unsupported operand type(s) for |=: 'dict' and 'dict' On Mon, 28 Nov 2022 at 03:06, Richard Purdie < richard.purdie@linuxfoundation.org> wrote: > Since bitbake now supports an official method to inject python modules, > switch to it. > > Anyone using OE_EXTRA_IMPORTS will need to adjust their code accordingly, > probably switching to their own module namespace. > > Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> > --- > [This patch is an RFC as part of a series on the bitbake-devel list] > > meta/classes-global/base.bbclass | 23 ----------------------- > meta/conf/layer.conf | 2 ++ > meta/lib/oe/__init__.py | 4 ++++ > 3 files changed, 6 insertions(+), 23 deletions(-) > > diff --git a/meta/classes-global/base.bbclass > b/meta/classes-global/base.bbclass > index c81aa51ef3f..c4ac43c5696 100644 > --- a/meta/classes-global/base.bbclass > +++ b/meta/classes-global/base.bbclass > @@ -15,31 +15,8 @@ inherit utils > inherit utility-tasks > inherit logging > > -OE_EXTRA_IMPORTS ?= "" > - > -OE_IMPORTS += "os sys time oe.path oe.utils oe.types oe.package > oe.packagegroup oe.sstatesig oe.lsb oe.cachedpath oe.license oe.qa > oe.reproducible oe.rust oe.buildcfg ${OE_EXTRA_IMPORTS}" > -OE_IMPORTS[type] = "list" > - > PACKAGECONFIG_CONFARGS ??= "" > > -def oe_import(d): > - import sys > - > - bbpath = [os.path.join(dir, "lib") for dir in > d.getVar("BBPATH").split(":")] > - sys.path[0:0] = [dir for dir in bbpath if dir not in sys.path] > - > - import oe.data > - for toimport in oe.data.typed_value("OE_IMPORTS", d): > - try: > - # Make a python object accessible from the metadata > - bb.utils._context[toimport.split(".", 1)[0]] = > __import__(toimport) > - except AttributeError as e: > - bb.error("Error importing OE modules: %s" % str(e)) > - return "" > - > -# We need the oe module name space early (before INHERITs get added) > -OE_IMPORTED := "${@oe_import(d)}" > - > inherit metadata_scm > > def lsb_distro_identifier(d): > diff --git a/meta/conf/layer.conf b/meta/conf/layer.conf > index 0ce90355baf..0fe05beebf0 100644 > --- a/meta/conf/layer.conf > +++ b/meta/conf/layer.conf > @@ -125,3 +125,5 @@ SSTATE_EXCLUDEDEPS_SYSROOT += > ".*->autoconf-archive-native" > # Avoid empty path entries > BITBAKEPATH := > "${@os.path.dirname(bb.utils.which(d.getVar('PATH'),'bitbake'))}" > PATH := "${@'${BITBAKEPATH}:' if '${BITBAKEPATH}' != '' else > ''}${HOSTTOOLS_DIR}" > + > +addpylib ${LAYERDIR}/lib oe > diff --git a/meta/lib/oe/__init__.py b/meta/lib/oe/__init__.py > index 92f002d8726..9166420b00e 100644 > --- a/meta/lib/oe/__init__.py > +++ b/meta/lib/oe/__init__.py > @@ -6,3 +6,7 @@ > > from pkgutil import extend_path > __path__ = extend_path(__path__, __name__) > + > +BBIMPORTS = ["os", "sys", "time", "oe.data", "oe.path", "oe.utils", > "oe.types", "oe.package", \ > + "oe.packagegroup", "oe.sstatesig", "oe.lsb", > "oe.cachedpath", "oe.license", \ > + "oe.qa", "oe.reproducible", "oe.rust", "oe.buildcfg"] > -- > 2.34.1 > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#173885): > https://lists.openembedded.org/g/openembedded-core/message/173885 > Mute This Topic: https://lists.openembedded.org/mt/95297771/3616781 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ > vishal.bhoj@linaro.org] > -=-=-=-=-=-=-=-=-=-=-=- > >
On Mon, 2022-11-28 at 19:29 +0530, Vishal Bhoj wrote: > I get the below backtrace with this patch on master-next. The same > patchset works fine on debian-11. Are there any specific versions of > packages that need to be installed as dependency ? > > Tested on host os: ubuntu-18.04/ubuntu-20.04 Which python versions are those? It looks like I perhaps assumed a piece of newer python syntax accidentally. I suspect this code only works with python 3.9+: https://peps.python.org/pep-0584/ I was wondering whether to remove this piece entirely anyway, I need to check the performance implications. Cheers, Richard
On Mon, 28 Nov 2022 at 20:14, Richard Purdie < richard.purdie@linuxfoundation.org> wrote: > On Mon, 2022-11-28 at 19:29 +0530, Vishal Bhoj wrote: > > I get the below backtrace with this patch on master-next. The same > > patchset works fine on debian-11. Are there any specific versions of > > packages that need to be installed as dependency ? > > > > Tested on host os: ubuntu-18.04/ubuntu-20.04 > Which python versions are those? It looks like I perhaps assumed a > piece of newer python syntax accidentally. I suspect this code only > works with python 3.9+: > Looks like it is the case: ubuntu-18.04: python 3.6 ubuntu-20.04: python 3.8 > > https://peps.python.org/pep-0584/ > > I was wondering whether to remove this piece entirely anyway, I need to > check the performance implications. > > Cheers, > > Richard > >
On Mon, 2022-11-28 at 21:00 +0530, Vishal Bhoj wrote: > > > On Mon, 28 Nov 2022 at 20:14, Richard Purdie > <richard.purdie@linuxfoundation.org> wrote: > > On Mon, 2022-11-28 at 19:29 +0530, Vishal Bhoj wrote: > > > I get the below backtrace with this patch on master-next. The > > > same > > > patchset works fine on debian-11. Are there any specific versions > > > of > > > packages that need to be installed as dependency ? > > > > > > Tested on host os: ubuntu-18.04/ubuntu-20.04 > > > > > > Which python versions are those? It looks like I perhaps assumed a > > piece of newer python syntax accidentally. I suspect this code only > > works with python 3.9+: > > > > > Looks like it is the case: > ubuntu-18.04: python 3.6 > ubuntu-20.04: python 3.8 > Thanks. I've tested and performance is the same without this chunk of code so I've sent an updated version. Cheers, Richard
diff --git a/meta/classes-global/base.bbclass b/meta/classes-global/base.bbclass index c81aa51ef3f..c4ac43c5696 100644 --- a/meta/classes-global/base.bbclass +++ b/meta/classes-global/base.bbclass @@ -15,31 +15,8 @@ inherit utils inherit utility-tasks inherit logging -OE_EXTRA_IMPORTS ?= "" - -OE_IMPORTS += "os sys time oe.path oe.utils oe.types oe.package oe.packagegroup oe.sstatesig oe.lsb oe.cachedpath oe.license oe.qa oe.reproducible oe.rust oe.buildcfg ${OE_EXTRA_IMPORTS}" -OE_IMPORTS[type] = "list" - PACKAGECONFIG_CONFARGS ??= "" -def oe_import(d): - import sys - - bbpath = [os.path.join(dir, "lib") for dir in d.getVar("BBPATH").split(":")] - sys.path[0:0] = [dir for dir in bbpath if dir not in sys.path] - - import oe.data - for toimport in oe.data.typed_value("OE_IMPORTS", d): - try: - # Make a python object accessible from the metadata - bb.utils._context[toimport.split(".", 1)[0]] = __import__(toimport) - except AttributeError as e: - bb.error("Error importing OE modules: %s" % str(e)) - return "" - -# We need the oe module name space early (before INHERITs get added) -OE_IMPORTED := "${@oe_import(d)}" - inherit metadata_scm def lsb_distro_identifier(d): diff --git a/meta/conf/layer.conf b/meta/conf/layer.conf index 0ce90355baf..0fe05beebf0 100644 --- a/meta/conf/layer.conf +++ b/meta/conf/layer.conf @@ -125,3 +125,5 @@ SSTATE_EXCLUDEDEPS_SYSROOT += ".*->autoconf-archive-native" # Avoid empty path entries BITBAKEPATH := "${@os.path.dirname(bb.utils.which(d.getVar('PATH'),'bitbake'))}" PATH := "${@'${BITBAKEPATH}:' if '${BITBAKEPATH}' != '' else ''}${HOSTTOOLS_DIR}" + +addpylib ${LAYERDIR}/lib oe diff --git a/meta/lib/oe/__init__.py b/meta/lib/oe/__init__.py index 92f002d8726..9166420b00e 100644 --- a/meta/lib/oe/__init__.py +++ b/meta/lib/oe/__init__.py @@ -6,3 +6,7 @@ from pkgutil import extend_path __path__ = extend_path(__path__, __name__) + +BBIMPORTS = ["os", "sys", "time", "oe.data", "oe.path", "oe.utils", "oe.types", "oe.package", \ + "oe.packagegroup", "oe.sstatesig", "oe.lsb", "oe.cachedpath", "oe.license", \ + "oe.qa", "oe.reproducible", "oe.rust", "oe.buildcfg"]
Since bitbake now supports an official method to inject python modules, switch to it. Anyone using OE_EXTRA_IMPORTS will need to adjust their code accordingly, probably switching to their own module namespace. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> --- [This patch is an RFC as part of a series on the bitbake-devel list] meta/classes-global/base.bbclass | 23 ----------------------- meta/conf/layer.conf | 2 ++ meta/lib/oe/__init__.py | 4 ++++ 3 files changed, 6 insertions(+), 23 deletions(-)