| Message ID | 20230928091304.4000026-1-archana.polampalli@windriver.com |
|---|---|
| State | Under Review |
| Headers | show |
| Series | [meta-oe,1/1] nodejs: upgrade 18.17.1 -> 20.5.1 | expand |
On Thu, Sep 28, 2023 at 11:13 AM Polampalli, Archana via lists.openembedded.org <archana.polampalli= windriver.com@lists.openembedded.org> wrote: > Update to latest release of latest LTS 20 release > > * node v20 introduces several new features and fixes many bugs and > CVEs as shown in [1] > > * Refresh 0001-liftoff-Correct-function-signatures.patch against 20.5.1 > > * License-Update: > - Change zlib version 1.2.13, October 13th, 2022 to version > 1.2.13.1, October xxth, 2022 [2] > - Change Copyright 2023 from Ada authors to Yagiz Nizipli and > Daniel Lemire [4] > > * Remove big-endian.patch as it is merged in v20.x [5] [6] > > * Remove below list of patches since mips32 is deleted from v8 as part > of update V8 to 10.7.193.13 [7] [8] > - mips-less-memory.patch > - 0001-mips-Use-32bit-cast-for-operand-on-mips32.patch > > * Update Using-native-binaries.patch for node_js2c, it resolved below > do_compile error [9] > Error: > /bin/sh: line 1: > build/tmp/work/core2-64-poky-linux/nodejs/20.5.1/node-v20.5.1/out/Release/node_js2c: > No such file or directory > > * Remove obsolete dtrace & etw configure options (we had: > --without-<feature>) from the recipe [10] FYI: (in case someone else are seeing similar issues now) With this upgrade I'm seeing some files created during the build with node not to be intercepted by pseudo and recorded in its database. This results in sstate_report_unihash failure, because appinfo.json is now owned by 1026:1026 (my build UID/GID) instead of 0:0 as before: DEBUG: Executing python function sstate_report_unihash WARNING: KeyError in ./package/usr/palm/applications/com.webos.app.videoplayer/appinfo.json ERROR: Error executing a python function in exec_func_python() autogenerated: The stack trace of python calls that resulted in this exception/failure was: File: 'exec_func_python() autogenerated', lineno: 2, function: <module> 0001: *** 0002:sstate_report_unihash(d) 0003: File: '/OE/build/luneos-nanbield/openembedded-core/meta/classes-global/sstate.bbclass', lineno: 880, function: sstate_report_unihash 0876: report_unihash = getattr(bb.parse.siggen, 'report_unihash', None) 0877: 0878: if report_unihash: 0879: ss = sstate_state_fromvars(d) *** 0880: report_unihash(os.getcwd(), ss['task'], d) 0881:} 0882: 0883:# 0884:# Shell function to decompress and prepare a package for installation File: '/OE/build/luneos-nanbield/bitbake/lib/bb/siggen.py', lineno: 659, function: report_unihash 0655: 0656: if "." in self.method: 0657: (module, method) = self.method.rsplit('.', 1) 0658: locs['method'] = getattr(importlib.import_module(module), method) *** 0659: outhash = bb.utils.better_eval('method(path, sigfile, task, d)', locs) 0660: else: 0661: outhash = bb.utils.better_eval(self.method + '(path, sigfile, task, d)', locs) 0662: 0663: try: File: '/OE/build/luneos-nanbield/bitbake/lib/bb/utils.py', lineno: 434, function: better_eval 0430: if extraglobals: 0431: ctx = copy.copy(ctx) 0432: for g in extraglobals: 0433: ctx[g] = extraglobals[g] *** 0434: return eval(source, ctx, locals) 0435: 0436:@contextmanager 0437:def fileslocked(files, *args, **kwargs): 0438: """Context manager for locking and unlocking file locks.""" File: '<string>', lineno: 1, function: <module> File "<string>", line 1, in <module> File: '/OE/build/luneos-nanbield/openembedded-core/meta/lib/oe/sstatesig.py', lineno: 649, function: OEOuthashBasic 0645: process(root) 0646: for f in files: 0647: if f == 'fixmepath': 0648: continue *** 0649: process(os.path.join(root, f)) 0650: 0651: for dir in dirs: 0652: if os.path.islink(os.path.join(root, dir)): 0653: process(os.path.join(root, dir)) File: '/OE/build/luneos-nanbield/openembedded-core/meta/lib/oe/sstatesig.py', lineno: 588, function: process 0584: except KeyError as e: 0585: bb.warn("KeyError in %s" % path) 0586: msg = ("KeyError: %s\nPath %s is owned by uid %d, gid %d, which doesn't match " 0587: "any user/group on target. This may be due to host contamination." % (e, path, s.st_uid, s.st_gid)) *** 0588: raise Exception(msg).with_traceback(e.__traceback__) 0589: 0590: if include_timestamps: 0591: update_hash(" %10d" % s.st_mtime) 0592: File: '/OE/build/luneos-nanbield/openembedded-core/meta/lib/oe/sstatesig.py', lineno: 582, function: process 0578: else: 0579: add_perm(stat.S_IXOTH, 'x') 0580: 0581: try: *** 0582: update_hash(" %10s" % pwd.getpwuid(s.st_uid).pw_name) 0583: update_hash(" %10s" % grp.getgrgid(s.st_gid).gr_name) 0584: except KeyError as e: 0585: bb.warn("KeyError in %s" % path) 0586: msg = ("KeyError: %s\nPath %s is owned by uid %d, gid %d, which doesn't match " Exception: Exception: KeyError: 'getpwuid(): uid not found: 1026' Path ./package/usr/palm/applications/com.webos.app.videoplayer/appinfo.json is owned by uid 1026, gid 1026, which doesn't match any user/group on target. This may be due to host contamination. martin@jama /OE/build/luneos-nanbield/tmp-glibc/work/corei7-64-webos-linux/com.webos.app.videoplayer $ sqlite3 0.0.1-12+git-nodejs-20/pseudo/files.db "select * from files where path like '%yer/appinfo.json'" 1107|/OE/build/luneos-nanbield/tmp-glibc/work/corei7-64-webos-linux/com.webos.app.videoplayer/0.0.1-12+git/sstate-build-package/package/usr/palm/applications/com.webos.app.videoplayer/appinfo.json|66305|41202418|1026|1026|33188|0|0 1192|/OE/build/luneos-nanbield/tmp-glibc/work/corei7-64-webos-linux/com.webos.app.videoplayer/0.0.1-12+git/sstate-build-package/packages-split/com.webos.app.videoplayer/usr/palm/applications/com.webos.app.videoplayer/appinfo.json|66305|41 202418|1026|1026|33188|0|0 martin@jama /OE/build/luneos-nanbield/tmp-glibc/work/corei7-64-webos-linux/com.webos.app.videoplayer $ sqlite3 0.0.1-12+git-nodejs-18/pseudo/files.db "select * from files where path like '%yer/appinfo.json'" 56|/OE/build/luneos-nanbield/tmp-glibc/work/corei7-64-webos-linux/com.webos.app.videoplayer/0.0.1-12+git/image/usr/palm/applications/com.webos.app.videoplayer/appinfo.json|66305|42878067|0|0|33188|0|0 374|/OE/build/luneos-nanbield/tmp-glibc/work/corei7-64-webos-linux/com.webos.app.videoplayer/0.0.1-12+git/package/usr/palm/applications/com.webos.app.videoplayer/appinfo.json|66305|42878641|0|0|33188|0|0 459|/OE/build/luneos-nanbield/tmp-glibc/work/corei7-64-webos-linux/com.webos.app.videoplayer/0.0.1-12+git/packages-split/com.webos.app.videoplayer/usr/palm/applications/com.webos.app.videoplayer/appinfo.json|66305|42878641|0|0|33188|0|0 I'll provide more details when I debug this further. Regards,
I've narrowed this down to simple reproducer with webpack creating pack with "hello world" as shown in: https://github.com/shr-project/com.webos.app.minimal/tree/recipes Then with this reproducer I've bisected it to nodejs-20.3.0 with libuv upgrade 1.45.0 which brings support for io_uring which doesn't work with pseudo in: https://github.com/libuv/libuv/pull/3952/commits/26c79a942b92573a1388c0ee8a6ad4397f009318 Now we have 3 options: 1) either every use of nodejs-native where io_uring might be used (like in webpack) needs to export UV_USE_IO_URING=0 before using node, but be aware that the PR says it's only temporary and this variable might be removed in future. Unfortunately many people are lucky enough not to know what pseudo even is, so they might not realize that host-user-contamination QA issues or "KeyError: 'getpwuid(): uid not found" might be related to this (and they probably won't find this e-mail as well). 2) revert the io_uring changes from bundled libuv, I've already verified this with nodejs-20.3.0, for 20.8.1 it would need couple more commits to be reverted as 20.8.1 uses even newer libuv-1.46.0. The changes are massive and it will be PIA to update them for every nodejs release, patch for keeping UV_USE_IO_URING support after it's removed from upstream libuv will be much smaller, but PIA to use UV_USE_IO_URING whenever using nodejs-native during build. 3) fix pseudo to correctly work with io_uring calls, definitely best long-term solution as more and more tools will now switch to io_uring, but without active maintainer of pseudo it might be _very_ difficult to implement this and there aren't many people willing to touch this beast. Any opinions? On Mon, Oct 2, 2023 at 7:08 PM Martin Jansa via lists.openembedded.org <martin.jansa=gmail.com@lists.openembedded.org> wrote: > On Thu, Sep 28, 2023 at 11:13 AM Polampalli, Archana via > lists.openembedded.org <archana.polampalli= > windriver.com@lists.openembedded.org> wrote: > >> Update to latest release of latest LTS 20 release >> >> * node v20 introduces several new features and fixes many bugs and >> CVEs as shown in [1] >> >> * Refresh 0001-liftoff-Correct-function-signatures.patch against >> 20.5.1 >> >> * License-Update: >> - Change zlib version 1.2.13, October 13th, 2022 to version >> 1.2.13.1, October xxth, 2022 [2] >> - Change Copyright 2023 from Ada authors to Yagiz Nizipli and >> Daniel Lemire [4] >> >> * Remove big-endian.patch as it is merged in v20.x [5] [6] >> >> * Remove below list of patches since mips32 is deleted from v8 as >> part of update V8 to 10.7.193.13 [7] [8] >> - mips-less-memory.patch >> - 0001-mips-Use-32bit-cast-for-operand-on-mips32.patch >> >> * Update Using-native-binaries.patch for node_js2c, it resolved below >> do_compile error [9] >> Error: >> /bin/sh: line 1: >> build/tmp/work/core2-64-poky-linux/nodejs/20.5.1/node-v20.5.1/out/Release/node_js2c: >> No such file or directory >> >> * Remove obsolete dtrace & etw configure options (we had: >> --without-<feature>) from the recipe [10] > > > FYI: (in case someone else are seeing similar issues now) > > With this upgrade I'm seeing some files created during the build with node > not to be intercepted by pseudo and recorded in its database. > > This results in sstate_report_unihash failure, because appinfo.json is now > owned by 1026:1026 (my build UID/GID) instead of 0:0 as before: > > DEBUG: Executing python function sstate_report_unihash > WARNING: KeyError in > ./package/usr/palm/applications/com.webos.app.videoplayer/appinfo.json > ERROR: Error executing a python function in exec_func_python() > autogenerated: > > The stack trace of python calls that resulted in this exception/failure > was: > File: 'exec_func_python() autogenerated', lineno: 2, function: <module> > 0001: > *** 0002:sstate_report_unihash(d) > 0003: > File: > '/OE/build/luneos-nanbield/openembedded-core/meta/classes-global/sstate.bbclass', > lineno: 880, function: sstate_report_unihash > 0876: report_unihash = getattr(bb.parse.siggen, 'report_unihash', > None) > 0877: > 0878: if report_unihash: > 0879: ss = sstate_state_fromvars(d) > *** 0880: report_unihash(os.getcwd(), ss['task'], d) > 0881:} > 0882: > 0883:# > 0884:# Shell function to decompress and prepare a package for > installation > File: '/OE/build/luneos-nanbield/bitbake/lib/bb/siggen.py', lineno: 659, > function: report_unihash > 0655: > 0656: if "." in self.method: > 0657: (module, method) = self.method.rsplit('.', 1) > 0658: locs['method'] = > getattr(importlib.import_module(module), method) > *** 0659: outhash = bb.utils.better_eval('method(path, > sigfile, task, d)', locs) > 0660: else: > 0661: outhash = bb.utils.better_eval(self.method + > '(path, sigfile, task, d)', locs) > 0662: > 0663: try: > File: '/OE/build/luneos-nanbield/bitbake/lib/bb/utils.py', lineno: 434, > function: better_eval > 0430: if extraglobals: > 0431: ctx = copy.copy(ctx) > 0432: for g in extraglobals: > 0433: ctx[g] = extraglobals[g] > *** 0434: return eval(source, ctx, locals) > 0435: > 0436:@contextmanager > 0437:def fileslocked(files, *args, **kwargs): > 0438: """Context manager for locking and unlocking file locks.""" > File: '<string>', lineno: 1, function: <module> > File "<string>", line 1, in <module> > > File: > '/OE/build/luneos-nanbield/openembedded-core/meta/lib/oe/sstatesig.py', > lineno: 649, function: OEOuthashBasic > 0645: process(root) > 0646: for f in files: > 0647: if f == 'fixmepath': > 0648: continue > *** 0649: process(os.path.join(root, f)) > 0650: > 0651: for dir in dirs: > 0652: if os.path.islink(os.path.join(root, dir)): > 0653: process(os.path.join(root, dir)) > File: > '/OE/build/luneos-nanbield/openembedded-core/meta/lib/oe/sstatesig.py', > lineno: 588, function: process > 0584: except KeyError as e: > 0585: bb.warn("KeyError in %s" % path) > 0586: msg = ("KeyError: %s\nPath %s is owned > by uid %d, gid %d, which doesn't match " > 0587: "any user/group on target. This may > be due to host contamination." % (e, path, s.st_uid, s.st_gid)) > *** 0588: raise > Exception(msg).with_traceback(e.__traceback__) > 0589: > 0590: if include_timestamps: > 0591: update_hash(" %10d" % s.st_mtime) > 0592: > File: > '/OE/build/luneos-nanbield/openembedded-core/meta/lib/oe/sstatesig.py', > lineno: 582, function: process > 0578: else: > 0579: add_perm(stat.S_IXOTH, 'x') > 0580: > 0581: try: > *** 0582: update_hash(" %10s" % > pwd.getpwuid(s.st_uid).pw_name) > 0583: update_hash(" %10s" % > grp.getgrgid(s.st_gid).gr_name) > 0584: except KeyError as e: > 0585: bb.warn("KeyError in %s" % path) > 0586: msg = ("KeyError: %s\nPath %s is owned > by uid %d, gid %d, which doesn't match " > Exception: Exception: KeyError: 'getpwuid(): uid not found: 1026' > Path > ./package/usr/palm/applications/com.webos.app.videoplayer/appinfo.json is > owned by uid 1026, gid 1026, which doesn't match any user/group on target. > This may be due to host contamination. > > martin@jama > /OE/build/luneos-nanbield/tmp-glibc/work/corei7-64-webos-linux/com.webos.app.videoplayer > $ sqlite3 0.0.1-12+git-nodejs-20/pseudo/files.db "select * from files where > path like '%yer/appinfo.json'" > > 1107|/OE/build/luneos-nanbield/tmp-glibc/work/corei7-64-webos-linux/com.webos.app.videoplayer/0.0.1-12+git/sstate-build-package/package/usr/palm/applications/com.webos.app.videoplayer/appinfo.json|66305|41202418|1026|1026|33188|0|0 > > 1192|/OE/build/luneos-nanbield/tmp-glibc/work/corei7-64-webos-linux/com.webos.app.videoplayer/0.0.1-12+git/sstate-build-package/packages-split/com.webos.app.videoplayer/usr/palm/applications/com.webos.app.videoplayer/appinfo.json|66305|41 > 202418|1026|1026|33188|0|0 > > martin@jama > /OE/build/luneos-nanbield/tmp-glibc/work/corei7-64-webos-linux/com.webos.app.videoplayer > $ sqlite3 0.0.1-12+git-nodejs-18/pseudo/files.db "select * from files where > path like '%yer/appinfo.json'" > > 56|/OE/build/luneos-nanbield/tmp-glibc/work/corei7-64-webos-linux/com.webos.app.videoplayer/0.0.1-12+git/image/usr/palm/applications/com.webos.app.videoplayer/appinfo.json|66305|42878067|0|0|33188|0|0 > > 374|/OE/build/luneos-nanbield/tmp-glibc/work/corei7-64-webos-linux/com.webos.app.videoplayer/0.0.1-12+git/package/usr/palm/applications/com.webos.app.videoplayer/appinfo.json|66305|42878641|0|0|33188|0|0 > > 459|/OE/build/luneos-nanbield/tmp-glibc/work/corei7-64-webos-linux/com.webos.app.videoplayer/0.0.1-12+git/packages-split/com.webos.app.videoplayer/usr/palm/applications/com.webos.app.videoplayer/appinfo.json|66305|42878641|0|0|33188|0|0 > > I'll provide more details when I debug this further. > > Regards, > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#105308): > https://lists.openembedded.org/g/openembedded-devel/message/105308 > Mute This Topic: https://lists.openembedded.org/mt/101098430/3617156 > Group Owner: openembedded-devel+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-devel/unsub [ > martin.jansa@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- > >
On Wed, Oct 18, 2023 at 1:41 PM Martin Jansa <martin.jansa@gmail.com> wrote: > I've narrowed this down to simple reproducer with webpack creating pack > with "hello world" as shown in: > https://github.com/shr-project/com.webos.app.minimal/tree/recipes > > Then with this reproducer I've bisected it to nodejs-20.3.0 with libuv > upgrade 1.45.0 which brings support for io_uring which doesn't work with > pseudo in: > > https://github.com/libuv/libuv/pull/3952/commits/26c79a942b92573a1388c0ee8a6ad4397f009318 > > Now we have 3 options: > > 1) either every use of nodejs-native where io_uring might be used (like in > webpack) needs to export UV_USE_IO_URING=0 before using node, but be aware > that the PR says it's only temporary and this variable might be removed in > future. Unfortunately many people are lucky enough not to know what pseudo > even is, so they might not realize that host-user-contamination QA issues > or "KeyError: 'getpwuid(): uid not found" might be related to this (and > they probably won't find this e-mail as well). > > 2) revert the io_uring changes from bundled libuv, I've already verified > this with nodejs-20.3.0, for 20.8.1 it would need couple more commits to be > reverted as 20.8.1 uses even newer libuv-1.46.0. The changes are massive > and it will be PIA to update them for every nodejs release, patch for > keeping UV_USE_IO_URING support after it's removed from upstream libuv will > be much smaller, but PIA to use UV_USE_IO_URING whenever using > nodejs-native during build. > > 3) fix pseudo to correctly work with io_uring calls, definitely best > long-term solution as more and more tools will now switch to io_uring, but > without active maintainer of pseudo it might be _very_ difficult to > implement this and there aren't many people willing to touch this beast. > > Any opinions? > > I would advise we not be overly aggressive about updates for important tools like NodeJS and Django. I don't think many consumers are building products on the latest greatest NodeJS release (which this isn't, latest is 21), many may not even be on the latest LTS (18 vs.16). While we can have a floating "latest" recipe, we should keep an LTS recipe. We tried to do this for Django in meta-python, but later commits wiped out the comments and the upstream check. https://git.openembedded.org/meta-openembedded/commit/meta-python/recipes-devtools/python?id=961755a7bd17f7347f7dd0f2de3d2400a9c36f09 On Mon, Oct 2, 2023 at 7:08 PM Martin Jansa via lists.openembedded.org > <martin.jansa=gmail.com@lists.openembedded.org> wrote: > >> On Thu, Sep 28, 2023 at 11:13 AM Polampalli, Archana via >> lists.openembedded.org <archana.polampalli= >> windriver.com@lists.openembedded.org> wrote: >> >>> Update to latest release of latest LTS 20 release >>> >>> * node v20 introduces several new features and fixes many bugs and >>> CVEs as shown in [1] >>> >>> * Refresh 0001-liftoff-Correct-function-signatures.patch against >>> 20.5.1 >>> >>> * License-Update: >>> - Change zlib version 1.2.13, October 13th, 2022 to version >>> 1.2.13.1, October xxth, 2022 [2] >>> - Change Copyright 2023 from Ada authors to Yagiz Nizipli and >>> Daniel Lemire [4] >>> >>> * Remove big-endian.patch as it is merged in v20.x [5] [6] >>> >>> * Remove below list of patches since mips32 is deleted from v8 as >>> part of update V8 to 10.7.193.13 [7] [8] >>> - mips-less-memory.patch >>> - 0001-mips-Use-32bit-cast-for-operand-on-mips32.patch >>> >>> * Update Using-native-binaries.patch for node_js2c, it resolved >>> below do_compile error [9] >>> Error: >>> /bin/sh: line 1: >>> build/tmp/work/core2-64-poky-linux/nodejs/20.5.1/node-v20.5.1/out/Release/node_js2c: >>> No such file or directory >>> >>> * Remove obsolete dtrace & etw configure options (we had: >>> --without-<feature>) from the recipe [10] >> >> >> FYI: (in case someone else are seeing similar issues now) >> >> With this upgrade I'm seeing some files created during the build with >> node not to be intercepted by pseudo and recorded in its database. >> >> This results in sstate_report_unihash failure, because appinfo.json is >> now owned by 1026:1026 (my build UID/GID) instead of 0:0 as before: >> >> DEBUG: Executing python function sstate_report_unihash >> WARNING: KeyError in >> ./package/usr/palm/applications/com.webos.app.videoplayer/appinfo.json >> ERROR: Error executing a python function in exec_func_python() >> autogenerated: >> >> The stack trace of python calls that resulted in this exception/failure >> was: >> File: 'exec_func_python() autogenerated', lineno: 2, function: <module> >> 0001: >> *** 0002:sstate_report_unihash(d) >> 0003: >> File: >> '/OE/build/luneos-nanbield/openembedded-core/meta/classes-global/sstate.bbclass', >> lineno: 880, function: sstate_report_unihash >> 0876: report_unihash = getattr(bb.parse.siggen, 'report_unihash', >> None) >> 0877: >> 0878: if report_unihash: >> 0879: ss = sstate_state_fromvars(d) >> *** 0880: report_unihash(os.getcwd(), ss['task'], d) >> 0881:} >> 0882: >> 0883:# >> 0884:# Shell function to decompress and prepare a package for >> installation >> File: '/OE/build/luneos-nanbield/bitbake/lib/bb/siggen.py', lineno: 659, >> function: report_unihash >> 0655: >> 0656: if "." in self.method: >> 0657: (module, method) = self.method.rsplit('.', 1) >> 0658: locs['method'] = >> getattr(importlib.import_module(module), method) >> *** 0659: outhash = bb.utils.better_eval('method(path, >> sigfile, task, d)', locs) >> 0660: else: >> 0661: outhash = bb.utils.better_eval(self.method + >> '(path, sigfile, task, d)', locs) >> 0662: >> 0663: try: >> File: '/OE/build/luneos-nanbield/bitbake/lib/bb/utils.py', lineno: 434, >> function: better_eval >> 0430: if extraglobals: >> 0431: ctx = copy.copy(ctx) >> 0432: for g in extraglobals: >> 0433: ctx[g] = extraglobals[g] >> *** 0434: return eval(source, ctx, locals) >> 0435: >> 0436:@contextmanager >> 0437:def fileslocked(files, *args, **kwargs): >> 0438: """Context manager for locking and unlocking file locks.""" >> File: '<string>', lineno: 1, function: <module> >> File "<string>", line 1, in <module> >> >> File: >> '/OE/build/luneos-nanbield/openembedded-core/meta/lib/oe/sstatesig.py', >> lineno: 649, function: OEOuthashBasic >> 0645: process(root) >> 0646: for f in files: >> 0647: if f == 'fixmepath': >> 0648: continue >> *** 0649: process(os.path.join(root, f)) >> 0650: >> 0651: for dir in dirs: >> 0652: if os.path.islink(os.path.join(root, dir)): >> 0653: process(os.path.join(root, dir)) >> File: >> '/OE/build/luneos-nanbield/openembedded-core/meta/lib/oe/sstatesig.py', >> lineno: 588, function: process >> 0584: except KeyError as e: >> 0585: bb.warn("KeyError in %s" % path) >> 0586: msg = ("KeyError: %s\nPath %s is owned >> by uid %d, gid %d, which doesn't match " >> 0587: "any user/group on target. This may >> be due to host contamination." % (e, path, s.st_uid, s.st_gid)) >> *** 0588: raise >> Exception(msg).with_traceback(e.__traceback__) >> 0589: >> 0590: if include_timestamps: >> 0591: update_hash(" %10d" % s.st_mtime) >> 0592: >> File: >> '/OE/build/luneos-nanbield/openembedded-core/meta/lib/oe/sstatesig.py', >> lineno: 582, function: process >> 0578: else: >> 0579: add_perm(stat.S_IXOTH, 'x') >> 0580: >> 0581: try: >> *** 0582: update_hash(" %10s" % >> pwd.getpwuid(s.st_uid).pw_name) >> 0583: update_hash(" %10s" % >> grp.getgrgid(s.st_gid).gr_name) >> 0584: except KeyError as e: >> 0585: bb.warn("KeyError in %s" % path) >> 0586: msg = ("KeyError: %s\nPath %s is owned >> by uid %d, gid %d, which doesn't match " >> Exception: Exception: KeyError: 'getpwuid(): uid not found: 1026' >> Path >> ./package/usr/palm/applications/com.webos.app.videoplayer/appinfo.json is >> owned by uid 1026, gid 1026, which doesn't match any user/group on target. >> This may be due to host contamination. >> >> martin@jama >> /OE/build/luneos-nanbield/tmp-glibc/work/corei7-64-webos-linux/com.webos.app.videoplayer >> $ sqlite3 0.0.1-12+git-nodejs-20/pseudo/files.db "select * from files where >> path like '%yer/appinfo.json'" >> >> 1107|/OE/build/luneos-nanbield/tmp-glibc/work/corei7-64-webos-linux/com.webos.app.videoplayer/0.0.1-12+git/sstate-build-package/package/usr/palm/applications/com.webos.app.videoplayer/appinfo.json|66305|41202418|1026|1026|33188|0|0 >> >> 1192|/OE/build/luneos-nanbield/tmp-glibc/work/corei7-64-webos-linux/com.webos.app.videoplayer/0.0.1-12+git/sstate-build-package/packages-split/com.webos.app.videoplayer/usr/palm/applications/com.webos.app.videoplayer/appinfo.json|66305|41 >> 202418|1026|1026|33188|0|0 >> >> martin@jama >> /OE/build/luneos-nanbield/tmp-glibc/work/corei7-64-webos-linux/com.webos.app.videoplayer >> $ sqlite3 0.0.1-12+git-nodejs-18/pseudo/files.db "select * from files where >> path like '%yer/appinfo.json'" >> >> 56|/OE/build/luneos-nanbield/tmp-glibc/work/corei7-64-webos-linux/com.webos.app.videoplayer/0.0.1-12+git/image/usr/palm/applications/com.webos.app.videoplayer/appinfo.json|66305|42878067|0|0|33188|0|0 >> >> 374|/OE/build/luneos-nanbield/tmp-glibc/work/corei7-64-webos-linux/com.webos.app.videoplayer/0.0.1-12+git/package/usr/palm/applications/com.webos.app.videoplayer/appinfo.json|66305|42878641|0|0|33188|0|0 >> >> 459|/OE/build/luneos-nanbield/tmp-glibc/work/corei7-64-webos-linux/com.webos.app.videoplayer/0.0.1-12+git/packages-split/com.webos.app.videoplayer/usr/palm/applications/com.webos.app.videoplayer/appinfo.json|66305|42878641|0|0|33188|0|0 >> >> I'll provide more details when I debug this further. >> >> Regards, >> >> >> >> > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#105583): > https://lists.openembedded.org/g/openembedded-devel/message/105583 > Mute This Topic: https://lists.openembedded.org/mt/101098430/924729 > Group Owner: openembedded-devel+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-devel/unsub [ > ticotimo@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- > >
On Wed, Oct 18, 2023 at 4:19 PM Tim Orling <ticotimo@gmail.com> wrote: > > > > On Wed, Oct 18, 2023 at 1:41 PM Martin Jansa <martin.jansa@gmail.com> wrote: >> >> I've narrowed this down to simple reproducer with webpack creating pack with "hello world" as shown in: >> https://github.com/shr-project/com.webos.app.minimal/tree/recipes >> >> Then with this reproducer I've bisected it to nodejs-20.3.0 with libuv upgrade 1.45.0 which brings support for io_uring which doesn't work with pseudo in: >> https://github.com/libuv/libuv/pull/3952/commits/26c79a942b92573a1388c0ee8a6ad4397f009318 >> >> Now we have 3 options: >> >> 1) either every use of nodejs-native where io_uring might be used (like in webpack) needs to export UV_USE_IO_URING=0 before using node, but be aware that the PR says it's only temporary and this variable might be removed in future. Unfortunately many people are lucky enough not to know what pseudo even is, so they might not realize that host-user-contamination QA issues or "KeyError: 'getpwuid(): uid not found" might be related to this (and they probably won't find this e-mail as well). >> >> 2) revert the io_uring changes from bundled libuv, I've already verified this with nodejs-20.3.0, for 20.8.1 it would need couple more commits to be reverted as 20.8.1 uses even newer libuv-1.46.0. The changes are massive and it will be PIA to update them for every nodejs release, patch for keeping UV_USE_IO_URING support after it's removed from upstream libuv will be much smaller, but PIA to use UV_USE_IO_URING whenever using nodejs-native during build. >> >> 3) fix pseudo to correctly work with io_uring calls, definitely best long-term solution as more and more tools will now switch to io_uring, but without active maintainer of pseudo it might be _very_ difficult to implement this and there aren't many people willing to touch this beast. >> >> Any opinions? >> > > I would advise we not be overly aggressive about updates for important tools like NodeJS and Django. > > I don't think many consumers are building products on the latest greatest NodeJS release (which this isn't, latest is 21), many may not even be on the latest LTS (18 vs.16). While we can have a floating "latest" recipe, we should keep an LTS recipe. > We tried to do this for Django in meta-python, but later commits wiped out the comments and the upstream check. > https://git.openembedded.org/meta-openembedded/commit/meta-python/recipes-devtools/python?id=961755a7bd17f7347f7dd0f2de3d2400a9c36f09 > 20.x is next LTS and we already branched nanbield with 18.x, for master and upcoming yocto LTS release in April 2024, I would like to see up move to 20.x and figure out problems during this time early enough so we can stick to 20.x for yocto LTS. Unless we run into blocker issues. > >> On Mon, Oct 2, 2023 at 7:08 PM Martin Jansa via lists.openembedded.org <martin.jansa=gmail.com@lists.openembedded.org> wrote: >>> >>> On Thu, Sep 28, 2023 at 11:13 AM Polampalli, Archana via lists.openembedded.org <archana.polampalli=windriver.com@lists.openembedded.org> wrote: >>>> >>>> Update to latest release of latest LTS 20 release >>>> >>>> * node v20 introduces several new features and fixes many bugs and CVEs as shown in [1] >>>> >>>> * Refresh 0001-liftoff-Correct-function-signatures.patch against 20.5.1 >>>> >>>> * License-Update: >>>> - Change zlib version 1.2.13, October 13th, 2022 to version 1.2.13.1, October xxth, 2022 [2] >>>> - Change Copyright 2023 from Ada authors to Yagiz Nizipli and Daniel Lemire [4] >>>> >>>> * Remove big-endian.patch as it is merged in v20.x [5] [6] >>>> >>>> * Remove below list of patches since mips32 is deleted from v8 as part of update V8 to 10.7.193.13 [7] [8] >>>> - mips-less-memory.patch >>>> - 0001-mips-Use-32bit-cast-for-operand-on-mips32.patch >>>> >>>> * Update Using-native-binaries.patch for node_js2c, it resolved below do_compile error [9] >>>> Error: >>>> /bin/sh: line 1: build/tmp/work/core2-64-poky-linux/nodejs/20.5.1/node-v20.5.1/out/Release/node_js2c: No such file or directory >>>> >>>> * Remove obsolete dtrace & etw configure options (we had: --without-<feature>) from the recipe [10] >>> >>> >>> FYI: (in case someone else are seeing similar issues now) >>> >>> With this upgrade I'm seeing some files created during the build with node not to be intercepted by pseudo and recorded in its database. >>> >>> This results in sstate_report_unihash failure, because appinfo.json is now owned by 1026:1026 (my build UID/GID) instead of 0:0 as before: >>> >>> DEBUG: Executing python function sstate_report_unihash >>> WARNING: KeyError in ./package/usr/palm/applications/com.webos.app.videoplayer/appinfo.json >>> ERROR: Error executing a python function in exec_func_python() autogenerated: >>> >>> The stack trace of python calls that resulted in this exception/failure was: >>> File: 'exec_func_python() autogenerated', lineno: 2, function: <module> >>> 0001: >>> *** 0002:sstate_report_unihash(d) >>> 0003: >>> File: '/OE/build/luneos-nanbield/openembedded-core/meta/classes-global/sstate.bbclass', lineno: 880, function: sstate_report_unihash >>> 0876: report_unihash = getattr(bb.parse.siggen, 'report_unihash', None) >>> 0877: >>> 0878: if report_unihash: >>> 0879: ss = sstate_state_fromvars(d) >>> *** 0880: report_unihash(os.getcwd(), ss['task'], d) >>> 0881:} >>> 0882: >>> 0883:# >>> 0884:# Shell function to decompress and prepare a package for installation >>> File: '/OE/build/luneos-nanbield/bitbake/lib/bb/siggen.py', lineno: 659, function: report_unihash >>> 0655: >>> 0656: if "." in self.method: >>> 0657: (module, method) = self.method.rsplit('.', 1) >>> 0658: locs['method'] = getattr(importlib.import_module(module), method) >>> *** 0659: outhash = bb.utils.better_eval('method(path, sigfile, task, d)', locs) >>> 0660: else: >>> 0661: outhash = bb.utils.better_eval(self.method + '(path, sigfile, task, d)', locs) >>> 0662: >>> 0663: try: >>> File: '/OE/build/luneos-nanbield/bitbake/lib/bb/utils.py', lineno: 434, function: better_eval >>> 0430: if extraglobals: >>> 0431: ctx = copy.copy(ctx) >>> 0432: for g in extraglobals: >>> 0433: ctx[g] = extraglobals[g] >>> *** 0434: return eval(source, ctx, locals) >>> 0435: >>> 0436:@contextmanager >>> 0437:def fileslocked(files, *args, **kwargs): >>> 0438: """Context manager for locking and unlocking file locks.""" >>> File: '<string>', lineno: 1, function: <module> >>> File "<string>", line 1, in <module> >>> >>> File: '/OE/build/luneos-nanbield/openembedded-core/meta/lib/oe/sstatesig.py', lineno: 649, function: OEOuthashBasic >>> 0645: process(root) >>> 0646: for f in files: >>> 0647: if f == 'fixmepath': >>> 0648: continue >>> *** 0649: process(os.path.join(root, f)) >>> 0650: >>> 0651: for dir in dirs: >>> 0652: if os.path.islink(os.path.join(root, dir)): >>> 0653: process(os.path.join(root, dir)) >>> File: '/OE/build/luneos-nanbield/openembedded-core/meta/lib/oe/sstatesig.py', lineno: 588, function: process >>> 0584: except KeyError as e: >>> 0585: bb.warn("KeyError in %s" % path) >>> 0586: msg = ("KeyError: %s\nPath %s is owned by uid %d, gid %d, which doesn't match " >>> 0587: "any user/group on target. This may be due to host contamination." % (e, path, s.st_uid, s.st_gid)) >>> *** 0588: raise Exception(msg).with_traceback(e.__traceback__) >>> 0589: >>> 0590: if include_timestamps: >>> 0591: update_hash(" %10d" % s.st_mtime) >>> 0592: >>> File: '/OE/build/luneos-nanbield/openembedded-core/meta/lib/oe/sstatesig.py', lineno: 582, function: process >>> 0578: else: >>> 0579: add_perm(stat.S_IXOTH, 'x') >>> 0580: >>> 0581: try: >>> *** 0582: update_hash(" %10s" % pwd.getpwuid(s.st_uid).pw_name) >>> 0583: update_hash(" %10s" % grp.getgrgid(s.st_gid).gr_name) >>> 0584: except KeyError as e: >>> 0585: bb.warn("KeyError in %s" % path) >>> 0586: msg = ("KeyError: %s\nPath %s is owned by uid %d, gid %d, which doesn't match " >>> Exception: Exception: KeyError: 'getpwuid(): uid not found: 1026' >>> Path ./package/usr/palm/applications/com.webos.app.videoplayer/appinfo.json is owned by uid 1026, gid 1026, which doesn't match any user/group on target. This may be due to host contamination. >>> >>> martin@jama /OE/build/luneos-nanbield/tmp-glibc/work/corei7-64-webos-linux/com.webos.app.videoplayer $ sqlite3 0.0.1-12+git-nodejs-20/pseudo/files.db "select * from files where path like '%yer/appinfo.json'" >>> 1107|/OE/build/luneos-nanbield/tmp-glibc/work/corei7-64-webos-linux/com.webos.app.videoplayer/0.0.1-12+git/sstate-build-package/package/usr/palm/applications/com.webos.app.videoplayer/appinfo.json|66305|41202418|1026|1026|33188|0|0 >>> 1192|/OE/build/luneos-nanbield/tmp-glibc/work/corei7-64-webos-linux/com.webos.app.videoplayer/0.0.1-12+git/sstate-build-package/packages-split/com.webos.app.videoplayer/usr/palm/applications/com.webos.app.videoplayer/appinfo.json|66305|41 >>> 202418|1026|1026|33188|0|0 >>> >>> martin@jama /OE/build/luneos-nanbield/tmp-glibc/work/corei7-64-webos-linux/com.webos.app.videoplayer $ sqlite3 0.0.1-12+git-nodejs-18/pseudo/files.db "select * from files where path like '%yer/appinfo.json'" >>> 56|/OE/build/luneos-nanbield/tmp-glibc/work/corei7-64-webos-linux/com.webos.app.videoplayer/0.0.1-12+git/image/usr/palm/applications/com.webos.app.videoplayer/appinfo.json|66305|42878067|0|0|33188|0|0 >>> 374|/OE/build/luneos-nanbield/tmp-glibc/work/corei7-64-webos-linux/com.webos.app.videoplayer/0.0.1-12+git/package/usr/palm/applications/com.webos.app.videoplayer/appinfo.json|66305|42878641|0|0|33188|0|0 >>> 459|/OE/build/luneos-nanbield/tmp-glibc/work/corei7-64-webos-linux/com.webos.app.videoplayer/0.0.1-12+git/packages-split/com.webos.app.videoplayer/usr/palm/applications/com.webos.app.videoplayer/appinfo.json|66305|42878641|0|0|33188|0|0 >>> >>> I'll provide more details when I debug this further. >>> >>> Regards, >>> >>> >>> >> >> >> > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#105585): https://lists.openembedded.org/g/openembedded-devel/message/105585 > Mute This Topic: https://lists.openembedded.org/mt/101098430/1997914 > Group Owner: openembedded-devel+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-devel/unsub [raj.khem@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- >
On Wed, Oct 18, 2023 at 8:40 PM Khem Raj <raj.khem@gmail.com> wrote: > On Wed, Oct 18, 2023 at 4:19 PM Tim Orling <ticotimo@gmail.com> wrote: > > > > > > > > On Wed, Oct 18, 2023 at 1:41 PM Martin Jansa <martin.jansa@gmail.com> > wrote: > >> > >> I've narrowed this down to simple reproducer with webpack creating pack > with "hello world" as shown in: > >> https://github.com/shr-project/com.webos.app.minimal/tree/recipes > >> > >> Then with this reproducer I've bisected it to nodejs-20.3.0 with libuv > upgrade 1.45.0 which brings support for io_uring which doesn't work with > pseudo in: > >> > https://github.com/libuv/libuv/pull/3952/commits/26c79a942b92573a1388c0ee8a6ad4397f009318 > >> > >> Now we have 3 options: > >> > >> 1) either every use of nodejs-native where io_uring might be used (like > in webpack) needs to export UV_USE_IO_URING=0 before using node, but be > aware that the PR says it's only temporary and this variable might be > removed in future. Unfortunately many people are lucky enough not to know > what pseudo even is, so they might not realize that host-user-contamination > QA issues or "KeyError: 'getpwuid(): uid not found" might be related to > this (and they probably won't find this e-mail as well). > >> > >> 2) revert the io_uring changes from bundled libuv, I've already > verified this with nodejs-20.3.0, for 20.8.1 it would need couple more > commits to be reverted as 20.8.1 uses even newer libuv-1.46.0. The changes > are massive and it will be PIA to update them for every nodejs release, > patch for keeping UV_USE_IO_URING support after it's removed from upstream > libuv will be much smaller, but PIA to use UV_USE_IO_URING whenever using > nodejs-native during build. > >> > >> 3) fix pseudo to correctly work with io_uring calls, definitely best > long-term solution as more and more tools will now switch to io_uring, but > without active maintainer of pseudo it might be _very_ difficult to > implement this and there aren't many people willing to touch this beast. > >> > >> Any opinions? > >> > > > > I would advise we not be overly aggressive about updates for important > tools like NodeJS and Django. > > > > I don't think many consumers are building products on the latest > greatest NodeJS release (which this isn't, latest is 21), many may not even > be on the latest LTS (18 vs.16). While we can have a floating "latest" > recipe, we should keep an LTS recipe. > > We tried to do this for Django in meta-python, but later commits wiped > out the comments and the upstream check. > > > https://git.openembedded.org/meta-openembedded/commit/meta-python/recipes-devtools/python?id=961755a7bd17f7347f7dd0f2de3d2400a9c36f09 > > > > 20.x is next LTS and we already branched nanbield with 18.x, for > master and upcoming yocto LTS release in April 2024, I would like to > see up move to 20.x and figure out > problems during this time early enough so we can stick to 20.x for > yocto LTS. Unless we run into blocker issues. > > For anyone else trying to catch up, in the "Active LTS Start" column you can see when a given release will become an LTS: https://github.com/nodejs/Release It is also in the release announcement for 21: https://nodejs.org/en/blog/announcements/v21-release-announce "Node.js 21 will replace Node.js 20 as our ‘Current’ release line when Node.js 20 enters long-term support (LTS) later this month. As per the release schedule, Node.js 21 will be ‘Current' release for the next 6 months, until April 2024" Unfortunately, the nodejs landing page doesn't have any indication. https://nodejs.org/en > > >> On Mon, Oct 2, 2023 at 7:08 PM Martin Jansa via lists.openembedded.org > <martin.jansa=gmail.com@lists.openembedded.org> wrote: > >>> > >>> On Thu, Sep 28, 2023 at 11:13 AM Polampalli, Archana via > lists.openembedded.org <archana.polampalli= > windriver.com@lists.openembedded.org> wrote: > >>>> > >>>> Update to latest release of latest LTS 20 release > >>>> > >>>> * node v20 introduces several new features and fixes many bugs > and CVEs as shown in [1] > >>>> > >>>> * Refresh 0001-liftoff-Correct-function-signatures.patch against > 20.5.1 > >>>> > >>>> * License-Update: > >>>> - Change zlib version 1.2.13, October 13th, 2022 to version > 1.2.13.1, October xxth, 2022 [2] > >>>> - Change Copyright 2023 from Ada authors to Yagiz Nizipli and > Daniel Lemire [4] > >>>> > >>>> * Remove big-endian.patch as it is merged in v20.x [5] [6] > >>>> > >>>> * Remove below list of patches since mips32 is deleted from v8 as > part of update V8 to 10.7.193.13 [7] [8] > >>>> - mips-less-memory.patch > >>>> - 0001-mips-Use-32bit-cast-for-operand-on-mips32.patch > >>>> > >>>> * Update Using-native-binaries.patch for node_js2c, it resolved > below do_compile error [9] > >>>> Error: > >>>> /bin/sh: line 1: > build/tmp/work/core2-64-poky-linux/nodejs/20.5.1/node-v20.5.1/out/Release/node_js2c: > No such file or directory > >>>> > >>>> * Remove obsolete dtrace & etw configure options (we had: > --without-<feature>) from the recipe [10] > >>> > >>> > >>> FYI: (in case someone else are seeing similar issues now) > >>> > >>> With this upgrade I'm seeing some files created during the build with > node not to be intercepted by pseudo and recorded in its database. > >>> > >>> This results in sstate_report_unihash failure, because appinfo.json is > now owned by 1026:1026 (my build UID/GID) instead of 0:0 as before: > >>> > >>> DEBUG: Executing python function sstate_report_unihash > >>> WARNING: KeyError in ./package/usr/palm/applications/com.webos.app > .videoplayer/appinfo.json > >>> ERROR: Error executing a python function in exec_func_python() > autogenerated: > >>> > >>> The stack trace of python calls that resulted in this > exception/failure was: > >>> File: 'exec_func_python() autogenerated', lineno: 2, function: <module> > >>> 0001: > >>> *** 0002:sstate_report_unihash(d) > >>> 0003: > >>> File: > '/OE/build/luneos-nanbield/openembedded-core/meta/classes-global/sstate.bbclass', > lineno: 880, function: sstate_report_unihash > >>> 0876: report_unihash = getattr(bb.parse.siggen, > 'report_unihash', None) > >>> 0877: > >>> 0878: if report_unihash: > >>> 0879: ss = sstate_state_fromvars(d) > >>> *** 0880: report_unihash(os.getcwd(), ss['task'], d) > >>> 0881:} > >>> 0882: > >>> 0883:# > >>> 0884:# Shell function to decompress and prepare a package for > installation > >>> File: '/OE/build/luneos-nanbield/bitbake/lib/bb/siggen.py', lineno: > 659, function: report_unihash > >>> 0655: > >>> 0656: if "." in self.method: > >>> 0657: (module, method) = self.method.rsplit('.', 1) > >>> 0658: locs['method'] = > getattr(importlib.import_module(module), method) > >>> *** 0659: outhash = bb.utils.better_eval('method(path, > sigfile, task, d)', locs) > >>> 0660: else: > >>> 0661: outhash = bb.utils.better_eval(self.method + > '(path, sigfile, task, d)', locs) > >>> 0662: > >>> 0663: try: > >>> File: '/OE/build/luneos-nanbield/bitbake/lib/bb/utils.py', lineno: > 434, function: better_eval > >>> 0430: if extraglobals: > >>> 0431: ctx = copy.copy(ctx) > >>> 0432: for g in extraglobals: > >>> 0433: ctx[g] = extraglobals[g] > >>> *** 0434: return eval(source, ctx, locals) > >>> 0435: > >>> 0436:@contextmanager > >>> 0437:def fileslocked(files, *args, **kwargs): > >>> 0438: """Context manager for locking and unlocking file > locks.""" > >>> File: '<string>', lineno: 1, function: <module> > >>> File "<string>", line 1, in <module> > >>> > >>> File: > '/OE/build/luneos-nanbield/openembedded-core/meta/lib/oe/sstatesig.py', > lineno: 649, function: OEOuthashBasic > >>> 0645: process(root) > >>> 0646: for f in files: > >>> 0647: if f == 'fixmepath': > >>> 0648: continue > >>> *** 0649: process(os.path.join(root, f)) > >>> 0650: > >>> 0651: for dir in dirs: > >>> 0652: if os.path.islink(os.path.join(root, dir)): > >>> 0653: process(os.path.join(root, dir)) > >>> File: > '/OE/build/luneos-nanbield/openembedded-core/meta/lib/oe/sstatesig.py', > lineno: 588, function: process > >>> 0584: except KeyError as e: > >>> 0585: bb.warn("KeyError in %s" % path) > >>> 0586: msg = ("KeyError: %s\nPath %s is > owned by uid %d, gid %d, which doesn't match " > >>> 0587: "any user/group on target. This > may be due to host contamination." % (e, path, s.st_uid, s.st_gid)) > >>> *** 0588: raise > Exception(msg).with_traceback(e.__traceback__) > >>> 0589: > >>> 0590: if include_timestamps: > >>> 0591: update_hash(" %10d" % s.st_mtime) > >>> 0592: > >>> File: > '/OE/build/luneos-nanbield/openembedded-core/meta/lib/oe/sstatesig.py', > lineno: 582, function: process > >>> 0578: else: > >>> 0579: add_perm(stat.S_IXOTH, 'x') > >>> 0580: > >>> 0581: try: > >>> *** 0582: update_hash(" %10s" % > pwd.getpwuid(s.st_uid).pw_name) > >>> 0583: update_hash(" %10s" % > grp.getgrgid(s.st_gid).gr_name) > >>> 0584: except KeyError as e: > >>> 0585: bb.warn("KeyError in %s" % path) > >>> 0586: msg = ("KeyError: %s\nPath %s is > owned by uid %d, gid %d, which doesn't match " > >>> Exception: Exception: KeyError: 'getpwuid(): uid not found: 1026' > >>> Path ./package/usr/palm/applications/com.webos.app.videoplayer/appinfo.json > is owned by uid 1026, gid 1026, which doesn't match any user/group on > target. This may be due to host contamination. > >>> > >>> martin@jama > /OE/build/luneos-nanbield/tmp-glibc/work/corei7-64-webos-linux/ > com.webos.app.videoplayer $ sqlite3 > 0.0.1-12+git-nodejs-20/pseudo/files.db "select * from files where path like > '%yer/appinfo.json'" > >>> > 1107|/OE/build/luneos-nanbield/tmp-glibc/work/corei7-64-webos-linux/com.webos.app.videoplayer/0.0.1-12+git/sstate-build-package/package/usr/palm/applications/com.webos.app.videoplayer/appinfo.json|66305|41202418|1026|1026|33188|0|0 > >>> > 1192|/OE/build/luneos-nanbield/tmp-glibc/work/corei7-64-webos-linux/com.webos.app.videoplayer/0.0.1-12+git/sstate-build-package/packages-split/ > com.webos.app.videoplayer/usr/palm/applications/com.webos.app > .videoplayer/appinfo.json|66305|41 > >>> 202418|1026|1026|33188|0|0 > >>> > >>> martin@jama > /OE/build/luneos-nanbield/tmp-glibc/work/corei7-64-webos-linux/ > com.webos.app.videoplayer $ sqlite3 > 0.0.1-12+git-nodejs-18/pseudo/files.db "select * from files where path like > '%yer/appinfo.json'" > >>> 56|/OE/build/luneos-nanbield/tmp-glibc/work/corei7-64-webos-linux/ > com.webos.app > .videoplayer/0.0.1-12+git/image/usr/palm/applications/com.webos.app.videoplayer/appinfo.json|66305|42878067|0|0|33188|0|0 > >>> 374|/OE/build/luneos-nanbield/tmp-glibc/work/corei7-64-webos-linux/ > com.webos.app > .videoplayer/0.0.1-12+git/package/usr/palm/applications/com.webos.app.videoplayer/appinfo.json|66305|42878641|0|0|33188|0|0 > >>> 459|/OE/build/luneos-nanbield/tmp-glibc/work/corei7-64-webos-linux/ > com.webos.app.videoplayer/0.0.1-12+git/packages-split/com.webos.app > .videoplayer/usr/palm/applications/com.webos.app > .videoplayer/appinfo.json|66305|42878641|0|0|33188|0|0 > >>> > >>> I'll provide more details when I debug this further. > >>> > >>> Regards, > >>> > >>> > >>> > >> > >> > >> > > > > -=-=-=-=-=-=-=-=-=-=-=- > > Links: You receive all messages sent to this group. > > View/Reply Online (#105585): > https://lists.openembedded.org/g/openembedded-devel/message/105585 > > Mute This Topic: https://lists.openembedded.org/mt/101098430/1997914 > > Group Owner: openembedded-devel+owner@lists.openembedded.org > > Unsubscribe: https://lists.openembedded.org/g/openembedded-devel/unsub [ > raj.khem@gmail.com] > > -=-=-=-=-=-=-=-=-=-=-=- > > >
diff --git a/meta-oe/recipes-devtools/nodejs/nodejs-oe-cache-18.17/oe-npm-cache b/meta-oe/recipes-devtools/nodejs/nodejs-oe-cache-20.5/oe-npm-cache similarity index 100% rename from meta-oe/recipes-devtools/nodejs/nodejs-oe-cache-18.17/oe-npm-cache rename to meta-oe/recipes-devtools/nodejs/nodejs-oe-cache-20.5/oe-npm-cache diff --git a/meta-oe/recipes-devtools/nodejs/nodejs-oe-cache-native_18.17.bb b/meta-oe/recipes-devtools/nodejs/nodejs-oe-cache-native_20.5.bb similarity index 100% rename from meta-oe/recipes-devtools/nodejs/nodejs-oe-cache-native_18.17.bb rename to meta-oe/recipes-devtools/nodejs/nodejs-oe-cache-native_20.5.bb diff --git a/meta-oe/recipes-devtools/nodejs/nodejs/0001-Using-native-binaries.patch b/meta-oe/recipes-devtools/nodejs/nodejs/0001-Using-native-binaries.patch index 66e10a0d0..0178cec77 100644 --- a/meta-oe/recipes-devtools/nodejs/nodejs/0001-Using-native-binaries.patch +++ b/meta-oe/recipes-devtools/nodejs/nodejs/0001-Using-native-binaries.patch @@ -12,7 +12,7 @@ https://git.openembedded.org/meta-openembedded/commit/?id=feeb172d1a8bf010490d22 Signed-off-by: Archana Polampalli <archana.polampalli@windriver.com> --- - node.gyp | 1 + + node.gyp | 3 + tools/v8_gypfiles/v8.gyp | 5 +++++ 2 files changed, 6 insertions(+) @@ -28,6 +28,23 @@ index e8e1d9f9..e60ccc10 100644 '<(node_mksnapshot_exec)', '<(node_snapshot_main)', ], +@@ -935,6 +935,7 @@ + 'action_name': 'node_js2c', + 'process_outputs_as_sources': 1, + 'inputs': [ ++ '<(PRODUCT_DIR)/v8-qemu-wrapper.sh', + '<(node_js2c_exec)', + '<@(library_files)', + '<@(deps_files)', +@@ -944,6 +945,7 @@ + '<(SHARED_INTERMEDIATE_DIR)/node_javascript.cc', + ], + 'action': [ ++ '<(PRODUCT_DIR)/v8-qemu-wrapper.sh', + '<(node_js2c_exec)', + '<@(_outputs)', + 'lib', + diff --git a/tools/v8_gypfiles/v8.gyp b/tools/v8_gypfiles/v8.gyp index 42e26cd9..bc721991 100644 --- a/tools/v8_gypfiles/v8.gyp diff --git a/meta-oe/recipes-devtools/nodejs/nodejs/0001-liftoff-Correct-function-signatures.patch b/meta-oe/recipes-devtools/nodejs/nodejs/0001-liftoff-Correct-function-signatures.patch index c9a522d2e..5af87d866 100644 --- a/meta-oe/recipes-devtools/nodejs/nodejs/0001-liftoff-Correct-function-signatures.patch +++ b/meta-oe/recipes-devtools/nodejs/nodejs/0001-liftoff-Correct-function-signatures.patch @@ -1,6 +1,6 @@ From dc3652c0abcdf8573fd044907b19d8eda7ca1124 Mon Sep 17 00:00:00 2001 From: Khem Raj <raj.khem@gmail.com> -Date: Wed, 20 Oct 2021 12:49:58 -0700 +Date: Mon, 3 Jul 2023 12:33:16 +0000 Subject: [PATCH] [liftoff] Correct function signatures Fixes builds on mips where clang reports an error @@ -13,16 +13,15 @@ Upstream-Status: Submitted [https://chromium-review.googlesource.com/c/v8/v8/+/3 Signed-off-by: Archana Polampalli <archana.polampalli@windriver.com> --- deps/v8/src/wasm/baseline/liftoff-assembler.h | 6 +++--- - deps/v8/src/wasm/baseline/mips/liftoff-assembler-mips.h | 2 +- deps/v8/src/wasm/baseline/mips64/liftoff-assembler-mips64.h | 2 +- - .../src/wasm/baseline/riscv64/liftoff-assembler-riscv64.h | 2 +- - 4 files changed, 6 insertions(+), 6 deletions(-) + deps/v8/src/wasm/baseline/riscv/liftoff-assembler-riscv64.h | 2 +- + 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/deps/v8/src/wasm/baseline/liftoff-assembler.h b/deps/v8/src/wasm/baseline/liftoff-assembler.h -index 22c7f73a..db4cb168 100644 +index aef63c64..f2a11b01 100644 --- a/deps/v8/src/wasm/baseline/liftoff-assembler.h +++ b/deps/v8/src/wasm/baseline/liftoff-assembler.h -@@ -646,7 +646,7 @@ class LiftoffAssembler : public TurboAssembler { +@@ -717,7 +717,7 @@ class LiftoffAssembler : public MacroAssembler { void FinishCall(const ValueKindSig*, compiler::CallDescriptor*); // Move {src} into {dst}. {src} and {dst} must be different. @@ -31,7 +30,7 @@ index 22c7f73a..db4cb168 100644 // Parallel register move: For a list of tuples <dst, src, kind>, move the // {src} register of kind {kind} into {dst}. If {src} equals {dst}, ignore -@@ -795,8 +795,8 @@ class LiftoffAssembler : public TurboAssembler { +@@ -884,8 +884,8 @@ class LiftoffAssembler : public MacroAssembler { inline void MoveStackValue(uint32_t dst_offset, uint32_t src_offset, ValueKind); @@ -42,24 +41,11 @@ index 22c7f73a..db4cb168 100644 inline void Spill(int offset, LiftoffRegister, ValueKind); inline void Spill(int offset, WasmValue); -diff --git a/deps/v8/src/wasm/baseline/mips/liftoff-assembler-mips.h b/deps/v8/src/wasm/baseline/mips/liftoff-assembler-mips.h -index c76fd2f4..0fffe231 100644 ---- a/deps/v8/src/wasm/baseline/mips/liftoff-assembler-mips.h -+++ b/deps/v8/src/wasm/baseline/mips/liftoff-assembler-mips.h -@@ -661,7 +661,7 @@ void LiftoffAssembler::Store(Register dst_addr, Register offset_reg, - pinned = pinned | LiftoffRegList{dst_op.rm(), src}; - LiftoffRegister tmp = GetUnusedRegister(src.reg_class(), pinned); - // Save original value. -- Move(tmp, src, type.value_type()); -+ Move(tmp, src, type.value_type().kind()); - - src = tmp; - pinned.set(tmp); diff --git a/deps/v8/src/wasm/baseline/mips64/liftoff-assembler-mips64.h b/deps/v8/src/wasm/baseline/mips64/liftoff-assembler-mips64.h -index 36413545..48207337 100644 +index 96cba24c..53e1842d 100644 --- a/deps/v8/src/wasm/baseline/mips64/liftoff-assembler-mips64.h +++ b/deps/v8/src/wasm/baseline/mips64/liftoff-assembler-mips64.h -@@ -593,7 +593,7 @@ void LiftoffAssembler::Store(Register dst_addr, Register offset_reg, +@@ -592,7 +592,7 @@ void LiftoffAssembler::Store(Register dst_addr, Register offset_reg, pinned.set(dst_op.rm()); LiftoffRegister tmp = GetUnusedRegister(src.reg_class(), pinned); // Save original value. @@ -68,11 +54,11 @@ index 36413545..48207337 100644 src = tmp; pinned.set(tmp); -diff --git a/deps/v8/src/wasm/baseline/riscv64/liftoff-assembler-riscv64.h b/deps/v8/src/wasm/baseline/riscv64/liftoff-assembler-riscv64.h -index 642a7d2a..56ffcc2a 100644 ---- a/deps/v8/src/wasm/baseline/riscv64/liftoff-assembler-riscv64.h -+++ b/deps/v8/src/wasm/baseline/riscv64/liftoff-assembler-riscv64.h -@@ -589,7 +589,7 @@ void LiftoffAssembler::Store(Register dst_addr, Register offset_reg, +diff --git a/deps/v8/src/wasm/baseline/riscv/liftoff-assembler-riscv64.h b/deps/v8/src/wasm/baseline/riscv/liftoff-assembler-riscv64.h +index 1d6ae09e..397e82b2 100644 +--- a/deps/v8/src/wasm/baseline/riscv/liftoff-assembler-riscv64.h ++++ b/deps/v8/src/wasm/baseline/riscv/liftoff-assembler-riscv64.h +@@ -286,7 +286,7 @@ void LiftoffAssembler::Store(Register dst_addr, Register offset_reg, pinned.set(dst_op.rm()); LiftoffRegister tmp = GetUnusedRegister(src.reg_class(), pinned); // Save original value. @@ -82,4 +68,4 @@ index 642a7d2a..56ffcc2a 100644 src = tmp; pinned.set(tmp); -- -2.34.1 +2.40.0 diff --git a/meta-oe/recipes-devtools/nodejs/nodejs/0001-mips-Use-32bit-cast-for-operand-on-mips32.patch b/meta-oe/recipes-devtools/nodejs/nodejs/0001-mips-Use-32bit-cast-for-operand-on-mips32.patch deleted file mode 100644 index a0242d8e1..000000000 --- a/meta-oe/recipes-devtools/nodejs/nodejs/0001-mips-Use-32bit-cast-for-operand-on-mips32.patch +++ /dev/null @@ -1,34 +0,0 @@ -From e65dde8db17da5acddeef7eb9316199c4e5e0811 Mon Sep 17 00:00:00 2001 -From: Khem Raj <raj.khem@gmail.com> -Date: Tue, 19 Apr 2022 12:40:25 -0700 -Subject: [PATCH] mips: Use 32bit cast for operand on mips32 - -Fixes -deps/v8/src/compiler/backend/mips/code-generator-mips.cc: In member function 'void v8::internal::compiler::CodeGenerator::AssembleReturn(v8::internal::compiler::InstructionOperand*)': -../deps/v8/src/compiler/backend/mips/code-generator-mips.cc:4233:48: error: call of overloaded 'Operand(int64_t)' is ambiguous - 4233 | Operand(static_cast<int64_t>(0))); - | ^ - -Upstream-Status: Pending - -Signed-off-by: Khem Raj <raj.khem@gmail.com> ---- - deps/v8/src/compiler/backend/mips/code-generator-mips.cc | 2 +- - 1 file changed, 1 insertion(+), 1 deletion(-) - -diff --git a/deps/v8/src/compiler/backend/mips/code-generator-mips.cc b/deps/v8/src/compiler/backend/mips/code-generator-mips.cc -index 2b8197e..b226140 100644 ---- a/deps/v8/src/compiler/backend/mips/code-generator-mips.cc -+++ b/deps/v8/src/compiler/backend/mips/code-generator-mips.cc -@@ -4230,7 +4230,7 @@ void CodeGenerator::AssembleReturn(InstructionOperand* additional_pop_count) { - } else if (FLAG_debug_code) { - __ Assert(eq, AbortReason::kUnexpectedAdditionalPopValue, - g.ToRegister(additional_pop_count), -- Operand(static_cast<int64_t>(0))); -+ Operand(static_cast<int32_t>(0))); - } - } - // Functions with JS linkage have at least one parameter (the receiver). --- -2.36.0 - diff --git a/meta-oe/recipes-devtools/nodejs/nodejs/big-endian.patch b/meta-oe/recipes-devtools/nodejs/nodejs/big-endian.patch deleted file mode 100644 index 529381842..000000000 --- a/meta-oe/recipes-devtools/nodejs/nodejs/big-endian.patch +++ /dev/null @@ -1,18 +0,0 @@ - -https://github.com/v8/v8/commit/878ccb33bd3cf0e6dc018ff8d15843f585ac07be - -did some automated cleanups but it missed big-endian code. - -Upstream-Status: Pending -Signed-off-by: Khem Raj <raj.khem@gmail.com> ---- a/deps/v8/src/runtime/runtime-utils.h -+++ b/deps/v8/src/runtime/runtime-utils.h -@@ -126,7 +126,7 @@ static inline ObjectPair MakePair(Object - #if defined(V8_TARGET_LITTLE_ENDIAN) - return x.ptr() | (static_cast<ObjectPair>(y.ptr()) << 32); - #elif defined(V8_TARGET_BIG_ENDIAN) -- return y->ptr() | (static_cast<ObjectPair>(x->ptr()) << 32); -+ return y.ptr() | (static_cast<ObjectPair>(x.ptr()) << 32); - #else - #error Unknown endianness - #endif diff --git a/meta-oe/recipes-devtools/nodejs/nodejs/mips-less-memory.patch b/meta-oe/recipes-devtools/nodejs/nodejs/mips-less-memory.patch deleted file mode 100644 index 16776cb76..000000000 --- a/meta-oe/recipes-devtools/nodejs/nodejs/mips-less-memory.patch +++ /dev/null @@ -1,37 +0,0 @@ -Description: mksnapshot uses too much memory on 32-bit mipsel -Author: Jérémy Lal <kapouer@melix.org> -Last-Update: 2020-06-03 -Forwarded: https://bugs.chromium.org/p/v8/issues/detail?id=10586 - -This ensures that we reserve 500M instead of 2G range for codegen -ensures that qemu-mips can allocate such large ranges - -Upstream-Status: Inappropriate [embedded specific] - -Imported from debian https://salsa.debian.org/js-team/nodejs/-/blob/master-12.x/debian/patches/mips-less-memory.patch -https://buildd.debian.org/status/fetch.php?pkg=nodejs&arch=mipsel&ver=12.17.0~dfsg-2&stamp=1591050388&raw=0 - -Signed-off-by: Khem Raj <raj.khem@gmail.com> - ---- a/deps/v8/src/common/globals.h -+++ b/deps/v8/src/common/globals.h -@@ -224,7 +224,7 @@ constexpr size_t kMinimumCodeRangeSize = - constexpr size_t kMinExpectedOSPageSize = 64 * KB; // OS page on PPC Linux - #elif V8_TARGET_ARCH_MIPS - constexpr bool kPlatformRequiresCodeRange = false; --constexpr size_t kMaximalCodeRangeSize = 2048LL * MB; -+constexpr size_t kMaximalCodeRangeSize = 512 * MB; - constexpr size_t kMinimumCodeRangeSize = 0 * MB; - constexpr size_t kMinExpectedOSPageSize = 4 * KB; // OS page. - #else ---- a/deps/v8/src/codegen/mips/constants-mips.h -+++ b/deps/v8/src/codegen/mips/constants-mips.h -@@ -140,7 +140,7 @@ const uint32_t kLeastSignificantByteInIn - namespace v8 { - namespace internal { - --constexpr size_t kMaxPCRelativeCodeRangeInMB = 4096; -+constexpr size_t kMaxPCRelativeCodeRangeInMB = 1024; - - // ----------------------------------------------------------------------------- - // Registers and FPURegisters. diff --git a/meta-oe/recipes-devtools/nodejs/nodejs_18.17.1.bb b/meta-oe/recipes-devtools/nodejs/nodejs_20.5.1.bb similarity index 94% rename from meta-oe/recipes-devtools/nodejs/nodejs_18.17.1.bb rename to meta-oe/recipes-devtools/nodejs/nodejs_20.5.1.bb index ee5e848bf..6bb0f7fab 100644 --- a/meta-oe/recipes-devtools/nodejs/nodejs_18.17.1.bb +++ b/meta-oe/recipes-devtools/nodejs/nodejs_20.5.1.bb @@ -1,7 +1,7 @@ DESCRIPTION = "nodeJS Evented I/O for V8 JavaScript" HOMEPAGE = "http://nodejs.org" LICENSE = "MIT & ISC & BSD-2-Clause & BSD-3-Clause & Artistic-2.0 & Apache-2.0" -LIC_FILES_CHKSUM = "file://LICENSE;md5=bc1f9ebe76be76f163e3b675303ad9cd" +LIC_FILES_CHKSUM = "file://LICENSE;md5=059ecf3a6f87111685e51b611b9563e5" CVE_PRODUCT = "nodejs node.js" @@ -22,11 +22,8 @@ COMPATIBLE_HOST:powerpc = "null" SRC_URI = "http://nodejs.org/dist/v${PV}/node-v${PV}.tar.xz \ file://0001-Disable-running-gyp-files-for-bundled-deps.patch \ file://0004-v8-don-t-override-ARM-CFLAGS.patch \ - file://big-endian.patch \ - file://mips-less-memory.patch \ file://system-c-ares.patch \ file://0001-liftoff-Correct-function-signatures.patch \ - file://0001-mips-Use-32bit-cast-for-operand-on-mips32.patch \ file://run-ptest \ " @@ -39,7 +36,7 @@ SRC_URI:append:toolchain-clang:x86 = " \ SRC_URI:append:toolchain-clang:powerpc64le = " \ file://0001-ppc64-Do-not-use-mminimal-toc-with-clang.patch \ " -SRC_URI[sha256sum] = "f215cf03d0f00f07ac0b674c6819f804c1542e16f152da04980022aeccf5e65a" +SRC_URI[sha256sum] = "439c71aa2f38c2861657bfa538e99191a571258066cbfd4548586049c8134190" S = "${WORKDIR}/node-v${PV}" @@ -148,8 +145,6 @@ do_configure () { # $TARGET_ARCH settings don't match --dest-cpu settings python3 configure.py --verbose --prefix=${prefix} \ --shared-openssl \ - --without-dtrace \ - --without-etw \ --dest-cpu="${@map_nodejs_arch(d.getVar('TARGET_ARCH'), d)}" \ --dest-os=linux \ --libdir=${baselib} \
Update to latest release of latest LTS 20 release * node v20 introduces several new features and fixes many bugs and CVEs as shown in [1] * Refresh 0001-liftoff-Correct-function-signatures.patch against 20.5.1 * License-Update: - Change zlib version 1.2.13, October 13th, 2022 to version 1.2.13.1, October xxth, 2022 [2] - Change Copyright 2023 from Ada authors to Yagiz Nizipli and Daniel Lemire [4] * Remove big-endian.patch as it is merged in v20.x [5] [6] * Remove below list of patches since mips32 is deleted from v8 as part of update V8 to 10.7.193.13 [7] [8] - mips-less-memory.patch - 0001-mips-Use-32bit-cast-for-operand-on-mips32.patch * Update Using-native-binaries.patch for node_js2c, it resolved below do_compile error [9] Error: /bin/sh: line 1: build/tmp/work/core2-64-poky-linux/nodejs/20.5.1/node-v20.5.1/out/Release/node_js2c: No such file or directory * Remove obsolete dtrace & etw configure options (we had: --without-<feature>) from the recipe [10] https://github.com/nodejs/node/releases/tag/v20.5.1 [1] https://github.com/nodejs/node/blob/main/doc/changelogs/CHANGELOG_V20.md [2] https://github.com/nodejs/node/commit/f1007325753e62b13083f6e9e499fdc33f753d7d [3] https://github.com/nodejs/node/commit/900ae1bda76caaa7213431a5268560b578ed2d55 [4] https://github.com/nodejs/node/commit/d2465369243bc4313d9840b62c2393c4f179ffbb [5] https://github.com/v8/v8/commit/3cea5d5425c3a0d7d1768b3758269ac98875de77 [6] https://github.com/nodejs/node/commit/f226350fcbebd4449fb0034fdaffa147e4de28ea [7] https://github.com/v8/v8/commit/a26ca5ed147dde47f02d70c5b38d8befc1c93cb3 [8] https://github.com/nodejs/node/commit/6bd756d7c6dfb7dc25daee329ad70df68c14223e [9] https://github.com/nodejs/node/commit/4da7bc915c714989eba7eab753db9eb9ebd8451a [10] https://github.com/nodejs/node/commit/aa3a572e6bee116cde69508dc29478b40f40551a Signed-off-by: Archana Polampalli <archana.polampalli@windriver.com> --- .../oe-npm-cache | 0 ...8.17.bb => nodejs-oe-cache-native_20.5.bb} | 0 .../nodejs/0001-Using-native-binaries.patch | 19 ++++++++- ...-liftoff-Correct-function-signatures.patch | 42 +++++++------------ ...Use-32bit-cast-for-operand-on-mips32.patch | 34 --------------- .../nodejs/nodejs/big-endian.patch | 18 -------- .../nodejs/nodejs/mips-less-memory.patch | 37 ---------------- .../{nodejs_18.17.1.bb => nodejs_20.5.1.bb} | 9 +--- 8 files changed, 34 insertions(+), 125 deletions(-) rename meta-oe/recipes-devtools/nodejs/{nodejs-oe-cache-18.17 => nodejs-oe-cache-20.5}/oe-npm-cache (100%) rename meta-oe/recipes-devtools/nodejs/{nodejs-oe-cache-native_18.17.bb => nodejs-oe-cache-native_20.5.bb} (100%) delete mode 100644 meta-oe/recipes-devtools/nodejs/nodejs/0001-mips-Use-32bit-cast-for-operand-on-mips32.patch delete mode 100644 meta-oe/recipes-devtools/nodejs/nodejs/big-endian.patch delete mode 100644 meta-oe/recipes-devtools/nodejs/nodejs/mips-less-memory.patch rename meta-oe/recipes-devtools/nodejs/{nodejs_18.17.1.bb => nodejs_20.5.1.bb} (94%)