| Message ID | 20230712190428.379909-1-p.lobacz@welotec.com |
|---|---|
| State | New |
| Headers | show |
| Series | [v5,1/5] bitbake.conf: add acl and xattr distro native features support | expand |
ACK this patchset, assuming the other maintainers are happy with how we're handling the DISTO_FEATURES. I chose not to include Piotr's opkg and opkg-utils ACL/xattr changes into the 0.6.2 release, which I have just pushed. They've been merged afterward, and can still be applied atop 0.6.2, and they'll be merged into the December opkg release. I have a branch ready to upgrade the opkg and opkg-utils recipes, but I'd like to wait for this patchset to clear, because they will otherwise conflict. On 7/12/23 15:04, Piotr Łobacz via lists.openembedded.org wrote: > Include support for ACLs and extended file attributes for native > builds, by default. > > Signed-off-by: Piotr Łobacz <p.lobacz@welotec.com> > --- > meta/conf/bitbake.conf | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf > index 9625a6fef4..8daaaad615 100644 > --- a/meta/conf/bitbake.conf > +++ b/meta/conf/bitbake.conf > @@ -904,7 +904,7 @@ IMAGE_FEATURES += "${EXTRA_IMAGE_FEATURES}" > > # Native distro features (will always be used for -native, even if they > # are not enabled for target) > -DISTRO_FEATURES_NATIVE ?= "x11 ipv6 xattr" > +DISTRO_FEATURES_NATIVE ?= "acl x11 ipv6 xattr" > DISTRO_FEATURES_NATIVESDK ?= "x11" > > # Normally target distro features will not be applied to native builds: > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#184214): https://lists.openembedded.org/g/openembedded-core/message/184214 > Mute This Topic: https://lists.openembedded.org/mt/100106217/3616788 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.stewart@ni.com] > -=-=-=-=-=-=-=-=-=-=-=- >
OK so, does any one have any thoughts regarding this patchset?
Hello, On 14/07/2023 10:14:50+0000, Piotr Łobacz wrote: > OK so, does any one have any thoughts regarding this patchset? This still fails on the autobuilder, most of the builds failed: https://autobuilder.yoctoproject.org/typhoon/#/builders/37/builds/7477/steps/11/logs/stdio ERROR: nativesdk-xcb-proto-1.15.2-r0 do_package_write_ipk: Fatal errors occurred in subprocesses: Command 'PATH="/home/pokybuild/yocto-worker/genericx86-64/build/scripts/nativesdk-intercept:/home/pokybuild/yocto-worker/genericx86-64/build/build/tmp/sysroots-uninative/x86_64-linux/usr/bin:/home/pokybuild/yocto-worker/genericx86-64/build/build/tmp/work/i686-nativesdk-pokysdk-linux/nativesdk-xcb-proto/1.15.2-r0/recipe-sysroot-native/usr/bin/python3-native:/home/pokybuild/yocto-worker/genericx86-64/build/build/tmp/work/i686-nativesdk-pokysdk-linux/nativesdk-xcb-proto/1.15.2-r0/recipe-sysroot-native/usr/bin/chrpath-native:/home/pokybuild/yocto-worker/genericx86-64/build/scripts:/home/pokybuild/yocto-worker/genericx86-64/build/build/tmp/work/i686-nativesdk-pokysdk-linux/nativesdk-xcb-proto/1.15.2-r0/recipe-sysroot-native/usr/bin/i686-pokysdk-linux:/home/pokybuild/yocto-worker/genericx86-64/build/build/tmp/work/i686-nativesdk-pokysdk-linux/nativesdk-xcb-proto/1.15.2-r0/recipe-sysroot/usr/local/oe-sdk-hardcoded-buildpath/sysroots/i686-pokysdk-linux/usr/bin/crossscripts:/home/pokybuild/yocto-worker/genericx86-64/build/build/tmp/work/i686-nativesdk-pokysdk-linux/nativesdk-xcb-proto/1.15.2-r0/recipe-sysroot-native/usr/sbin:/home/pokybuild/yocto-worker/genericx86-64/build/build/tmp/work/i686-nativesdk-pokysdk-linux/nativesdk-xcb-proto/1.15.2-r0/recipe-sysroot-native/usr/bin:/home/pokybuild/yocto-worker/genericx86-64/build/build/tmp/work/i686-nativesdk-pokysdk-linux/nativesdk-xcb-proto/1.15.2-r0/recipe-sysroot-native/sbin:/home/pokybuild/yocto-worker/genericx86-64/build/build/tmp/work/i686-nativesdk-pokysdk-linux/nativesdk-xcb-proto/1.15.2-r0/recipe-sysroot-native/bin:/home/pokybuild/yocto-worker/genericx86-64/build/bitbake/bin:/home/pokybuild/yocto-worker/genericx86-64/build/build/tmp/hosttools" opkg-build -Z xz -a "--memlimit=5% --threads=8" "" "" nativesdk-python-xcbgen /home/pokybuild/yocto-worker/genericx86-64/build/build/tmp/work/i686-nativesdk-pokysdk-linux/nativesdk-xcb-proto/1.15.2-r0/deploy-ipks/i686-nativesdk' returned non-zero exit status 1. Subprocess output:Usage: /home/pokybuild/yocto-worker/genericx86-64/build/build/tmp/work/i686-nativesdk-pokysdk-linux/nativesdk-xcb-proto/1.15.2-r0/recipe-sysroot-native/usr/bin/opkg-build [-A] [-X] [-c] [-C] [-Z compressor] [-a compressor_args] [-O] [-o owner] [-g group] <pkg_directory> [<destination_directory>] Command 'PATH="/home/pokybuild/yocto-worker/genericx86-64/build/scripts/nativesdk-intercept:/home/pokybuild/yocto-worker/genericx86-64/build/build/tmp/sysroots-uninative/x86_64-linux/usr/bin:/home/pokybuild/yocto-worker/genericx86-64/build/build/tmp/work/i686-nativesdk-pokysdk-linux/nativesdk-xcb-proto/1.15.2-r0/recipe-sysroot-native/usr/bin/python3-native:/home/pokybuild/yocto-worker/genericx86-64/build/build/tmp/work/i686-nativesdk-pokysdk-linux/nativesdk-xcb-proto/1.15.2-r0/recipe-sysroot-native/usr/bin/chrpath-native:/home/pokybuild/yocto-worker/genericx86-64/build/scripts:/home/pokybuild/yocto-worker/genericx86-64/build/build/tmp/work/i686-nativesdk-pokysdk-linux/nativesdk-xcb-proto/1.15.2-r0/recipe-sysroot-native/usr/bin/i686-pokysdk-linux:/home/pokybuild/yocto-worker/genericx86-64/build/build/tmp/work/i686-nativesdk-pokysdk-linux/nativesdk-xcb-proto/1.15.2-r0/recipe-sysroot/usr/local/oe-sdk-hardcoded-buildpath/sysroots/i686-pokysdk-linux/usr/bin/crossscripts:/home/pokybuild/yocto-worker/genericx86-64/build/build/tmp/work/i686-nativesdk-pokysdk-linux/nativesdk-xcb-proto/1.15.2-r0/recipe-sysroot-native/usr/sbin:/home/pokybuild/yocto-worker/genericx86-64/build/build/tmp/work/i686-nativesdk-pokysdk-linux/nativesdk-xcb-proto/1.15.2-r0/recipe-sysroot-native/usr/bin:/home/pokybuild/yocto-worker/genericx86-64/build/build/tmp/work/i686-nativesdk-pokysdk-linux/nativesdk-xcb-proto/1.15.2-r0/recipe-sysroot-native/sbin:/home/pokybuild/yocto-worker/genericx86-64/build/build/tmp/work/i686-nativesdk-pokysdk-linux/nativesdk-xcb-proto/1.15.2-r0/recipe-sysroot-native/bin:/home/pokybuild/yocto-worker/genericx86-64/build/bitbake/bin:/home/pokybuild/yocto-worker/genericx86-64/build/build/tmp/hosttools" opkg-build -Z xz -a "--memlimit=5% --threads=8" "" "" nativesdk-xcb-proto-dev /home/pokybuild/yocto-worker/genericx86-64/build/build/tmp/work/i686-nativesdk-pokysdk-linux/nativesdk-xcb-proto/1.15.2-r0/deploy-ipks/i686-nativesdk' returned non-zero exit status 1. Subprocess output:Usage: /home/pokybuild/yocto-worker/genericx86-64/build/build/tmp/work/i686-nativesdk-pokysdk-linux/nativesdk-xcb-proto/1.15.2-r0/recipe-sysroot-native/usr/bin/opkg-build [-A] [-X] [-c] [-C] [-Z compressor] [-a compressor_args] [-O] [-o owner] [-g group] <pkg_directory> [<destination_directory>] Command 'PATH="/home/pokybuild/yocto-worker/genericx86-64/build/scripts/nativesdk-intercept:/home/pokybuild/yocto-worker/genericx86-64/build/build/tmp/sysroots-uninative/x86_64-linux/usr/bin:/home/pokybuild/yocto-worker/genericx86-64/build/build/tmp/work/i686-nativesdk-pokysdk-linux/nativesdk-xcb-proto/1.15.2-r0/recipe-sysroot-native/usr/bin/python3-native:/home/pokybuild/yocto-worker/genericx86-64/build/build/tmp/work/i686-nativesdk-pokysdk-linux/nativesdk-xcb-proto/1.15.2-r0/recipe-sysroot-native/usr/bin/chrpath-native:/home/pokybuild/yocto-worker/genericx86-64/build/scripts:/home/pokybuild/yocto-worker/genericx86-64/build/build/tmp/work/i686-nativesdk-pokysdk-linux/nativesdk-xcb-proto/1.15.2-r0/recipe-sysroot-native/usr/bin/i686-pokysdk-linux:/home/pokybuild/yocto-worker/genericx86-64/build/build/tmp/work/i686-nativesdk-pokysdk-linux/nativesdk-xcb-proto/1.15.2-r0/recipe-sysroot/usr/local/oe-sdk-hardcoded-buildpath/sysroots/i686-pokysdk-linux/usr/bin/crossscripts:/home/pokybuild/yocto-worker/genericx86-64/build/build/tmp/work/i686-nativesdk-pokysdk-linux/nativesdk-xcb-proto/1.15.2-r0/recipe-sysroot-native/usr/sbin:/home/pokybuild/yocto-worker/genericx86-64/build/build/tmp/work/i686-nativesdk-pokysdk-linux/nativesdk-xcb-proto/1.15.2-r0/recipe-sysroot-native/usr/bin:/home/pokybuild/yocto-worker/genericx86-64/build/build/tmp/work/i686-nativesdk-pokysdk-linux/nativesdk-xcb-proto/1.15.2-r0/recipe-sysroot-native/sbin:/home/pokybuild/yocto-worker/genericx86-64/build/build/tmp/work/i686-nativesdk-pokysdk-linux/nativesdk-xcb-proto/1.15.2-r0/recipe-sysroot-native/bin:/home/pokybuild/yocto-worker/genericx86-64/build/bitbake/bin:/home/pokybuild/yocto-worker/genericx86-64/build/build/tmp/hosttools" opkg-build -Z xz -a "--memlimit=5% --threads=8" "" "" nativesdk-xcb-proto-dbg /home/pokybuild/yocto-worker/genericx86-64/build/build/tmp/work/i686-nativesdk-pokysdk-linux/nativesdk-xcb-proto/1.15.2-r0/deploy-ipks/i686-nativesdk' returned non-zero exit status 1. Subprocess output:Usage: /home/pokybuild/yocto-worker/genericx86-64/build/build/tmp/work/i686-nativesdk-pokysdk-linux/nativesdk-xcb-proto/1.15.2-r0/recipe-sysroot-native/usr/bin/opkg-build [-A] [-X] [-c] [-C] [-Z compressor] [-a compressor_args] [-O] [-o owner] [-g group] <pkg_directory> [<destination_directory>] > ________________________________ > Od: Alex Stewart <alex.stewart@ni.com> > Wysłane: czwartek, 13 lipca 2023 00:04 > Do: Piotr Łobacz <p.lobacz@welotec.com>; openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org> > Temat: Re: [OE-Core][PATCH v5 1/5] bitbake.conf: add acl and xattr distro native features support > > ACK this patchset, assuming the other maintainers are happy with how > we're handling the DISTO_FEATURES. > > I chose not to include Piotr's opkg and opkg-utils ACL/xattr changes > into the 0.6.2 release, which I have just pushed. They've been merged > afterward, and can still be applied atop 0.6.2, and they'll be merged > into the December opkg release. > > I have a branch ready to upgrade the opkg and opkg-utils recipes, but > I'd like to wait for this patchset to clear, because they will otherwise > conflict. > > On 7/12/23 15:04, Piotr Łobacz via lists.openembedded.org wrote: > > Include support for ACLs and extended file attributes for native > > builds, by default. > > > > Signed-off-by: Piotr Łobacz <p.lobacz@welotec.com> > > --- > > meta/conf/bitbake.conf | 2 +- > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf > > index 9625a6fef4..8daaaad615 100644 > > --- a/meta/conf/bitbake.conf > > +++ b/meta/conf/bitbake.conf > > @@ -904,7 +904,7 @@ IMAGE_FEATURES += "${EXTRA_IMAGE_FEATURES}" > > > > # Native distro features (will always be used for -native, even if they > > # are not enabled for target) > > -DISTRO_FEATURES_NATIVE ?= "x11 ipv6 xattr" > > +DISTRO_FEATURES_NATIVE ?= "acl x11 ipv6 xattr" > > DISTRO_FEATURES_NATIVESDK ?= "x11" > > > > # Normally target distro features will not be applied to native builds: > > > > > > > > -- > Alex Stewart > Software Engineer - NI Real-Time OS > NI (National Instruments) > > alex.stewart@ni.com > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#184256): https://lists.openembedded.org/g/openembedded-core/message/184256 > Mute This Topic: https://lists.openembedded.org/mt/100138221/3617179 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alexandre.belloni@bootlin.com] > -=-=-=-=-=-=-=-=-=-=-=- >
Ok, this is odd. Can you tell me how can I reproduce this, step by step on my local machine? Because I have to admit that it is not happening for me… BR Piotr Wysyłane z aplikacji Outlook dla systemu iOS<https://aka.ms/o0ukef>
Alex, from what I'm seeing the issue touches opkg-build command: opkg-build -Z xz -a "--memlimit=5% --threads=8" "" "" nativesdk-xcb-proto-dbg /home/pokybuild/yocto-worker/genericx86-64/build/build/tmp/work/i686-nativesdk-pokysdk-linux/nativesdk-xcb-proto/1.15.2-r0/deploy-ipks/i686-nativesdk' returned non-zero exit status 1. which causes you an error. This may happen with bad tar hosttools command. Can you please post me which version is on yocto autobuilder? BR Piotr
Alexander, this message: > Alex, > from what I'm seeing the issue touches opkg-build command: > > opkg-build -Z xz -a "--memlimit=5% --threads=8" "" "" nativesdk-xcb-proto-dbg /home/pokybuild/yocto-worker/genericx86-64/build/build/tmp/work/i686-nativesdk-pokysdk-linux/nativesdk-xcb-proto/1.15.2-r0/deploy-ipks/i686-nativesdk' returned non-zero exit status 1. > > which causes you an error. This may happen with bad tar hosttools command. Can you please post me which version is on yocto autobuilder? > BR > Piotr was meant for you, sorry for the confusion. Can you please verify/check what version of `tar` is being used on the autobuilder? This is really important for me, if we're going to move forward with it, because I have suspicions that it may not support posix or it may not be patched with --acls and --xattrs attributes https://www.mail-archive.com/bug-tar@gnu.org/msg06198.html BR Piotr
On 18/07/2023 08:05:12+0000, Piotr Łobacz wrote: > Alexander, this message: > > > Alex, > > from what I'm seeing the issue touches opkg-build command: > > > > opkg-build -Z xz -a "--memlimit=5% --threads=8" "" "" nativesdk-xcb-proto-dbg /home/pokybuild/yocto-worker/genericx86-64/build/build/tmp/work/i686-nativesdk-pokysdk-linux/nativesdk-xcb-proto/1.15.2-r0/deploy-ipks/i686-nativesdk' returned non-zero exit status 1. > > > > which causes you an error. This may happen with bad tar hosttools command. Can you please post me which version is on yocto autobuilder? > > > BR > > Piotr > > was meant for you, sorry for the confusion. Can you please verify/check what version of `tar` is being used on the autobuilder? This is really important for me, if we're going to move forward with it, because I have suspicions that it may not support posix or it may not be patched with --acls and --xattrs attributes https://www.mail-archive.com/bug-tar@gnu.org/msg06198.html > This fails at least on: fedora38-ty-4: tar (GNU tar) 1.34, has --acls and --xattrs stream8-ty-1: tar (GNU tar) 1.30, has --acls and --xattrs rocky9-ty-1: tar (GNU tar) 1.34, has --acls and --xattrs debian12-ty-1: tar (GNU tar) 1.34, has --acls and --xattrs ubuntu2204-ty-3: tar (GNU tar) 1.34, has --acls and --xattrs ubuntu2004-arm-1: tar (GNU tar) 1.30, has --acls and --xattrs opensuse154-ty-3: tar (GNU tar) 1.34, has --acls and --xattrs alma9-ty-1: tar (GNU tar) 1.34, has --acls and --xattrs ubuntu2210-ty-1: tar (GNU tar) 1.34, has --acls and --xattrs ubuntu2004-ty-1: tar (GNU tar) 1.30, has --acls and --xattrs Really, the question is more on which host this is working.
Hi Alexandre, I'm running this on docker with ubuntu 22.04 LTS and I have patched tar 1.34 with the patch I have given to you. I know that it contains these parameters, but they are faulty - meaning the ACLs do not preserve uid/gid in tar archive. Nevertheless this concerns me that opkg-build command should not fail. Additionally I have tested yesterday running my build without acl and xattr in DISTRO_FEATURES and it also worked for me - without applaying acls and xattrs to tar archive. Generally I need to reproduce your error and it seems to me that there is no other option as to re-create exactly the same environment on my machine locally, so I could investigate it. My question is how can we achive it in the simplest way? Does autobuilder use a docker for building? Because if so, I could take these docker scripts, run them and check what's happening. BR Piotr
On Wed, 2023-07-19 at 07:53 +0000, Piotr Łobacz wrote: > I'm running this on docker with ubuntu 22.04 LTS and I have patched > tar 1.34 with the patch I have given to you. I know that it contains > these parameters, but they are faulty - meaning the ACLs do not > preserve uid/gid in tar archive. Are you saying we need to patch tar in order for these patches to work? > Nevertheless this concerns me that opkg-build command should not > fail. Additionally I have tested yesterday running my build without > acl and xattr in DISTRO_FEATURES and it also worked for me - without > applaying acls and xattrs to tar archive. > Generally I need to reproduce your error and it seems to me that > there is no other option as to re-create exactly the same environment > on my machine locally, so I could investigate it. > > My question is how can we achive it in the simplest way? Does > autobuilder use a docker for building?Because if so, I could take > these docker scripts, run them and check what's happening. The autobuilders are standard installs of various distros. We keep them matching upstream and the list of installed software minimal. There are more issues in this patch series. For example there is this reproducibility issue: https://autobuilder.yoctoproject.org/typhoon/#/builders/117/builds/3219/steps/13/logs/stdio which is saying there were two sets of different ipk packages produced. These are available here: http://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20230718-bn_npkdc/packages/ sadly the automatic diffoscope output wasn't generated. Worryingly this report suggests the opkg-build change is not deterministic. FWIW this is just with the opkg-build change and the bitbake.conf change, not the other patches. These two changes alone also cause these warnings: https://autobuilder.yoctoproject.org/typhoon/#/builders/76/builds/7436/steps/12/logs/stdio https://autobuilder.yoctoproject.org/typhoon/#/builders/76/builds/7436/steps/18/logs/stdio https://autobuilder.yoctoproject.org/typhoon/#/builders/44/builds/7509/steps/21/logs/stdio e.g. things like: WARNING: core-image-sato-1.0-r0 do_rootfs: [log_check] core-image-sato: found 4 warning messages in the logfile: [log_check] Warning when reading ar archive header: Pathname can't be converted from UTF-8 to current locale. (errno=84) [log_check] Warning when reading ar archive header: Pathname can't be converted from UTF-8 to current locale. (errno=84) [log_check] Warning when reading ar archive header: Pathname can't be converted from UTF-8 to current locale. (errno=84) [log_check] Warning when reading ar archive header: Pathname can't be converted from UTF-8 to current locale. (errno=84) so there is a further issue to investigate there. All in all, there are a lot of issues with this patch series :(. Cheers, Richard
HI all, Hi Richard thx for quick response. Generally this patch for tar has been already applied to the upstream http://git.savannah.gnu.org/cgit/tar.git/commit/?id=5461025569c2d946fb31b79f16f60e923bbd79f9 Additionally a new version 1.35 has been released which has this fix applied as well. > Are you saying we need to patch tar in order for these patches to work? Unfortunately yes, but still this error, which occurs on autobuilder should not happen, because the patch for ACLs and xattrs only changes the writing algorithm to the tar file - meaning, when --numeric-owner parameter is being used all uid(s)/gid(s) are written with numbers instead of names. > WARNING: core-image-sato-1.0-r0 do_rootfs: [log_check] core-image-sato: found 4 warning messages in the logfile: > [log_check] Warning when reading ar archive header: Pathname can't be converted from UTF-8 to current locale. (errno=84) > [log_check] Warning when reading ar archive header: Pathname can't be converted from UTF-8 to current locale. (errno=84) > [log_check] Warning when reading ar archive header: Pathname can't be converted from UTF-8 to current locale. (errno=84) > [log_check] Warning when reading ar archive header: Pathname can't be converted from UTF-8 to current locale. (errno=84) This one is also occurring on our side, but I thought that it is irrelevant as it's just a warning and everything is working fine. Generally I have investigated it and it occurs that this error occurs from opkg source code https://git.yoctoproject.org/opkg/tree/libopkg/opkg_archive.c#n272 I added a debug code in here like that: opkg_msg(NOTICE, "Warning when reading ar archive header: %s (errno=%d) LC_CTYPE=%s LC_ALL=%s\n", archive_error_string(ar), archive_errno(ar), setlocale(LC_CTYPE, NULL), setlocale(LC_ALL, NULL)); to actually see what are the LC values and it occurred that it is equal C. Afterwards my investigations were focused on whom is calling the opkg command, maybe it is changing somehow locales and this is being executed in here http://git.openembedded.org/openembedded-core/tree/meta/lib/oe/package_manager/ipk/__init__.py#n368 but to my surprise LC_ALL which is being passed through os.environ is proper and equals LC_ALL: 'en_US.UTF-8' My guess is that opkg runs fork process to actually install all depends, because in log.do_rootfs file from this http://git.openembedded.org/openembedded-core/tree/meta/lib/oe/package_manager/ipk/__init__.py#n366 line I don't see the whole list of all packages that are being installed. A stupid fix for that was to set LC_ALL in opkg with: setlocale(LC_ALL, "en_US.UTF-8"); to check if it fixes the issue and yes it fixes but this is no go. Question is probably now to Alex does opkg can run fork instances for depends? Because I have found that it uses this vfork, however I dunno if it uses parent's process env variables or it's own. I think that this needs further investigations... BR Piotr Od: Richard Purdie <richard.purdie@linuxfoundation.org> Wysłane: środa, 19 lipca 2023 10:37 Do: Piotr Łobacz <p.lobacz@welotec.com>; Alexandre Belloni <alexandre.belloni@bootlin.com> DW: Alex Stewart <alex.stewart@ni.com>; openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org> Temat: Re: [OE-Core][PATCH v5 1/5] bitbake.conf: add acl and xattr distro native features support On Wed, 2023-07-19 at 07:53 +0000, Piotr Łobacz wrote: > I'm running this on docker with ubuntu 22.04 LTS and I have patched > tar 1.34 with the patch I have given to you. I know that it contains > these parameters, but they are faulty - meaning the ACLs do not > preserve uid/gid in tar archive. Are you saying we need to patch tar in order for these patches to work? > Nevertheless this concerns me that opkg-build command should not > fail. Additionally I have tested yesterday running my build without > acl and xattr in DISTRO_FEATURES and it also worked for me - without > applaying acls and xattrs to tar archive. > Generally I need to reproduce your error and it seems to me that > there is no other option as to re-create exactly the same environment > on my machine locally, so I could investigate it. > > My question is how can we achive it in the simplest way? Does > autobuilder use a docker for building?Because if so, I could take > these docker scripts, run them and check what's happening. The autobuilders are standard installs of various distros. We keep them matching upstream and the list of installed software minimal. There are more issues in this patch series. For example there is this reproducibility issue: https://autobuilder.yoctoproject.org/typhoon/#/builders/117/builds/3219/steps/13/logs/stdio which is saying there were two sets of different ipk packages produced. These are available here: http://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20230718-bn_npkdc/packages/ sadly the automatic diffoscope output wasn't generated. Worryingly this report suggests the opkg-build change is not deterministic. FWIW this is just with the opkg-build change and the bitbake.conf change, not the other patches. These two changes alone also cause these warnings: https://autobuilder.yoctoproject.org/typhoon/#/builders/76/builds/7436/steps/12/logs/stdio https://autobuilder.yoctoproject.org/typhoon/#/builders/76/builds/7436/steps/18/logs/stdio https://autobuilder.yoctoproject.org/typhoon/#/builders/44/builds/7509/steps/21/logs/stdio e.g. things like: WARNING: core-image-sato-1.0-r0 do_rootfs: [log_check] core-image-sato: found 4 warning messages in the logfile: [log_check] Warning when reading ar archive header: Pathname can't be converted from UTF-8 to current locale. (errno=84) [log_check] Warning when reading ar archive header: Pathname can't be converted from UTF-8 to current locale. (errno=84) [log_check] Warning when reading ar archive header: Pathname can't be converted from UTF-8 to current locale. (errno=84) [log_check] Warning when reading ar archive header: Pathname can't be converted from UTF-8 to current locale. (errno=84) so there is a further issue to investigate there. All in all, there are a lot of issues with this patch series :(. Cheers, Richard
Ok I have found the root cause for this warning. I will post it later on and this is another fix to opkg. BR Piotr
diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf index 9625a6fef4..8daaaad615 100644 --- a/meta/conf/bitbake.conf +++ b/meta/conf/bitbake.conf @@ -904,7 +904,7 @@ IMAGE_FEATURES += "${EXTRA_IMAGE_FEATURES}" # Native distro features (will always be used for -native, even if they # are not enabled for target) -DISTRO_FEATURES_NATIVE ?= "x11 ipv6 xattr" +DISTRO_FEATURES_NATIVE ?= "acl x11 ipv6 xattr" DISTRO_FEATURES_NATIVESDK ?= "x11" # Normally target distro features will not be applied to native builds:
Include support for ACLs and extended file attributes for native builds, by default. Signed-off-by: Piotr Łobacz <p.lobacz@welotec.com> --- meta/conf/bitbake.conf | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)