| Message ID | 20230724204639.353571-3-p.lobacz@welotec.com |
|---|---|
| State | New |
| Headers | show |
| Series | [v10,master-next,1/5] package_ipk.bbclass: add support for ACLs and xattr | expand |
On 7/24/23 1:46 PM, Piotr Łobacz wrote: > Add support for tar archives created with --acls and/or --xattrs options, > PAX header format. > > GNU tar and libarchive already supports ACLs and extended attributes. > We can now add this support as well to opkg-build script in order to use > fsetattr or setcap inside do_install command and end up with a file in > an image with the relevant ACLs and xattrs. this patch series is regressing ipk backends with errors like below I am using archlinux for by build system if that matters. ERROR: linux-libc-headers-6.4-r0 do_package_write_ipk: Fatal errors occurred in subprocesses: Command 'PATH="/mnt/b/yoe/master/build/tmp/sysroots-uninative/x86_64-linux/usr/bin:/mnt/b/yoe/master/sources/poky/scripts:/mnt/b/yoe/master/build/tmp/work/cortexa72-cortexa53-crypto-yoe-linux/linux-libc-headers/6.4-r0/recipe-sysroot-native/usr/bin/aarch64-yoe-linux:/mnt/b/yoe/master/build/tmp/work/cortexa72-cortexa53-crypto-yoe-linux/linux-libc-headers/6.4-r0/recipe-sysroot/usr/bin/crossscripts:/mnt/b/yoe/master/build/tmp/work/cortexa72-cortexa53-crypto-yoe-linux/linux-libc-headers/6.4-r0/recipe-sysroot-native/usr/sbin:/mnt/b/yoe/master/build/tmp/work/cortexa72-cortexa53-crypto-yoe-linux/linux-libc-headers/6.4-r0/recipe-sysroot-native/usr/bin:/mnt/b/yoe/master/build/tmp/work/cortexa72-cortexa53-crypto-yoe-linux/linux-libc-headers/6.4-r0/recipe-sysroot-native/sbin:/mnt/b/yoe/master/build/tmp/work/cortexa72-cortexa53-crypto-yoe-linux/linux-libc-headers/6.4-r0/recipe-sysroot-native/bin:/mnt/b/yoe/master/sources/poky/bitbake/bin:/mnt/b/yoe/master/build/tmp/hosttools" opkg-build -Z zstd -a "--threads=44" linux-libc-headers-dbg /mnt/b/yoe/master/build/tmp/work/cortexa72-cortexa53-crypto-yoe-linux/linux-libc-headers/6.4-r0/deploy-ipks/cortexa72-cortexa53-crypto' returned non-zero exit status 1. Subprocess output:find: paths must precede expression: `BUILD_CCLD=gcc ' Command 'PATH="/mnt/b/yoe/master/build/tmp/sysroots-uninative/x86_64-linux/usr/bin:/mnt/b/yoe/master/sources/poky/scripts:/mnt/b/yoe/master/build/tmp/work/cortexa72-cortexa53-crypto-yoe-linux/linux-libc-headers/6.4-r0/recipe-sysroot-native/usr/bin/aarch64-yoe-linux:/mnt/b/yoe/master/build/tmp/work/cortexa72-cortexa53-crypto-yoe-linux/linux-libc-headers/6.4-r0/recipe-sysroot/usr/bin/crossscripts:/mnt/b/yoe/master/build/tmp/work/cortexa72-cortexa53-crypto-yoe-linux/linux-libc-headers/6.4-r0/recipe-sysroot-native/usr/sbin:/mnt/b/yoe/master/build/tmp/work/cortexa72-cortexa53-crypto-yoe-linux/linux-libc-headers/6.4-r0/recipe-sysroot-native/usr/bin:/mnt/b/yoe/master/build/tmp/work/cortexa72-cortexa53-crypto-yoe-linux/linux-libc-headers/6.4-r0/recipe-sysroot-native/sbin:/mnt/b/yoe/master/build/tmp/work/cortexa72-cortexa53-crypto-yoe-linux/linux-libc-headers/6.4-r0/recipe-sysroot-native/bin:/mnt/b/yoe/master/sources/poky/bitbake/bin:/mnt/b/yoe/master/build/tmp/hosttools" opkg-build -Z zstd -a "--threads=44" linux-libc-headers-dev /mnt/b/yoe/master/build/tmp/work/cortexa72-cortexa53-crypto-yoe-linux/linux-libc-headers/6.4-r0/deploy-ipks/cortexa72-cortexa53-crypto' returned non-zero exit status 1. Subprocess output:find: paths must precede expression: `BUILD_CCLD=gcc ' > > Signed-off-by: Piotr Łobacz <p.lobacz@welotec.com> > --- > ...kg-build-Add-acls-and-xattrs-support.patch | 164 ++++++++++++++++++ > .../opkg-utils/opkg-utils_0.6.2.bb | 1 + > 2 files changed, 165 insertions(+) > create mode 100644 meta/recipes-devtools/opkg-utils/opkg-utils/0002-opkg-build-Add-acls-and-xattrs-support.patch > > diff --git a/meta/recipes-devtools/opkg-utils/opkg-utils/0002-opkg-build-Add-acls-and-xattrs-support.patch b/meta/recipes-devtools/opkg-utils/opkg-utils/0002-opkg-build-Add-acls-and-xattrs-support.patch > new file mode 100644 > index 0000000000..0874d3f75c > --- /dev/null > +++ b/meta/recipes-devtools/opkg-utils/opkg-utils/0002-opkg-build-Add-acls-and-xattrs-support.patch > @@ -0,0 +1,164 @@ > +From 5a5901f703bfac7376cfef3d4734c37400db03f1 Mon Sep 17 00:00:00 2001 > +From: =?UTF-8?q?Piotr=20=C5=81obacz?= <p.lobacz@welotec.com> > +Date: Wed, 5 Jul 2023 10:31:13 +0200 > +Subject: [PATCH] opkg-build: Add acls and xattrs support > +MIME-Version: 1.0 > +Content-Type: text/plain; charset=UTF-8 > +Content-Transfer-Encoding: 8bit > + > +Add support for tar archives created with --acls and/or --xattrs options, > +PAX header format. > + > +GNU tar and libarchive already supports ACLs and extended attributes. > +We can now add this support as well to opkg-build script in order to use > +fsetattr or setcap inside do_install command and end up with a file in > +an image with the relevant ACLs and xattrs. > + > +Upstream-Status: Submitted [https://groups.google.com/g/opkg-devel/c/dYNHrLjDwg8] > + > +[1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=15097 > +[2] https://groups.google.com/g/opkg-devel/c/aEGL7XRXfaA > + > +Signed-off-by: Piotr Łobacz <p.lobacz@welotec.com> > +--- > + opkg-build | 76 +++++++++++++++++++++++++++++++++++------------------- > + 1 file changed, 50 insertions(+), 26 deletions(-) > + > +diff --git a/opkg-build b/opkg-build > +index a9e45d4..b3127e0 100755 > +--- a/opkg-build > ++++ b/opkg-build > +@@ -145,6 +145,7 @@ You probably want to chown these to a system user: " >&2 > + ### > + # opkg-build "main" > + ### > ++attributesargs="" > + ogargs="" > + outer=ar > + noclean=0 > +@@ -153,22 +154,6 @@ compressor=gzip > + zipargs="-9n" > + compressorargs="" > + > +-# Determine if tar supports the --format argument by checking the help output. > +-# > +-# This is needed because: > +-# - Busybox tar doesn't support '--format' > +-# - On some Linux distros, tar now defaults to posix format if '--format' > +-# isn't explicitly specified > +-# - Opkg doesn't currently support posix format archives > +-# > +-# It's easier to check for mention of the '--format' option than to detect the > +-# tar implementation and maintain a list of which support '--format'. > +-tarformat="" > +-if tar --help 2>&1 | grep -- "--format" > /dev/null; > +-then > +- tarformat="--format=gnu" > +-fi > +- > + compressor_ext() { > + case $1 in > + gzip|pigz) > +@@ -197,13 +182,17 @@ compressor_ext() { > + : <<=cut > + =head1 SYNOPSIS > + > +-B<opkg-build> [B<-c>] [B<-C>] [B<-Z> I<compressor>] [B<-a>] [B<-O>] [B<-o> I<owner>] [B<-g> I<group>] I<pkg_directory> [I<destination_directory>] > ++B<opkg-build> [B<-A>] [B<-X>] [B<-c>] [B<-C>] [B<-Z> I<compressor>] [B<-a>] [B<-O>] [B<-o> I<owner>] [B<-g> I<group>] I<pkg_directory> [I<destination_directory>] > + > + =cut > + > +-usage="Usage: $0 [-c] [-C] [-Z compressor] [-a compressor_args] [-O] [-o owner] [-g group] <pkg_directory> [<destination_directory>]" > +-while getopts "a:cCg:ho:vOZ:" opt; do > ++usage="Usage: $0 [-A] [-X] [-c] [-C] [-Z compressor] [-a compressor_args] [-O] [-o owner] [-g group] <pkg_directory> [<destination_directory>]" > ++while getopts "Aa:cCg:ho:vOXZ:" opt; do > + case $opt in > ++ A ) attributesargs="--acls" > ++ ;; > ++ X ) attributesargs="$attributesargs --xattrs" > ++ ;; > + o ) owner=$OPTARG > + ogargs="--owner=$owner" > + ;; > +@@ -232,6 +221,31 @@ while getopts "a:cCg:ho:vOZ:" opt; do > + esac > + done > + > ++# Determine if tar supports the --format argument by checking the help output. > ++# > ++# This is needed because: > ++# - Busybox tar doesn't support '--format' > ++# - On some Linux distros, tar now defaults to posix format if '--format' > ++# isn't explicitly specified > ++# - Opkg doesn't currently support posix format archives > ++# > ++# It's easier to check for mention of the '--format' option than to detect the > ++# tar implementation and maintain a list of which support '--format'. > ++tarformat="" > ++if tar --help 2>&1 | grep -- "--format" > /dev/null; > ++then > ++ # For ACLs or xattr support, gnu format will not work > ++ # we need to set posix format instead > ++ if [ ! -z "$attributesargs" ] ; then > ++ tarformat="--format=posix" > ++ else > ++ tarformat="--format=gnu" > ++ fi > ++elif [ ! -z "$attributesargs" ] ; then > ++ echo "*** Error: Attributes: $attributesargs, doesn't' work, without posix format, which is not supported by tar command." >&2 > ++ exit 1 > ++fi > ++ > + cext=$(compressor_ext $compressor) > + > + # pgzip requires -T to avoid timestamps on the gzip archive > +@@ -301,21 +315,31 @@ fi > + tmp_dir=$dest_dir/IPKG_BUILD.$$ > + mkdir $tmp_dir > + > +-build_date="${SOURCE_DATE_EPOCH:-$(date +%s)}" > +- > +-mtime_args="" > ++mtime_args="--mtime=@${SOURCE_DATE_EPOCH:-$(date +%s)}" > + # --clamp-mtime requires tar > 1.28. Only use it if SOURCE_DATE_EPOCH is set, to avoid having a generic case dependency on tar > 1.28. > + # this setting will make sure files generated at build time have consistent mtimes, for reproducible builds. > + if [ ! -z "$SOURCE_DATE_EPOCH" ]; then > +- mtime_args="--mtime=@$build_date --clamp-mtime" > ++ mtime_args="$mtime_args --clamp-mtime" > ++fi > ++ > ++# Notice, that if you create an archive in POSIX format (see section GNU tar and POSIX tar) and the environment variable POSIXLY_CORRECT is set, > ++# then the two archives created using the same options on the same set of files will not be byte-to-byte equivalent even with the above option. > ++# This is because the posix default for extended header names includes the PID of the tar process, which is different at each run. To produce > ++# byte-to-byte equivalent archives in this case, either unset POSIXLY_CORRECT, or use the following option: > ++# > ++# --pax-option=exthdr.name=%d/PaxHeaders/%f,atime:=0,ctime:=0 > ++# > ++# https://www.gnu.org/software/tar/manual/html_node/PAX-keywords.html > ++if [ $tarformat == "--format=posix" ]; then > ++ mtime_args="$mtime_args --pax-option=exthdr.name=%d/PaxHeaders/%f,atime:=0,ctime:=0" > + fi > + > + export LANG=C > + export LC_ALL=C > + ( cd $pkg_dir/$CONTROL && find . -type f | sort > $tmp_dir/control_list ) > + ( cd $pkg_dir && find . -path ./$CONTROL -prune -o -path . -o -print | sort > $tmp_dir/file_list ) > +-( cd $pkg_dir && tar $ogargs $tsortargs --no-recursion $mtime_args -c $tarformat -T $tmp_dir/file_list | $compressor $compressorargs > $tmp_dir/data.tar.$cext ) > +-( cd $pkg_dir/$CONTROL && tar $ogargs $tsortargs --no-recursion --mtime=@$build_date -c $tarformat -T $tmp_dir/control_list | gzip $zipargs > $tmp_dir/control.tar.gz ) > ++( cd $pkg_dir && tar $attributesargs $ogargs $tsortargs --numeric-owner --no-recursion $mtime_args -c $tarformat -T $tmp_dir/file_list | $compressor $compressorargs > $tmp_dir/data.tar.$cext ) > ++( cd $pkg_dir/$CONTROL && tar $ogargs $tsortargs --no-recursion $mtime_args -c $tarformat -T $tmp_dir/control_list | gzip $zipargs > $tmp_dir/control.tar.gz ) > + rm $tmp_dir/file_list > + rm $tmp_dir/control_list > + > +@@ -331,7 +355,7 @@ rm -f $pkg_file > + if [ "$outer" = "ar" ] ; then > + ( cd $tmp_dir && ar -crfD $pkg_file ./debian-binary ./control.tar.gz ./data.tar.$cext ) > + else > +- ( cd $tmp_dir && tar -c $tsortargs --mtime=@$build_date $tarformat ./debian-binary ./control.tar.gz ./data.tar.$cext | gzip $zipargs > $pkg_file ) > ++ ( cd $tmp_dir && tar -c $tsortargs $mtime_args $tarformat ./debian-binary ./control.tar.gz ./data.tar.$cext | gzip $zipargs > $pkg_file ) > + fi > + > + rm $tmp_dir/debian-binary $tmp_dir/data.tar.$cext $tmp_dir/control.tar.gz > +-- > +2.34.1 > + > diff --git a/meta/recipes-devtools/opkg-utils/opkg-utils_0.6.2.bb b/meta/recipes-devtools/opkg-utils/opkg-utils_0.6.2.bb > index eb88b9b734..d5ce2cfbe2 100644 > --- a/meta/recipes-devtools/opkg-utils/opkg-utils_0.6.2.bb > +++ b/meta/recipes-devtools/opkg-utils/opkg-utils_0.6.2.bb > @@ -9,6 +9,7 @@ PROVIDES += "${@bb.utils.contains('PACKAGECONFIG', 'update-alternatives', 'virtu > > SRC_URI = "git://git.yoctoproject.org/opkg-utils;protocol=https;branch=master \ > file://0001-update-alternatives-correctly-match-priority.patch \ > + file://0002-opkg-build-Add-acls-and-xattrs-support.patch \ > " > SRCREV = "67994e62dc598282830385da75ba9b1abbbda941" > > > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#184815): https://lists.openembedded.org/g/openembedded-core/message/184815 > Mute This Topic: https://lists.openembedded.org/mt/100337842/1997914 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [raj.khem@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- >
On Fri, Aug 4, 2023 at 8:27 AM Khem Raj <raj.khem@gmail.com> wrote: > > On 7/24/23 1:46 PM, Piotr Łobacz wrote: > > Add support for tar archives created with --acls and/or --xattrs options, > > PAX header format. > > > > GNU tar and libarchive already supports ACLs and extended attributes. > > We can now add this support as well to opkg-build script in order to use > > fsetattr or setcap inside do_install command and end up with a file in > > an image with the relevant ACLs and xattrs. > > this patch series is regressing ipk backends with errors like below > I am using archlinux for by build system if that matters. > > ERROR: linux-libc-headers-6.4-r0 do_package_write_ipk: Fatal errors > occurred in subprocesses: > Command > 'PATH="/mnt/b/yoe/master/build/tmp/sysroots-uninative/x86_64-linux/usr/bin:/mnt/b/yoe/master/sources/poky/scripts:/mnt/b/yoe/master/build/tmp/work/cortexa72-cortexa53-crypto-yoe-linux/linux-libc-headers/6.4-r0/recipe-sysroot-native/usr/bin/aarch64-yoe-linux:/mnt/b/yoe/master/build/tmp/work/cortexa72-cortexa53-crypto-yoe-linux/linux-libc-headers/6.4-r0/recipe-sysroot/usr/bin/crossscripts:/mnt/b/yoe/master/build/tmp/work/cortexa72-cortexa53-crypto-yoe-linux/linux-libc-headers/6.4-r0/recipe-sysroot-native/usr/sbin:/mnt/b/yoe/master/build/tmp/work/cortexa72-cortexa53-crypto-yoe-linux/linux-libc-headers/6.4-r0/recipe-sysroot-native/usr/bin:/mnt/b/yoe/master/build/tmp/work/cortexa72-cortexa53-crypto-yoe-linux/linux-libc-headers/6.4-r0/recipe-sysroot-native/sbin:/mnt/b/yoe/master/build/tmp/work/cortexa72-cortexa53-crypto-yoe-linux/linux-libc-headers/6.4-r0/recipe-sysroot-native/bin:/mnt/b/yoe/master/sources/poky/bitbake/bin:/mnt/b/yoe/master/build/tmp/hosttools" > opkg-build -Z zstd -a "--threads=44" linux-libc-headers-dbg > /mnt/b/yoe/master/build/tmp/work/cortexa72-cortexa53-crypto-yoe-linux/linux-libc-headers/6.4-r0/deploy-ipks/cortexa72-cortexa53-crypto' > returned non-zero exit status 1. > Subprocess output:find: paths must precede expression: `BUILD_CCLD=gcc ' > Command > 'PATH="/mnt/b/yoe/master/build/tmp/sysroots-uninative/x86_64-linux/usr/bin:/mnt/b/yoe/master/sources/poky/scripts:/mnt/b/yoe/master/build/tmp/work/cortexa72-cortexa53-crypto-yoe-linux/linux-libc-headers/6.4-r0/recipe-sysroot-native/usr/bin/aarch64-yoe-linux:/mnt/b/yoe/master/build/tmp/work/cortexa72-cortexa53-crypto-yoe-linux/linux-libc-headers/6.4-r0/recipe-sysroot/usr/bin/crossscripts:/mnt/b/yoe/master/build/tmp/work/cortexa72-cortexa53-crypto-yoe-linux/linux-libc-headers/6.4-r0/recipe-sysroot-native/usr/sbin:/mnt/b/yoe/master/build/tmp/work/cortexa72-cortexa53-crypto-yoe-linux/linux-libc-headers/6.4-r0/recipe-sysroot-native/usr/bin:/mnt/b/yoe/master/build/tmp/work/cortexa72-cortexa53-crypto-yoe-linux/linux-libc-headers/6.4-r0/recipe-sysroot-native/sbin:/mnt/b/yoe/master/build/tmp/work/cortexa72-cortexa53-crypto-yoe-linux/linux-libc-headers/6.4-r0/recipe-sysroot-native/bin:/mnt/b/yoe/master/sources/poky/bitbake/bin:/mnt/b/yoe/master/build/tmp/hosttools" > opkg-build -Z zstd -a "--threads=44" linux-libc-headers-dev > /mnt/b/yoe/master/build/tmp/work/cortexa72-cortexa53-crypto-yoe-linux/linux-libc-headers/6.4-r0/deploy-ipks/cortexa72-cortexa53-crypto' > returned non-zero exit status 1. > Subprocess output:find: paths must precede expression: `BUILD_CCLD=gcc ' > I think this is more related to building on hosts with glibc 2.38 rather than opkg issue. So this patch might not be the cause of this problem but maybe something in pseudo or some such layer. > > > > Signed-off-by: Piotr Łobacz <p.lobacz@welotec.com> > > --- > > ...kg-build-Add-acls-and-xattrs-support.patch | 164 ++++++++++++++++++ > > .../opkg-utils/opkg-utils_0.6.2.bb | 1 + > > 2 files changed, 165 insertions(+) > > create mode 100644 meta/recipes-devtools/opkg-utils/opkg-utils/0002-opkg-build-Add-acls-and-xattrs-support.patch > > > > diff --git a/meta/recipes-devtools/opkg-utils/opkg-utils/0002-opkg-build-Add-acls-and-xattrs-support.patch b/meta/recipes-devtools/opkg-utils/opkg-utils/0002-opkg-build-Add-acls-and-xattrs-support.patch > > new file mode 100644 > > index 0000000000..0874d3f75c > > --- /dev/null > > +++ b/meta/recipes-devtools/opkg-utils/opkg-utils/0002-opkg-build-Add-acls-and-xattrs-support.patch > > @@ -0,0 +1,164 @@ > > +From 5a5901f703bfac7376cfef3d4734c37400db03f1 Mon Sep 17 00:00:00 2001 > > +From: =?UTF-8?q?Piotr=20=C5=81obacz?= <p.lobacz@welotec.com> > > +Date: Wed, 5 Jul 2023 10:31:13 +0200 > > +Subject: [PATCH] opkg-build: Add acls and xattrs support > > +MIME-Version: 1.0 > > +Content-Type: text/plain; charset=UTF-8 > > +Content-Transfer-Encoding: 8bit > > + > > +Add support for tar archives created with --acls and/or --xattrs options, > > +PAX header format. > > + > > +GNU tar and libarchive already supports ACLs and extended attributes. > > +We can now add this support as well to opkg-build script in order to use > > +fsetattr or setcap inside do_install command and end up with a file in > > +an image with the relevant ACLs and xattrs. > > + > > +Upstream-Status: Submitted [https://groups.google.com/g/opkg-devel/c/dYNHrLjDwg8] > > + > > +[1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=15097 > > +[2] https://groups.google.com/g/opkg-devel/c/aEGL7XRXfaA > > + > > +Signed-off-by: Piotr Łobacz <p.lobacz@welotec.com> > > +--- > > + opkg-build | 76 +++++++++++++++++++++++++++++++++++------------------- > > + 1 file changed, 50 insertions(+), 26 deletions(-) > > + > > +diff --git a/opkg-build b/opkg-build > > +index a9e45d4..b3127e0 100755 > > +--- a/opkg-build > > ++++ b/opkg-build > > +@@ -145,6 +145,7 @@ You probably want to chown these to a system user: " >&2 > > + ### > > + # opkg-build "main" > > + ### > > ++attributesargs="" > > + ogargs="" > > + outer=ar > > + noclean=0 > > +@@ -153,22 +154,6 @@ compressor=gzip > > + zipargs="-9n" > > + compressorargs="" > > + > > +-# Determine if tar supports the --format argument by checking the help output. > > +-# > > +-# This is needed because: > > +-# - Busybox tar doesn't support '--format' > > +-# - On some Linux distros, tar now defaults to posix format if '--format' > > +-# isn't explicitly specified > > +-# - Opkg doesn't currently support posix format archives > > +-# > > +-# It's easier to check for mention of the '--format' option than to detect the > > +-# tar implementation and maintain a list of which support '--format'. > > +-tarformat="" > > +-if tar --help 2>&1 | grep -- "--format" > /dev/null; > > +-then > > +- tarformat="--format=gnu" > > +-fi > > +- > > + compressor_ext() { > > + case $1 in > > + gzip|pigz) > > +@@ -197,13 +182,17 @@ compressor_ext() { > > + : <<=cut > > + =head1 SYNOPSIS > > + > > +-B<opkg-build> [B<-c>] [B<-C>] [B<-Z> I<compressor>] [B<-a>] [B<-O>] [B<-o> I<owner>] [B<-g> I<group>] I<pkg_directory> [I<destination_directory>] > > ++B<opkg-build> [B<-A>] [B<-X>] [B<-c>] [B<-C>] [B<-Z> I<compressor>] [B<-a>] [B<-O>] [B<-o> I<owner>] [B<-g> I<group>] I<pkg_directory> [I<destination_directory>] > > + > > + =cut > > + > > +-usage="Usage: $0 [-c] [-C] [-Z compressor] [-a compressor_args] [-O] [-o owner] [-g group] <pkg_directory> [<destination_directory>]" > > +-while getopts "a:cCg:ho:vOZ:" opt; do > > ++usage="Usage: $0 [-A] [-X] [-c] [-C] [-Z compressor] [-a compressor_args] [-O] [-o owner] [-g group] <pkg_directory> [<destination_directory>]" > > ++while getopts "Aa:cCg:ho:vOXZ:" opt; do > > + case $opt in > > ++ A ) attributesargs="--acls" > > ++ ;; > > ++ X ) attributesargs="$attributesargs --xattrs" > > ++ ;; > > + o ) owner=$OPTARG > > + ogargs="--owner=$owner" > > + ;; > > +@@ -232,6 +221,31 @@ while getopts "a:cCg:ho:vOZ:" opt; do > > + esac > > + done > > + > > ++# Determine if tar supports the --format argument by checking the help output. > > ++# > > ++# This is needed because: > > ++# - Busybox tar doesn't support '--format' > > ++# - On some Linux distros, tar now defaults to posix format if '--format' > > ++# isn't explicitly specified > > ++# - Opkg doesn't currently support posix format archives > > ++# > > ++# It's easier to check for mention of the '--format' option than to detect the > > ++# tar implementation and maintain a list of which support '--format'. > > ++tarformat="" > > ++if tar --help 2>&1 | grep -- "--format" > /dev/null; > > ++then > > ++ # For ACLs or xattr support, gnu format will not work > > ++ # we need to set posix format instead > > ++ if [ ! -z "$attributesargs" ] ; then > > ++ tarformat="--format=posix" > > ++ else > > ++ tarformat="--format=gnu" > > ++ fi > > ++elif [ ! -z "$attributesargs" ] ; then > > ++ echo "*** Error: Attributes: $attributesargs, doesn't' work, without posix format, which is not supported by tar command." >&2 > > ++ exit 1 > > ++fi > > ++ > > + cext=$(compressor_ext $compressor) > > + > > + # pgzip requires -T to avoid timestamps on the gzip archive > > +@@ -301,21 +315,31 @@ fi > > + tmp_dir=$dest_dir/IPKG_BUILD.$$ > > + mkdir $tmp_dir > > + > > +-build_date="${SOURCE_DATE_EPOCH:-$(date +%s)}" > > +- > > +-mtime_args="" > > ++mtime_args="--mtime=@${SOURCE_DATE_EPOCH:-$(date +%s)}" > > + # --clamp-mtime requires tar > 1.28. Only use it if SOURCE_DATE_EPOCH is set, to avoid having a generic case dependency on tar > 1.28. > > + # this setting will make sure files generated at build time have consistent mtimes, for reproducible builds. > > + if [ ! -z "$SOURCE_DATE_EPOCH" ]; then > > +- mtime_args="--mtime=@$build_date --clamp-mtime" > > ++ mtime_args="$mtime_args --clamp-mtime" > > ++fi > > ++ > > ++# Notice, that if you create an archive in POSIX format (see section GNU tar and POSIX tar) and the environment variable POSIXLY_CORRECT is set, > > ++# then the two archives created using the same options on the same set of files will not be byte-to-byte equivalent even with the above option. > > ++# This is because the posix default for extended header names includes the PID of the tar process, which is different at each run. To produce > > ++# byte-to-byte equivalent archives in this case, either unset POSIXLY_CORRECT, or use the following option: > > ++# > > ++# --pax-option=exthdr.name=%d/PaxHeaders/%f,atime:=0,ctime:=0 > > ++# > > ++# https://www.gnu.org/software/tar/manual/html_node/PAX-keywords.html > > ++if [ $tarformat == "--format=posix" ]; then > > ++ mtime_args="$mtime_args --pax-option=exthdr.name=%d/PaxHeaders/%f,atime:=0,ctime:=0" > > + fi > > + > > + export LANG=C > > + export LC_ALL=C > > + ( cd $pkg_dir/$CONTROL && find . -type f | sort > $tmp_dir/control_list ) > > + ( cd $pkg_dir && find . -path ./$CONTROL -prune -o -path . -o -print | sort > $tmp_dir/file_list ) > > +-( cd $pkg_dir && tar $ogargs $tsortargs --no-recursion $mtime_args -c $tarformat -T $tmp_dir/file_list | $compressor $compressorargs > $tmp_dir/data.tar.$cext ) > > +-( cd $pkg_dir/$CONTROL && tar $ogargs $tsortargs --no-recursion --mtime=@$build_date -c $tarformat -T $tmp_dir/control_list | gzip $zipargs > $tmp_dir/control.tar.gz ) > > ++( cd $pkg_dir && tar $attributesargs $ogargs $tsortargs --numeric-owner --no-recursion $mtime_args -c $tarformat -T $tmp_dir/file_list | $compressor $compressorargs > $tmp_dir/data.tar.$cext ) > > ++( cd $pkg_dir/$CONTROL && tar $ogargs $tsortargs --no-recursion $mtime_args -c $tarformat -T $tmp_dir/control_list | gzip $zipargs > $tmp_dir/control.tar.gz ) > > + rm $tmp_dir/file_list > > + rm $tmp_dir/control_list > > + > > +@@ -331,7 +355,7 @@ rm -f $pkg_file > > + if [ "$outer" = "ar" ] ; then > > + ( cd $tmp_dir && ar -crfD $pkg_file ./debian-binary ./control.tar.gz ./data.tar.$cext ) > > + else > > +- ( cd $tmp_dir && tar -c $tsortargs --mtime=@$build_date $tarformat ./debian-binary ./control.tar.gz ./data.tar.$cext | gzip $zipargs > $pkg_file ) > > ++ ( cd $tmp_dir && tar -c $tsortargs $mtime_args $tarformat ./debian-binary ./control.tar.gz ./data.tar.$cext | gzip $zipargs > $pkg_file ) > > + fi > > + > > + rm $tmp_dir/debian-binary $tmp_dir/data.tar.$cext $tmp_dir/control.tar.gz > > +-- > > +2.34.1 > > + > > diff --git a/meta/recipes-devtools/opkg-utils/opkg-utils_0.6.2.bb b/meta/recipes-devtools/opkg-utils/opkg-utils_0.6.2.bb > > index eb88b9b734..d5ce2cfbe2 100644 > > --- a/meta/recipes-devtools/opkg-utils/opkg-utils_0.6.2.bb > > +++ b/meta/recipes-devtools/opkg-utils/opkg-utils_0.6.2.bb > > @@ -9,6 +9,7 @@ PROVIDES += "${@bb.utils.contains('PACKAGECONFIG', 'update-alternatives', 'virtu > > > > SRC_URI = "git://git.yoctoproject.org/opkg-utils;protocol=https;branch=master \ > > file://0001-update-alternatives-correctly-match-priority.patch \ > > + file://0002-opkg-build-Add-acls-and-xattrs-support.patch \ > > " > > SRCREV = "67994e62dc598282830385da75ba9b1abbbda941" > > > > > > > > > > -=-=-=-=-=-=-=-=-=-=-=- > > Links: You receive all messages sent to this group. > > View/Reply Online (#184815): https://lists.openembedded.org/g/openembedded-core/message/184815 > > Mute This Topic: https://lists.openembedded.org/mt/100337842/1997914 > > Group Owner: openembedded-core+owner@lists.openembedded.org > > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [raj.khem@gmail.com] > > -=-=-=-=-=-=-=-=-=-=-=- > >
diff --git a/meta/recipes-devtools/opkg-utils/opkg-utils/0002-opkg-build-Add-acls-and-xattrs-support.patch b/meta/recipes-devtools/opkg-utils/opkg-utils/0002-opkg-build-Add-acls-and-xattrs-support.patch new file mode 100644 index 0000000000..0874d3f75c --- /dev/null +++ b/meta/recipes-devtools/opkg-utils/opkg-utils/0002-opkg-build-Add-acls-and-xattrs-support.patch @@ -0,0 +1,164 @@ +From 5a5901f703bfac7376cfef3d4734c37400db03f1 Mon Sep 17 00:00:00 2001 +From: =?UTF-8?q?Piotr=20=C5=81obacz?= <p.lobacz@welotec.com> +Date: Wed, 5 Jul 2023 10:31:13 +0200 +Subject: [PATCH] opkg-build: Add acls and xattrs support +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +Add support for tar archives created with --acls and/or --xattrs options, +PAX header format. + +GNU tar and libarchive already supports ACLs and extended attributes. +We can now add this support as well to opkg-build script in order to use +fsetattr or setcap inside do_install command and end up with a file in +an image with the relevant ACLs and xattrs. + +Upstream-Status: Submitted [https://groups.google.com/g/opkg-devel/c/dYNHrLjDwg8] + +[1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=15097 +[2] https://groups.google.com/g/opkg-devel/c/aEGL7XRXfaA + +Signed-off-by: Piotr Łobacz <p.lobacz@welotec.com> +--- + opkg-build | 76 +++++++++++++++++++++++++++++++++++------------------- + 1 file changed, 50 insertions(+), 26 deletions(-) + +diff --git a/opkg-build b/opkg-build +index a9e45d4..b3127e0 100755 +--- a/opkg-build ++++ b/opkg-build +@@ -145,6 +145,7 @@ You probably want to chown these to a system user: " >&2 + ### + # opkg-build "main" + ### ++attributesargs="" + ogargs="" + outer=ar + noclean=0 +@@ -153,22 +154,6 @@ compressor=gzip + zipargs="-9n" + compressorargs="" + +-# Determine if tar supports the --format argument by checking the help output. +-# +-# This is needed because: +-# - Busybox tar doesn't support '--format' +-# - On some Linux distros, tar now defaults to posix format if '--format' +-# isn't explicitly specified +-# - Opkg doesn't currently support posix format archives +-# +-# It's easier to check for mention of the '--format' option than to detect the +-# tar implementation and maintain a list of which support '--format'. +-tarformat="" +-if tar --help 2>&1 | grep -- "--format" > /dev/null; +-then +- tarformat="--format=gnu" +-fi +- + compressor_ext() { + case $1 in + gzip|pigz) +@@ -197,13 +182,17 @@ compressor_ext() { + : <<=cut + =head1 SYNOPSIS + +-B<opkg-build> [B<-c>] [B<-C>] [B<-Z> I<compressor>] [B<-a>] [B<-O>] [B<-o> I<owner>] [B<-g> I<group>] I<pkg_directory> [I<destination_directory>] ++B<opkg-build> [B<-A>] [B<-X>] [B<-c>] [B<-C>] [B<-Z> I<compressor>] [B<-a>] [B<-O>] [B<-o> I<owner>] [B<-g> I<group>] I<pkg_directory> [I<destination_directory>] + + =cut + +-usage="Usage: $0 [-c] [-C] [-Z compressor] [-a compressor_args] [-O] [-o owner] [-g group] <pkg_directory> [<destination_directory>]" +-while getopts "a:cCg:ho:vOZ:" opt; do ++usage="Usage: $0 [-A] [-X] [-c] [-C] [-Z compressor] [-a compressor_args] [-O] [-o owner] [-g group] <pkg_directory> [<destination_directory>]" ++while getopts "Aa:cCg:ho:vOXZ:" opt; do + case $opt in ++ A ) attributesargs="--acls" ++ ;; ++ X ) attributesargs="$attributesargs --xattrs" ++ ;; + o ) owner=$OPTARG + ogargs="--owner=$owner" + ;; +@@ -232,6 +221,31 @@ while getopts "a:cCg:ho:vOZ:" opt; do + esac + done + ++# Determine if tar supports the --format argument by checking the help output. ++# ++# This is needed because: ++# - Busybox tar doesn't support '--format' ++# - On some Linux distros, tar now defaults to posix format if '--format' ++# isn't explicitly specified ++# - Opkg doesn't currently support posix format archives ++# ++# It's easier to check for mention of the '--format' option than to detect the ++# tar implementation and maintain a list of which support '--format'. ++tarformat="" ++if tar --help 2>&1 | grep -- "--format" > /dev/null; ++then ++ # For ACLs or xattr support, gnu format will not work ++ # we need to set posix format instead ++ if [ ! -z "$attributesargs" ] ; then ++ tarformat="--format=posix" ++ else ++ tarformat="--format=gnu" ++ fi ++elif [ ! -z "$attributesargs" ] ; then ++ echo "*** Error: Attributes: $attributesargs, doesn't' work, without posix format, which is not supported by tar command." >&2 ++ exit 1 ++fi ++ + cext=$(compressor_ext $compressor) + + # pgzip requires -T to avoid timestamps on the gzip archive +@@ -301,21 +315,31 @@ fi + tmp_dir=$dest_dir/IPKG_BUILD.$$ + mkdir $tmp_dir + +-build_date="${SOURCE_DATE_EPOCH:-$(date +%s)}" +- +-mtime_args="" ++mtime_args="--mtime=@${SOURCE_DATE_EPOCH:-$(date +%s)}" + # --clamp-mtime requires tar > 1.28. Only use it if SOURCE_DATE_EPOCH is set, to avoid having a generic case dependency on tar > 1.28. + # this setting will make sure files generated at build time have consistent mtimes, for reproducible builds. + if [ ! -z "$SOURCE_DATE_EPOCH" ]; then +- mtime_args="--mtime=@$build_date --clamp-mtime" ++ mtime_args="$mtime_args --clamp-mtime" ++fi ++ ++# Notice, that if you create an archive in POSIX format (see section GNU tar and POSIX tar) and the environment variable POSIXLY_CORRECT is set, ++# then the two archives created using the same options on the same set of files will not be byte-to-byte equivalent even with the above option. ++# This is because the posix default for extended header names includes the PID of the tar process, which is different at each run. To produce ++# byte-to-byte equivalent archives in this case, either unset POSIXLY_CORRECT, or use the following option: ++# ++# --pax-option=exthdr.name=%d/PaxHeaders/%f,atime:=0,ctime:=0 ++# ++# https://www.gnu.org/software/tar/manual/html_node/PAX-keywords.html ++if [ $tarformat == "--format=posix" ]; then ++ mtime_args="$mtime_args --pax-option=exthdr.name=%d/PaxHeaders/%f,atime:=0,ctime:=0" + fi + + export LANG=C + export LC_ALL=C + ( cd $pkg_dir/$CONTROL && find . -type f | sort > $tmp_dir/control_list ) + ( cd $pkg_dir && find . -path ./$CONTROL -prune -o -path . -o -print | sort > $tmp_dir/file_list ) +-( cd $pkg_dir && tar $ogargs $tsortargs --no-recursion $mtime_args -c $tarformat -T $tmp_dir/file_list | $compressor $compressorargs > $tmp_dir/data.tar.$cext ) +-( cd $pkg_dir/$CONTROL && tar $ogargs $tsortargs --no-recursion --mtime=@$build_date -c $tarformat -T $tmp_dir/control_list | gzip $zipargs > $tmp_dir/control.tar.gz ) ++( cd $pkg_dir && tar $attributesargs $ogargs $tsortargs --numeric-owner --no-recursion $mtime_args -c $tarformat -T $tmp_dir/file_list | $compressor $compressorargs > $tmp_dir/data.tar.$cext ) ++( cd $pkg_dir/$CONTROL && tar $ogargs $tsortargs --no-recursion $mtime_args -c $tarformat -T $tmp_dir/control_list | gzip $zipargs > $tmp_dir/control.tar.gz ) + rm $tmp_dir/file_list + rm $tmp_dir/control_list + +@@ -331,7 +355,7 @@ rm -f $pkg_file + if [ "$outer" = "ar" ] ; then + ( cd $tmp_dir && ar -crfD $pkg_file ./debian-binary ./control.tar.gz ./data.tar.$cext ) + else +- ( cd $tmp_dir && tar -c $tsortargs --mtime=@$build_date $tarformat ./debian-binary ./control.tar.gz ./data.tar.$cext | gzip $zipargs > $pkg_file ) ++ ( cd $tmp_dir && tar -c $tsortargs $mtime_args $tarformat ./debian-binary ./control.tar.gz ./data.tar.$cext | gzip $zipargs > $pkg_file ) + fi + + rm $tmp_dir/debian-binary $tmp_dir/data.tar.$cext $tmp_dir/control.tar.gz +-- +2.34.1 + diff --git a/meta/recipes-devtools/opkg-utils/opkg-utils_0.6.2.bb b/meta/recipes-devtools/opkg-utils/opkg-utils_0.6.2.bb index eb88b9b734..d5ce2cfbe2 100644 --- a/meta/recipes-devtools/opkg-utils/opkg-utils_0.6.2.bb +++ b/meta/recipes-devtools/opkg-utils/opkg-utils_0.6.2.bb @@ -9,6 +9,7 @@ PROVIDES += "${@bb.utils.contains('PACKAGECONFIG', 'update-alternatives', 'virtu SRC_URI = "git://git.yoctoproject.org/opkg-utils;protocol=https;branch=master \ file://0001-update-alternatives-correctly-match-priority.patch \ + file://0002-opkg-build-Add-acls-and-xattrs-support.patch \ " SRCREV = "67994e62dc598282830385da75ba9b1abbbda941"
Add support for tar archives created with --acls and/or --xattrs options, PAX header format. GNU tar and libarchive already supports ACLs and extended attributes. We can now add this support as well to opkg-build script in order to use fsetattr or setcap inside do_install command and end up with a file in an image with the relevant ACLs and xattrs. Signed-off-by: Piotr Łobacz <p.lobacz@welotec.com> --- ...kg-build-Add-acls-and-xattrs-support.patch | 164 ++++++++++++++++++ .../opkg-utils/opkg-utils_0.6.2.bb | 1 + 2 files changed, 165 insertions(+) create mode 100644 meta/recipes-devtools/opkg-utils/opkg-utils/0002-opkg-build-Add-acls-and-xattrs-support.patch