diff mbox series

ODP: [OE-Core][PATCH v12 1/7] path.py: add support for ACLs and all additional attributes

Message ID VI1PR04MB53733C1B75A5F09E4AD466E99109A@VI1PR04MB5373.eurprd04.prod.outlook.com
State New
Headers show
Series ODP: [OE-Core][PATCH v12 1/7] path.py: add support for ACLs and all additional attributes | expand

Commit Message

Piotr Łobacz Aug. 4, 2023, 8:48 a.m. UTC 
Hi Alexandre, Richard,
I have finally fixed it and testes with `oe-selftest -r reproducible` tests. which resulted me with:

plobacz@latitude:/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib$ oe-selftest -r reproducible
2023-08-03 10:31:57,100 - oe-selftest - INFO - Changing cwd to /media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build
2023-08-03 10:31:57,100 - oe-selftest - INFO - Adding layer libraries:
2023-08-03 10:31:57,101 - oe-selftest - INFO - 	/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/meta/lib
2023-08-03 10:31:57,101 - oe-selftest - INFO - 	/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/meta-yocto-bsp/lib
2023-08-03 10:31:57,101 - oe-selftest - INFO - 	/media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/meta-selftest/lib
2023-08-03 10:31:57,101 - oe-selftest - INFO - Running bitbake -e to test the configuration is valid/parsable
NOTE: Reconnecting to bitbake server...
NOTE: Retrying server connection (#1)... (10:31:58.417913)
NOTE: Reconnecting to bitbake server...
NOTE: Reconnecting to bitbake server...
NOTE: Retrying server connection (#1)... (10:31:58.417913)
NOTE: Retrying server connection (#1)... (10:31:58.417913)
NOTE: Starting bitbake server...
podpowiedź: Użycie „master” jako nawy początkowej gałęzi. Ta domyślna nazwa gałęzi
podpowiedź: może się zmienić. Aby ustawić nazwę początkowej gałęzi do użycia we
podpowiedź: wszystkich nowych repozytoriach, co uciszy ten komunikat, wykonaj:
podpowiedź: 
podpowiedź: 	git config --global init.defaultBranch <nazwa>
podpowiedź: 
podpowiedź: Tradycyjnie było „master”, a inne popularne to „main”, „trunk”
podpowiedź: i „development”. Nazwę właśnie utworzonej gałęzi można zmienić tym poleceniem:
podpowiedź: 
podpowiedź: 	git branch -m <nazwa>
2023-08-03 10:32:00,173 - oe-selftest - INFO - Adding: "include selftest.inc" in /media/plobacz/9582beb9-8774-4d4d-976f-63a6f1c87c13/poky-contrib/build-st/conf/local.conf
2023-08-03 10:32:00,174 - oe-selftest - INFO - Adding: "include bblayers.inc" in bblayers.conf
2023-08-03 10:32:00,174 - oe-selftest - INFO - test_diffoscope (reproducible.DiffoscopeTests)
2023-08-03 10:47:31,527 - oe-selftest - INFO -  ... ok
2023-08-03 10:47:31,528 - oe-selftest - INFO - test_reproducible_builds (reproducible.ReproducibleTests)
2023-08-03 10:48:34,404 - oe-selftest - INFO - Building reproducibleA (sstate allowed)...
2023-08-03 12:11:54,419 - oe-selftest - INFO - Keepalive message
2023-08-03 13:35:14,459 - oe-selftest - INFO - Keepalive message
2023-08-03 14:58:34,500 - oe-selftest - INFO - Keepalive message
2023-08-03 16:21:54,537 - oe-selftest - INFO - Keepalive message
2023-08-03 17:45:14,574 - oe-selftest - INFO - Keepalive message
2023-08-03 19:08:34,618 - oe-selftest - INFO - Keepalive message
2023-08-03 19:31:24,469 - oe-selftest - INFO - Building reproducibleB (sstate NOT allowed)...
2023-08-03 20:54:44,489 - oe-selftest - INFO - Keepalive message
2023-08-03 22:18:04,509 - oe-selftest - INFO - Keepalive message
2023-08-03 23:41:24,579 - oe-selftest - INFO - Keepalive message
2023-08-04 01:04:44,794 - oe-selftest - INFO - Keepalive message
2023-08-04 02:28:04,857 - oe-selftest - INFO - Keepalive message
2023-08-04 03:51:24,904 - oe-selftest - INFO - Keepalive message
2023-08-04 04:19:26,061 - oe-selftest - INFO - Checking deb packages for differences...
2023-08-04 04:19:34,627 - oe-selftest - INFO - Reproducibility summary for deb: same=12137 different=0 different_excluded=2 missing=0 total=12139
unused_exclusions=['rust-dbg']
2023-08-04 04:19:34,661 - oe-selftest - INFO - Checking ipk packages for differences...
2023-08-04 04:19:42,786 - oe-selftest - INFO - Reproducibility summary for ipk: same=12127 different=0 different_excluded=2 missing=0 total=12129
unused_exclusions=['rust-dbg']
2023-08-04 04:19:42,830 - oe-selftest - INFO - Checking rpm packages for differences...
2023-08-04 04:19:52,644 - oe-selftest - INFO - Reproducibility summary for rpm: same=12127 different=0 different_excluded=2 missing=0 total=12129
unused_exclusions=['rust-dbg']
2023-08-04 04:19:52,738 - oe-selftest - INFO -  ... ok
2023-08-04 04:27:21,489 - oe-selftest - INFO - ----------------------------------------------------------------------
2023-08-04 04:27:21,490 - oe-selftest - INFO - Ran 2 tests in 64523.097s
2023-08-04 04:27:21,490 - oe-selftest - INFO - OK
2023-08-04 04:27:25,783 - oe-selftest - INFO - RESULTS:
2023-08-04 04:27:25,783 - oe-selftest - INFO - RESULTS - reproducible.DiffoscopeTests.test_diffoscope: PASSED (931.35s)
2023-08-04 04:27:25,783 - oe-selftest - INFO - RESULTS - reproducible.ReproducibleTests.test_reproducible_builds: PASSED (63141.21s)
2023-08-04 04:27:27,913 - oe-selftest - INFO - SUMMARY:
2023-08-04 04:27:27,913 - oe-selftest - INFO - oe-selftest () - Ran 2 tests in 64523.097s
2023-08-04 04:27:27,913 - oe-selftest - INFO - oe-selftest - OK - All required tests passed (successes=2, skipped=0, failures=0, errors=0)

So I think that this will finally work as well for you.

BR
Piotr

Od: Piotr Łobacz <p.lobacz@welotec.com>
Wysłane: piątek, 4 sierpnia 2023 10:43
Do: openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org>
DW: Piotr Łobacz <p.lobacz@welotec.com>
Temat: [OE-Core][PATCH v12 1/7] path.py: add support for ACLs and all additional attributes 
 
Extend `tar` command inside copytree() and copyhardlinktree() functions,
with additional parameters, in order to support ACLs and xattr. The posix
format doesn't need to be set in here, as GNU tar switches to it whenewer
is uses --acls and/or --xattrs parameters.

Additionaly change preservation of additional attributes for `cp` command
to all (meaning mode, ownership, timestamps etc.) not only xattrs, which
were previously.

Signed-off-by: Piotr Łobacz <p.lobacz@welotec.com>
---
 meta/lib/oe/path.py | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
diff mbox series

Patch

diff --git a/meta/lib/oe/path.py b/meta/lib/oe/path.py
index 0dc8f172d5..56e8e214ac 100644
--- a/meta/lib/oe/path.py
+++ b/meta/lib/oe/path.py
@@ -92,7 +92,7 @@  def copytree(src, dst):
     # This way we also preserve hardlinks between files in the tree.
 
     bb.utils.mkdirhier(dst)
-    cmd = "tar --xattrs --xattrs-include='*' -cf - -S -C %s -p . | tar --xattrs --xattrs-include='*' -xf - -C %s" % (src, dst)
+    cmd = "tar --acls --xattrs --xattrs-include='*' -cf - -S -C %s -p . | tar --acls --xattrs --xattrs-include='*' -xf - -C %s" % (src, dst)
     subprocess.check_output(cmd, shell=True, stderr=subprocess.STDOUT)
 
 def copyhardlinktree(src, dst):
@@ -119,7 +119,7 @@  def copyhardlinktree(src, dst):
     if (canhard):
         # Need to copy directories only with tar first since cp will error if two 
         # writers try and create a directory at the same time
-        cmd = "cd %s; find . -type d -print | tar --xattrs --xattrs-include='*' -cf - -S -C %s -p --no-recursion --files-from - | tar --xattrs --xattrs-include='*' -xhf - -C %s" % (src, src, dst)
+        cmd = "cd %s; find . -type d -print | tar --acls --xattrs --xattrs-include='*' -cf - -S -C %s -p --no-recursion --files-from - | tar --acls --xattrs --xattrs-include='*' -xhf - -C %s" % (src, src, dst)
         subprocess.check_output(cmd, shell=True, stderr=subprocess.STDOUT)
         source = ''
         if os.path.isdir(src):
@@ -130,7 +130,7 @@  def copyhardlinktree(src, dst):
         else:
             source = src
             s_dir = os.getcwd()
-        cmd = 'cp -afl --preserve=xattr %s %s' % (source, os.path.realpath(dst))
+        cmd = 'cp -afl --preserve=all %s %s' % (source, os.path.realpath(dst))
         subprocess.check_output(cmd, shell=True, cwd=s_dir, stderr=subprocess.STDOUT)
     else:
         copytree(src, dst)