| Message ID | 20220609163907.40205-1-jose.quaresma@foundries.io |
|---|---|
| State | New |
| Headers | show |
| Series | [v3] archiver: avoid race conditions when using multiconfig and sharing the same TMPDIR | expand |
On Thu, 2022-06-09 at 17:39 +0100, Jose Quaresma wrote: > The archiver doesn't support using multiconfig sharing the same TMPDIR. > When in the multiconfig and using the same TMPDIR: the two machines can make > the same copy at the same time to the same destination, which will not work. > > This issiue can be replicated with meta-ti layer: > > # use a multiconfig machine > > MACHINE = "am64xx-evm" > > # archive sources for target recipes in conf/local.conf > > INHERIT += "archiver" > > COPYLEFT_RECIPE_TYPES = "target" > > ARCHIVER_MODE[src] = "original" > > ARCHIVER_MODE[diff] = "1" > > # and run bitbake with: > > bitbake mc:k3r5:gcc-source-11.2.0 gcc-source-11.2.0 -c do_unpack_and_patch -f > > Fixes the following: > > ERROR: mc:k3r5:gcc-source-11.2.0-11.2.0-r0 do_unpack_and_patch: Error executing a python function in exec_func_python() autogenerated: > > > > The stack trace of python calls that resulted in this exception/failure was: > > File: 'exec_func_python() autogenerated', lineno: 2, function: <module> > > 0001: > > *** 0002:do_unpack_and_patch(d) > > 0003: > > File: '/build/conf/../../layers/openembedded-core/meta/classes/archiver.bbclass', lineno: 489, function: do_unpack_and_patch > > 0485: # Save the original source for creating the patches > > 0486: if d.getVarFlag('ARCHIVER_MODE', 'diff') == '1': > > 0487: src = d.getVar('S').rstrip('/') > > 0488: src_orig = '%s.orig' % src > > *** 0489: oe.path.copytree(src, src_orig) > > 0490: > > 0491: if bb.data.inherits_class('dos2unix', d): > > 0492: bb.build.exec_func('do_convert_crlf_to_lf', d) > > 0493: > > File: '/build/conf/../../layers/openembedded-core/meta/lib/oe/path.py', lineno: 94, function: copytree > > 0090: # This way we also preserve hardlinks between files in the tree. > > 0091: > > 0092: bb.utils.mkdirhier(dst) > > 0093: cmd = "tar --xattrs --xattrs-include='*' -cf - -S -C %s -p . | tar --xattrs --xattrs-include='*' -xf - -C %s" % (src, dst) > > *** 0094: subprocess.check_output(cmd, shell=True, stderr=subprocess.STDOUT) > > 0095: > > 0096:def copyhardlinktree(src, dst): > > 0097: """Make a tree of hard links when possible, otherwise copy.""" > > 0098: bb.utils.mkdirhier(dst) > > File: '/usr/lib/python3.10/subprocess.py', lineno: 420, function: check_output > > 0416: else: > > 0417: empty = b'' > > 0418: kwargs['input'] = empty > > 0419: > > *** 0420: return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, > > 0421: **kwargs).stdout > > 0422: > > 0423: > > 0424:class CompletedProcess(object): > > File: '/usr/lib/python3.10/subprocess.py', lineno: 524, function: run > > 0520: # We don't call process.wait() as .__exit__ does that for us. > > 0521: raise > > 0522: retcode = process.poll() > > 0523: if check and retcode: > > *** 0524: raise CalledProcessError(retcode, process.args, > > 0525: output=stdout, stderr=stderr) > > 0526: return CompletedProcess(process.args, retcode, stdout, stderr) > > 0527: > > 0528: > > Exception: subprocess.CalledProcessError: Command 'tar --xattrs --xattrs-include='*' -cf - -S -C /build/tmp-lmp/work-shared/gcc-11.2.0-r0/gcc-11.2.0 -p . | tar --xattrs --xattrs-include='*' |-xf - -C /build/tmp-lmp/work-shared/gcc-11.2.0-r0/gcc-11.2.0.orig' returned non-zero exit status 2. > > > > Subprocess output: > > tar: ./gcc/config/moxie/predicates.md: file changed as we read it > > tar: ./patches: Cannot stat: No such file or directory > > tar: Exiting with failure status due to previous errors > > > > ERROR: Logfile of failure stored in: /build/tmp-lmp/work-shared/gcc-11.2.0-r0/temp/log.do_unpack_and_patch.291906 > > ERROR: Task (mc:k3r5:/build/conf/../../layers/openembedded-core/meta/recipes-devtools/gcc/gcc-source_11.2.bb:do_unpack_and_patch) failed with exit code '1' > > Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io> > --- > > v2: replace "bb.error" with "bb.fatal" > v3: instead of failing make it more resilient > > meta/classes/archiver.bbclass | 3 ++- > 1 file changed, 2 insertions(+), 1 deletion(-) > > diff --git a/meta/classes/archiver.bbclass b/meta/classes/archiver.bbclass > index c19c770d11..b1d7aa0247 100644 > --- a/meta/classes/archiver.bbclass > +++ b/meta/classes/archiver.bbclass > @@ -485,7 +485,8 @@ python do_unpack_and_patch() { > # Save the original source for creating the patches > if d.getVarFlag('ARCHIVER_MODE', 'diff') == '1': > src = d.getVar('S').rstrip('/') > - src_orig = '%s.orig' % src > + # append machine name to avoid races in multiconfig when using the same TMPDIR > + src_orig = '%s.orig.%s' % (src, d.getVar('MACHINE')) > oe.path.copytree(src, src_orig) > > if bb.data.inherits_class('dos2unix', d): I'm not convinced this is correct. I did have a quick look at that multiconfig (a pointer to the config helps a lot!) and it looks correct in that it is just two different MACHINE settings being built within the same TMPDIR. That should mean there is only one gcc-source and that gcc-source should only run once. Obviously that isn't true so something is broken. This patch just works around it rather than identifying what is broken and fixing that. We need to find the real problem. Cheers, Richard
On Thu, 2022-06-09 at 17:39 +0100, Jose Quaresma wrote: > The archiver doesn't support using multiconfig sharing the same TMPDIR. > When in the multiconfig and using the same TMPDIR: the two machines can make > the same copy at the same time to the same destination, which will not work. > > This issiue can be replicated with meta-ti layer: > > # use a multiconfig machine > > MACHINE = "am64xx-evm" > > # archive sources for target recipes in conf/local.conf > > INHERIT += "archiver" > > COPYLEFT_RECIPE_TYPES = "target" > > ARCHIVER_MODE[src] = "original" > > ARCHIVER_MODE[diff] = "1" > > # and run bitbake with: > > bitbake mc:k3r5:gcc-source-11.2.0 gcc-source-11.2.0 -c do_unpack_and_patch -f > I went digging. If I enable this as above (using master) and run $ bitbake mc:k3r5:gcc-source-12.1.0 gcc-source-12.1.0 -c do_unpack_and_patch -S none then: $ bitbake-diffsigs tmp/stamps/work-shared/gcc-12.1.0-r0.do_unpack_and_patch.sigdata.* NOTE: Starting bitbake server... Task dependencies changed from: ['ABIEXTENSION', 'ARCHIVER_MODE[diff-exclude]', 'ARCHIVER_MODE[diff]', 'ARCHIVER_MODE[src]', 'ARCHIVER_OUTDIR', 'ARCHIVER_TOPDIR', 'ARCHIVER_WORKDIR', 'B', 'BASEURI', 'EXTENDPE', 'GNU_MIRROR', 'HOST_ARCH', 'HOST_OS', 'HOST_SYS', 'HOST_VENDOR', 'INSANE_SKIP', 'LIBCEXTENSION', 'PATCHTOOL', 'PE', 'PF', 'PN', 'PR', 'PV', 'RECIPE_SYSROOT_NATIVE', 'S', 'SDE_DIR', 'SDE_FILE', 'SOURCEDIR', 'SRC_URI', 'SRC_URI[sha256sum]', 'STAGING_DIR_NATIVE', 'T', 'TARGET_ARCH', 'TARGET_OS', 'TARGET_SYS', 'TARGET_VENDOR', 'TUNE_ARCH', 'UNPACK_DIR', 'base_do_unpack', 'base_do_unpack[cleandirs]', 'create_diff_gz', 'create_source_date_epoch_stamp', 'do_patch', 'do_qa_patch', 'do_qa_unpack', 'do_unpack', 'do_unpack[cleandirs]', 'is_work_shared', 'patch_do_patch', 'src_patches', 'unpack_check_src_uri'] to: ['ABIEXTENSION', 'ABIEXTENSION_32', 'ABIEXTENSION_64', 'ARCHIVER_MODE[diff-exclude]', 'ARCHIVER_MODE[diff]', 'ARCHIVER_MODE[src]', 'ARCHIVER_OUTDIR', 'ARCHIVER_TOPDIR', 'ARCHIVER_WORKDIR', 'ARMPKGSFX_ENDIAN_64', 'B', 'BASEURI', 'EXTENDPE', 'GNU_MIRROR', 'HOST_ARCH', 'HOST_OS', 'HOST_SYS', 'HOST_VENDOR', 'INSANE_SKIP', 'LIBCEXTENSION', 'PATCHTOOL', 'PE', 'PF', 'PN', 'PR', 'PV', 'RECIPE_SYSROOT_NATIVE', 'S', 'SDE_DIR', 'SDE_FILE', 'SOURCEDIR', 'SRC_URI', 'SRC_URI[sha256sum]', 'STAGING_DIR_NATIVE', 'T', 'TARGET_ARCH', 'TARGET_OS', 'TARGET_SYS', 'TARGET_VENDOR', 'TUNE_ARCH', 'TUNE_ARCH_32', 'TUNE_ARCH_64', 'UNPACK_DIR', 'base_do_unpack', 'base_do_unpack[cleandirs]', 'create_diff_gz', 'create_source_date_epoch_stamp', 'do_patch', 'do_qa_patch', 'do_qa_unpack', 'do_unpack', 'do_unpack[cleandirs]', 'is_work_shared', 'patch_do_patch', 'src_patches', 'unpack_check_src_uri'] basehash changed from 005580a7a5a1ab1ff554d0a9e47c2ad4ba932f3fe185d8d349b06c83e5e58b6c to d7b2e696005a578d5958ef4e73f1fc83e22612a3ae4294fb0540b0fadcb6c50f List of dependencies for variable ABIEXTENSION changed from 'set()' to '{'ABIEXTENSION_64', 'ABIEXTENSION_32'}' List of dependencies for variable TUNE_ARCH changed from 'set()' to '{'TUNE_ARCH_32', 'TUNE_ARCH_64'}' Dependency on variable ABIEXTENSION_32 was added Dependency on variable ABIEXTENSION_64 was added Dependency on variable ARMPKGSFX_ENDIAN_64 was added Dependency on variable TUNE_ARCH_32 was added Dependency on variable TUNE_ARCH_64 was added Variable ABIEXTENSION value changed: @@ -1 +1,2 @@ -eabi +${@bb.utils.contains('TUNE_FEATURES', 'aarch64', '${ABIEXTENSION_64}', '${ABIEXTENSION_32}', d)} +TUNE_FEATURES{aarch64} = Set Variable TUNE_ARCH value changed: @@ -1,2 +1,2 @@ -${@bb.utils.contains('TUNE_FEATURES', 'bigendian', 'armeb', 'arm', d)} -TUNE_FEATURES{bigendian} = Unset +${@bb.utils.contains('TUNE_FEATURES', 'aarch64', '${TUNE_ARCH_64}', '${TUNE_ARCH_32}', d)} +TUNE_FEATURES{aarch64} = Set this tells us the real issue. The task is changing due to toolchain issues. These stamps should be the same and aren't and this is the real issue that needs fixing. Cheers, Richard
On Fri, 2022-06-10 at 11:43 +0100, Richard Purdie via lists.openembedded.org wrote: > On Thu, 2022-06-09 at 17:39 +0100, Jose Quaresma wrote: > > The archiver doesn't support using multiconfig sharing the same TMPDIR. > > When in the multiconfig and using the same TMPDIR: the two machines can make > > the same copy at the same time to the same destination, which will not work. > > > > This issiue can be replicated with meta-ti layer: > > > # use a multiconfig machine > > > MACHINE = "am64xx-evm" > > > # archive sources for target recipes in conf/local.conf > > > INHERIT += "archiver" > > > COPYLEFT_RECIPE_TYPES = "target" > > > ARCHIVER_MODE[src] = "original" > > > ARCHIVER_MODE[diff] = "1" > > > # and run bitbake with: > > > bitbake mc:k3r5:gcc-source-11.2.0 gcc-source-11.2.0 -c do_unpack_and_patch -f > > > > I went digging. If I enable this as above (using master) and run > > $ bitbake mc:k3r5:gcc-source-12.1.0 gcc-source-12.1.0 -c do_unpack_and_patch -S none > > then: > > $ bitbake-diffsigs tmp/stamps/work-shared/gcc-12.1.0-r0.do_unpack_and_patch.sigdata.* > NOTE: Starting bitbake server... > Task dependencies changed from: > ['ABIEXTENSION', 'ARCHIVER_MODE[diff-exclude]', 'ARCHIVER_MODE[diff]', 'ARCHIVER_MODE[src]', 'ARCHIVER_OUTDIR', 'ARCHIVER_TOPDIR', 'ARCHIVER_WORKDIR', 'B', 'BASEURI', 'EXTENDPE', 'GNU_MIRROR', 'HOST_ARCH', 'HOST_OS', 'HOST_SYS', 'HOST_VENDOR', 'INSANE_SKIP', 'LIBCEXTENSION', 'PATCHTOOL', 'PE', 'PF', 'PN', 'PR', 'PV', 'RECIPE_SYSROOT_NATIVE', 'S', 'SDE_DIR', 'SDE_FILE', 'SOURCEDIR', 'SRC_URI', 'SRC_URI[sha256sum]', 'STAGING_DIR_NATIVE', 'T', 'TARGET_ARCH', 'TARGET_OS', 'TARGET_SYS', 'TARGET_VENDOR', 'TUNE_ARCH', 'UNPACK_DIR', 'base_do_unpack', 'base_do_unpack[cleandirs]', 'create_diff_gz', 'create_source_date_epoch_stamp', 'do_patch', 'do_qa_patch', 'do_qa_unpack', 'do_unpack', 'do_unpack[cleandirs]', 'is_work_shared', 'patch_do_patch', 'src_patches', 'unpack_check_src_uri'] > to: > ['ABIEXTENSION', 'ABIEXTENSION_32', 'ABIEXTENSION_64', 'ARCHIVER_MODE[diff-exclude]', 'ARCHIVER_MODE[diff]', 'ARCHIVER_MODE[src]', 'ARCHIVER_OUTDIR', 'ARCHIVER_TOPDIR', 'ARCHIVER_WORKDIR', 'ARMPKGSFX_ENDIAN_64', 'B', 'BASEURI', 'EXTENDPE', 'GNU_MIRROR', 'HOST_ARCH', 'HOST_OS', 'HOST_SYS', 'HOST_VENDOR', 'INSANE_SKIP', 'LIBCEXTENSION', 'PATCHTOOL', 'PE', 'PF', 'PN', 'PR', 'PV', 'RECIPE_SYSROOT_NATIVE', 'S', 'SDE_DIR', 'SDE_FILE', 'SOURCEDIR', 'SRC_URI', 'SRC_URI[sha256sum]', 'STAGING_DIR_NATIVE', 'T', 'TARGET_ARCH', 'TARGET_OS', 'TARGET_SYS', 'TARGET_VENDOR', 'TUNE_ARCH', 'TUNE_ARCH_32', 'TUNE_ARCH_64', 'UNPACK_DIR', 'base_do_unpack', 'base_do_unpack[cleandirs]', 'create_diff_gz', 'create_source_date_epoch_stamp', 'do_patch', 'do_qa_patch', 'do_qa_unpack', 'do_unpack', 'do_unpack[cleandirs]', 'is_work_shared', 'patch_do_patch', 'src_patches', 'unpack_check_src_uri'] > basehash changed from 005580a7a5a1ab1ff554d0a9e47c2ad4ba932f3fe185d8d349b06c83e5e58b6c to d7b2e696005a578d5958ef4e73f1fc83e22612a3ae4294fb0540b0fadcb6c50f > List of dependencies for variable ABIEXTENSION changed from 'set()' to '{'ABIEXTENSION_64', 'ABIEXTENSION_32'}' > List of dependencies for variable TUNE_ARCH changed from 'set()' to '{'TUNE_ARCH_32', 'TUNE_ARCH_64'}' > Dependency on variable ABIEXTENSION_32 was added > Dependency on variable ABIEXTENSION_64 was added > Dependency on variable ARMPKGSFX_ENDIAN_64 was added > Dependency on variable TUNE_ARCH_32 was added > Dependency on variable TUNE_ARCH_64 was added > Variable ABIEXTENSION value changed: > @@ -1 +1,2 @@ > -eabi > +${@bb.utils.contains('TUNE_FEATURES', 'aarch64', '${ABIEXTENSION_64}', '${ABIEXTENSION_32}', d)} > +TUNE_FEATURES{aarch64} = Set > Variable TUNE_ARCH value changed: > @@ -1,2 +1,2 @@ > -${@bb.utils.contains('TUNE_FEATURES', 'bigendian', 'armeb', 'arm', d)} > -TUNE_FEATURES{bigendian} = Unset > +${@bb.utils.contains('TUNE_FEATURES', 'aarch64', '${TUNE_ARCH_64}', '${TUNE_ARCH_32}', d)} > +TUNE_FEATURES{aarch64} = Set > > this tells us the real issue. The task is changing due to toolchain > issues. These stamps should be the same and aren't and this is the real > issue that needs fixing. To further follow up, there are two issues: a) the task uses ${B} which for gcc includes HOST_SYS and TARGET_SYS. We can just set that in the gcc-source recipe to B = "${WORKDIR}/build". b) the archiver uses: ARCHIVER_OUTDIR = ${ARCHIVER_TOPDIR}/${TARGET_SYS}/${PF}/ which includes TARGET_SYS and between the two different MACHINE values, this changes from 'arm-poky-linux-gnueabi' to 'aarch64-poky-linux'. This leads to the task running twice, once for each multiconfig. The first is easily solved as I mentioned. The question in the second case is whether the task should run twice and where the output should go since it is common to both builds. I do wonder if the archiver shouldn't set: ARCHIVER_OUTPUT:pn-gcc-source = "${ARCHIVER_TOPDIR}/allarch/${PF}" Cheers, Richard
Hi Richard, Richard Purdie <richard.purdie@linuxfoundation.org> escreveu no dia sexta, 10/06/2022 à(s) 11:59: > On Fri, 2022-06-10 at 11:43 +0100, Richard Purdie via > lists.openembedded.org wrote: > > On Thu, 2022-06-09 at 17:39 +0100, Jose Quaresma wrote: > > > The archiver doesn't support using multiconfig sharing the same TMPDIR. > > > When in the multiconfig and using the same TMPDIR: the two machines > can make > > > the same copy at the same time to the same destination, which will not > work. > > > > > > This issiue can be replicated with meta-ti layer: > > > > # use a multiconfig machine > > > > MACHINE = "am64xx-evm" > > > > # archive sources for target recipes in conf/local.conf > > > > INHERIT += "archiver" > > > > COPYLEFT_RECIPE_TYPES = "target" > > > > ARCHIVER_MODE[src] = "original" > > > > ARCHIVER_MODE[diff] = "1" > > > > # and run bitbake with: > > > > bitbake mc:k3r5:gcc-source-11.2.0 gcc-source-11.2.0 -c > do_unpack_and_patch -f > > > > > > > I went digging. If I enable this as above (using master) and run > > > > $ bitbake mc:k3r5:gcc-source-12.1.0 gcc-source-12.1.0 -c > do_unpack_and_patch -S none > > > > then: > > > > $ bitbake-diffsigs > tmp/stamps/work-shared/gcc-12.1.0-r0.do_unpack_and_patch.sigdata.* > > NOTE: Starting bitbake server... > > Task dependencies changed from: > > ['ABIEXTENSION', 'ARCHIVER_MODE[diff-exclude]', 'ARCHIVER_MODE[diff]', > 'ARCHIVER_MODE[src]', 'ARCHIVER_OUTDIR', 'ARCHIVER_TOPDIR', > 'ARCHIVER_WORKDIR', 'B', 'BASEURI', 'EXTENDPE', 'GNU_MIRROR', 'HOST_ARCH', > 'HOST_OS', 'HOST_SYS', 'HOST_VENDOR', 'INSANE_SKIP', 'LIBCEXTENSION', > 'PATCHTOOL', 'PE', 'PF', 'PN', 'PR', 'PV', 'RECIPE_SYSROOT_NATIVE', 'S', > 'SDE_DIR', 'SDE_FILE', 'SOURCEDIR', 'SRC_URI', 'SRC_URI[sha256sum]', > 'STAGING_DIR_NATIVE', 'T', 'TARGET_ARCH', 'TARGET_OS', 'TARGET_SYS', > 'TARGET_VENDOR', 'TUNE_ARCH', 'UNPACK_DIR', 'base_do_unpack', > 'base_do_unpack[cleandirs]', 'create_diff_gz', > 'create_source_date_epoch_stamp', 'do_patch', 'do_qa_patch', > 'do_qa_unpack', 'do_unpack', 'do_unpack[cleandirs]', 'is_work_shared', > 'patch_do_patch', 'src_patches', 'unpack_check_src_uri'] > > to: > > ['ABIEXTENSION', 'ABIEXTENSION_32', 'ABIEXTENSION_64', > 'ARCHIVER_MODE[diff-exclude]', 'ARCHIVER_MODE[diff]', 'ARCHIVER_MODE[src]', > 'ARCHIVER_OUTDIR', 'ARCHIVER_TOPDIR', 'ARCHIVER_WORKDIR', > 'ARMPKGSFX_ENDIAN_64', 'B', 'BASEURI', 'EXTENDPE', 'GNU_MIRROR', > 'HOST_ARCH', 'HOST_OS', 'HOST_SYS', 'HOST_VENDOR', 'INSANE_SKIP', > 'LIBCEXTENSION', 'PATCHTOOL', 'PE', 'PF', 'PN', 'PR', 'PV', > 'RECIPE_SYSROOT_NATIVE', 'S', 'SDE_DIR', 'SDE_FILE', 'SOURCEDIR', > 'SRC_URI', 'SRC_URI[sha256sum]', 'STAGING_DIR_NATIVE', 'T', 'TARGET_ARCH', > 'TARGET_OS', 'TARGET_SYS', 'TARGET_VENDOR', 'TUNE_ARCH', 'TUNE_ARCH_32', > 'TUNE_ARCH_64', 'UNPACK_DIR', 'base_do_unpack', > 'base_do_unpack[cleandirs]', 'create_diff_gz', > 'create_source_date_epoch_stamp', 'do_patch', 'do_qa_patch', > 'do_qa_unpack', 'do_unpack', 'do_unpack[cleandirs]', 'is_work_shared', > 'patch_do_patch', 'src_patches', 'unpack_check_src_uri'] > > basehash changed from > 005580a7a5a1ab1ff554d0a9e47c2ad4ba932f3fe185d8d349b06c83e5e58b6c to > d7b2e696005a578d5958ef4e73f1fc83e22612a3ae4294fb0540b0fadcb6c50f > > List of dependencies for variable ABIEXTENSION changed from 'set()' to > '{'ABIEXTENSION_64', 'ABIEXTENSION_32'}' > > List of dependencies for variable TUNE_ARCH changed from 'set()' to > '{'TUNE_ARCH_32', 'TUNE_ARCH_64'}' > > Dependency on variable ABIEXTENSION_32 was added > > Dependency on variable ABIEXTENSION_64 was added > > Dependency on variable ARMPKGSFX_ENDIAN_64 was added > > Dependency on variable TUNE_ARCH_32 was added > > Dependency on variable TUNE_ARCH_64 was added > > Variable ABIEXTENSION value changed: > > @@ -1 +1,2 @@ > > -eabi > > +${@bb.utils.contains('TUNE_FEATURES', 'aarch64', '${ABIEXTENSION_64}', > '${ABIEXTENSION_32}', d)} > > +TUNE_FEATURES{aarch64} = Set > > Variable TUNE_ARCH value changed: > > @@ -1,2 +1,2 @@ > > -${@bb.utils.contains('TUNE_FEATURES', 'bigendian', 'armeb', 'arm', d)} > > -TUNE_FEATURES{bigendian} = Unset > > +${@bb.utils.contains('TUNE_FEATURES', 'aarch64', '${TUNE_ARCH_64}', > '${TUNE_ARCH_32}', d)} > > +TUNE_FEATURES{aarch64} = Set > > > > this tells us the real issue. The task is changing due to toolchain > > issues. These stamps should be the same and aren't and this is the real > > issue that needs fixing. > Many thanks for this tip that helps me a lot. I will use this procedure to see what are the changes in some other issues in the rm_work class that I found as well related with multiconfig. But first is the gcc-sources shared recipe. > > To further follow up, there are two issues: > > a) the task uses ${B} which for gcc includes HOST_SYS and TARGET_SYS. > We can just set that in the gcc-source recipe to B = > "${WORKDIR}/build". > > b) the archiver uses: > ARCHIVER_OUTDIR = ${ARCHIVER_TOPDIR}/${TARGET_SYS}/${PF}/ > which includes TARGET_SYS and between the two different MACHINE values, > this changes from 'arm-poky-linux-gnueabi' to 'aarch64-poky-linux'. > This leads to the task running twice, once for each multiconfig. > > The first is easily solved as I mentioned. The question in the second > case is whether the task should run twice and where the output should > go since it is common to both builds. I do wonder if the archiver > shouldn't set: > > ARCHIVER_OUTPUT:pn-gcc-source = "${ARCHIVER_TOPDIR}/allarch/${PF}" > > Cheers, > > Richard > > > I am debugging this issue today with your a) and b) suggestions applied but is not enough. However I realized that this problem does not depend on archiver class and it is always reproducible in oe-core when multiconfig is enabled using the same temp dir, what leads me to conclude is that the shared gcc-source depends on the machine configuration. So in multiconfig it will run twice, once for each machine. Replicated in oe-core with the default configuration using the layer meta-ti: | rm tmp/stamps/work-shared/gcc-12.1.0-r0.do_build.*sigdata.* | bitbake mc:k3r5:gcc-source-12.1.0 gcc-source-11.2.0 -S none | bitbake-diffsigs tmp/stamps/work-shared/gcc-12.1.0-r0.do_build.*sigdata.* Task dependencies changed from: ['ABIEXTENSION', 'AR', 'AS', 'ASNEEDED', 'BUILD_AR', 'BUILD_AS', 'BUILD_AS_ARCH', 'BUILD_CC', 'BUILD_CCLD', 'BUILD_CC_ARCH', 'BUILD_CFLAGS', 'BUILD_CPP', 'BUILD_CPPFLAGS', 'BUILD_CXX', 'BUILD_CXXF LAGS', 'BUILD_FC', 'BUILD_LD', 'BUILD_LDFLAGS', 'BUILD_LD_ARCH', 'BUILD_NM', 'BUILD_OPTIMIZATION', 'BUILD_PREFIX', 'BUILD_RANLIB', 'BUILD_STRIP', 'CC', 'CCLD', 'CC_FOR_BUILD', 'CFLAGS', 'CFLAGS_FO R_BUILD', 'CONFIG_SITE', 'CPP', 'CPPFLAGS', 'CPPFLAGS_FOR_BUILD', 'CPP_FOR_BUILD', 'CXX', 'CXXFLAGS', 'CXXFLAGS_FOR_BUILD', 'CXX_FOR_BUILD', 'DEBUG_BUILD', 'DEBUG_FLAGS', 'DEBUG_OPTIMIZATION', 'DE BUG_PREFIX_MAP', 'EXTENDPE', 'FC', 'FULL_OPTIMIZATION', 'GCCPIE', 'GETTEXTDATADIRS', 'HOST_AS_ARCH', 'HOST_CC_ARCH', 'HOST_LD_ARCH', 'HOST_PREFIX', 'LC_ALL', 'LD', 'LDFLAGS', 'LDFLAGS_FOR_BUILD', 'LD_FOR_BUILD', 'LIBCEXTENSION', 'LINKER_HASH_STYLE', 'MAKE', 'NM', 'OBJCOPY', 'OBJDUMP', 'OPTLEVEL', 'PE', 'PERL_HASH_SEED', 'PKG_CONFIG_DIR', 'PKG_CONFIG_DISABLE_UNINSTALLED', 'PKG_CONFIG_LIBDIR ', 'PKG_CONFIG_PATH', 'PKG_CONFIG_SYSROOT_DIR', 'PKG_CONFIG_SYSTEM_INCLUDE_PATH', 'PKG_CONFIG_SYSTEM_LIBRARY_PATH', 'PN', 'PR', 'PSEUDO_DISABLED', 'PV', 'PYTHONHASHSEED', 'RANLIB', 'READELF', 'REC IPE_SYSROOT_NATIVE', 'SECURITY_CFLAGS', 'SECURITY_LDFLAGS', 'SECURITY_PIE_CFLAGS', 'SECURITY_STACK_PROTECTOR', 'SECURITY_STRINGFORMAT', 'SELECTED_OPTIMIZATION', 'SOURCE_DATE_EPOCH_FALLBACK', 'STAG ING_BASE_LIBDIR_NATIVE', 'STAGING_DATADIR', 'STAGING_DIR_NATIVE', 'STAGING_INCDIR_NATIVE', 'STAGING_LIBDIR_NATIVE', 'STRINGS', 'STRIP', 'TARGET_ARCH', 'TARGET_AS_ARCH', 'TARGET_CC_ARCH', 'TARGET_C FLAGS', 'TARGET_CPPFLAGS', 'TARGET_CXXFLAGS', 'TARGET_LDFLAGS', 'TARGET_LD_ARCH', 'TARGET_LINK_HASH_STYLE', 'TARGET_OS', 'TARGET_PREFIX', 'TARGET_SYS', 'TARGET_VENDOR', 'TOOLCHAIN_OPTIONS', 'TUNE_ ARCH', 'TUNE_ASARGS', 'TUNE_CCARGS', 'TUNE_LDARGS', 'TZ', 'base_bindir', 'base_libdir', 'base_libdir_native', 'base_prefix', 'base_sbindir', 'baselib', 'bindir', 'datadir', 'do_build[noexec]', 'do cdir', 'exec_prefix', 'includedir', 'includedir_native', 'infodir', 'lcl_maybe_fortify', 'libdir', 'libdir_native', 'libexecdir', 'localstatedir', 'lt_cv_sys_lib_dlsearch_path_spec', 'mandir', 'no narch_base_libdir', 'nonarch_libdir', 'oldincludedir', 'prefix', 'prefix_native', 'root_prefix', 'sbindir', 'servicedir', 'sharedstatedir', 'sysconfdir', 'systemd_system_unitdir', 'systemd_unitdir ', 'systemd_user_unitdir'] to: ['ABIEXTENSION', 'ABIEXTENSION_32', 'ABIEXTENSION_64', 'AR', 'ARMPKGSFX_ENDIAN_64', 'AS', 'ASNEEDED', 'BUILD_AR', 'BUILD_AS', 'BUILD_AS_ARCH', 'BUILD_CC', 'BUILD_CCLD', 'BUILD_CC_ARCH', 'BUILD_CFL AGS', 'BUILD_CPP', 'BUILD_CPPFLAGS', 'BUILD_CXX', 'BUILD_CXXFLAGS', 'BUILD_FC', 'BUILD_LD', 'BUILD_LDFLAGS', 'BUILD_LD_ARCH', 'BUILD_NM', 'BUILD_OPTIMIZATION', 'BUILD_PREFIX', 'BUILD_RANLIB', 'BUI LD_STRIP', 'CC', 'CCLD', 'CC_FOR_BUILD', 'CFLAGS', 'CFLAGS_FOR_BUILD', 'CONFIG_SITE', 'CPP', 'CPPFLAGS', 'CPPFLAGS_FOR_BUILD', 'CPP_FOR_BUILD', 'CXX', 'CXXFLAGS', 'CXXFLAGS_FOR_BUILD', 'CXX_FOR_BU ILD', 'DEBUG_BUILD', 'DEBUG_FLAGS', 'DEBUG_OPTIMIZATION', 'DEBUG_PREFIX_MAP', 'EXTENDPE', 'FC', 'FULL_OPTIMIZATION', 'GCCPIE', 'GETTEXTDATADIRS', 'HOST_AS_ARCH', 'HOST_CC_ARCH', 'HOST_LD_ARCH', 'H OST_PREFIX', 'LC_ALL', 'LD', 'LDFLAGS', 'LDFLAGS_FOR_BUILD', 'LD_FOR_BUILD', 'LIBCEXTENSION', 'LINKER_HASH_STYLE', 'MAKE', 'NM', 'OBJCOPY', 'OBJDUMP', 'OPTLEVEL', 'PE', 'PERL_HASH_SEED', 'PKG_CONF IG_DIR', 'PKG_CONFIG_DISABLE_UNINSTALLED', 'PKG_CONFIG_LIBDIR', 'PKG_CONFIG_PATH', 'PKG_CONFIG_SYSROOT_DIR', 'PKG_CONFIG_SYSTEM_INCLUDE_PATH', 'PKG_CONFIG_SYSTEM_LIBRARY_PATH', 'PN', 'PR', 'PSEUDO _DISABLED', 'PV', 'PYTHONHASHSEED', 'RANLIB', 'READELF', 'RECIPE_SYSROOT_NATIVE', 'SECURITY_CFLAGS', 'SECURITY_LDFLAGS', 'SECURITY_PIE_CFLAGS', 'SECURITY_STACK_PROTECTOR', 'SECURITY_STRINGFORMAT', 'SELECTED_OPTIMIZATION', 'SOURCE_DATE_EPOCH_FALLBACK', 'STAGING_BASE_LIBDIR_NATIVE', 'STAGING_DATADIR', 'STAGING_DIR_NATIVE', 'STAGING_INCDIR_NATIVE', 'STAGING_LIBDIR_NATIVE', 'STRINGS', 'STRIP', 'TARGET_ARCH', 'TARGET_AS_ARCH', 'TARGET_CC_ARCH', 'TARGET_CFLAGS', 'TARGET_CPPFLAGS', 'TARGET_CXXFLAGS', 'TARGET_LDFLAGS', 'TARGET_LD_ARCH', 'TARGET_LINK_HASH_STYLE', 'TARGET_OS', 'TARGET_PREFIX ', 'TARGET_SYS', 'TARGET_VENDOR', 'TOOLCHAIN_OPTIONS', 'TUNE_ARCH', 'TUNE_ARCH_32', 'TUNE_ARCH_64', 'TUNE_ASARGS', 'TUNE_CCARGS', 'TUNE_LDARGS', 'TZ', 'base_bindir', 'base_libdir', 'base_libdir_na tive', 'base_prefix', 'base_sbindir', 'baselib', 'bindir', 'datadir', 'do_build[noexec]', 'docdir', 'exec_prefix', 'includedir', 'includedir_native', 'infodir', 'lcl_maybe_fortify', 'libdir', 'lib dir_native', 'libexecdir', 'localstatedir', 'lt_cv_sys_lib_dlsearch_path_spec', 'mandir', 'nonarch_base_libdir', 'nonarch_libdir', 'oldincludedir', 'prefix', 'prefix_native', 'root_prefix', 'sbind ir', 'servicedir', 'sharedstatedir', 'sysconfdir', 'systemd_system_unitdir', 'systemd_unitdir', 'systemd_user_unitdir'] basehash changed from 8ebbebb9e1848d2d90d237100288175cedafc5fe9a5da92cdba1ffcb2556a7fc to 50842f3a6f7b6c1f7fe3e1bccc306e26ddb72ab8d771779007ce85be4db29fea List of dependencies for variable ABIEXTENSION changed from 'set()' to '{'ABIEXTENSION_64', 'ABIEXTENSION_32'}' List of dependencies for variable TUNE_ARCH changed from 'set()' to '{'TUNE_ARCH_32', 'TUNE_ARCH_64'}' Dependency on variable ABIEXTENSION_32 was added Dependency on variable ABIEXTENSION_64 was added Dependency on variable ARMPKGSFX_ENDIAN_64 was added Dependency on variable TUNE_ARCH_32 was added Dependency on variable TUNE_ARCH_64 was added Variable ABIEXTENSION value changed: @@ -1 +1,2 @@ -eabi +${@bb.utils.contains('TUNE_FEATURES', 'aarch64', '${ABIEXTENSION_64}', '${ABIEXTENSION_32}', d)} +TUNE_FEATURES{aarch64} = Set Variable TUNE_ARCH value changed: @@ -1,2 +1,2 @@ -${@bb.utils.contains('TUNE_FEATURES', 'bigendian', 'armeb', 'arm', d)} -TUNE_FEATURES{bigendian} = Unset +${@bb.utils.contains('TUNE_FEATURES', 'aarch64', '${TUNE_ARCH_64}', '${TUNE_ARCH_32}', d)} +TUNE_FEATURES{aarch64} = Set Variable TUNE_CCARGS value changed from ' -march=armv7-a -mthumb -mfpu=vfp -mfloat-abi=hard' to '' Jose
Jose Quaresma via lists.openembedded.org <quaresma.jose= gmail.com@lists.openembedded.org> escreveu no dia segunda, 13/06/2022 à(s) 23:12: > Hi Richard, > > Richard Purdie <richard.purdie@linuxfoundation.org> escreveu no dia > sexta, 10/06/2022 à(s) 11:59: > >> On Fri, 2022-06-10 at 11:43 +0100, Richard Purdie via >> lists.openembedded.org wrote: >> > On Thu, 2022-06-09 at 17:39 +0100, Jose Quaresma wrote: >> > > The archiver doesn't support using multiconfig sharing the same >> TMPDIR. >> > > When in the multiconfig and using the same TMPDIR: the two machines >> can make >> > > the same copy at the same time to the same destination, which will >> not work. >> > > >> > > This issiue can be replicated with meta-ti layer: >> > > > # use a multiconfig machine >> > > > MACHINE = "am64xx-evm" >> > > > # archive sources for target recipes in conf/local.conf >> > > > INHERIT += "archiver" >> > > > COPYLEFT_RECIPE_TYPES = "target" >> > > > ARCHIVER_MODE[src] = "original" >> > > > ARCHIVER_MODE[diff] = "1" >> > > > # and run bitbake with: >> > > > bitbake mc:k3r5:gcc-source-11.2.0 gcc-source-11.2.0 -c >> do_unpack_and_patch -f >> > > >> > >> > I went digging. If I enable this as above (using master) and run >> > >> > $ bitbake mc:k3r5:gcc-source-12.1.0 gcc-source-12.1.0 -c >> do_unpack_and_patch -S none >> > >> > then: >> > >> > $ bitbake-diffsigs >> tmp/stamps/work-shared/gcc-12.1.0-r0.do_unpack_and_patch.sigdata.* >> > NOTE: Starting bitbake server... >> > Task dependencies changed from: >> > ['ABIEXTENSION', 'ARCHIVER_MODE[diff-exclude]', 'ARCHIVER_MODE[diff]', >> 'ARCHIVER_MODE[src]', 'ARCHIVER_OUTDIR', 'ARCHIVER_TOPDIR', >> 'ARCHIVER_WORKDIR', 'B', 'BASEURI', 'EXTENDPE', 'GNU_MIRROR', 'HOST_ARCH', >> 'HOST_OS', 'HOST_SYS', 'HOST_VENDOR', 'INSANE_SKIP', 'LIBCEXTENSION', >> 'PATCHTOOL', 'PE', 'PF', 'PN', 'PR', 'PV', 'RECIPE_SYSROOT_NATIVE', 'S', >> 'SDE_DIR', 'SDE_FILE', 'SOURCEDIR', 'SRC_URI', 'SRC_URI[sha256sum]', >> 'STAGING_DIR_NATIVE', 'T', 'TARGET_ARCH', 'TARGET_OS', 'TARGET_SYS', >> 'TARGET_VENDOR', 'TUNE_ARCH', 'UNPACK_DIR', 'base_do_unpack', >> 'base_do_unpack[cleandirs]', 'create_diff_gz', >> 'create_source_date_epoch_stamp', 'do_patch', 'do_qa_patch', >> 'do_qa_unpack', 'do_unpack', 'do_unpack[cleandirs]', 'is_work_shared', >> 'patch_do_patch', 'src_patches', 'unpack_check_src_uri'] >> > to: >> > ['ABIEXTENSION', 'ABIEXTENSION_32', 'ABIEXTENSION_64', >> 'ARCHIVER_MODE[diff-exclude]', 'ARCHIVER_MODE[diff]', 'ARCHIVER_MODE[src]', >> 'ARCHIVER_OUTDIR', 'ARCHIVER_TOPDIR', 'ARCHIVER_WORKDIR', >> 'ARMPKGSFX_ENDIAN_64', 'B', 'BASEURI', 'EXTENDPE', 'GNU_MIRROR', >> 'HOST_ARCH', 'HOST_OS', 'HOST_SYS', 'HOST_VENDOR', 'INSANE_SKIP', >> 'LIBCEXTENSION', 'PATCHTOOL', 'PE', 'PF', 'PN', 'PR', 'PV', >> 'RECIPE_SYSROOT_NATIVE', 'S', 'SDE_DIR', 'SDE_FILE', 'SOURCEDIR', >> 'SRC_URI', 'SRC_URI[sha256sum]', 'STAGING_DIR_NATIVE', 'T', 'TARGET_ARCH', >> 'TARGET_OS', 'TARGET_SYS', 'TARGET_VENDOR', 'TUNE_ARCH', 'TUNE_ARCH_32', >> 'TUNE_ARCH_64', 'UNPACK_DIR', 'base_do_unpack', >> 'base_do_unpack[cleandirs]', 'create_diff_gz', >> 'create_source_date_epoch_stamp', 'do_patch', 'do_qa_patch', >> 'do_qa_unpack', 'do_unpack', 'do_unpack[cleandirs]', 'is_work_shared', >> 'patch_do_patch', 'src_patches', 'unpack_check_src_uri'] >> > basehash changed from >> 005580a7a5a1ab1ff554d0a9e47c2ad4ba932f3fe185d8d349b06c83e5e58b6c to >> d7b2e696005a578d5958ef4e73f1fc83e22612a3ae4294fb0540b0fadcb6c50f >> > List of dependencies for variable ABIEXTENSION changed from 'set()' to >> '{'ABIEXTENSION_64', 'ABIEXTENSION_32'}' >> > List of dependencies for variable TUNE_ARCH changed from 'set()' to >> '{'TUNE_ARCH_32', 'TUNE_ARCH_64'}' >> > Dependency on variable ABIEXTENSION_32 was added >> > Dependency on variable ABIEXTENSION_64 was added >> > Dependency on variable ARMPKGSFX_ENDIAN_64 was added >> > Dependency on variable TUNE_ARCH_32 was added >> > Dependency on variable TUNE_ARCH_64 was added >> > Variable ABIEXTENSION value changed: >> > @@ -1 +1,2 @@ >> > -eabi >> > +${@bb.utils.contains('TUNE_FEATURES', 'aarch64', '${ABIEXTENSION_64}', >> '${ABIEXTENSION_32}', d)} >> > +TUNE_FEATURES{aarch64} = Set >> > Variable TUNE_ARCH value changed: >> > @@ -1,2 +1,2 @@ >> > -${@bb.utils.contains('TUNE_FEATURES', 'bigendian', 'armeb', 'arm', d)} >> > -TUNE_FEATURES{bigendian} = Unset >> > +${@bb.utils.contains('TUNE_FEATURES', 'aarch64', '${TUNE_ARCH_64}', >> '${TUNE_ARCH_32}', d)} >> > +TUNE_FEATURES{aarch64} = Set >> > >> > this tells us the real issue. The task is changing due to toolchain >> > issues. These stamps should be the same and aren't and this is the real >> > issue that needs fixing. >> > > Many thanks for this tip that helps me a lot. > I will use this procedure to see what are the changes in some other issues > in the rm_work class that I found as well related with multiconfig. > But first is the gcc-sources shared recipe. > > >> >> To further follow up, there are two issues: >> >> a) the task uses ${B} which for gcc includes HOST_SYS and TARGET_SYS. >> We can just set that in the gcc-source recipe to B = >> "${WORKDIR}/build". >> >> b) the archiver uses: >> ARCHIVER_OUTDIR = ${ARCHIVER_TOPDIR}/${TARGET_SYS}/${PF}/ >> which includes TARGET_SYS and between the two different MACHINE values, >> this changes from 'arm-poky-linux-gnueabi' to 'aarch64-poky-linux'. >> This leads to the task running twice, once for each multiconfig. >> >> The first is easily solved as I mentioned. The question in the second >> case is whether the task should run twice and where the output should >> go since it is common to both builds. I do wonder if the archiver >> shouldn't set: >> >> ARCHIVER_OUTPUT:pn-gcc-source = "${ARCHIVER_TOPDIR}/allarch/${PF}" >> >> Cheers, >> >> Richard >> >> >> > I am debugging this issue today with your a) and b) suggestions > applied but is not enough. > > However I realized that this problem does not depend on archiver class and > it is always reproducible > in oe-core when multiconfig is enabled using the same temp dir, what leads > me to conclude is that the > shared gcc-source depends on the machine configuration. So in multiconfig > it will run twice, once for each machine. > I was wrong in my last analizes, the only task of gcc-sources that is used by other recipes is the do_preconfigure. The stamps produced for that are the same so the problem is really in the archiver. Maybe can be related with this task, do_preconfigure, that is called in other places on the archiver but not on the do_unpack_and_patch. TASK=do_preconfigure rm tmp-lmp/stamps/work-shared/gcc-11.2.0-r0.$TASK.sigdata.* ; \ bitbake mc:k3r5:gcc-source-11.2.0 gcc-source-11.2.0 -c $TASK -S none && \ cat locked-sigs.inc && \ bitbake-diffsigs tmp-lmp/stamps/work-shared/gcc-11.2.0-r0.$TASK.sigdata.* Produces the same stamp in the two machines. SIGGEN_LOCKEDSIGS_t-aarch64 = "\ gcc-source-11.2.0:do_fetch:99156079ba9189e31a0990e3b1232613a449ad217034bfcb0e4abb2b1847ea77 \ gcc-source-11.2.0:do_patch:0ab19cac7bffb236412ec3f0bb8270cc077721162c01a0ff783f119459270881 \ gcc-source-11.2.0:do_preconfigure:f4d7cbe00a06eb28b417235981f600251f5eff4f3afd791a7d63ef34a0d3484f \ gcc-source-11.2.0:do_unpack:cd754360e80c5b00ab1fc47aad830f5a3dd396b6711d60c4a17e39844bb5a079 \ " SIGGEN_LOCKEDSIGS_t-armv7at2hf-vfp = "\ gcc-source-11.2.0:do_fetch:99156079ba9189e31a0990e3b1232613a449ad217034bfcb0e4abb2b1847ea77 \ gcc-source-11.2.0:do_patch:0ab19cac7bffb236412ec3f0bb8270cc077721162c01a0ff783f119459270881 \ gcc-source-11.2.0:do_preconfigure:f4d7cbe00a06eb28b417235981f600251f5eff4f3afd791a7d63ef34a0d3484f \ gcc-source-11.2.0:do_unpack:cd754360e80c5b00ab1fc47aad830f5a3dd396b6711d60c4a17e39844bb5a079 \ Jose > > Replicated in oe-core with the default configuration using the layer > meta-ti: > > | rm tmp/stamps/work-shared/gcc-12.1.0-r0.do_build.*sigdata.* > | bitbake mc:k3r5:gcc-source-12.1.0 gcc-source-11.2.0 -S none > | bitbake-diffsigs tmp/stamps/work-shared/gcc-12.1 > .0-r0.do_build.*sigdata.* > > Task dependencies changed from: > ['ABIEXTENSION', 'AR', 'AS', 'ASNEEDED', 'BUILD_AR', 'BUILD_AS', > 'BUILD_AS_ARCH', 'BUILD_CC', 'BUILD_CCLD', 'BUILD_CC_ARCH', 'BUILD_CFLAGS', > 'BUILD_CPP', 'BUILD_CPPFLAGS', 'BUILD_CXX', 'BUILD_CXXF > LAGS', 'BUILD_FC', 'BUILD_LD', 'BUILD_LDFLAGS', 'BUILD_LD_ARCH', > 'BUILD_NM', 'BUILD_OPTIMIZATION', 'BUILD_PREFIX', 'BUILD_RANLIB', > 'BUILD_STRIP', 'CC', 'CCLD', 'CC_FOR_BUILD', 'CFLAGS', 'CFLAGS_FO > R_BUILD', 'CONFIG_SITE', 'CPP', 'CPPFLAGS', 'CPPFLAGS_FOR_BUILD', > 'CPP_FOR_BUILD', 'CXX', 'CXXFLAGS', 'CXXFLAGS_FOR_BUILD', 'CXX_FOR_BUILD', > 'DEBUG_BUILD', 'DEBUG_FLAGS', 'DEBUG_OPTIMIZATION', 'DE > BUG_PREFIX_MAP', 'EXTENDPE', 'FC', 'FULL_OPTIMIZATION', 'GCCPIE', > 'GETTEXTDATADIRS', 'HOST_AS_ARCH', 'HOST_CC_ARCH', 'HOST_LD_ARCH', > 'HOST_PREFIX', 'LC_ALL', 'LD', 'LDFLAGS', 'LDFLAGS_FOR_BUILD', > 'LD_FOR_BUILD', 'LIBCEXTENSION', 'LINKER_HASH_STYLE', 'MAKE', 'NM', > 'OBJCOPY', 'OBJDUMP', 'OPTLEVEL', 'PE', 'PERL_HASH_SEED', 'PKG_CONFIG_DIR', > 'PKG_CONFIG_DISABLE_UNINSTALLED', 'PKG_CONFIG_LIBDIR > ', 'PKG_CONFIG_PATH', 'PKG_CONFIG_SYSROOT_DIR', > 'PKG_CONFIG_SYSTEM_INCLUDE_PATH', 'PKG_CONFIG_SYSTEM_LIBRARY_PATH', 'PN', > 'PR', 'PSEUDO_DISABLED', 'PV', 'PYTHONHASHSEED', 'RANLIB', 'READELF', 'REC > IPE_SYSROOT_NATIVE', 'SECURITY_CFLAGS', 'SECURITY_LDFLAGS', > 'SECURITY_PIE_CFLAGS', 'SECURITY_STACK_PROTECTOR', 'SECURITY_STRINGFORMAT', > 'SELECTED_OPTIMIZATION', 'SOURCE_DATE_EPOCH_FALLBACK', 'STAG > ING_BASE_LIBDIR_NATIVE', 'STAGING_DATADIR', 'STAGING_DIR_NATIVE', > 'STAGING_INCDIR_NATIVE', 'STAGING_LIBDIR_NATIVE', 'STRINGS', 'STRIP', > 'TARGET_ARCH', 'TARGET_AS_ARCH', 'TARGET_CC_ARCH', 'TARGET_C > FLAGS', 'TARGET_CPPFLAGS', 'TARGET_CXXFLAGS', 'TARGET_LDFLAGS', > 'TARGET_LD_ARCH', 'TARGET_LINK_HASH_STYLE', 'TARGET_OS', 'TARGET_PREFIX', > 'TARGET_SYS', 'TARGET_VENDOR', 'TOOLCHAIN_OPTIONS', 'TUNE_ > ARCH', 'TUNE_ASARGS', 'TUNE_CCARGS', 'TUNE_LDARGS', 'TZ', 'base_bindir', > 'base_libdir', 'base_libdir_native', 'base_prefix', 'base_sbindir', > 'baselib', 'bindir', 'datadir', 'do_build[noexec]', 'do > cdir', 'exec_prefix', 'includedir', 'includedir_native', 'infodir', > 'lcl_maybe_fortify', 'libdir', 'libdir_native', 'libexecdir', > 'localstatedir', 'lt_cv_sys_lib_dlsearch_path_spec', 'mandir', 'no > narch_base_libdir', 'nonarch_libdir', 'oldincludedir', 'prefix', > 'prefix_native', 'root_prefix', 'sbindir', 'servicedir', 'sharedstatedir', > 'sysconfdir', 'systemd_system_unitdir', 'systemd_unitdir > ', 'systemd_user_unitdir'] > to: > ['ABIEXTENSION', 'ABIEXTENSION_32', 'ABIEXTENSION_64', 'AR', > 'ARMPKGSFX_ENDIAN_64', 'AS', 'ASNEEDED', 'BUILD_AR', 'BUILD_AS', > 'BUILD_AS_ARCH', 'BUILD_CC', 'BUILD_CCLD', 'BUILD_CC_ARCH', 'BUILD_CFL > AGS', 'BUILD_CPP', 'BUILD_CPPFLAGS', 'BUILD_CXX', 'BUILD_CXXFLAGS', > 'BUILD_FC', 'BUILD_LD', 'BUILD_LDFLAGS', 'BUILD_LD_ARCH', 'BUILD_NM', > 'BUILD_OPTIMIZATION', 'BUILD_PREFIX', 'BUILD_RANLIB', 'BUI > LD_STRIP', 'CC', 'CCLD', 'CC_FOR_BUILD', 'CFLAGS', 'CFLAGS_FOR_BUILD', > 'CONFIG_SITE', 'CPP', 'CPPFLAGS', 'CPPFLAGS_FOR_BUILD', 'CPP_FOR_BUILD', > 'CXX', 'CXXFLAGS', 'CXXFLAGS_FOR_BUILD', 'CXX_FOR_BU > ILD', 'DEBUG_BUILD', 'DEBUG_FLAGS', 'DEBUG_OPTIMIZATION', > 'DEBUG_PREFIX_MAP', 'EXTENDPE', 'FC', 'FULL_OPTIMIZATION', 'GCCPIE', > 'GETTEXTDATADIRS', 'HOST_AS_ARCH', 'HOST_CC_ARCH', 'HOST_LD_ARCH', 'H > OST_PREFIX', 'LC_ALL', 'LD', 'LDFLAGS', 'LDFLAGS_FOR_BUILD', > 'LD_FOR_BUILD', 'LIBCEXTENSION', 'LINKER_HASH_STYLE', 'MAKE', 'NM', > 'OBJCOPY', 'OBJDUMP', 'OPTLEVEL', 'PE', 'PERL_HASH_SEED', 'PKG_CONF > IG_DIR', 'PKG_CONFIG_DISABLE_UNINSTALLED', 'PKG_CONFIG_LIBDIR', > 'PKG_CONFIG_PATH', 'PKG_CONFIG_SYSROOT_DIR', > 'PKG_CONFIG_SYSTEM_INCLUDE_PATH', 'PKG_CONFIG_SYSTEM_LIBRARY_PATH', 'PN', > 'PR', 'PSEUDO > _DISABLED', 'PV', 'PYTHONHASHSEED', 'RANLIB', 'READELF', > 'RECIPE_SYSROOT_NATIVE', 'SECURITY_CFLAGS', 'SECURITY_LDFLAGS', > 'SECURITY_PIE_CFLAGS', 'SECURITY_STACK_PROTECTOR', 'SECURITY_STRINGFORMAT', > 'SELECTED_OPTIMIZATION', 'SOURCE_DATE_EPOCH_FALLBACK', > 'STAGING_BASE_LIBDIR_NATIVE', 'STAGING_DATADIR', 'STAGING_DIR_NATIVE', > 'STAGING_INCDIR_NATIVE', 'STAGING_LIBDIR_NATIVE', 'STRINGS', 'STRIP', > 'TARGET_ARCH', 'TARGET_AS_ARCH', 'TARGET_CC_ARCH', 'TARGET_CFLAGS', > 'TARGET_CPPFLAGS', 'TARGET_CXXFLAGS', 'TARGET_LDFLAGS', 'TARGET_LD_ARCH', > 'TARGET_LINK_HASH_STYLE', 'TARGET_OS', 'TARGET_PREFIX > ', 'TARGET_SYS', 'TARGET_VENDOR', 'TOOLCHAIN_OPTIONS', 'TUNE_ARCH', > 'TUNE_ARCH_32', 'TUNE_ARCH_64', 'TUNE_ASARGS', 'TUNE_CCARGS', > 'TUNE_LDARGS', 'TZ', 'base_bindir', 'base_libdir', 'base_libdir_na > tive', 'base_prefix', 'base_sbindir', 'baselib', 'bindir', 'datadir', > 'do_build[noexec]', 'docdir', 'exec_prefix', 'includedir', > 'includedir_native', 'infodir', 'lcl_maybe_fortify', 'libdir', 'lib > dir_native', 'libexecdir', 'localstatedir', > 'lt_cv_sys_lib_dlsearch_path_spec', 'mandir', 'nonarch_base_libdir', > 'nonarch_libdir', 'oldincludedir', 'prefix', 'prefix_native', > 'root_prefix', 'sbind > ir', 'servicedir', 'sharedstatedir', 'sysconfdir', > 'systemd_system_unitdir', 'systemd_unitdir', 'systemd_user_unitdir'] > basehash changed from > 8ebbebb9e1848d2d90d237100288175cedafc5fe9a5da92cdba1ffcb2556a7fc to > 50842f3a6f7b6c1f7fe3e1bccc306e26ddb72ab8d771779007ce85be4db29fea > List of dependencies for variable ABIEXTENSION changed from 'set()' to '{'ABIEXTENSION_64', > 'ABIEXTENSION_32'}' > List of dependencies for variable TUNE_ARCH changed from 'set()' to '{'TUNE_ARCH_32', > 'TUNE_ARCH_64'}' > Dependency on variable ABIEXTENSION_32 was added > Dependency on variable ABIEXTENSION_64 was added > Dependency on variable ARMPKGSFX_ENDIAN_64 was added > Dependency on variable TUNE_ARCH_32 was added > Dependency on variable TUNE_ARCH_64 was added > Variable ABIEXTENSION value changed: > @@ -1 +1,2 @@ > -eabi > +${@bb.utils.contains('TUNE_FEATURES', 'aarch64', '${ABIEXTENSION_64}', > '${ABIEXTENSION_32}', d)} > +TUNE_FEATURES{aarch64} = Set > Variable TUNE_ARCH value changed: > @@ -1,2 +1,2 @@ > -${@bb.utils.contains('TUNE_FEATURES', 'bigendian', 'armeb', 'arm', d)} > -TUNE_FEATURES{bigendian} = Unset > +${@bb.utils.contains('TUNE_FEATURES', 'aarch64', '${TUNE_ARCH_64}', > '${TUNE_ARCH_32}', d)} > +TUNE_FEATURES{aarch64} = Set > Variable TUNE_CCARGS value changed from ' -march=armv7-a -mthumb > -mfpu=vfp -mfloat-abi=hard' to '' > > Jose > > -- > Best regards, > > José Quaresma > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#166871): > https://lists.openembedded.org/g/openembedded-core/message/166871 > Mute This Topic: https://lists.openembedded.org/mt/91650171/5052612 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ > quaresma.jose@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- > >
Hi, v4 fixes the issue Jose Quaresma via lists.openembedded.org <quaresma.jose= gmail.com@lists.openembedded.org> escreveu no dia terça, 14/06/2022 à(s) 10:43: > > > Jose Quaresma via lists.openembedded.org <quaresma.jose= > gmail.com@lists.openembedded.org> escreveu no dia segunda, 13/06/2022 > à(s) 23:12: > >> Hi Richard, >> >> Richard Purdie <richard.purdie@linuxfoundation.org> escreveu no dia >> sexta, 10/06/2022 à(s) 11:59: >> >>> On Fri, 2022-06-10 at 11:43 +0100, Richard Purdie via >>> lists.openembedded.org wrote: >>> > On Thu, 2022-06-09 at 17:39 +0100, Jose Quaresma wrote: >>> > > The archiver doesn't support using multiconfig sharing the same >>> TMPDIR. >>> > > When in the multiconfig and using the same TMPDIR: the two machines >>> can make >>> > > the same copy at the same time to the same destination, which will >>> not work. >>> > > >>> > > This issiue can be replicated with meta-ti layer: >>> > > > # use a multiconfig machine >>> > > > MACHINE = "am64xx-evm" >>> > > > # archive sources for target recipes in conf/local.conf >>> > > > INHERIT += "archiver" >>> > > > COPYLEFT_RECIPE_TYPES = "target" >>> > > > ARCHIVER_MODE[src] = "original" >>> > > > ARCHIVER_MODE[diff] = "1" >>> > > > # and run bitbake with: >>> > > > bitbake mc:k3r5:gcc-source-11.2.0 gcc-source-11.2.0 -c >>> do_unpack_and_patch -f >>> > > >>> > >>> > I went digging. If I enable this as above (using master) and run >>> > >>> > $ bitbake mc:k3r5:gcc-source-12.1.0 gcc-source-12.1.0 -c >>> do_unpack_and_patch -S none >>> > >>> > then: >>> > >>> > $ bitbake-diffsigs >>> tmp/stamps/work-shared/gcc-12.1.0-r0.do_unpack_and_patch.sigdata.* >>> > NOTE: Starting bitbake server... >>> > Task dependencies changed from: >>> > ['ABIEXTENSION', 'ARCHIVER_MODE[diff-exclude]', 'ARCHIVER_MODE[diff]', >>> 'ARCHIVER_MODE[src]', 'ARCHIVER_OUTDIR', 'ARCHIVER_TOPDIR', >>> 'ARCHIVER_WORKDIR', 'B', 'BASEURI', 'EXTENDPE', 'GNU_MIRROR', 'HOST_ARCH', >>> 'HOST_OS', 'HOST_SYS', 'HOST_VENDOR', 'INSANE_SKIP', 'LIBCEXTENSION', >>> 'PATCHTOOL', 'PE', 'PF', 'PN', 'PR', 'PV', 'RECIPE_SYSROOT_NATIVE', 'S', >>> 'SDE_DIR', 'SDE_FILE', 'SOURCEDIR', 'SRC_URI', 'SRC_URI[sha256sum]', >>> 'STAGING_DIR_NATIVE', 'T', 'TARGET_ARCH', 'TARGET_OS', 'TARGET_SYS', >>> 'TARGET_VENDOR', 'TUNE_ARCH', 'UNPACK_DIR', 'base_do_unpack', >>> 'base_do_unpack[cleandirs]', 'create_diff_gz', >>> 'create_source_date_epoch_stamp', 'do_patch', 'do_qa_patch', >>> 'do_qa_unpack', 'do_unpack', 'do_unpack[cleandirs]', 'is_work_shared', >>> 'patch_do_patch', 'src_patches', 'unpack_check_src_uri'] >>> > to: >>> > ['ABIEXTENSION', 'ABIEXTENSION_32', 'ABIEXTENSION_64', >>> 'ARCHIVER_MODE[diff-exclude]', 'ARCHIVER_MODE[diff]', 'ARCHIVER_MODE[src]', >>> 'ARCHIVER_OUTDIR', 'ARCHIVER_TOPDIR', 'ARCHIVER_WORKDIR', >>> 'ARMPKGSFX_ENDIAN_64', 'B', 'BASEURI', 'EXTENDPE', 'GNU_MIRROR', >>> 'HOST_ARCH', 'HOST_OS', 'HOST_SYS', 'HOST_VENDOR', 'INSANE_SKIP', >>> 'LIBCEXTENSION', 'PATCHTOOL', 'PE', 'PF', 'PN', 'PR', 'PV', >>> 'RECIPE_SYSROOT_NATIVE', 'S', 'SDE_DIR', 'SDE_FILE', 'SOURCEDIR', >>> 'SRC_URI', 'SRC_URI[sha256sum]', 'STAGING_DIR_NATIVE', 'T', 'TARGET_ARCH', >>> 'TARGET_OS', 'TARGET_SYS', 'TARGET_VENDOR', 'TUNE_ARCH', 'TUNE_ARCH_32', >>> 'TUNE_ARCH_64', 'UNPACK_DIR', 'base_do_unpack', >>> 'base_do_unpack[cleandirs]', 'create_diff_gz', >>> 'create_source_date_epoch_stamp', 'do_patch', 'do_qa_patch', >>> 'do_qa_unpack', 'do_unpack', 'do_unpack[cleandirs]', 'is_work_shared', >>> 'patch_do_patch', 'src_patches', 'unpack_check_src_uri'] >>> > basehash changed from >>> 005580a7a5a1ab1ff554d0a9e47c2ad4ba932f3fe185d8d349b06c83e5e58b6c to >>> d7b2e696005a578d5958ef4e73f1fc83e22612a3ae4294fb0540b0fadcb6c50f >>> > List of dependencies for variable ABIEXTENSION changed from 'set()' to >>> '{'ABIEXTENSION_64', 'ABIEXTENSION_32'}' >>> > List of dependencies for variable TUNE_ARCH changed from 'set()' to >>> '{'TUNE_ARCH_32', 'TUNE_ARCH_64'}' >>> > Dependency on variable ABIEXTENSION_32 was added >>> > Dependency on variable ABIEXTENSION_64 was added >>> > Dependency on variable ARMPKGSFX_ENDIAN_64 was added >>> > Dependency on variable TUNE_ARCH_32 was added >>> > Dependency on variable TUNE_ARCH_64 was added >>> > Variable ABIEXTENSION value changed: >>> > @@ -1 +1,2 @@ >>> > -eabi >>> > +${@bb.utils.contains('TUNE_FEATURES', 'aarch64', >>> '${ABIEXTENSION_64}', '${ABIEXTENSION_32}', d)} >>> > +TUNE_FEATURES{aarch64} = Set >>> > Variable TUNE_ARCH value changed: >>> > @@ -1,2 +1,2 @@ >>> > -${@bb.utils.contains('TUNE_FEATURES', 'bigendian', 'armeb', 'arm', d)} >>> > -TUNE_FEATURES{bigendian} = Unset >>> > +${@bb.utils.contains('TUNE_FEATURES', 'aarch64', '${TUNE_ARCH_64}', >>> '${TUNE_ARCH_32}', d)} >>> > +TUNE_FEATURES{aarch64} = Set >>> > >>> > this tells us the real issue. The task is changing due to toolchain >>> > issues. These stamps should be the same and aren't and this is the real >>> > issue that needs fixing. >>> >> >> Many thanks for this tip that helps me a lot. >> I will use this procedure to see what are the changes in some other >> issues in the rm_work class that I found as well related with multiconfig. >> But first is the gcc-sources shared recipe. >> >> >>> >>> To further follow up, there are two issues: >>> >>> a) the task uses ${B} which for gcc includes HOST_SYS and TARGET_SYS. >>> We can just set that in the gcc-source recipe to B = >>> "${WORKDIR}/build". >>> >>> b) the archiver uses: >>> ARCHIVER_OUTDIR = ${ARCHIVER_TOPDIR}/${TARGET_SYS}/${PF}/ >>> which includes TARGET_SYS and between the two different MACHINE values, >>> this changes from 'arm-poky-linux-gnueabi' to 'aarch64-poky-linux'. >>> This leads to the task running twice, once for each multiconfig. >>> >>> The first is easily solved as I mentioned. The question in the second >>> case is whether the task should run twice and where the output should >>> go since it is common to both builds. I do wonder if the archiver >>> shouldn't set: >>> >>> ARCHIVER_OUTPUT:pn-gcc-source = "${ARCHIVER_TOPDIR}/allarch/${PF}" >>> >>> Cheers, >>> >>> Richard >>> >>> >>> >> With an adaptation of a) and b) the issue is fixed for the do_unpack_and_patch task, sent in v4. The last broken stamp is now the task do_deploy_archives that does nothing but saves the artifacts in the sstate cache, more debug is needed but is mostly fixed. TASK=do_deploy_archives rm tmp-lmp/stamps/work-shared/gcc-11.2.0-r0.$TASK.sigdata.* ; \ bitbake mc:k3r5:gcc-source-11.2.0 gcc-source-11.2.0 -c $TASK -S none && \ bitbake-diffsigs tmp-lmp/stamps/work-shared/gcc-11.2.0-r0.$TASK.sigdata.* cat locked-sigs.inc SIGGEN_LOCKEDSIGS_t-aarch64 = "\ gcc-source-11.2.0:do_ar_original:9eda3eec4d43d6ecca5633f52c27b337f55395d0c1a2003cd44b1edaf0b5a626 \ gcc-source-11.2.0:do_deploy_archives:170e3258863184af73b4c3168e52abd989edaa139befefeba0151844474f6bc6 \ gcc-source-11.2.0:do_fetch:99156079ba9189e31a0990e3b1232613a449ad217034bfcb0e4abb2b1847ea77 \ gcc-source-11.2.0:do_patch:0ab19cac7bffb236412ec3f0bb8270cc077721162c01a0ff783f119459270881 \ gcc-source-11.2.0:do_preconfigure:f4d7cbe00a06eb28b417235981f600251f5eff4f3afd791a7d63ef34a0d3484f \ gcc-source-11.2.0:do_unpack:cd754360e80c5b00ab1fc47aad830f5a3dd396b6711d60c4a17e39844bb5a079 \ gcc-source-11.2.0:do_unpack_and_patch:21166df787f00293ebab9ad2468ea378fdb6da32723dd2e805473184c11d9ae5 \ " SIGGEN_LOCKEDSIGS_t-armv7at2hf-vfp = "\ gcc-source-11.2.0:do_ar_original:9eda3eec4d43d6ecca5633f52c27b337f55395d0c1a2003cd44b1edaf0b5a626 \ gcc-source-11.2.0:do_deploy_archives:87b9d1e697453afbcca435a5a6729293ed2be22960627c3a35580115dbe8ed22 \ gcc-source-11.2.0:do_fetch:99156079ba9189e31a0990e3b1232613a449ad217034bfcb0e4abb2b1847ea77 \ gcc-source-11.2.0:do_patch:0ab19cac7bffb236412ec3f0bb8270cc077721162c01a0ff783f119459270881 \ gcc-source-11.2.0:do_preconfigure:f4d7cbe00a06eb28b417235981f600251f5eff4f3afd791a7d63ef34a0d3484f \ gcc-source-11.2.0:do_unpack:cd754360e80c5b00ab1fc47aad830f5a3dd396b6711d60c4a17e39844bb5a079 \ gcc-source-11.2.0:do_unpack_and_patch:21166df787f00293ebab9ad2468ea378fdb6da32723dd2e805473184c11d9ae5 \ " Jose I am debugging this issue today with your a) and b) suggestions applied but >> is not enough. >> >> However I realized that this problem does not depend on archiver class >> and it is always reproducible >> in oe-core when multiconfig is enabled using the same temp dir, what >> leads me to conclude is that the >> shared gcc-source depends on the machine configuration. So in multiconfig >> it will run twice, once for each machine. >> > > I was wrong in my last analizes, the only task of gcc-sources that is used > by other recipes is the do_preconfigure. > The stamps produced for that are the same so the problem is really in the > archiver. Maybe can be related with > this task, do_preconfigure, that is called in other places on the archiver > but not on the do_unpack_and_patch. > > > TASK=do_preconfigure > rm tmp-lmp/stamps/work-shared/gcc-11.2.0-r0.$TASK.sigdata.* ; \ > bitbake mc:k3r5:gcc-source-11.2.0 gcc-source-11.2.0 -c $TASK -S none && \ > cat locked-sigs.inc && \ > bitbake-diffsigs tmp-lmp/stamps/work-shared/gcc-11.2.0-r0.$TASK.sigdata.* > > > Produces the same stamp in the two machines. > > > SIGGEN_LOCKEDSIGS_t-aarch64 = "\ > gcc-source-11.2.0:do_fetch:99156079ba9189e31a0990e3b1232613a449ad217034bfcb0e4abb2b1847ea77 > \ > gcc-source-11.2.0:do_patch:0ab19cac7bffb236412ec3f0bb8270cc077721162c01a0ff783f119459270881 > \ > gcc-source-11.2.0:do_preconfigure:f4d7cbe00a06eb28b417235981f600251f5eff4f3afd791a7d63ef34a0d3484f > \ > gcc-source-11.2.0:do_unpack:cd754360e80c5b00ab1fc47aad830f5a3dd396b6711d60c4a17e39844bb5a079 > \ > " > SIGGEN_LOCKEDSIGS_t-armv7at2hf-vfp = "\ > gcc-source-11.2.0:do_fetch:99156079ba9189e31a0990e3b1232613a449ad217034bfcb0e4abb2b1847ea77 > \ > gcc-source-11.2.0:do_patch:0ab19cac7bffb236412ec3f0bb8270cc077721162c01a0ff783f119459270881 > \ > gcc-source-11.2.0:do_preconfigure:f4d7cbe00a06eb28b417235981f600251f5eff4f3afd791a7d63ef34a0d3484f > \ > gcc-source-11.2.0:do_unpack:cd754360e80c5b00ab1fc47aad830f5a3dd396b6711d60c4a17e39844bb5a079 > \ > > Jose > > >> >> Replicated in oe-core with the default configuration using the layer >> meta-ti: >> >> | rm tmp/stamps/work-shared/gcc-12.1.0-r0.do_build.*sigdata.* >> | bitbake mc:k3r5:gcc-source-12.1.0 gcc-source-11.2.0 -S none >> | bitbake-diffsigs tmp/stamps/work-shared/gcc-12.1 >> .0-r0.do_build.*sigdata.* >> >> Task dependencies changed from: >> ['ABIEXTENSION', 'AR', 'AS', 'ASNEEDED', 'BUILD_AR', 'BUILD_AS', >> 'BUILD_AS_ARCH', 'BUILD_CC', 'BUILD_CCLD', 'BUILD_CC_ARCH', 'BUILD_CFLAGS', >> 'BUILD_CPP', 'BUILD_CPPFLAGS', 'BUILD_CXX', 'BUILD_CXXF >> LAGS', 'BUILD_FC', 'BUILD_LD', 'BUILD_LDFLAGS', 'BUILD_LD_ARCH', >> 'BUILD_NM', 'BUILD_OPTIMIZATION', 'BUILD_PREFIX', 'BUILD_RANLIB', >> 'BUILD_STRIP', 'CC', 'CCLD', 'CC_FOR_BUILD', 'CFLAGS', 'CFLAGS_FO >> R_BUILD', 'CONFIG_SITE', 'CPP', 'CPPFLAGS', 'CPPFLAGS_FOR_BUILD', >> 'CPP_FOR_BUILD', 'CXX', 'CXXFLAGS', 'CXXFLAGS_FOR_BUILD', 'CXX_FOR_BUILD', >> 'DEBUG_BUILD', 'DEBUG_FLAGS', 'DEBUG_OPTIMIZATION', 'DE >> BUG_PREFIX_MAP', 'EXTENDPE', 'FC', 'FULL_OPTIMIZATION', 'GCCPIE', >> 'GETTEXTDATADIRS', 'HOST_AS_ARCH', 'HOST_CC_ARCH', 'HOST_LD_ARCH', >> 'HOST_PREFIX', 'LC_ALL', 'LD', 'LDFLAGS', 'LDFLAGS_FOR_BUILD', >> 'LD_FOR_BUILD', 'LIBCEXTENSION', 'LINKER_HASH_STYLE', 'MAKE', 'NM', >> 'OBJCOPY', 'OBJDUMP', 'OPTLEVEL', 'PE', 'PERL_HASH_SEED', 'PKG_CONFIG_DIR', >> 'PKG_CONFIG_DISABLE_UNINSTALLED', 'PKG_CONFIG_LIBDIR >> ', 'PKG_CONFIG_PATH', 'PKG_CONFIG_SYSROOT_DIR', >> 'PKG_CONFIG_SYSTEM_INCLUDE_PATH', 'PKG_CONFIG_SYSTEM_LIBRARY_PATH', 'PN', >> 'PR', 'PSEUDO_DISABLED', 'PV', 'PYTHONHASHSEED', 'RANLIB', 'READELF', 'REC >> IPE_SYSROOT_NATIVE', 'SECURITY_CFLAGS', 'SECURITY_LDFLAGS', >> 'SECURITY_PIE_CFLAGS', 'SECURITY_STACK_PROTECTOR', 'SECURITY_STRINGFORMAT', >> 'SELECTED_OPTIMIZATION', 'SOURCE_DATE_EPOCH_FALLBACK', 'STAG >> ING_BASE_LIBDIR_NATIVE', 'STAGING_DATADIR', 'STAGING_DIR_NATIVE', >> 'STAGING_INCDIR_NATIVE', 'STAGING_LIBDIR_NATIVE', 'STRINGS', 'STRIP', >> 'TARGET_ARCH', 'TARGET_AS_ARCH', 'TARGET_CC_ARCH', 'TARGET_C >> FLAGS', 'TARGET_CPPFLAGS', 'TARGET_CXXFLAGS', 'TARGET_LDFLAGS', >> 'TARGET_LD_ARCH', 'TARGET_LINK_HASH_STYLE', 'TARGET_OS', 'TARGET_PREFIX', >> 'TARGET_SYS', 'TARGET_VENDOR', 'TOOLCHAIN_OPTIONS', 'TUNE_ >> ARCH', 'TUNE_ASARGS', 'TUNE_CCARGS', 'TUNE_LDARGS', 'TZ', 'base_bindir', >> 'base_libdir', 'base_libdir_native', 'base_prefix', 'base_sbindir', >> 'baselib', 'bindir', 'datadir', 'do_build[noexec]', 'do >> cdir', 'exec_prefix', 'includedir', 'includedir_native', 'infodir', >> 'lcl_maybe_fortify', 'libdir', 'libdir_native', 'libexecdir', >> 'localstatedir', 'lt_cv_sys_lib_dlsearch_path_spec', 'mandir', 'no >> narch_base_libdir', 'nonarch_libdir', 'oldincludedir', 'prefix', >> 'prefix_native', 'root_prefix', 'sbindir', 'servicedir', 'sharedstatedir', >> 'sysconfdir', 'systemd_system_unitdir', 'systemd_unitdir >> ', 'systemd_user_unitdir'] >> to: >> ['ABIEXTENSION', 'ABIEXTENSION_32', 'ABIEXTENSION_64', 'AR', >> 'ARMPKGSFX_ENDIAN_64', 'AS', 'ASNEEDED', 'BUILD_AR', 'BUILD_AS', >> 'BUILD_AS_ARCH', 'BUILD_CC', 'BUILD_CCLD', 'BUILD_CC_ARCH', 'BUILD_CFL >> AGS', 'BUILD_CPP', 'BUILD_CPPFLAGS', 'BUILD_CXX', 'BUILD_CXXFLAGS', >> 'BUILD_FC', 'BUILD_LD', 'BUILD_LDFLAGS', 'BUILD_LD_ARCH', 'BUILD_NM', >> 'BUILD_OPTIMIZATION', 'BUILD_PREFIX', 'BUILD_RANLIB', 'BUI >> LD_STRIP', 'CC', 'CCLD', 'CC_FOR_BUILD', 'CFLAGS', 'CFLAGS_FOR_BUILD', >> 'CONFIG_SITE', 'CPP', 'CPPFLAGS', 'CPPFLAGS_FOR_BUILD', 'CPP_FOR_BUILD', >> 'CXX', 'CXXFLAGS', 'CXXFLAGS_FOR_BUILD', 'CXX_FOR_BU >> ILD', 'DEBUG_BUILD', 'DEBUG_FLAGS', 'DEBUG_OPTIMIZATION', >> 'DEBUG_PREFIX_MAP', 'EXTENDPE', 'FC', 'FULL_OPTIMIZATION', 'GCCPIE', >> 'GETTEXTDATADIRS', 'HOST_AS_ARCH', 'HOST_CC_ARCH', 'HOST_LD_ARCH', 'H >> OST_PREFIX', 'LC_ALL', 'LD', 'LDFLAGS', 'LDFLAGS_FOR_BUILD', >> 'LD_FOR_BUILD', 'LIBCEXTENSION', 'LINKER_HASH_STYLE', 'MAKE', 'NM', >> 'OBJCOPY', 'OBJDUMP', 'OPTLEVEL', 'PE', 'PERL_HASH_SEED', 'PKG_CONF >> IG_DIR', 'PKG_CONFIG_DISABLE_UNINSTALLED', 'PKG_CONFIG_LIBDIR', >> 'PKG_CONFIG_PATH', 'PKG_CONFIG_SYSROOT_DIR', >> 'PKG_CONFIG_SYSTEM_INCLUDE_PATH', 'PKG_CONFIG_SYSTEM_LIBRARY_PATH', 'PN', >> 'PR', 'PSEUDO >> _DISABLED', 'PV', 'PYTHONHASHSEED', 'RANLIB', 'READELF', >> 'RECIPE_SYSROOT_NATIVE', 'SECURITY_CFLAGS', 'SECURITY_LDFLAGS', >> 'SECURITY_PIE_CFLAGS', 'SECURITY_STACK_PROTECTOR', 'SECURITY_STRINGFORMAT', >> 'SELECTED_OPTIMIZATION', 'SOURCE_DATE_EPOCH_FALLBACK', >> 'STAGING_BASE_LIBDIR_NATIVE', 'STAGING_DATADIR', 'STAGING_DIR_NATIVE', >> 'STAGING_INCDIR_NATIVE', 'STAGING_LIBDIR_NATIVE', 'STRINGS', 'STRIP', >> 'TARGET_ARCH', 'TARGET_AS_ARCH', 'TARGET_CC_ARCH', 'TARGET_CFLAGS', >> 'TARGET_CPPFLAGS', 'TARGET_CXXFLAGS', 'TARGET_LDFLAGS', 'TARGET_LD_ARCH', >> 'TARGET_LINK_HASH_STYLE', 'TARGET_OS', 'TARGET_PREFIX >> ', 'TARGET_SYS', 'TARGET_VENDOR', 'TOOLCHAIN_OPTIONS', 'TUNE_ARCH', >> 'TUNE_ARCH_32', 'TUNE_ARCH_64', 'TUNE_ASARGS', 'TUNE_CCARGS', >> 'TUNE_LDARGS', 'TZ', 'base_bindir', 'base_libdir', 'base_libdir_na >> tive', 'base_prefix', 'base_sbindir', 'baselib', 'bindir', 'datadir', >> 'do_build[noexec]', 'docdir', 'exec_prefix', 'includedir', >> 'includedir_native', 'infodir', 'lcl_maybe_fortify', 'libdir', 'lib >> dir_native', 'libexecdir', 'localstatedir', >> 'lt_cv_sys_lib_dlsearch_path_spec', 'mandir', 'nonarch_base_libdir', >> 'nonarch_libdir', 'oldincludedir', 'prefix', 'prefix_native', >> 'root_prefix', 'sbind >> ir', 'servicedir', 'sharedstatedir', 'sysconfdir', >> 'systemd_system_unitdir', 'systemd_unitdir', 'systemd_user_unitdir'] >> basehash changed from >> 8ebbebb9e1848d2d90d237100288175cedafc5fe9a5da92cdba1ffcb2556a7fc to >> 50842f3a6f7b6c1f7fe3e1bccc306e26ddb72ab8d771779007ce85be4db29fea >> List of dependencies for variable ABIEXTENSION changed from 'set()' to '{'ABIEXTENSION_64', >> 'ABIEXTENSION_32'}' >> List of dependencies for variable TUNE_ARCH changed from 'set()' to '{'TUNE_ARCH_32', >> 'TUNE_ARCH_64'}' >> Dependency on variable ABIEXTENSION_32 was added >> Dependency on variable ABIEXTENSION_64 was added >> Dependency on variable ARMPKGSFX_ENDIAN_64 was added >> Dependency on variable TUNE_ARCH_32 was added >> Dependency on variable TUNE_ARCH_64 was added >> Variable ABIEXTENSION value changed: >> @@ -1 +1,2 @@ >> -eabi >> +${@bb.utils.contains('TUNE_FEATURES', 'aarch64', '${ABIEXTENSION_64}', >> '${ABIEXTENSION_32}', d)} >> +TUNE_FEATURES{aarch64} = Set >> Variable TUNE_ARCH value changed: >> @@ -1,2 +1,2 @@ >> -${@bb.utils.contains('TUNE_FEATURES', 'bigendian', 'armeb', 'arm', d)} >> -TUNE_FEATURES{bigendian} = Unset >> +${@bb.utils.contains('TUNE_FEATURES', 'aarch64', '${TUNE_ARCH_64}', >> '${TUNE_ARCH_32}', d)} >> +TUNE_FEATURES{aarch64} = Set >> Variable TUNE_CCARGS value changed from ' -march=armv7-a -mthumb >> -mfpu=vfp -mfloat-abi=hard' to '' >> >> Jose >> >> -- >> Best regards, >> >> José Quaresma >> >> >> >> > > -- > Best regards, > > José Quaresma > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#166881): > https://lists.openembedded.org/g/openembedded-core/message/166881 > Mute This Topic: https://lists.openembedded.org/mt/91650171/5052612 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ > quaresma.jose@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- > >
diff --git a/meta/classes/archiver.bbclass b/meta/classes/archiver.bbclass index c19c770d11..b1d7aa0247 100644 --- a/meta/classes/archiver.bbclass +++ b/meta/classes/archiver.bbclass @@ -485,7 +485,8 @@ python do_unpack_and_patch() { # Save the original source for creating the patches if d.getVarFlag('ARCHIVER_MODE', 'diff') == '1': src = d.getVar('S').rstrip('/') - src_orig = '%s.orig' % src + # append machine name to avoid races in multiconfig when using the same TMPDIR + src_orig = '%s.orig.%s' % (src, d.getVar('MACHINE')) oe.path.copytree(src, src_orig) if bb.data.inherits_class('dos2unix', d):
The archiver doesn't support using multiconfig sharing the same TMPDIR. When in the multiconfig and using the same TMPDIR: the two machines can make the same copy at the same time to the same destination, which will not work. This issiue can be replicated with meta-ti layer: |# use a multiconfig machine |MACHINE = "am64xx-evm" |# archive sources for target recipes in conf/local.conf |INHERIT += "archiver" |COPYLEFT_RECIPE_TYPES = "target" |ARCHIVER_MODE[src] = "original" |ARCHIVER_MODE[diff] = "1" |# and run bitbake with: |bitbake mc:k3r5:gcc-source-11.2.0 gcc-source-11.2.0 -c do_unpack_and_patch -f Fixes the following: |ERROR: mc:k3r5:gcc-source-11.2.0-11.2.0-r0 do_unpack_and_patch: Error executing a python function in exec_func_python() autogenerated: | |The stack trace of python calls that resulted in this exception/failure was: |File: 'exec_func_python() autogenerated', lineno: 2, function: <module> | 0001: | *** 0002:do_unpack_and_patch(d) | 0003: |File: '/build/conf/../../layers/openembedded-core/meta/classes/archiver.bbclass', lineno: 489, function: do_unpack_and_patch | 0485: # Save the original source for creating the patches | 0486: if d.getVarFlag('ARCHIVER_MODE', 'diff') == '1': | 0487: src = d.getVar('S').rstrip('/') | 0488: src_orig = '%s.orig' % src | *** 0489: oe.path.copytree(src, src_orig) | 0490: | 0491: if bb.data.inherits_class('dos2unix', d): | 0492: bb.build.exec_func('do_convert_crlf_to_lf', d) | 0493: |File: '/build/conf/../../layers/openembedded-core/meta/lib/oe/path.py', lineno: 94, function: copytree | 0090: # This way we also preserve hardlinks between files in the tree. | 0091: | 0092: bb.utils.mkdirhier(dst) | 0093: cmd = "tar --xattrs --xattrs-include='*' -cf - -S -C %s -p . | tar --xattrs --xattrs-include='*' -xf - -C %s" % (src, dst) | *** 0094: subprocess.check_output(cmd, shell=True, stderr=subprocess.STDOUT) | 0095: | 0096:def copyhardlinktree(src, dst): | 0097: """Make a tree of hard links when possible, otherwise copy.""" | 0098: bb.utils.mkdirhier(dst) |File: '/usr/lib/python3.10/subprocess.py', lineno: 420, function: check_output | 0416: else: | 0417: empty = b'' | 0418: kwargs['input'] = empty | 0419: | *** 0420: return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, | 0421: **kwargs).stdout | 0422: | 0423: | 0424:class CompletedProcess(object): |File: '/usr/lib/python3.10/subprocess.py', lineno: 524, function: run | 0520: # We don't call process.wait() as .__exit__ does that for us. | 0521: raise | 0522: retcode = process.poll() | 0523: if check and retcode: | *** 0524: raise CalledProcessError(retcode, process.args, | 0525: output=stdout, stderr=stderr) | 0526: return CompletedProcess(process.args, retcode, stdout, stderr) | 0527: | 0528: |Exception: subprocess.CalledProcessError: Command 'tar --xattrs --xattrs-include='*' -cf - -S -C /build/tmp-lmp/work-shared/gcc-11.2.0-r0/gcc-11.2.0 -p . | tar --xattrs --xattrs-include='*' |-xf - -C /build/tmp-lmp/work-shared/gcc-11.2.0-r0/gcc-11.2.0.orig' returned non-zero exit status 2. | |Subprocess output: |tar: ./gcc/config/moxie/predicates.md: file changed as we read it |tar: ./patches: Cannot stat: No such file or directory |tar: Exiting with failure status due to previous errors | |ERROR: Logfile of failure stored in: /build/tmp-lmp/work-shared/gcc-11.2.0-r0/temp/log.do_unpack_and_patch.291906 |ERROR: Task (mc:k3r5:/build/conf/../../layers/openembedded-core/meta/recipes-devtools/gcc/gcc-source_11.2.bb:do_unpack_and_patch) failed with exit code '1' Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io> --- v2: replace "bb.error" with "bb.fatal" v3: instead of failing make it more resilient meta/classes/archiver.bbclass | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)