| Message ID | 20221113084429.329094-1-vince@underview.tech |
|---|---|
| State | New |
| Headers | show |
| Series | llvm: fix native llvm-config RUNPATH | expand |
I'm not sure I understand the desired outcome properly. Does this patch native llvm-config according to the target libdir, or according to the build host libdir? The first one is not correct (native items must be reusable between targets), the second one is not solving the problem? Alex On Sun, 13 Nov 2022 at 09:44, Vincent Davis Jr <vince@underview.tech> wrote: > > Problem occurs when native llvm-config binary is > required by another recipe. RUNPATH is hardcoded > to $ORIGIN/../lib:$ORIGIN/../../lib which depending > upon architecture 'lib' directory name may vary > (i.e 'lib64'). > > Commit fixes issue by updating rpath on binary to include > architecture dependent directory name. > > Commit also fixes issue with llvm-config --libdir command > by setting LLVM_LIBDIR_SUFFIX. Command will return proper > architecture dependent directory name. > > Signed-off-by: Vincent Davis Jr <vince@underview.tech> > --- > meta/recipes-devtools/llvm/llvm_git.bb | 26 ++++++++++++++++++++++++++ > 1 file changed, 26 insertions(+) > > diff --git a/meta/recipes-devtools/llvm/llvm_git.bb b/meta/recipes-devtools/llvm/llvm_git.bb > index 1f47e5d0fb..29eb119be5 100644 > --- a/meta/recipes-devtools/llvm/llvm_git.bb > +++ b/meta/recipes-devtools/llvm/llvm_git.bb > @@ -56,6 +56,14 @@ def get_llvm_arch(bb, d, arch_var): > def get_llvm_host_arch(bb, d): > return get_llvm_arch(bb, d, 'HOST_ARCH') > > +def get_llvm_libdir_suffix(bb, d): > + import re > + arch = d.getVar('HOST_ARCH') > + if re.match(r'(x86.64)$', arch): > + return 64 > + else: > + return "" > + > PACKAGECONFIG ??= "" > # if optviewer OFF, force the modules to be not found or the ones on the host would be found > PACKAGECONFIG[optviewer] = ",-DPY_PYGMENTS_FOUND=OFF -DPY_PYGMENTS_LEXERS_C_CPP_FOUND=OFF -DPY_YAML_FOUND=OFF,python3-pygments python3-pyyaml,python3-pygments python3-pyyaml" > @@ -64,6 +72,7 @@ PACKAGECONFIG[optviewer] = ",-DPY_PYGMENTS_FOUND=OFF -DPY_PYGMENTS_LEXERS_C_CPP_ > # Default to build all OE-Core supported target arches (user overridable). > # > LLVM_TARGETS ?= "AMDGPU;${@get_llvm_host_arch(bb, d)}" > +LLVM_TARGET_LIBDIR_SUFFIX ?= "${@get_llvm_libdir_suffix(bb, d)}" > > ARM_INSTRUCTION_SET:armv5 = "arm" > ARM_INSTRUCTION_SET:armv4t = "arm" > @@ -83,6 +92,10 @@ EXTRA_OECMAKE += "-DLLVM_ENABLE_ASSERTIONS=OFF \ > -DCMAKE_BUILD_TYPE=Release \ > -G Ninja" > > +EXTRA_OECMAKE:append:class-native = "\ > + -DLLVM_LIBDIR_SUFFIX=${LLVM_TARGET_LIBDIR_SUFFIX} \ > + " > + > EXTRA_OECMAKE:append:class-target = "\ > -DCMAKE_CROSSCOMPILING:BOOL=ON \ > -DLLVM_TABLEGEN=${STAGING_BINDIR_NATIVE}/llvm-tblgen${PV} \ > @@ -129,6 +142,7 @@ do_install:class-native() { > } > > SYSROOT_PREPROCESS_FUNCS:append:class-target = " llvm_sysroot_preprocess" > +SYSROOT_PREPROCESS_FUNCS:append:class-native = " llvm_sysroot_preprocess_native" > > llvm_sysroot_preprocess() { > install -d ${SYSROOT_DESTDIR}${bindir_crossscripts}/ > @@ -136,6 +150,18 @@ llvm_sysroot_preprocess() { > ln -sf llvm-config ${SYSROOT_DESTDIR}${bindir_crossscripts}/llvm-config${PV} > } > > +# All of LLVM's shipping binaries have an rpath pointing at ``$ORIGIN/../lib``. > +# Recipe llvm requires llvm-native to compile. When compiling llvm build libraries > +# are located in $ORIGIN/../lib directory. When the llvm-config binary required > +# in another recipe depending upon HOST_ARCH variable libraries may be located > +# in a different directory. Account for both situations here. > +llvm_sysroot_preprocess_native() { > + chrpath --replace '$ORIGIN'/../lib${LLVM_TARGET_LIBDIR_SUFFIX}:'$ORIGIN'/../../lib${LLVM_TARGET_LIBDIR_SUFFIX}:'$ORIGIN'/../lib:'$ORIGIN'/../../lib \ > + ${SYSROOT_DESTDIR}${bindir}/llvm-tblgen${PV} > + chrpath --replace '$ORIGIN'/../lib${LLVM_TARGET_LIBDIR_SUFFIX}:'$ORIGIN'/../../lib${LLVM_TARGET_LIBDIR_SUFFIX}:'$ORIGIN'/../lib:'$ORIGIN'/../../lib \ > + ${SYSROOT_DESTDIR}${bindir}/llvm-config${PV} > +} > + > PACKAGES =+ "${PN}-bugpointpasses ${PN}-llvmhello ${PN}-libllvm ${PN}-liboptremarks ${PN}-liblto" > > RRECOMMENDS:${PN}-dev += "${PN}-bugpointpasses ${PN}-llvmhello ${PN}-liboptremarks" > -- > 2.34.1 > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#173187): https://lists.openembedded.org/g/openembedded-core/message/173187 > Mute This Topic: https://lists.openembedded.org/mt/94995332/1686489 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- >
Hey,
Desired outcome is for mesa meson configure to succeeded by updating
RUNPATH on llvm-config native binary required by mesa when gallium-llvm
included. When running bitbake -c devshell mesa. Then chrpath -l on llvm-config
Should return on TARGET_ARCH x86_64
$ORIGIN/../lib64:$ORIGIN/../../lib64
not
$ORIGIN/../lib:$ORIGIN/../../lib
SNAPSHOT OF FAILURE
*************************************************************************************************************************
File "/../../../mesa/<version>/recipe-sysroot-native/usr/lib/python3.11/re/__init__.py", line 223, in finditer
return _compile(pattern, flags).finditer(string)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: expected string or bytes-like object, got 'NoneType'
*************************************************************************************************************************
Reason for above failure, at least for me, when TARGET_ARCH = x86_64 you run into
bellow when running llvm-config --help or variant commands directly.
llvm-config: error while loading shared libraries: libtinfo.so.5: cannot open shared object file: No such file or directory
Because of that when mesa configure executes llvm-config --version the string is not returned. Thus, leading
to above error.
If you run chrpath -l on llvm-config you see
$ORIGIN/../lib:$ORIGIN/../../lib:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
When target architecture is x86_64 those folders don't exists in recipe-sysroot of a recipe that requires llvm-native.
The llvm_sysroot_preprocess_native function in current MR addresses that problem by
making RUNPATH target architecture dependent.
As to why RUNPATH includes both the original RUNPATH and the update target architecture
dependent directory name the llvm recipe requires the llvm-native llvm-config binary to
compile, but the build libs are stored in lib directory not lib64. Tried everything I could
think of change that not much can be done about that. When llvm-native gets included
in another recipe however libs are located at $ORIGIN/../lib64:$ORIGIN/../../lib64, at least
for TARGET ARCH x86_64.
As for the second problem llvm-config --libdir returns
/../../../../../../mesa/2_22.2.2-r0/recipe-sysroot/usr/lib
instead of
/../../../../../../mesa/2_22.2.2-r0/recipe-sysroot/usr/lib64
Which causes mesa configure to fail as ../../mesa/2_22.2.2-r0/recipe-sysroot/usr/lib doesn't exist.
According to https://github.com/llvm/llvm-project/blob/a11cd0d94ed3cabf0998a0289aead05da94c86eb/llvm/CMakeLists.txt#L399
and https://github.com/llvm/llvm-project/blob/a11cd0d94ed3cabf0998a0289aead05da94c86eb/llvm/CMakeLists.txt#L389
LLVM_LIBDIR_SUFFIX variable is used to set command. MR addresses that by setting the LLVM_LIBDIR_SUFFIX
variable. The reason I decided to use HOST_ARCH instead of TARGET_ARCH is because bitbake sets
HOST_ARCH = "${TARGET_ARCH}" in bitbake.conf, but I'll admit there's ignorance on my part in terms
of what I think that variable is set to and what class-native it used for.
:)
Wait. When you invoke native llvm-config it must be able to locate *native* libraries that it is linked with. And so rpath must be set to where they are, which is $ORIGIN/../lib. Why does it need to be changed to something defined differently for some target? Then you seem to say that 'llvm-config --libdir' returns something incorrect, but is that an actual problem? Where in the mesa build is it used, and how can the issue be observed? We regularly build mesa with llvm, and it does not fail. Alex On Sun, 13 Nov 2022 at 20:17, Vincent Davis Jr <vince@underview.tech> wrote: > > Hey, > > Desired outcome is for mesa meson configure to succeeded by updating > RUNPATH on llvm-config native binary required by mesa when gallium-llvm > included. When running bitbake -c devshell mesa. Then chrpath -l on llvm-config > > Should return on TARGET_ARCH x86_64 > $ORIGIN/../lib64:$ORIGIN/../../lib64 > not > $ORIGIN/../lib:$ORIGIN/../../lib > > SNAPSHOT OF FAILURE > ************************************************************************************************************************* > File "/../../../mesa/<version>/recipe-sysroot-native/usr/lib/python3.11/re/__init__.py", line 223, in finditer > return _compile(pattern, flags).finditer(string) > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > TypeError: expected string or bytes-like object, got 'NoneType' > ************************************************************************************************************************* > > Reason for above failure, at least for me, when TARGET_ARCH = x86_64 you run into > bellow when running llvm-config --help or variant commands directly. > > llvm-config: error while loading shared libraries: libtinfo.so.5: cannot open shared object file: No such file or directory > > Because of that when mesa configure executes llvm-config --version the string is not returned. Thus, leading > to above error. > > If you run chrpath -l on llvm-config you see > > $ORIGIN/../lib:$ORIGIN/../../lib::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: > > When target architecture is x86_64 those folders don't exists in recipe-sysroot of a recipe that requires llvm-native. > > The llvm_sysroot_preprocess_native function in current MR addresses that problem by > making RUNPATH target architecture dependent. > > As to why RUNPATH includes both the original RUNPATH and the update target architecture > dependent directory name the llvm recipe requires the llvm-native llvm-config binary to > compile, but the build libs are stored in lib directory not lib64. Tried everything I could > think of change that not much can be done about that. When llvm-native gets included > in another recipe however libs are located at $ORIGIN/../lib64:$ORIGIN/../../lib64, at least > for TARGET ARCH x86_64. > > As for the second problem llvm-config --libdir returns > > /../../../../../../mesa/2_22.2.2-r0/recipe-sysroot/usr/lib > > instead of > > /../../../../../../mesa/2_22.2.2-r0/recipe-sysroot/usr/lib64 > > Which causes mesa configure to fail as ../../mesa/2_22.2.2-r0/recipe-sysroot/usr/lib doesn't exist. > > According to https://github.com/llvm/llvm-project/blob/a11cd0d94ed3cabf0998a0289aead05da94c86eb/llvm/CMakeLists.txt#L399 > and https://github.com/llvm/llvm-project/blob/a11cd0d94ed3cabf0998a0289aead05da94c86eb/llvm/CMakeLists.txt#L389 > > LLVM_LIBDIR_SUFFIX variable is used to set command. MR addresses that by setting the LLVM_LIBDIR_SUFFIX > variable. The reason I decided to use HOST_ARCH instead of TARGET_ARCH is because bitbake sets > HOST_ARCH = "${TARGET_ARCH}" in bitbake.conf, but I'll admit there's ignorance on my part in terms > of what I think that variable is set to and what class-native it used for. > > :) > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#173209): https://lists.openembedded.org/g/openembedded-core/message/173209 > Mute This Topic: https://lists.openembedded.org/mt/94995332/1686489 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- >
To be specific:
alex@Zen2:/srv/storage/alex/yocto/build-64-alt$ ldd
tmp/work/core2-64-poky-linux/mesa/2_22.2.2-r0/recipe-sysroot-native/usr/bin/llvm-config
linux-vdso.so.1 (0x00007fffe9964000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0
(0x00007f415335d000)
libtinfo.so.5 =>
/srv/storage/alex/yocto/build-64-alt/tmp/work/core2-64-poky-linux/mesa/2_22.2.2-r0/recipe-sysroot-native/usr/bin/../lib/libtinfo.so.5
(0x00007f415332e000)
libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f4153161000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f415301d000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f4153001000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f4152e2c000)
/srv/storage/alex/yocto/build-64-alt/tmp/sysroots-uninative/x86_64-linux/lib/ld-linux-x86-64.so.2
=> /lib64/ld-linux-x86-64.so.2 (0x00007f41533c3000)
See how it's able to link with libtinfo in sysroot-native? That's
precisely because of rpath telling it to.
Alex
On Sun, 13 Nov 2022 at 22:54, Alexander Kanavin <alex.kanavin@gmail.com> wrote:
>
> Wait. When you invoke native llvm-config it must be able to locate
> *native* libraries that it is linked with. And so rpath must be set to
> where they are, which is $ORIGIN/../lib. Why does it need to be
> changed to something defined differently for some target?
>
> Then you seem to say that 'llvm-config --libdir' returns something
> incorrect, but is that an actual problem? Where in the mesa build is
> it used, and how can the issue be observed? We regularly build mesa
> with llvm, and it does not fail.
>
> Alex
>
>
> On Sun, 13 Nov 2022 at 20:17, Vincent Davis Jr <vince@underview.tech> wrote:
> >
> > Hey,
> >
> > Desired outcome is for mesa meson configure to succeeded by updating
> > RUNPATH on llvm-config native binary required by mesa when gallium-llvm
> > included. When running bitbake -c devshell mesa. Then chrpath -l on llvm-config
> >
> > Should return on TARGET_ARCH x86_64
> > $ORIGIN/../lib64:$ORIGIN/../../lib64
> > not
> > $ORIGIN/../lib:$ORIGIN/../../lib
> >
> > SNAPSHOT OF FAILURE
> > *************************************************************************************************************************
> > File "/../../../mesa/<version>/recipe-sysroot-native/usr/lib/python3.11/re/__init__.py", line 223, in finditer
> > return _compile(pattern, flags).finditer(string)
> > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> > TypeError: expected string or bytes-like object, got 'NoneType'
> > *************************************************************************************************************************
> >
> > Reason for above failure, at least for me, when TARGET_ARCH = x86_64 you run into
> > bellow when running llvm-config --help or variant commands directly.
> >
> > llvm-config: error while loading shared libraries: libtinfo.so.5: cannot open shared object file: No such file or directory
> >
> > Because of that when mesa configure executes llvm-config --version the string is not returned. Thus, leading
> > to above error.
> >
> > If you run chrpath -l on llvm-config you see
> >
> > $ORIGIN/../lib:$ORIGIN/../../lib:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
> >
> > When target architecture is x86_64 those folders don't exists in recipe-sysroot of a recipe that requires llvm-native.
> >
> > The llvm_sysroot_preprocess_native function in current MR addresses that problem by
> > making RUNPATH target architecture dependent.
> >
> > As to why RUNPATH includes both the original RUNPATH and the update target architecture
> > dependent directory name the llvm recipe requires the llvm-native llvm-config binary to
> > compile, but the build libs are stored in lib directory not lib64. Tried everything I could
> > think of change that not much can be done about that. When llvm-native gets included
> > in another recipe however libs are located at $ORIGIN/../lib64:$ORIGIN/../../lib64, at least
> > for TARGET ARCH x86_64.
> >
> > As for the second problem llvm-config --libdir returns
> >
> > /../../../../../../mesa/2_22.2.2-r0/recipe-sysroot/usr/lib
> >
> > instead of
> >
> > /../../../../../../mesa/2_22.2.2-r0/recipe-sysroot/usr/lib64
> >
> > Which causes mesa configure to fail as ../../mesa/2_22.2.2-r0/recipe-sysroot/usr/lib doesn't exist.
> >
> > According to https://github.com/llvm/llvm-project/blob/a11cd0d94ed3cabf0998a0289aead05da94c86eb/llvm/CMakeLists.txt#L399
> > and https://github.com/llvm/llvm-project/blob/a11cd0d94ed3cabf0998a0289aead05da94c86eb/llvm/CMakeLists.txt#L389
> >
> > LLVM_LIBDIR_SUFFIX variable is used to set command. MR addresses that by setting the LLVM_LIBDIR_SUFFIX
> > variable. The reason I decided to use HOST_ARCH instead of TARGET_ARCH is because bitbake sets
> > HOST_ARCH = "${TARGET_ARCH}" in bitbake.conf, but I'll admit there's ignorance on my part in terms
> > of what I think that variable is set to and what class-native it used for.
> >
> > :)
> >
> > -=-=-=-=-=-=-=-=-=-=-=-
> > Links: You receive all messages sent to this group.
> > View/Reply Online (#173209): https://lists.openembedded.org/g/openembedded-core/message/173209
> > Mute This Topic: https://lists.openembedded.org/mt/94995332/1686489
> > Group Owner: openembedded-core+owner@lists.openembedded.org
> > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com]
> > -=-=-=-=-=-=-=-=-=-=-=-
> >
O I see usr/lib directory does exists for me in recipe-sysroot-native too, but not in recipe-sysroot. Think mesa configure may be utilizing the version of llvm-config in recipe-sysroot and not recipe-sysroot-native. mesa/2_22.2.2-r0/mesa-22.2.2# ldd ../recipe-sysroot-native/usr/bin/llvm-config linux-vdso.so.1 (0x00007ffe7b5cb000) libpseudo.so => /../../../../../../../tmp/sysroots-components/x86_64/pseudo-native/usr/lib/pseudo/lib64/libpseudo.so (0x00007f4d138ed000) libtinfo.so.5 => /../../../../../../../tmp/work/amd64-northstar-linux/mesa/2_22.2.2-r0/mesa-22.2.2/../recipe-sysroot-native/usr/bin/../lib/libtinfo.so.5 (0x00007f4d138bd000) libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f4d1367d000) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f4d13594000) libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f4d13574000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f4d1334c000) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f4d13347000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f4d13342000) /../../../../../../../tmp/sysroots-uninative/x86_64-linux/lib/ld-linux-x86-64.so.2 => /lib64/ld-linux-x86-64.so.2 (0x00007f4d13994000) mesa/2_22.2.2-r0/mesa-22.2.2# ldd ../recipe-sysroot/usr/bin/llvm-config linux-vdso.so.1 (0x00007ffc5bc95000) libpseudo.so => /../../../../../../../tmp/sysroots-components/x86_64/pseudo-native/usr/lib/pseudo/lib64/libpseudo.so (0x00007f1c6c02b000) libtinfo.so.5 => /../../../../../../../tmp/work/amd64-northstar-linux/mesa/2_22.2.2-r0/mesa-22.2.2/../recipe-sysroot/usr/bin/../../lib64/libtinfo.so.5 (0x00007f1c6bffb000) libstdc++.so.6 => /../../../../../../../tmp/work/amd64-northstar-linux/mesa/2_22.2.2-r0/mesa-22.2.2/../recipe-sysroot/usr/bin/../lib64/libstdc++.so.6 (0x00007f1c6bdd7000) libm.so.6 => /../../../../../../../tmp/work/amd64-northstar-linux/mesa/2_22.2.2-r0/mesa-22.2.2/../recipe-sysroot/usr/bin/../../lib64/libm.so.6 (0x00007f1c6bcfb000) libgcc_s.so.1 => /../../../../../../../tmp/work/amd64-northstar-linux/mesa/2_22.2.2-r0/mesa-22.2.2/../recipe-sysroot/usr/bin/../../lib64/libgcc_s.so.1 (0x00007f1c6bcdb000) libc.so.6 => /../../../../../../../tmp/work/amd64-northstar-linux/mesa/2_22.2.2-r0/mesa-22.2.2/../recipe-sysroot/usr/bin/../../lib64/libc.so.6 (0x00007f1c6bafd000) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f1c6bae2000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f1c6badb000) /../../../../../../../tmp/sysroots-uninative/x86_64-linux/lib/ld-linux-x86-64.so.2 => /lib64/ld-linux-x86-64.so.2 (0x00007f1c6c0d2000) ../mesa/2_22.2.2-r0/mesa-22.2.2# which llvm-config /../../../../../../../tmp/work/amd64-northstar-linux/mesa/2_22.2.2-r0/recipe-sysroot/usr/bin/crossscripts/llvm-config ../mesa/2_22.2.2-r0/mesa-22.2.2# llvm-config --version 15.0.1 ../mesa/2_22.2.2-r0/mesa-22.2.2# llvm-config --libdir /../../../../../../../tmp/work/amd64-northstar-linux/mesa/2_22.2.2-r0/recipe-sysroot/usr/lib64 ../mesa/2_22.2.2-r0/mesa-22.2.2# ls /../../../../../../../mesa/2_22.2.2-r0/recipe-sysroot/usr/ bin include lib64 share ***************************************************************************************************************************** ../mesa/2_22.2.2-r0/mesa-22.2.2# ../recipe-sysroot-native/usr/bin/llvm-config --version 15.0.1 ../mesa/2_22.2.2-r0/mesa-22.2.2# ../recipe-sysroot-native/usr/bin/llvm-config --libdir /../../../../../../../tmp/work/amd64-northstar-linux/mesa/2_22.2.2-r0/recipe-sysroot-native/usr/lib64 ../mesa/2_22.2.2-r0/mesa-22.2.2# ls /../../../../../../../tmp/work/amd64-northstar-linux/mesa/2_22.2.2-r0/recipe-sysroot-native/usr/ bin include lib libexec sbin share My apologies mesa configure utilizes the bin in recipe-sysroot not recipe-sysroot-native.
To answer second question. When you don't include EXTRA_OECMAKE:append:class-native = "\ -DLLVM_LIBDIR_SUFFIX=64 \ " meson configure fails with .... .... .... | llvm-config found: YES (/../../../../../../../../tmp/work/amd64-northstar-linux/mesa/2_22.2.2-r0/recipe-sysroot/usr/bin/crossscripts/llvm-config15.0.1) 15.0.1 | Run-time dependency LLVM (modules: amdgpu, asmparser, bitreader, bitwriter, core, engine, executionengine, instcombine, ipo, mcdisassembler, mcjit, native, scalaropts, transformutils, coroutines) found: NO (tried cmake and config-tool) | Building fallback subproject with default_library=shared | | ../mesa-22.2.2/meson.build:1783:2: ERROR: Neither a subproject directory nor a llvm.wrap file was found. | | A full log can be found at /../../.../../../../../../../tmp/work/amd64-northstar-linux/mesa/2_22.2.2-r0/build/meson-logs/meson-log.txt .... .... .... Now if you $ cat < /../../.../../../../../../../tmp/work/amd64-northstar-linux/mesa/2_22.2.2-r0/build/meson-logs/meson-log.txt You see llvm-config found: YES (/../../.../../../../../../../tmp/work/amd64-northstar-linux/mesa/2_22.2.2-r0/recipe-sysroot/usr/bin/crossscripts/llvm-config15.0.1) 15.0.1 llvm-config --shared-mode returned an error Run-time dependency LLVM (modules: amdgpu, asmparser, bitreader, bitwriter, core, engine, executionengine, instcombine, ipo, mcdisassembler, mcjit, native, scalaropts, transformutils, coroutines) found: NO (tried cmake and config-tool) Now run llvm-config --shared-mode directly you see $ llvm-config --shared-mode llvm-config: error: component libraries and shared library .... .... .... .... llvm-config: error: missing: /../../.../../../../../../../tmp/work/amd64-northstar-linux/mesa/2_22.2.2-r0/recipe-sysroot/usr/lib/libLLVMMCParser.a llvm-config: error: missing: /../../.../../../../../../../tmp/work/amd64-northstar-linux/mesa/2_22.2.2-r0/recipe-sysroot/usr/lib/libLLVMTextAPI.a llvm-config: error: missing: /../../.../../../../../../../tmp/work/amd64-northstar-linux/mesa/2_22.2.2-r0/recipe-sysroot/usr/lib/libLLVMObject.a llvm-config: error: missing: /../../.../../../../../../../tmp/work/amd64-northstar-linux/mesa/2_22.2.2-r0/recipe-sysroot/usr/lib/libLLVMDebugInfoDWARF.a llvm-config: error: missing: /../../.../../../../../../../tmp/work/amd64-northstar-linux/mesa/2_22.2.2-r0/recipe-sysroot/usr/lib/libLLVMDebugInfoMSF.a llvm-config: error: missing: /../../.../../../../../../../tmp/work/amd64-northstar-linux/mesa/2_22.2.2-r0/recipe-sysroot/usr/lib/libLLVMDebugInfoPDB.a .... .... .... .... ../mesa/2_22.2.2-r0/mesa-22.2.2# llvm-config --libdir /../../.../../../../../../../tmp/work/amd64-northstar-linux/mesa/2_22.2.2-r0/recipe-sysroot/usr/lib ../mesa/2_22.2.2-r0/mesa-22.2.2# ls /../../.../../../../../../../tmp/work/amd64-northstar-linux/mesa/2_22.2.2-r0/recipe-sysroot/usr bin include lib64 share That folder doesn't exists for me. Also another false statement on my end its llvm-config that hard sets directory where to load libs. mesa never calls llvm-config --libdir.
Okay, I figured out what is happening here. There are two issues: 1. llvm-config is placed into the target sysroot (by mesa recipe), without adjusting rpath to point to the native sysroot. This may work if native and target are both x86, but will fail if they are not, or if the native and target use different libdir paths (e.g. lib and lib64). It needs to be able to find libraries from the native sysroot. 2. llvm-config hardcodes the lib location inside of itself to lib+LLVM_LIBDIR_SUFFIX. This one is harder to solve, as llvm-config should be built once, and must be reusable between target builds, even if those builds have different lib suffixes. What we can do is adjust llvm-config wrapper (one in crossscripts) to have more special-cases. Maybe this will even make installing llvm-config native binary into target bindir unnecessary. I'll try to come up with a patchset. Alex On Mon, 14 Nov 2022 at 01:40, Vincent Davis Jr <vince@underview.tech> wrote: > > To answer second question. When you don't include > > EXTRA_OECMAKE:append:class-native = "\ > -DLLVM_LIBDIR_SUFFIX=64 \ > " > > meson configure fails with > .... > .... > .... > | llvm-config found: YES (/../../../../../../../../tmp/work/amd64-northstar-linux/mesa/2_22.2.2-r0/recipe-sysroot/usr/bin/crossscripts/llvm-config15.0.1) 15.0.1 > | Run-time dependency LLVM (modules: amdgpu, asmparser, bitreader, bitwriter, core, engine, executionengine, instcombine, ipo, mcdisassembler, mcjit, native, scalaropts, transformutils, coroutines) found: NO (tried cmake and config-tool) > | Building fallback subproject with default_library=shared > | > | ../mesa-22.2.2/meson.build:1783:2: ERROR: Neither a subproject directory nor a llvm.wrap file was found. > | > | A full log can be found at /../../.../../../../../../../tmp/work/amd64-northstar-linux/mesa/2_22.2.2-r0/build/meson-logs/meson-log.txt > .... > .... > .... > > Now if you > > $ cat < /../../.../../../../../../../tmp/work/amd64-northstar-linux/mesa/2_22.2.2-r0/build/meson-logs/meson-log.txt > > You see > > llvm-config found: YES (/../../.../../../../../../../tmp/work/amd64-northstar-linux/mesa/2_22.2.2-r0/recipe-sysroot/usr/bin/crossscripts/llvm-config15.0.1) 15.0.1 > llvm-config --shared-mode returned an error > Run-time dependency LLVM (modules: amdgpu, asmparser, bitreader, bitwriter, core, engine, executionengine, instcombine, ipo, mcdisassembler, mcjit, native, scalaropts, transformutils, coroutines) found: NO (tried cmake and config-tool) > > Now run llvm-config --shared-mode directly you see > > $ llvm-config --shared-mode > llvm-config: error: component libraries and shared library > .... > .... > .... > .... > llvm-config: error: missing: /../../.../../../../../../../tmp/work/amd64-northstar-linux/mesa/2_22.2.2-r0/recipe-sysroot/usr/lib/libLLVMMCParser.a > llvm-config: error: missing: /../../.../../../../../../../tmp/work/amd64-northstar-linux/mesa/2_22.2.2-r0/recipe-sysroot/usr/lib/libLLVMTextAPI.a > llvm-config: error: missing: /../../.../../../../../../../tmp/work/amd64-northstar-linux/mesa/2_22.2.2-r0/recipe-sysroot/usr/lib/libLLVMObject.a > llvm-config: error: missing: /../../.../../../../../../../tmp/work/amd64-northstar-linux/mesa/2_22.2.2-r0/recipe-sysroot/usr/lib/libLLVMDebugInfoDWARF.a > llvm-config: error: missing: /../../.../../../../../../../tmp/work/amd64-northstar-linux/mesa/2_22.2.2-r0/recipe-sysroot/usr/lib/libLLVMDebugInfoMSF.a > llvm-config: error: missing: /../../.../../../../../../../tmp/work/amd64-northstar-linux/mesa/2_22.2.2-r0/recipe-sysroot/usr/lib/libLLVMDebugInfoPDB.a > .... > .... > .... > .... > > > ../mesa/2_22.2.2-r0/mesa-22.2.2# llvm-config --libdir > /../../.../../../../../../../tmp/work/amd64-northstar-linux/mesa/2_22.2.2-r0/recipe-sysroot/usr/lib > > ../mesa/2_22.2.2-r0/mesa-22.2.2# ls /../../.../../../../../../../tmp/work/amd64-northstar-linux/mesa/2_22.2.2-r0/recipe-sysroot/usr > bin include lib64 share > > That folder doesn't exists for me. > > Also another false statement on my end its llvm-config that hard sets directory where to load libs. > mesa never calls llvm-config --libdir. > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#173215): https://lists.openembedded.org/g/openembedded-core/message/173215 > Mute This Topic: https://lists.openembedded.org/mt/94995332/1686489 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- >
I just sent a patch. Can you try it? Alex On Mon, 14 Nov 2022 at 13:17, Alexander Kanavin via lists.openembedded.org <alex.kanavin=gmail.com@lists.openembedded.org> wrote: > > Okay, I figured out what is happening here. There are two issues: > > 1. llvm-config is placed into the target sysroot (by mesa recipe), > without adjusting rpath to point to the native sysroot. This may work > if native and target are both x86, but will fail if they are not, or > if the native and target use different libdir paths (e.g. lib and > lib64). It needs to be able to find libraries from the native sysroot. > > 2. llvm-config hardcodes the lib location inside of itself to > lib+LLVM_LIBDIR_SUFFIX. This one is harder to solve, as llvm-config > should be built once, and must be reusable between target builds, even > if those builds have different lib suffixes. What we can do is adjust > llvm-config wrapper (one in crossscripts) to have more special-cases. > Maybe this will even make installing llvm-config native binary into > target bindir unnecessary. > > I'll try to come up with a patchset. > > Alex > > On Mon, 14 Nov 2022 at 01:40, Vincent Davis Jr <vince@underview.tech> wrote: > > > > To answer second question. When you don't include > > > > EXTRA_OECMAKE:append:class-native = "\ > > -DLLVM_LIBDIR_SUFFIX=64 \ > > " > > > > meson configure fails with > > .... > > .... > > .... > > | llvm-config found: YES (/../../../../../../../../tmp/work/amd64-northstar-linux/mesa/2_22.2.2-r0/recipe-sysroot/usr/bin/crossscripts/llvm-config15.0.1) 15.0.1 > > | Run-time dependency LLVM (modules: amdgpu, asmparser, bitreader, bitwriter, core, engine, executionengine, instcombine, ipo, mcdisassembler, mcjit, native, scalaropts, transformutils, coroutines) found: NO (tried cmake and config-tool) > > | Building fallback subproject with default_library=shared > > | > > | ../mesa-22.2.2/meson.build:1783:2: ERROR: Neither a subproject directory nor a llvm.wrap file was found. > > | > > | A full log can be found at /../../.../../../../../../../tmp/work/amd64-northstar-linux/mesa/2_22.2.2-r0/build/meson-logs/meson-log.txt > > .... > > .... > > .... > > > > Now if you > > > > $ cat < /../../.../../../../../../../tmp/work/amd64-northstar-linux/mesa/2_22.2.2-r0/build/meson-logs/meson-log.txt > > > > You see > > > > llvm-config found: YES (/../../.../../../../../../../tmp/work/amd64-northstar-linux/mesa/2_22.2.2-r0/recipe-sysroot/usr/bin/crossscripts/llvm-config15.0.1) 15.0.1 > > llvm-config --shared-mode returned an error > > Run-time dependency LLVM (modules: amdgpu, asmparser, bitreader, bitwriter, core, engine, executionengine, instcombine, ipo, mcdisassembler, mcjit, native, scalaropts, transformutils, coroutines) found: NO (tried cmake and config-tool) > > > > Now run llvm-config --shared-mode directly you see > > > > $ llvm-config --shared-mode > > llvm-config: error: component libraries and shared library > > .... > > .... > > .... > > .... > > llvm-config: error: missing: /../../.../../../../../../../tmp/work/amd64-northstar-linux/mesa/2_22.2.2-r0/recipe-sysroot/usr/lib/libLLVMMCParser.a > > llvm-config: error: missing: /../../.../../../../../../../tmp/work/amd64-northstar-linux/mesa/2_22.2.2-r0/recipe-sysroot/usr/lib/libLLVMTextAPI.a > > llvm-config: error: missing: /../../.../../../../../../../tmp/work/amd64-northstar-linux/mesa/2_22.2.2-r0/recipe-sysroot/usr/lib/libLLVMObject.a > > llvm-config: error: missing: /../../.../../../../../../../tmp/work/amd64-northstar-linux/mesa/2_22.2.2-r0/recipe-sysroot/usr/lib/libLLVMDebugInfoDWARF.a > > llvm-config: error: missing: /../../.../../../../../../../tmp/work/amd64-northstar-linux/mesa/2_22.2.2-r0/recipe-sysroot/usr/lib/libLLVMDebugInfoMSF.a > > llvm-config: error: missing: /../../.../../../../../../../tmp/work/amd64-northstar-linux/mesa/2_22.2.2-r0/recipe-sysroot/usr/lib/libLLVMDebugInfoPDB.a > > .... > > .... > > .... > > .... > > > > > > ../mesa/2_22.2.2-r0/mesa-22.2.2# llvm-config --libdir > > /../../.../../../../../../../tmp/work/amd64-northstar-linux/mesa/2_22.2.2-r0/recipe-sysroot/usr/lib > > > > ../mesa/2_22.2.2-r0/mesa-22.2.2# ls /../../.../../../../../../../tmp/work/amd64-northstar-linux/mesa/2_22.2.2-r0/recipe-sysroot/usr > > bin include lib64 share > > > > That folder doesn't exists for me. > > > > Also another false statement on my end its llvm-config that hard sets directory where to load libs. > > mesa never calls llvm-config --libdir. > > > > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#173237): https://lists.openembedded.org/g/openembedded-core/message/173237 > Mute This Topic: https://lists.openembedded.org/mt/94995332/1686489 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- >
Yes can try now
diff --git a/meta/recipes-devtools/llvm/llvm_git.bb b/meta/recipes-devtools/llvm/llvm_git.bb index 1f47e5d0fb..29eb119be5 100644 --- a/meta/recipes-devtools/llvm/llvm_git.bb +++ b/meta/recipes-devtools/llvm/llvm_git.bb @@ -56,6 +56,14 @@ def get_llvm_arch(bb, d, arch_var): def get_llvm_host_arch(bb, d): return get_llvm_arch(bb, d, 'HOST_ARCH') +def get_llvm_libdir_suffix(bb, d): + import re + arch = d.getVar('HOST_ARCH') + if re.match(r'(x86.64)$', arch): + return 64 + else: + return "" + PACKAGECONFIG ??= "" # if optviewer OFF, force the modules to be not found or the ones on the host would be found PACKAGECONFIG[optviewer] = ",-DPY_PYGMENTS_FOUND=OFF -DPY_PYGMENTS_LEXERS_C_CPP_FOUND=OFF -DPY_YAML_FOUND=OFF,python3-pygments python3-pyyaml,python3-pygments python3-pyyaml" @@ -64,6 +72,7 @@ PACKAGECONFIG[optviewer] = ",-DPY_PYGMENTS_FOUND=OFF -DPY_PYGMENTS_LEXERS_C_CPP_ # Default to build all OE-Core supported target arches (user overridable). # LLVM_TARGETS ?= "AMDGPU;${@get_llvm_host_arch(bb, d)}" +LLVM_TARGET_LIBDIR_SUFFIX ?= "${@get_llvm_libdir_suffix(bb, d)}" ARM_INSTRUCTION_SET:armv5 = "arm" ARM_INSTRUCTION_SET:armv4t = "arm" @@ -83,6 +92,10 @@ EXTRA_OECMAKE += "-DLLVM_ENABLE_ASSERTIONS=OFF \ -DCMAKE_BUILD_TYPE=Release \ -G Ninja" +EXTRA_OECMAKE:append:class-native = "\ + -DLLVM_LIBDIR_SUFFIX=${LLVM_TARGET_LIBDIR_SUFFIX} \ + " + EXTRA_OECMAKE:append:class-target = "\ -DCMAKE_CROSSCOMPILING:BOOL=ON \ -DLLVM_TABLEGEN=${STAGING_BINDIR_NATIVE}/llvm-tblgen${PV} \ @@ -129,6 +142,7 @@ do_install:class-native() { } SYSROOT_PREPROCESS_FUNCS:append:class-target = " llvm_sysroot_preprocess" +SYSROOT_PREPROCESS_FUNCS:append:class-native = " llvm_sysroot_preprocess_native" llvm_sysroot_preprocess() { install -d ${SYSROOT_DESTDIR}${bindir_crossscripts}/ @@ -136,6 +150,18 @@ llvm_sysroot_preprocess() { ln -sf llvm-config ${SYSROOT_DESTDIR}${bindir_crossscripts}/llvm-config${PV} } +# All of LLVM's shipping binaries have an rpath pointing at ``$ORIGIN/../lib``. +# Recipe llvm requires llvm-native to compile. When compiling llvm build libraries +# are located in $ORIGIN/../lib directory. When the llvm-config binary required +# in another recipe depending upon HOST_ARCH variable libraries may be located +# in a different directory. Account for both situations here. +llvm_sysroot_preprocess_native() { + chrpath --replace '$ORIGIN'/../lib${LLVM_TARGET_LIBDIR_SUFFIX}:'$ORIGIN'/../../lib${LLVM_TARGET_LIBDIR_SUFFIX}:'$ORIGIN'/../lib:'$ORIGIN'/../../lib \ + ${SYSROOT_DESTDIR}${bindir}/llvm-tblgen${PV} + chrpath --replace '$ORIGIN'/../lib${LLVM_TARGET_LIBDIR_SUFFIX}:'$ORIGIN'/../../lib${LLVM_TARGET_LIBDIR_SUFFIX}:'$ORIGIN'/../lib:'$ORIGIN'/../../lib \ + ${SYSROOT_DESTDIR}${bindir}/llvm-config${PV} +} + PACKAGES =+ "${PN}-bugpointpasses ${PN}-llvmhello ${PN}-libllvm ${PN}-liboptremarks ${PN}-liblto" RRECOMMENDS:${PN}-dev += "${PN}-bugpointpasses ${PN}-llvmhello ${PN}-liboptremarks"
Problem occurs when native llvm-config binary is required by another recipe. RUNPATH is hardcoded to $ORIGIN/../lib:$ORIGIN/../../lib which depending upon architecture 'lib' directory name may vary (i.e 'lib64'). Commit fixes issue by updating rpath on binary to include architecture dependent directory name. Commit also fixes issue with llvm-config --libdir command by setting LLVM_LIBDIR_SUFFIX. Command will return proper architecture dependent directory name. Signed-off-by: Vincent Davis Jr <vince@underview.tech> --- meta/recipes-devtools/llvm/llvm_git.bb | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+)