| Message ID | 20231212140820.2900963-2-ross.burton@arm.com |
|---|---|
| State | Accepted, archived |
| Commit | f560ac0a5ccced02b84df337f0f26209cd4b6474 |
| Headers | show |
| Series | Let autoreconf run gtkdocize | expand |
Hello Ross, Not obvious but this is the cause of: https://autobuilder.yoctoproject.org/typhoon/#/builders/20/builds/8697 On 12/12/2023 14:08:12+0000, Ross Burton wrote: > From: Ross Burton <ross.burton@arm.com> > > nativesdk aims to run in a slightly different environment than the > target build, so it unsets MACHINE_FEATURES and lets DISTRO_FEATURES be > filtered by DISTRO_FEATURES_NATIVESDK (in bitbake.conf). > > However, feature backfill happens _after_ these operations: > > $ bitbake-getvar -r nativesdk-glib-2.0 MACHINE_FEATURES > # > # $MACHINE_FEATURES [5 operations] > # set /home/ross/Yocto/poky/meta/conf/machine/include/qemu.inc:14 > # "alsa bluetooth usbgadget screen vfat" > # set /home/ross/Yocto/poky/meta/conf/documentation.conf:284 > # [doc] "Specifies the list of hardware features the MACHINE supports." > # set? /home/ross/Yocto/poky/meta/conf/bitbake.conf:893 > # "" > # set /home/ross/Yocto/poky/meta/classes-recipe/nativesdk.bbclass:18 > # "" > # append utils.py:132 [features_backfill] > # " rtc qemu-usermode" > # pre-expansion value: > # " rtc qemu-usermode" > MACHINE_FEATURES=" rtc qemu-usermode" > > I don't believe this is intentional or desired as the machine features > are unrelated to the nativesdk environment. Specifically, this means > that recipes that look for the qemu-usermode machine feature will fail > in nativesdk as there may not actually be qemu-user support for the > nativesdk environment. > > Signed-off-by: Ross Burton <ross.burton@arm.com> > --- > meta/classes-recipe/nativesdk.bbclass | 3 +++ > 1 file changed, 3 insertions(+) > > diff --git a/meta/classes-recipe/nativesdk.bbclass b/meta/classes-recipe/nativesdk.bbclass > index 08288fdb733..ccaf411f0d7 100644 > --- a/meta/classes-recipe/nativesdk.bbclass > +++ b/meta/classes-recipe/nativesdk.bbclass > @@ -15,7 +15,10 @@ NATIVESDKLIBC ?= "libc-glibc" > LIBCOVERRIDE = ":${NATIVESDKLIBC}" > CLASSOVERRIDE = "class-nativesdk" > MACHINEOVERRIDES = "" > + > MACHINE_FEATURES = "" > +DISTRO_FEATURES_BACKFILL = "" > +MACHINE_FEATURES_BACKFILL = "" > > MULTILIBS = "" > > -- > 2.34.1 > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#192193): https://lists.openembedded.org/g/openembedded-core/message/192193 > Mute This Topic: https://lists.openembedded.org/mt/103129922/3617179 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alexandre.belloni@bootlin.com] > -=-=-=-=-=-=-=-=-=-=-=- >
On 14 Dec 2023, at 07:49, Alexandre Belloni <alexandre.belloni@bootlin.com> wrote: > Not obvious but this is the cause of: > > https://autobuilder.yoctoproject.org/typhoon/#/builders/20/builds/8697 That was a fun rabbithole to dig. The error from the log is: ERROR: /home/pokybuild/yocto-worker/buildtools/build/meta/recipes-devtools/python/python3_3.11.5.bb: pgo cannot be enabled as there is no qemu-usermode support for this architecture/machine Which was intentional fallout: MACHINE_FEATURES don’t describe the SDK so that this ever worked was more luck than judgement. Also the python3 recipe will write the qemu-calling wrapper script using the architecture only for target builds, so we’ve not had PGO nativesdk python since the 3.7.2 upgrade in 2019[1] which changed this from all builds to target only. Maybe we should allow machine-sdk configurations to extend SDK_MACHINE_FEATURES and then nativesdk can use that? This might allow the MACHINE_FEATURES to “work” without being based on the target machine. Alternatively, I can remove the pgo from the buildtools build, as it doesn’t actually do anything right now anyway. Ross [1] oe-core 02714c105426b0d687620913c1a7401b386428b6
On Thu, 2023-12-14 at 13:31 +0000, Ross Burton wrote: > On 14 Dec 2023, at 07:49, Alexandre Belloni > <alexandre.belloni@bootlin.com> wrote: > > Not obvious but this is the cause of: > > > > https://autobuilder.yoctoproject.org/typhoon/#/builders/20/builds/8697 > > That was a fun rabbithole to dig. > > The error from the log is: > > ERROR: /home/pokybuild/yocto-worker/buildtools/build/meta/recipes- > devtools/python/python3_3.11.5.bb: pgo cannot be enabled as there is > no qemu-usermode support for this architecture/machine > > Which was intentional fallout: MACHINE_FEATURES don’t describe the > SDK so that this ever worked was more luck than judgement. Also the > python3 recipe will write the qemu-calling wrapper script using the > architecture only for target builds, so we’ve not had PGO nativesdk > python since the 3.7.2 upgrade in 2019[1] which changed this from all > builds to target only. > > Maybe we should allow machine-sdk configurations to extend > SDK_MACHINE_FEATURES and then nativesdk can use that? This might > allow the MACHINE_FEATURES to “work” without being based on the > target machine. Sadly I suspect we're going to have to go down this route eventually in some form. > Alternatively, I can remove the pgo from the buildtools build, as it > doesn’t actually do anything right now anyway. It was a decent win at one point so that would seem sad. Cheers, Richard
FWIW in the upcoming python 3.12 update I have removed the pgo patch (0001-Makefile.pre-use-qemu-wrapper-when-gathering-profile.patch) with the rationale that "pgo support has been rewritten upstream and needs to be reenabled and tested separately". I simply don't have bandwidth to make it work, with a ton of recipes in meta-oe that still need fixing with 3.12. Alex On Thu, 14 Dec 2023 at 14:35, Richard Purdie <richard.purdie@linuxfoundation.org> wrote: > > On Thu, 2023-12-14 at 13:31 +0000, Ross Burton wrote: > > On 14 Dec 2023, at 07:49, Alexandre Belloni > > <alexandre.belloni@bootlin.com> wrote: > > > Not obvious but this is the cause of: > > > > > > https://autobuilder.yoctoproject.org/typhoon/#/builders/20/builds/8697 > > > > That was a fun rabbithole to dig. > > > > The error from the log is: > > > > ERROR: /home/pokybuild/yocto-worker/buildtools/build/meta/recipes- > > devtools/python/python3_3.11.5.bb: pgo cannot be enabled as there is > > no qemu-usermode support for this architecture/machine > > > > Which was intentional fallout: MACHINE_FEATURES don’t describe the > > SDK so that this ever worked was more luck than judgement. Also the > > python3 recipe will write the qemu-calling wrapper script using the > > architecture only for target builds, so we’ve not had PGO nativesdk > > python since the 3.7.2 upgrade in 2019[1] which changed this from all > > builds to target only. > > > > Maybe we should allow machine-sdk configurations to extend > > SDK_MACHINE_FEATURES and then nativesdk can use that? This might > > allow the MACHINE_FEATURES to “work” without being based on the > > target machine. > > Sadly I suspect we're going to have to go down this route eventually in > some form. > > > Alternatively, I can remove the pgo from the buildtools build, as it > > doesn’t actually do anything right now anyway. > > It was a decent win at one point so that would seem sad. > > Cheers, > > Richard > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#192370): https://lists.openembedded.org/g/openembedded-core/message/192370 > Mute This Topic: https://lists.openembedded.org/mt/103129922/1686489 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- >
Correction: they didn't rewrite it, they just moved around the lines: https://github.com/python/cpython/commit/9084e1b04f8d35721b535a3acd550b920d50f11a and I didn't look thoroughly enough for the new location. Neverthless, given that a-full still passed with the patch removed means pgo can be completely broken or completely working, and we would have no idea. I lean towards 'completely broken' but I'll reinstate the patch. Alex On Thu, 14 Dec 2023 at 14:49, Alexander Kanavin via lists.openembedded.org <alex.kanavin=gmail.com@lists.openembedded.org> wrote: > > FWIW in the upcoming python 3.12 update I have removed the pgo patch > (0001-Makefile.pre-use-qemu-wrapper-when-gathering-profile.patch) with > the rationale that "pgo support has been rewritten upstream and needs > to be reenabled and tested separately". I simply don't have bandwidth > to make it work, with a ton of recipes in meta-oe that still need > fixing with 3.12. > > Alex > > On Thu, 14 Dec 2023 at 14:35, Richard Purdie > <richard.purdie@linuxfoundation.org> wrote: > > > > On Thu, 2023-12-14 at 13:31 +0000, Ross Burton wrote: > > > On 14 Dec 2023, at 07:49, Alexandre Belloni > > > <alexandre.belloni@bootlin.com> wrote: > > > > Not obvious but this is the cause of: > > > > > > > > https://autobuilder.yoctoproject.org/typhoon/#/builders/20/builds/8697 > > > > > > That was a fun rabbithole to dig. > > > > > > The error from the log is: > > > > > > ERROR: /home/pokybuild/yocto-worker/buildtools/build/meta/recipes- > > > devtools/python/python3_3.11.5.bb: pgo cannot be enabled as there is > > > no qemu-usermode support for this architecture/machine > > > > > > Which was intentional fallout: MACHINE_FEATURES don’t describe the > > > SDK so that this ever worked was more luck than judgement. Also the > > > python3 recipe will write the qemu-calling wrapper script using the > > > architecture only for target builds, so we’ve not had PGO nativesdk > > > python since the 3.7.2 upgrade in 2019[1] which changed this from all > > > builds to target only. > > > > > > Maybe we should allow machine-sdk configurations to extend > > > SDK_MACHINE_FEATURES and then nativesdk can use that? This might > > > allow the MACHINE_FEATURES to “work” without being based on the > > > target machine. > > > > Sadly I suspect we're going to have to go down this route eventually in > > some form. > > > > > Alternatively, I can remove the pgo from the buildtools build, as it > > > doesn’t actually do anything right now anyway. > > > > It was a decent win at one point so that would seem sad. > > > > Cheers, > > > > Richard > > > > > > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#192380): https://lists.openembedded.org/g/openembedded-core/message/192380 > Mute This Topic: https://lists.openembedded.org/mt/103129922/1686489 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- >
diff --git a/meta/classes-recipe/nativesdk.bbclass b/meta/classes-recipe/nativesdk.bbclass index 08288fdb733..ccaf411f0d7 100644 --- a/meta/classes-recipe/nativesdk.bbclass +++ b/meta/classes-recipe/nativesdk.bbclass @@ -15,7 +15,10 @@ NATIVESDKLIBC ?= "libc-glibc" LIBCOVERRIDE = ":${NATIVESDKLIBC}" CLASSOVERRIDE = "class-nativesdk" MACHINEOVERRIDES = "" + MACHINE_FEATURES = "" +DISTRO_FEATURES_BACKFILL = "" +MACHINE_FEATURES_BACKFILL = "" MULTILIBS = ""