| Message ID | 20231106142217.3837516-1-rasmus.villemoes@prevas.dk |
|---|---|
| State | Accepted, archived |
| Commit | df1905294690682496d8f8e8284964ab897f0cd4 |
| Headers | show |
| Series | perf: add jevents PACKAGECONFIG item | expand |
On Mon, Nov 6, 2023 at 9:22 AM Rasmus Villemoes <rasmus.villemoes@prevas.dk> wrote: > > From: Rasmus Villemoes <rasmus.villemoes@prevas.dk> > > Building for an arm64 target, e.g. qemuarm64 or a raspberrypi3, > without "python" in PACKAGECONFIG, results in > > | Makefile.config:892: *** ERROR: No python interpreter needed for jevents generation. Install python or build with NO_JEVENTS=1.. Stop. > > Signed-off-by: Rasmus Villemoes <rasmus.villemoes@prevas.dk> > --- > meta/recipes-kernel/perf/perf.bb | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/meta/recipes-kernel/perf/perf.bb b/meta/recipes-kernel/perf/perf.bb > index a392166e73..37b53ee7df 100644 > --- a/meta/recipes-kernel/perf/perf.bb > +++ b/meta/recipes-kernel/perf/perf.bb > @@ -28,6 +28,7 @@ PACKAGECONFIG[audit] = ",NO_LIBAUDIT=1,audit" > PACKAGECONFIG[manpages] = ",,xmlto-native asciidoc-native" > PACKAGECONFIG[cap] = ",,libcap" > PACKAGECONFIG[libtraceevent] = ",NO_LIBTRACEEVENT=1,libtraceevent" > +PACKAGECONFIG[jevents] = ",NO_JEVENTS=1,python3-native" Since we already have a python packageconfig and it takes care of the dependencies that we need, we shouldn't be repeating the python3-native in this one. We'll end up having to maintain that dependency in two locations. i.e. we don't have python3-native as a dependency of our main python packageconfig .. is that an error (the python packageconfig) or should this be depending on python3 python3-setuptools-native, just like the python one ? Since there's no way to make packageconfig 'a' depend on packageconfig 'b' that I know of (RP will correct me if I'm missing or forgetting something here), we could always add a test on the final PACKAGECONFIG and error/warn if jevents was enabled and python wasn't. It is always a bit annoying that perf turns these on, and we have to explicitly opt-out .. but that is out of our control :) Bruce > # Arm CoreSight > PACKAGECONFIG[coresight] = "CORESIGHT=1,,opencsd" > PACKAGECONFIG[pfm4] = ",NO_LIBPFM4=1,libpfm4" > -- > 2.40.1.1.g1c60b9335d >
Somewhat related, but maybe not really:
When I set PACKAGECONFIG for perf in a perf.bbappend file, without
python being in there, the build breaks:
| Makefile.config:277: ***
.../tmp/work/qemuarm64-poky-linux/perf/1.0/recipe-sysroot-native/usr/bin/python3-native/python3-config
not found. Stop.
| make[1]: *** [Makefile.perf:242: sub-make] Error 2
If I instead set the same PACKAGECONFIG value, but by doing
PACKAGECONFIG:pn-perf = "tui libunwind libtraceevent"
in some global configuration file, it works as expected.
====
I think the difference is due to the
inherit ${@bb.utils.contains('PACKAGECONFIG', 'python',
'python3targetconfig', '', d)}
line in the perf recipe. At the time where this line is evaluated, if
PACKAGECONFIG is set via a .bbappend, this ends up evaluated using the
??= value which does include python, and hence we end up inheriting
python3targetconfig, which in turn inherits python3native, which does
export PYTHON
And since PYTHON is now exported, perf's logic ends up in this fragment
in tools/perf/Makefile.config:
# If PYTHON is defined but PYTHON_CONFIG isn't, then take
$(PYTHON)-config as if it was the user
# supplied value for PYTHON_CONFIG. Because it's "user supplied", error
out if it doesn't exist.
ifdef PYTHON
ifndef PYTHON_CONFIG
PYTHON_CONFIG_AUTO := $(call get-executable,$(PYTHON)-config)
PYTHON_CONFIG := $(if $(PYTHON_CONFIG_AUTO),$(PYTHON_CONFIG_AUTO),\
$(call $(error $(PYTHON)-config not found)))
endif
endif
which breaks because the value of PACKAGECONFIG that ended up
determining buildtime dependencies did not pull in that python3-config.
====
This difference is, to put it mildly, quite confusing and surprising
(and took quite some time to track down). I don't know if there's
anything to do about it. Is setting PACKAGECONFIG via :pn-foo overrides
the only supported method, or is this just a quirk of the perf recipe
that one has to live with?
Rasmus
On Mon, 2023-11-06 at 16:00 +0100, Rasmus Villemoes wrote: > Somewhat related, but maybe not really: > > When I set PACKAGECONFIG for perf in a perf.bbappend file, without > python being in there, the build breaks: > > > Makefile.config:277: *** > .../tmp/work/qemuarm64-poky-linux/perf/1.0/recipe-sysroot-native/usr/bin/python3-native/python3-config > not found. Stop. > > make[1]: *** [Makefile.perf:242: sub-make] Error 2 > > If I instead set the same PACKAGECONFIG value, but by doing > > PACKAGECONFIG:pn-perf = "tui libunwind libtraceevent" > > in some global configuration file, it works as expected. > > ==== > > I think the difference is due to the > > inherit ${@bb.utils.contains('PACKAGECONFIG', 'python', > 'python3targetconfig', '', d)} > > line in the perf recipe. At the time where this line is evaluated, if > PACKAGECONFIG is set via a .bbappend, this ends up evaluated using the > ??= value which does include python, and hence we end up inheriting > python3targetconfig, which in turn inherits python3native, which does > > export PYTHON > > And since PYTHON is now exported, perf's logic ends up in this fragment > in tools/perf/Makefile.config: > > # If PYTHON is defined but PYTHON_CONFIG isn't, then take > $(PYTHON)-config as if it was the user > # supplied value for PYTHON_CONFIG. Because it's "user supplied", error > out if it doesn't exist. > ifdef PYTHON > ifndef PYTHON_CONFIG > PYTHON_CONFIG_AUTO := $(call get-executable,$(PYTHON)-config) > PYTHON_CONFIG := $(if $(PYTHON_CONFIG_AUTO),$(PYTHON_CONFIG_AUTO),\ > $(call $(error $(PYTHON)-config not found))) > endif > endif > > which breaks because the value of PACKAGECONFIG that ended up > determining buildtime dependencies did not pull in that python3-config. > > ==== > > This difference is, to put it mildly, quite confusing and surprising > (and took quite some time to track down). I don't know if there's > anything to do about it. Is setting PACKAGECONFIG via :pn-foo overrides > the only supported method, or is this just a quirk of the perf recipe > that one has to live with? That is a pretty horrible interaction of things. As you say, that inherit is effectively an immediate expansion of the data and wouldn't "see" changes in any bbappend. Could you file a bug please? I'm wondering if we should have some other form of inherit which applies at the end of parsing which would avoid that issue. Sometimes order is important, sometimes it is less important with the inherits. Cheers, Richard
On 06/11/2023 15.49, Bruce Ashfield wrote: > On Mon, Nov 6, 2023 at 9:22 AM Rasmus Villemoes > <rasmus.villemoes@prevas.dk> wrote: >> >> From: Rasmus Villemoes <rasmus.villemoes@prevas.dk> >> >> Building for an arm64 target, e.g. qemuarm64 or a raspberrypi3, >> without "python" in PACKAGECONFIG, results in >> >> | Makefile.config:892: *** ERROR: No python interpreter needed for jevents generation. Install python or build with NO_JEVENTS=1.. Stop. >> >> Signed-off-by: Rasmus Villemoes <rasmus.villemoes@prevas.dk> >> --- >> meta/recipes-kernel/perf/perf.bb | 1 + >> 1 file changed, 1 insertion(+) >> >> diff --git a/meta/recipes-kernel/perf/perf.bb b/meta/recipes-kernel/perf/perf.bb >> index a392166e73..37b53ee7df 100644 >> --- a/meta/recipes-kernel/perf/perf.bb >> +++ b/meta/recipes-kernel/perf/perf.bb >> @@ -28,6 +28,7 @@ PACKAGECONFIG[audit] = ",NO_LIBAUDIT=1,audit" >> PACKAGECONFIG[manpages] = ",,xmlto-native asciidoc-native" >> PACKAGECONFIG[cap] = ",,libcap" >> PACKAGECONFIG[libtraceevent] = ",NO_LIBTRACEEVENT=1,libtraceevent" >> +PACKAGECONFIG[jevents] = ",NO_JEVENTS=1,python3-native" > > Since we already have a python packageconfig and it takes care of > the dependencies that we need, we shouldn't be repeating the python3-native > in this one. We'll end up having to maintain that dependency in > two locations. > > i.e. we don't have python3-native as a dependency of our main python > packageconfig .. is that an error (the python packageconfig) or should > this be depending on python3 python3-setuptools-native, just like the > python one ? The two are as I understand it entirely unrelated. The existing python knob controls whether some stuff that requires python at runtime is built/installed (and that build may of course itself require some native python, in this case in the guise of setuptools-native). The jevents stuff is purely about generating some tables [as a .c file] at build time, and that requires native python, but does not impose a dependency on python on target. > Since there's no way to make packageconfig 'a' depend on > packageconfig 'b' that I know of (RP will correct me if I'm missing > or forgetting something here), we could always add a test on the > final PACKAGECONFIG and error/warn if jevents was enabled and > python wasn't. I don't think it's correct to force jevents pc to include python pc, as the latter will make the perf binary pull in the whole python interpreter etc. If feature A depends on foo-native and foo and bar, and feature B requires foo-native, I don't see why one should be forced to enable feature A in order to get B. [Conversely, even if the dependencies of feature A today happen to be a superset of those of feature B, I don't think selecting feature A should automatically imply feature B.] > It is always a bit annoying that perf turns these on, and we have > to explicitly opt-out .. but that is out of our control :) Indeed. Rasmus
On Mon, Nov 6, 2023 at 10:15 AM Rasmus Villemoes <rasmus.villemoes@prevas.dk> wrote: > > On 06/11/2023 15.49, Bruce Ashfield wrote: > > On Mon, Nov 6, 2023 at 9:22 AM Rasmus Villemoes > > <rasmus.villemoes@prevas.dk> wrote: > >> > >> From: Rasmus Villemoes <rasmus.villemoes@prevas.dk> > >> > >> Building for an arm64 target, e.g. qemuarm64 or a raspberrypi3, > >> without "python" in PACKAGECONFIG, results in > >> > >> | Makefile.config:892: *** ERROR: No python interpreter needed for jevents generation. Install python or build with NO_JEVENTS=1.. Stop. > >> > >> Signed-off-by: Rasmus Villemoes <rasmus.villemoes@prevas.dk> > >> --- > >> meta/recipes-kernel/perf/perf.bb | 1 + > >> 1 file changed, 1 insertion(+) > >> > >> diff --git a/meta/recipes-kernel/perf/perf.bb b/meta/recipes-kernel/perf/perf.bb > >> index a392166e73..37b53ee7df 100644 > >> --- a/meta/recipes-kernel/perf/perf.bb > >> +++ b/meta/recipes-kernel/perf/perf.bb > >> @@ -28,6 +28,7 @@ PACKAGECONFIG[audit] = ",NO_LIBAUDIT=1,audit" > >> PACKAGECONFIG[manpages] = ",,xmlto-native asciidoc-native" > >> PACKAGECONFIG[cap] = ",,libcap" > >> PACKAGECONFIG[libtraceevent] = ",NO_LIBTRACEEVENT=1,libtraceevent" > >> +PACKAGECONFIG[jevents] = ",NO_JEVENTS=1,python3-native" > > > > Since we already have a python packageconfig and it takes care of > > the dependencies that we need, we shouldn't be repeating the python3-native > > in this one. We'll end up having to maintain that dependency in > > two locations. > > > > i.e. we don't have python3-native as a dependency of our main python > > packageconfig .. is that an error (the python packageconfig) or should > > this be depending on python3 python3-setuptools-native, just like the > > python one ? > > The two are as I understand it entirely unrelated. The existing python > knob controls whether some stuff that requires python at runtime is > built/installed (and that build may of course itself require some native > python, in this case in the guise of setuptools-native). > > The jevents stuff is purely about generating some tables [as a .c file] > at build time, and that requires native python, but does not impose a > dependency on python on target. Fair enough. So it is definitely a host-side feature. A comment above the packageconfig explaining that would be ideal. In a perfect world, all of our packageconfig options would have a docstring along with them .. since it is easy to lose track of things. > > > Since there's no way to make packageconfig 'a' depend on > > packageconfig 'b' that I know of (RP will correct me if I'm missing > > or forgetting something here), we could always add a test on the > > final PACKAGECONFIG and error/warn if jevents was enabled and > > python wasn't. > > I don't think it's correct to force jevents pc to include python pc, as > the latter will make the perf binary pull in the whole python > interpreter etc. > > If feature A depends on foo-native and foo and bar, and feature B > requires foo-native, I don't see why one should be forced to enable > feature A in order to get B. [Conversely, even if the dependencies of > feature A today happen to be a superset of those of feature B, I don't > think selecting feature A should automatically imply feature B.] I don't disagree with that logic, but you went too far with it .. since it was based solely on my first comment about it being target python. If there actually are dependent features, I'd always argue for having a dependency versus duplicating the details, but that isn't the case here. Bruce > > > It is always a bit annoying that perf turns these on, and we have > > to explicitly opt-out .. but that is out of our control :) > > Indeed. > > Rasmus >
On 06/11/2023 16.07, Richard Purdie wrote: > On Mon, 2023-11-06 at 16:00 +0100, Rasmus Villemoes wrote: >> >> When I set PACKAGECONFIG for perf in a perf.bbappend file, without >> python being in there, the build breaks: >> [...] >> This difference is, to put it mildly, quite confusing and surprising >> (and took quite some time to track down). I don't know if there's >> anything to do about it. Is setting PACKAGECONFIG via :pn-foo overrides >> the only supported method, or is this just a quirk of the perf recipe >> that one has to live with? > > That is a pretty horrible interaction of things. As you say, that > inherit is effectively an immediate expansion of the data and wouldn't > "see" changes in any bbappend. > > Could you file a bug please? Done. https://bugzilla.yoctoproject.org/show_bug.cgi?id=15276 I now see that the same problem probably exists for the perl item (just the other way around because perl is not initially in PACKAGECONFIG); the line include ${@bb.utils.contains('PACKAGECONFIG', 'perl', 'perf-perl.inc', '', d)} won't have effect if perl is only set in PACKAGECONFIG in a .bbappend. > I'm wondering if we should have some other form of inherit which > applies at the end of parsing which would avoid that issue. Sometimes > order is important, sometimes it is less important with the inherits. Dunno. Maybe a new bitbake keyword "defer" (or "late" or whatever): defer inherit ${@something(PACKAGECONFIG)} that would parse the rest of the line as usual, but instead of adding directly to the statements list, instead to some new deferred_statements (which StatementGroup could track itself, it would probably just need to be taught .append_deferred and .eval_deferred methods and have a .deferred member). I don't really know much about bitbake internals, it doesn't sound too hard to do, but I'm probably missing something. Also, perhaps it's too subtle. Rasmus
diff --git a/meta/recipes-kernel/perf/perf.bb b/meta/recipes-kernel/perf/perf.bb index a392166e73..37b53ee7df 100644 --- a/meta/recipes-kernel/perf/perf.bb +++ b/meta/recipes-kernel/perf/perf.bb @@ -28,6 +28,7 @@ PACKAGECONFIG[audit] = ",NO_LIBAUDIT=1,audit" PACKAGECONFIG[manpages] = ",,xmlto-native asciidoc-native" PACKAGECONFIG[cap] = ",,libcap" PACKAGECONFIG[libtraceevent] = ",NO_LIBTRACEEVENT=1,libtraceevent" +PACKAGECONFIG[jevents] = ",NO_JEVENTS=1,python3-native" # Arm CoreSight PACKAGECONFIG[coresight] = "CORESIGHT=1,,opencsd" PACKAGECONFIG[pfm4] = ",NO_LIBPFM4=1,libpfm4"