| Message ID | 20230828124834.376779-4-martin@geanix.com |
|---|---|
| State | New |
| Headers | show |
| Series | [RFC,1/5] classes: jobserver: support gnu make fifo jobserver | expand |
Thanks for the patch set Martin! On 2023-08-28 08:48, Martin Hundeb?ll via lists.openembedded.org wrote: > If the jobserver class is enabled, the PARALLEL_MAKE variable is unset in > favor of configuring a shared jobserver in the MAKEFLAGS variable. However, > the qemu makefile translates the missing `-j<N>` argument to `-j1` when > calling into meson / ninja. Avoid this by setting `-j` without a value. > For normal/GNU make, this can result in a fork bomb, but for ninja, it > simply makes it use the jobserver fifa instead. small typo: s/fifa/fifo/ > > Signed-off-by: Martin Hundebøll<martin@geanix.com> > --- > meta/recipes-devtools/qemu/qemu.inc | 5 +++++ > 1 file changed, 5 insertions(+) > > diff --git a/meta/recipes-devtools/qemu/qemu.inc b/meta/recipes-devtools/qemu/qemu.inc > index ea02bf0c73..7da05fcbf4 100644 > --- a/meta/recipes-devtools/qemu/qemu.inc > +++ b/meta/recipes-devtools/qemu/qemu.inc > @@ -110,6 +110,11 @@ EXTRA_OECONF = " \ > ${PACKAGECONFIG_CONFARGS} \ > " > > +# Avoid the qemu makefile treating a missing `-j<N>` argument as `-j1` when > +# calling meson / ninja. This happens when the `jobserver` class is used, since > +# it manages parallelism in the MAKEFLAGS variable instead of PARALLEL_MAKE. > +EXTRA_OEMAKE:append ="${@' -j' if d.getVar('JOBSERVER_FIFO') else ''}" > + Is there, or should there be, an upstream bug report/PR for this? To save other from checking, this is from 2020: qemu.git on master ❯ git blame Makefile | rg j1 09e93326e44 (Paolo Bonzini 2020-08-13 09:28:11 -0400 145) $(filter-out -j, $(lastword -j1 $(filter -l% -j%, $(MAKEFLAGS)))) \ 09e93326e4 build: replace ninjatool with ninja https://gitlab.com/qemu-project/qemu/-/commit/09e93326e4 I've applied the patch set and will do a few builds on a old 24 core system and maybe other systems. Do you have any information on the impact when using the job server? I suspect that adding openssl to an image might be a good use case since it's is a highly parallel build. Are there other packages that don't depend on openssl that typically run at the same time and can use loads of cores for a long time? If not, I may try out a Nodejs + chromium + webkitgtk + ??? build! ../Randy > B = "${WORKDIR}/build" > > #EXTRA_OECONF:append = " --python=${HOSTTOOLS_DIR}/python3" > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#186827):https://lists.openembedded.org/g/openembedded-core/message/186827 > Mute This Topic:https://lists.openembedded.org/mt/101009095/3616765 > Group Owner:openembedded-core+owner@lists.openembedded.org > Unsubscribe:https://lists.openembedded.org/g/openembedded-core/unsub [randy.macleod@windriver.com] > -=-=-=-=-=-=-=-=-=-=-=- >
diff --git a/meta/recipes-devtools/qemu/qemu.inc b/meta/recipes-devtools/qemu/qemu.inc index ea02bf0c73..7da05fcbf4 100644 --- a/meta/recipes-devtools/qemu/qemu.inc +++ b/meta/recipes-devtools/qemu/qemu.inc @@ -110,6 +110,11 @@ EXTRA_OECONF = " \ ${PACKAGECONFIG_CONFARGS} \ " +# Avoid the qemu makefile treating a missing `-j<N>` argument as `-j1` when +# calling meson / ninja. This happens when the `jobserver` class is used, since +# it manages parallelism in the MAKEFLAGS variable instead of PARALLEL_MAKE. +EXTRA_OEMAKE:append = "${@' -j' if d.getVar('JOBSERVER_FIFO') else ''}" + B = "${WORKDIR}/build" #EXTRA_OECONF:append = " --python=${HOSTTOOLS_DIR}/python3"
If the jobserver class is enabled, the PARALLEL_MAKE variable is unset in favor of configuring a shared jobserver in the MAKEFLAGS variable. However, the qemu makefile translates the missing `-j<N>` argument to `-j1` when calling into meson / ninja. Avoid this by setting `-j` without a value. For normal/GNU make, this can result in a fork bomb, but for ninja, it simply makes it use the jobserver fifa instead. Signed-off-by: Martin Hundebøll <martin@geanix.com> --- meta/recipes-devtools/qemu/qemu.inc | 5 +++++ 1 file changed, 5 insertions(+)