| Message ID | 20220622191615.10168-1-niko.mauno@vaisala.com |
|---|---|
| State | Accepted, archived |
| Commit | 5c09d2a017add8bd34142c6c1543e2d0f9ead286 |
| Headers | show |
| Series | [1/3] ptest.bbclass: Honor PARALLEL_MAKE, PARALLEL_MAKEINST | expand |
On Wed, 2022-06-22 at 22:16 +0300, niko.mauno@vaisala.com via lists.openembedded.org wrote: > Since oe_runmake wrapper doesn't by default involve PARALLEL_MAKE > outside do_compile() nor PARALLEL_MAKEINST outside do_install(), > enable parallellized make by default when oe_runmake is invoked from > do_compile_ptest() or do_install_ptest() by declaring wrapper task > specific EXTRA_OEMAKE overrides in fashion similar to do_compile and > do_install overrides in meta/conf/bitbake.conf. > > Parallel make can still be disabled by resetting bbclass specific > PTEST_PARALLEL_MAKE and PTEST_PARALLEL_MAKEINST variables in recipe > e.g. if a race issue needs to be avoided without modifying source code. > > Tested by issuing following command sequence on a 32-core build host: > > $ bitbake -c clean util-linux && bitbake --skip-setscene -c compile util-linux && time bitbake --skip-setscene -c compile_ptest_base util-linux > > and found that before this change the result was > > real 0m34.684s > user 0m0.753s > sys 0m0.131s > > and after this change > > real 0m9.868s > user 0m0.749s > sys 0m0.150s > > Signed-off-by: Niko Mauno <niko.mauno@vaisala.com> > --- > meta/classes/ptest.bbclass | 4 ++++ > 1 file changed, 4 insertions(+) This has merged but is causing a few problems. So far, elfutils and flex both ran into make races with a ton of failed builds as a result. I've a patch to disable paralell compile for those two recipes in the ptest compile. Ideally we'd look into and get this issues fixed upstream. Some random build failure links (there are many more in that build): https://autobuilder.yoctoproject.org/typhoon/#/builders/100/builds/3045 https://autobuilder.yoctoproject.org/typhoon/#/builders/65/builds/5446 https://autobuilder.yoctoproject.org/typhoon/#/builders/62/builds/5417 https://autobuilder.yoctoproject.org/typhoon/#/builders/95/builds/3428 Cheers, Richard
diff --git a/meta/classes/ptest.bbclass b/meta/classes/ptest.bbclass index 1ec23c0923..c162f5d934 100644 --- a/meta/classes/ptest.bbclass +++ b/meta/classes/ptest.bbclass @@ -5,6 +5,10 @@ This package contains a test directory ${PTEST_PATH} for package test purposes." PTEST_PATH ?= "${libdir}/${BPN}/ptest" PTEST_BUILD_HOST_FILES ?= "Makefile" PTEST_BUILD_HOST_PATTERN ?= "" +PTEST_PARALLEL_MAKE ?= "${PARALLEL_MAKE}" +PTEST_PARALLEL_MAKEINST ?= "${PARALLEL_MAKEINST}" +EXTRA_OEMAKE:prepend:task-compile-ptest-base = "${PTEST_PARALLEL_MAKE} " +EXTRA_OEMAKE:prepend:task-install-ptest-base = "${PTEST_PARALLEL_MAKEINST} " FILES:${PN}-ptest += "${PTEST_PATH}" SECTION:${PN}-ptest = "devel"
Since oe_runmake wrapper doesn't by default involve PARALLEL_MAKE outside do_compile() nor PARALLEL_MAKEINST outside do_install(), enable parallellized make by default when oe_runmake is invoked from do_compile_ptest() or do_install_ptest() by declaring wrapper task specific EXTRA_OEMAKE overrides in fashion similar to do_compile and do_install overrides in meta/conf/bitbake.conf. Parallel make can still be disabled by resetting bbclass specific PTEST_PARALLEL_MAKE and PTEST_PARALLEL_MAKEINST variables in recipe e.g. if a race issue needs to be avoided without modifying source code. Tested by issuing following command sequence on a 32-core build host: $ bitbake -c clean util-linux && bitbake --skip-setscene -c compile util-linux && time bitbake --skip-setscene -c compile_ptest_base util-linux and found that before this change the result was real 0m34.684s user 0m0.753s sys 0m0.131s and after this change real 0m9.868s user 0m0.749s sys 0m0.150s Signed-off-by: Niko Mauno <niko.mauno@vaisala.com> --- meta/classes/ptest.bbclass | 4 ++++ 1 file changed, 4 insertions(+)