| Message ID | 20220520121617.2993631-1-thomas.perrot@bootlin.com |
|---|---|
| State | New |
| Headers | show |
| Series | webkitgtk: limit the number of compilation threads | expand |
On 5/20/22 14:16, Thomas Perrot via lists.openembedded.org wrote: > Otherwise the compilation may lead to out-of-memories. > > Signed-off-by: Thomas Perrot <thomas.perrot@bootlin.com> > --- > meta/recipes-sato/webkit/webkitgtk_2.36.1.bb | 3 ++- > 1 file changed, 2 insertions(+), 1 deletion(-) > > diff --git a/meta/recipes-sato/webkit/webkitgtk_2.36.1.bb b/meta/recipes-sato/webkit/webkitgtk_2.36.1.bb > index 65757c36a733..e7cfe49e2b87 100644 > --- a/meta/recipes-sato/webkit/webkitgtk_2.36.1.bb > +++ b/meta/recipes-sato/webkit/webkitgtk_2.36.1.bb > @@ -80,6 +80,8 @@ setup_python_link() { > fi > } > > +PARALLEL_MAKE = "" > + > EXTRA_OECMAKE = " \ > -DPORT=GTK \ > ${@bb.utils.contains('GI_DATA_ENABLED', 'True', '-DENABLE_INTROSPECTION=ON', '-DENABLE_INTROSPECTION=OFF', d)} \ > @@ -164,4 +166,3 @@ src_package_preprocess () { > ${B}/WebKit2Gtk/DerivedSources/webkit2/*.h > > } > - > Why punish build machines with lots of RAM ? Maybe it could be set depending on available RAM/number of cores ? Jacob
The correct way to address this is to restrict the amount of webkit make jobs in site.conf, specific to the build host. Or add RAM. Ale On Fri 20. May 2022 at 14.20, Jacob Kroon <jacob.kroon@gmail.com> wrote: > On 5/20/22 14:16, Thomas Perrot via lists.openembedded.org wrote: > > Otherwise the compilation may lead to out-of-memories. > > > > Signed-off-by: Thomas Perrot <thomas.perrot@bootlin.com> > > --- > > meta/recipes-sato/webkit/webkitgtk_2.36.1.bb | 3 ++- > > 1 file changed, 2 insertions(+), 1 deletion(-) > > > > diff --git a/meta/recipes-sato/webkit/webkitgtk_2.36.1.bb > b/meta/recipes-sato/webkit/webkitgtk_2.36.1.bb > > index 65757c36a733..e7cfe49e2b87 100644 > > --- a/meta/recipes-sato/webkit/webkitgtk_2.36.1.bb > > +++ b/meta/recipes-sato/webkit/webkitgtk_2.36.1.bb > > @@ -80,6 +80,8 @@ setup_python_link() { > > fi > > } > > > > +PARALLEL_MAKE = "" > > + > > EXTRA_OECMAKE = " \ > > -DPORT=GTK \ > > ${@bb.utils.contains('GI_DATA_ENABLED', 'True', > '-DENABLE_INTROSPECTION=ON', '-DENABLE_INTROSPECTION=OFF', d)} \ > > @@ -164,4 +166,3 @@ src_package_preprocess () { > > ${B}/WebKit2Gtk/DerivedSources/webkit2/*.h > > > > } > > - > > > > Why punish build machines with lots of RAM ? Maybe it could be set > depending on available RAM/number of cores ? > > Jacob > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#165928): > https://lists.openembedded.org/g/openembedded-core/message/165928 > Mute This Topic: https://lists.openembedded.org/mt/91229473/1686489 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ > alex.kanavin@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- > >
Hello Jacob, On Fri, 2022-05-20 at 14:19 +0200, Jacob Kroon wrote: > On 5/20/22 14:16, Thomas Perrot via lists.openembedded.org wrote: > > Otherwise the compilation may lead to out-of-memories. > > > > Signed-off-by: Thomas Perrot <thomas.perrot@bootlin.com> > > --- > > meta/recipes-sato/webkit/webkitgtk_2.36.1.bb | 3 ++- > > 1 file changed, 2 insertions(+), 1 deletion(-) > > > > diff --git a/meta/recipes-sato/webkit/webkitgtk_2.36.1.bb > > b/meta/recipes-sato/webkit/webkitgtk_2.36.1.bb > > index 65757c36a733..e7cfe49e2b87 100644 > > --- a/meta/recipes-sato/webkit/webkitgtk_2.36.1.bb > > +++ b/meta/recipes-sato/webkit/webkitgtk_2.36.1.bb > > @@ -80,6 +80,8 @@ setup_python_link() { > > fi > > } > > > > +PARALLEL_MAKE = "" > > + > > EXTRA_OECMAKE = " \ > > -DPORT=GTK \ > > ${@bb.utils.contains('GI_DATA_ENABLED', 'True', '- > > DENABLE_INTROSPECTION=ON', '-DENABLE_INTROSPECTION=OFF', d)} \ > > @@ -164,4 +166,3 @@ src_package_preprocess () { > > ${B}/WebKit2Gtk/DerivedSources/webkit2/*.h > > > > } > > - > > > > Why punish build machines with lots of RAM ? Maybe it could be set > depending on available RAM/number of cores ? > Why not. I tested on a build server with more than ~12GB ram available and 16 cores, then I tried to reduce the number of threads and when the value is greater than 1 the build fails. Kind regards, Thomas Perrot > Jacob > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#165928): > https://lists.openembedded.org/g/openembedded-core/message/165928 > Mute This Topic: https://lists.openembedded.org/mt/91229473/5443093 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: > https://lists.openembedded.org/g/openembedded-core/unsub [ > thomas.perrot@bootlin.com] > -=-=-=-=-=-=-=-=-=-=-=- >
On 5/20/22 14:30, Thomas Perrot wrote: > Hello Jacob, > > On Fri, 2022-05-20 at 14:19 +0200, Jacob Kroon wrote: >> On 5/20/22 14:16, Thomas Perrot via lists.openembedded.org wrote: >>> Otherwise the compilation may lead to out-of-memories. >>> >>> Signed-off-by: Thomas Perrot <thomas.perrot@bootlin.com> >>> --- >>> meta/recipes-sato/webkit/webkitgtk_2.36.1.bb | 3 ++- >>> 1 file changed, 2 insertions(+), 1 deletion(-) >>> >>> diff --git a/meta/recipes-sato/webkit/webkitgtk_2.36.1.bb >>> b/meta/recipes-sato/webkit/webkitgtk_2.36.1.bb >>> index 65757c36a733..e7cfe49e2b87 100644 >>> --- a/meta/recipes-sato/webkit/webkitgtk_2.36.1.bb >>> +++ b/meta/recipes-sato/webkit/webkitgtk_2.36.1.bb >>> @@ -80,6 +80,8 @@ setup_python_link() { >>> fi >>> } >>> >>> +PARALLEL_MAKE = "" >>> + >>> EXTRA_OECMAKE = " \ >>> -DPORT=GTK \ >>> ${@bb.utils.contains('GI_DATA_ENABLED', 'True', '- >>> DENABLE_INTROSPECTION=ON', '-DENABLE_INTROSPECTION=OFF', d)} \ >>> @@ -164,4 +166,3 @@ src_package_preprocess () { >>> ${B}/WebKit2Gtk/DerivedSources/webkit2/*.h >>> >>> } >>> - >>> >> >> Why punish build machines with lots of RAM ? Maybe it could be set >> depending on available RAM/number of cores ? >> > > Why not. > Because it slows down the build for them, which is not good. > I tested on a build server with more than ~12GB ram available and 16 > cores, then I tried to reduce the number of threads and when the value > is greater than 1 the build fails. > 12GB RAM does not sound like much for a 16 core system. Jacob
Hi Alexander, On Fri, 2022-05-20 at 14:27 +0200, Alexander Kanavin wrote: > The correct way to address this is to restrict the amount of webkit > make jobs in site.conf, specific to the build host. Or add RAM. > I heard you Kind regards, Thomas Perrot > Ale > > On Fri 20. May 2022 at 14.20, Jacob Kroon <jacob.kroon@gmail.com> > wrote: > > On 5/20/22 14:16, Thomas Perrot via lists.openembedded.org wrote: > > > Otherwise the compilation may lead to out-of-memories. > > > > > > Signed-off-by: Thomas Perrot <thomas.perrot@bootlin.com> > > > --- > > > meta/recipes-sato/webkit/webkitgtk_2.36.1.bb | 3 ++- > > > 1 file changed, 2 insertions(+), 1 deletion(-) > > > > > > diff --git a/meta/recipes-sato/webkit/webkitgtk_2.36.1.bb > > b/meta/recipes-sato/webkit/webkitgtk_2.36.1.bb > > > index 65757c36a733..e7cfe49e2b87 100644 > > > --- a/meta/recipes-sato/webkit/webkitgtk_2.36.1.bb > > > +++ b/meta/recipes-sato/webkit/webkitgtk_2.36.1.bb > > > @@ -80,6 +80,8 @@ setup_python_link() { > > > fi > > > } > > > > > > +PARALLEL_MAKE = "" > > > + > > > EXTRA_OECMAKE = " \ > > > -DPORT=GTK \ > > > ${@bb.utils.contains('GI_DATA_ENABLED', 'True', '- > > DENABLE_INTROSPECTION=ON', '-DENABLE_INTROSPECTION=OFF', d)} \ > > > @@ -164,4 +166,3 @@ src_package_preprocess () { > > > ${B}/WebKit2Gtk/DerivedSources/webkit2/*.h > > > > > > } > > > - > > > > > > > Why punish build machines with lots of RAM ? Maybe it could be set > > depending on available RAM/number of cores ? > > > > Jacob > > > > > > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#165929): > https://lists.openembedded.org/g/openembedded-core/message/165929 > Mute This Topic: https://lists.openembedded.org/mt/91229473/5443093 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: > https://lists.openembedded.org/g/openembedded-core/unsub [ > thomas.perrot@bootlin.com] > -=-=-=-=-=-=-=-=-=-=-=- >
On Fri, 2022-05-20 at 14:35 +0200, Thomas Perrot via lists.openembedded.org wrote: > Hi Alexander, > > On Fri, 2022-05-20 at 14:27 +0200, Alexander Kanavin wrote: > > The correct way to address this is to restrict the amount of webkit > > make jobs in site.conf, specific to the build host. Or add RAM. > > > > I heard you I do agree with Alex and Jacob here. Whilst this may not work on your system, the same could be said for other recipes too. If you're going to run builds on resource constrained systems, you will need to configure the parallelism options accordingly and setting limits on webkit isn't unusual. We do set load based limits on paralellism on the autobuilder for example. Cheers, Richard
diff --git a/meta/recipes-sato/webkit/webkitgtk_2.36.1.bb b/meta/recipes-sato/webkit/webkitgtk_2.36.1.bb index 65757c36a733..e7cfe49e2b87 100644 --- a/meta/recipes-sato/webkit/webkitgtk_2.36.1.bb +++ b/meta/recipes-sato/webkit/webkitgtk_2.36.1.bb @@ -80,6 +80,8 @@ setup_python_link() { fi } +PARALLEL_MAKE = "" + EXTRA_OECMAKE = " \ -DPORT=GTK \ ${@bb.utils.contains('GI_DATA_ENABLED', 'True', '-DENABLE_INTROSPECTION=ON', '-DENABLE_INTROSPECTION=OFF', d)} \ @@ -164,4 +166,3 @@ src_package_preprocess () { ${B}/WebKit2Gtk/DerivedSources/webkit2/*.h } -
Otherwise the compilation may lead to out-of-memories. Signed-off-by: Thomas Perrot <thomas.perrot@bootlin.com> --- meta/recipes-sato/webkit/webkitgtk_2.36.1.bb | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)