| Message ID | 20220607151123.3588947-4-ptsneves@gmail.com |
|---|---|
| State | Accepted, archived |
| Commit | 280f68528c93b5ffab888c99190accf59e807a3f |
| Headers | show |
| Series | [1/3] insane.bbclass: Make do_qa_staging check shebangs | expand |
Hi Paulo, On Tue, 7 Jun 2022 17:11:22 +0200 "Paulo Neves" <ptsneves@gmail.com> wrote: > Make sure we do not stage any executable with a bigger shebang > than 128. Fixes [1] > > [1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=11053 > > Signed-off-by: Paulo Neves <ptsneves@gmail.com> This check seems to be working very well!! It triggered a huge amount of build failures on the autobuilders due to libcheck having a shebang too long in the checkmk script, e.g.: #! /home/pokybuild/yocto-worker/genericx86-64-alt/build/build/tmp/work/x86_64-linux/libcheck-native/0.15.2-r0/recipe-sysroot-native/usr/bin/gawk -f Here are a few logs: https://autobuilder.yoctoproject.org/typhoon/#/builders/65/builds/5367/steps/14/logs/errors https://autobuilder.yoctoproject.org/typhoon/#/builders/65/builds/5367/steps/11/logs/errors https://autobuilder.yoctoproject.org/typhoon/#/builders/20/builds/5689/steps/11/logs/errors https://autobuilder.yoctoproject.org/typhoon/#/builders/37/builds/5340/steps/12/logs/errors It would be great if you could add another patch to your series to fix libcheck, and also to do 'bitbake world' to test as many packages as possible before discovering from the autobuilders. Thank you!
On 6/8/22 14:53, Luca Ceresoli wrote: > Hi Paulo, > > On Tue, 7 Jun 2022 17:11:22 +0200 > "Paulo Neves" <ptsneves@gmail.com> wrote: > >> Make sure we do not stage any executable with a bigger shebang >> than 128. Fixes [1] >> >> [1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=11053 >> >> Signed-off-by: Paulo Neves <ptsneves@gmail.com> > This check seems to be working very well!! It triggered a huge amount > of build failures on the autobuilders due to libcheck having a shebang > too long in the checkmk script, e.g.: > > #! /home/pokybuild/yocto-worker/genericx86-64-alt/build/build/tmp/work/x86_64-linux/libcheck-native/0.15.2-r0/recipe-sysroot-native/usr/bin/gawk -f > > Here are a few logs: > > https://autobuilder.yoctoproject.org/typhoon/#/builders/65/builds/5367/steps/14/logs/errors > https://autobuilder.yoctoproject.org/typhoon/#/builders/65/builds/5367/steps/11/logs/errors > https://autobuilder.yoctoproject.org/typhoon/#/builders/20/builds/5689/steps/11/logs/errors > https://autobuilder.yoctoproject.org/typhoon/#/builders/37/builds/5340/steps/12/logs/errors I am not seeing an immediate way to fix this optimally. The go-to solution to this class of issues is to just make the #!/usr/bin/env <interpreter>. The issue is that there is an extra -f, which with /usr/bin/env, will not work. The awk manual also implies this is not optional for standalone scripts. I think we can create a wrapper, or maybe we already have such a wrapper? > > It would be great if you could add another patch to your series to fix > libcheck, and also to do 'bitbake world' to test as many packages as > possible before discovering from the autobuilders. It takes quite a while on my computer and often i get out of disk space. I tried asking the linaro guys for a tuxsuite token but no answer yet. If you have some way of getting resources to make builds let me know. > Thank you! Thank you Paulo Neves
On Wed, 8 Jun 2022 16:45:20 +0200 "Paulo Neves" <ptsneves@gmail.com> wrote: > On 6/8/22 14:53, Luca Ceresoli wrote: > > Hi Paulo, > > > > On Tue, 7 Jun 2022 17:11:22 +0200 > > "Paulo Neves" <ptsneves@gmail.com> wrote: > > > >> Make sure we do not stage any executable with a bigger shebang > >> than 128. Fixes [1] > >> > >> [1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=11053 > >> > >> Signed-off-by: Paulo Neves <ptsneves@gmail.com> > > This check seems to be working very well!! It triggered a huge amount > > of build failures on the autobuilders due to libcheck having a shebang > > too long in the checkmk script, e.g.: > > > > #! /home/pokybuild/yocto-worker/genericx86-64-alt/build/build/tmp/work/x86_64-linux/libcheck-native/0.15.2-r0/recipe-sysroot-native/usr/bin/gawk -f > > > > Here are a few logs: > > > > https://autobuilder.yoctoproject.org/typhoon/#/builders/65/builds/5367/steps/14/logs/errors > > https://autobuilder.yoctoproject.org/typhoon/#/builders/65/builds/5367/steps/11/logs/errors > > https://autobuilder.yoctoproject.org/typhoon/#/builders/20/builds/5689/steps/11/logs/errors > > https://autobuilder.yoctoproject.org/typhoon/#/builders/37/builds/5340/steps/12/logs/errors > > I am not seeing an immediate way to fix this optimally. The go-to > solution to this class of issues is to just make > the #!/usr/bin/env <interpreter>. The issue is that there is an extra > -f, which with /usr/bin/env, will not work. The awk manual also implies > this is not optional for standalone scripts. I think we can create a > wrapper, or maybe we already have such a wrapper? > > > > It would be great if you could add another patch to your series to fix > > libcheck, and also to do 'bitbake world' to test as many packages as > > possible before discovering from the autobuilders. > It takes quite a while on my computer and often i get out of disk space. > I tried asking the linaro guys for a tuxsuite token but no answer yet. > If you have some way of getting resources to make builds let me know. Did you enable rm_work?
Hi Paulo, On Wed, 8 Jun 2022 14:53:05 +0200 "Luca Ceresoli via lists.openembedded.org" <luca.ceresoli=bootlin.com@lists.openembedded.org> wrote: > Hi Paulo, > > On Tue, 7 Jun 2022 17:11:22 +0200 > "Paulo Neves" <ptsneves@gmail.com> wrote: > > > Make sure we do not stage any executable with a bigger shebang > > than 128. Fixes [1] > > > > [1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=11053 > > > > Signed-off-by: Paulo Neves <ptsneves@gmail.com> > > This check seems to be working very well!! It triggered a huge amount > of build failures on the autobuilders due to libcheck having a shebang > too long in the checkmk script, e.g.: > > #! /home/pokybuild/yocto-worker/genericx86-64-alt/build/build/tmp/work/x86_64-linux/libcheck-native/0.15.2-r0/recipe-sysroot-native/usr/bin/gawk -f > > Here are a few logs: > > https://autobuilder.yoctoproject.org/typhoon/#/builders/65/builds/5367/steps/14/logs/errors > https://autobuilder.yoctoproject.org/typhoon/#/builders/65/builds/5367/steps/11/logs/errors > https://autobuilder.yoctoproject.org/typhoon/#/builders/20/builds/5689/steps/11/logs/errors > https://autobuilder.yoctoproject.org/typhoon/#/builders/37/builds/5340/steps/12/logs/errors > > It would be great if you could add another patch to your series to fix > libcheck, and also to do 'bitbake world' to test as many packages as > possible before discovering from the autobuilders. Here are more failures: https://autobuilder.yoctoproject.org/typhoon/#/builders/20/builds/5689/steps/12/logs/errors https://autobuilder.yoctoproject.org/typhoon/#/builders/20/builds/5689/steps/12/logs/stdio https://autobuilder.yoctoproject.org/typhoon/#/builders/62/builds/5340/steps/12/logs/errors https://autobuilder.yoctoproject.org/typhoon/#/builders/62/builds/5340/steps/12/logs/stdio https://autobuilder.yoctoproject.org/typhoon/#/builders/37/builds/5340/steps/13/logs/errors https://autobuilder.yoctoproject.org/typhoon/#/builders/37/builds/5340/steps/13/logs/stdio This time the error is: stdio: ERROR: core-image-sato-1.0-r0 do_testsdk: The toolchain <...> is not built. Build it before running the tests: 'bitbake <image> -c populate_sdk' . I'm not sure exactly how your code triggers such error, but it appeared when testing on the autobuilders with this patch series and disappeared when I removed only these 5 patches, thus it seems related.
Hey Luca, I sent a new patchset and from my tests it all should be good now. Paulo Neves On 6/8/22 17:06, Luca Ceresoli wrote: > Hi Paulo, > > On Wed, 8 Jun 2022 14:53:05 +0200 > "Luca Ceresoli via lists.openembedded.org" > <luca.ceresoli=bootlin.com@lists.openembedded.org> wrote: > >> Hi Paulo, >> >> On Tue, 7 Jun 2022 17:11:22 +0200 >> "Paulo Neves" <ptsneves@gmail.com> wrote: >> >>> Make sure we do not stage any executable with a bigger shebang >>> than 128. Fixes [1] >>> >>> [1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=11053 >>> >>> Signed-off-by: Paulo Neves <ptsneves@gmail.com> >> This check seems to be working very well!! It triggered a huge amount >> of build failures on the autobuilders due to libcheck having a shebang >> too long in the checkmk script, e.g.: >> >> #! /home/pokybuild/yocto-worker/genericx86-64-alt/build/build/tmp/work/x86_64-linux/libcheck-native/0.15.2-r0/recipe-sysroot-native/usr/bin/gawk -f >> >> Here are a few logs: >> >> https://autobuilder.yoctoproject.org/typhoon/#/builders/65/builds/5367/steps/14/logs/errors >> https://autobuilder.yoctoproject.org/typhoon/#/builders/65/builds/5367/steps/11/logs/errors >> https://autobuilder.yoctoproject.org/typhoon/#/builders/20/builds/5689/steps/11/logs/errors >> https://autobuilder.yoctoproject.org/typhoon/#/builders/37/builds/5340/steps/12/logs/errors >> >> It would be great if you could add another patch to your series to fix >> libcheck, and also to do 'bitbake world' to test as many packages as >> possible before discovering from the autobuilders. > Here are more failures: > > https://autobuilder.yoctoproject.org/typhoon/#/builders/20/builds/5689/steps/12/logs/errors > https://autobuilder.yoctoproject.org/typhoon/#/builders/20/builds/5689/steps/12/logs/stdio > https://autobuilder.yoctoproject.org/typhoon/#/builders/62/builds/5340/steps/12/logs/errors > https://autobuilder.yoctoproject.org/typhoon/#/builders/62/builds/5340/steps/12/logs/stdio > https://autobuilder.yoctoproject.org/typhoon/#/builders/37/builds/5340/steps/13/logs/errors > https://autobuilder.yoctoproject.org/typhoon/#/builders/37/builds/5340/steps/13/logs/stdio > > This time the error is: > > stdio: ERROR: core-image-sato-1.0-r0 do_testsdk: The toolchain <...> is not built. Build it before running the tests: 'bitbake <image> -c populate_sdk' . > > I'm not sure exactly how your code triggers such error, but it appeared > when testing on the autobuilders with this patch series and disappeared > when I removed only these 5 patches, thus it seems related. >
> + expected = "maximum shebang size exceeded, the maximum size is 128. [shebang-size]" > + res = bitbake("sysroot-shebang-test-native -c populate_sysroot", ignore_status=True) > + self.assertTrue(expected in res.output, msg=res.output) I err towards paranoia and would also check that res.status != 0. Ross IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
diff --git a/meta-selftest/recipes-test/sysroot-test/sysroot-shebang-test_1.0.bb b/meta-selftest/recipes-test/sysroot-test/sysroot-shebang-test_1.0.bb new file mode 100644 index 0000000000..6c834be897 --- /dev/null +++ b/meta-selftest/recipes-test/sysroot-test/sysroot-shebang-test_1.0.bb @@ -0,0 +1,12 @@ +SUMMARY = "Check that shebang does not exceed 128 characters" +LICENSE = "CLOSED" +INHIBIT_DEFAULT_DEPS = "1" + +EXCLUDE_FROM_WORLD = "1" +do_install() { + install -d ${D}${bindir} + echo '#!BiM3cnVd1Amtv6PG+FynrQiVMbZnX5ELgF21q3EkuB+44JEGWtq8TvBJ7EGidfVs3eR3wVOUbLnjYDlKUWcm7YC/ute7f+KDHbwxziRUSUBZAUqgjiQdfQ0HnxajI0ozbM863E9JV9k13yZKYfh9/zR77Y6Dl4Dd3zOWS75LSpkAXV' > ${D}${bindir}/max-shebang + chmod 755 ${D}${bindir}/max-shebang +} + +BBCLASSEXTEND = "native" diff --git a/meta/lib/oeqa/selftest/cases/sysroot.py b/meta/lib/oeqa/selftest/cases/sysroot.py index 315d1a61c2..9457f1e3ac 100644 --- a/meta/lib/oeqa/selftest/cases/sysroot.py +++ b/meta/lib/oeqa/selftest/cases/sysroot.py @@ -35,3 +35,13 @@ TESTSTRING:pn-sysroot-test-arch1 = "%s" TESTSTRING:pn-sysroot-test-arch2 = "%s" """ % (uuid1, uuid2)) bitbake("sysroot-test") + + def test_sysroot_max_shebang(self): + """ + Summary: Check max shebang triggers. To confirm [YOCTO #11053] is closed. + Expected: Fail when a shebang bigger than the max shebang-size is reached. + Author: Paulo Neves <ptsneves@gmail.com> + """ + expected = "maximum shebang size exceeded, the maximum size is 128. [shebang-size]" + res = bitbake("sysroot-shebang-test-native -c populate_sysroot", ignore_status=True) + self.assertTrue(expected in res.output, msg=res.output)
Make sure we do not stage any executable with a bigger shebang than 128. Fixes [1] [1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=11053 Signed-off-by: Paulo Neves <ptsneves@gmail.com> --- .../sysroot-test/sysroot-shebang-test_1.0.bb | 12 ++++++++++++ meta/lib/oeqa/selftest/cases/sysroot.py | 10 ++++++++++ 2 files changed, 22 insertions(+) create mode 100644 meta-selftest/recipes-test/sysroot-test/sysroot-shebang-test_1.0.bb