| Message ID | 20220614151105.1890454-6-ptsneves@gmail.com |
|---|---|
| State | Accepted, archived |
| Commit | 6edc1fffcbe1405d8c309a75643d7d6cd9a92848 |
| Headers | show |
| Series | [v5,1/7] python: Avoid shebang overflow on python-config.py | expand |
On 14 Jun 2022, at 16:11, Paulo Neves via lists.openembedded.org <ptsneves=gmail.com@lists.openembedded.org> wrote: > +create_cmdline_shebang_wrapper () { The indentation in this function is a mix of tabs and spaces, please unify. > + argument="$(basename "$(head -n1 $cmd | sed -e 's|#![ ]*||g' )”)” [ ]* seems overly, complex, no need for the brackets. I’d anchor to the beginning of the line, and remove the g as there’s only one expected. Might also be wise to assert that you found something here, so running this on an ELF fails with an obvious error. Ross IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
> -----Original Message----- > From: openembedded-core@lists.openembedded.org <openembedded- > core@lists.openembedded.org> On Behalf Of Ross Burton > Sent: den 15 juni 2022 12:52 > To: ptsneves@gmail.com > Cc: openembedded-core@lists.openembedded.org > Subject: Re: [OE-core] [PATCH v5 6/7] utils: Add cmdline_shebang_wrapper > util. > > On 14 Jun 2022, at 16:11, Paulo Neves via lists.openembedded.org > <ptsneves=gmail.com@lists.openembedded.org> wrote: > > +create_cmdline_shebang_wrapper () { > > The indentation in this function is a mix of tabs and spaces, please > unify. > > > + argument="$(basename "$(head -n1 $cmd | sed -e 's|#![ ]*||g' )”)” > > [ ]* seems overly, complex, no need for the brackets. I’d anchor to the > beginning of the line, and remove the g as there’s only one expected. This should work: argument=$(basename "$(sed -ne 's/^#! *//p;q')" $cmd) However, if the first line is something like "#!/usr/bin/env python3", then $argument will be "env python3" and somehow I assume that is not exactly what the rest of the code expects. > > Might also be wise to assert that you found something here, so running > this on an ELF fails with an obvious error. > > Ross //Peter
On 6/15/22 15:11, Peter Kjellerstedt wrote: >> -----Original Message----- >> From: openembedded-core@lists.openembedded.org <openembedded- >> core@lists.openembedded.org> On Behalf Of Ross Burton >> Sent: den 15 juni 2022 12:52 >> To: ptsneves@gmail.com >> Cc: openembedded-core@lists.openembedded.org >> Subject: Re: [OE-core] [PATCH v5 6/7] utils: Add cmdline_shebang_wrapper >> util. >> >> On 14 Jun 2022, at 16:11, Paulo Neves via lists.openembedded.org >> <ptsneves=gmail.com@lists.openembedded.org> wrote: >>> +create_cmdline_shebang_wrapper () { >> The indentation in this function is a mix of tabs and spaces, please >> unify. >> >>> + argument="$(basename "$(head -n1 $cmd | sed -e 's|#![ ]*||g' )”)” >> [ ]* seems overly, complex, no need for the brackets. I’d anchor to the >> beginning of the line, and remove the g as there’s only one expected. > This should work: > > argument=$(basename "$(sed -ne 's/^#! *//p;q')" $cmd) > > However, if the first line is something like "#!/usr/bin/env python3", then > $argument will be "env python3" and somehow I assume that is not exactly > what the rest of the code expects. The use case is that whatever is in shebang should be extracted and ran verbatim, so truncating like your suggestion can work but has slightly different semantic. >> Might also be wise to assert that you found something here, so running >> this on an ELF fails with an obvious error. >> >> Ross > //Peter > Paulo Neves
> -----Original Message----- > From: Paulo Neves <ptsneves@gmail.com> > Sent: den 15 juni 2022 15:51 > To: Peter Kjellerstedt <peter.kjellerstedt@axis.com>; Ross Burton > <ross.burton@arm.com> > Cc: openembedded-core@lists.openembedded.org > Subject: Re: [OE-core] [PATCH v5 6/7] utils: Add cmdline_shebang_wrapper > util. > > On 6/15/22 15:11, Peter Kjellerstedt wrote: > >> -----Original Message----- > >> From: openembedded-core@lists.openembedded.org <openembedded- > >> core@lists.openembedded.org> On Behalf Of Ross Burton > >> Sent: den 15 juni 2022 12:52 > >> To: ptsneves@gmail.com > >> Cc: openembedded-core@lists.openembedded.org > >> Subject: Re: [OE-core] [PATCH v5 6/7] utils: Add > cmdline_shebang_wrapper > >> util. > >> > >> On 14 Jun 2022, at 16:11, Paulo Neves via lists.openembedded.org > >> <ptsneves=gmail.com@lists.openembedded.org> wrote: > >>> +create_cmdline_shebang_wrapper () { > >> The indentation in this function is a mix of tabs and spaces, please > >> unify. > >> > >>> + argument="$(basename "$(head -n1 $cmd | sed -e 's|#![ ]*||g' )”)” > >> [ ]* seems overly, complex, no need for the brackets. I’d anchor to the > >> beginning of the line, and remove the g as there’s only one expected. > > This should work: > > > > argument=$(basename "$(sed -ne 's/^#! *//p;q')" $cmd) > > > > However, if the first line is something like "#!/usr/bin/env python3", then > > $argument will be "env python3" and somehow I assume that is not exactly > > what the rest of the code expects. > > The use case is that whatever is in shebang should be extracted and ran > verbatim, so truncating like your suggestion can work but has slightly > different semantic. Umm, my suggested line above should give exactly the same output as yours, only with fewer commands... > >> Might also be wise to assert that you found something here, so running > >> this on an ELF fails with an obvious error. > >> > >> Ross > > //Peter > > > Paulo Neves //Peter
You are right never mind. Paulo Neves On 6/16/22 00:16, Peter Kjellerstedt wrote: >> -----Original Message----- >> From: Paulo Neves <ptsneves@gmail.com> >> Sent: den 15 juni 2022 15:51 >> To: Peter Kjellerstedt <peter.kjellerstedt@axis.com>; Ross Burton >> <ross.burton@arm.com> >> Cc: openembedded-core@lists.openembedded.org >> Subject: Re: [OE-core] [PATCH v5 6/7] utils: Add cmdline_shebang_wrapper >> util. >> >> On 6/15/22 15:11, Peter Kjellerstedt wrote: >>>> -----Original Message----- >>>> From: openembedded-core@lists.openembedded.org <openembedded- >>>> core@lists.openembedded.org> On Behalf Of Ross Burton >>>> Sent: den 15 juni 2022 12:52 >>>> To: ptsneves@gmail.com >>>> Cc: openembedded-core@lists.openembedded.org >>>> Subject: Re: [OE-core] [PATCH v5 6/7] utils: Add >> cmdline_shebang_wrapper >>>> util. >>>> >>>> On 14 Jun 2022, at 16:11, Paulo Neves via lists.openembedded.org >>>> <ptsneves=gmail.com@lists.openembedded.org> wrote: >>>>> +create_cmdline_shebang_wrapper () { >>>> The indentation in this function is a mix of tabs and spaces, please >>>> unify. >>>> >>>>> + argument="$(basename "$(head -n1 $cmd | sed -e 's|#![ ]*||g' )”)” >>>> [ ]* seems overly, complex, no need for the brackets. I’d anchor to the >>>> beginning of the line, and remove the g as there’s only one expected. >>> This should work: >>> >>> argument=$(basename "$(sed -ne 's/^#! *//p;q')" $cmd) >>> >>> However, if the first line is something like "#!/usr/bin/env python3", then >>> $argument will be "env python3" and somehow I assume that is not exactly >>> what the rest of the code expects. >> The use case is that whatever is in shebang should be extracted and ran >> verbatim, so truncating like your suggestion can work but has slightly >> different semantic. > Umm, my suggested line above should give exactly the same output as yours, > only with fewer commands... > >>>> Might also be wise to assert that you found something here, so running >>>> this on an ELF fails with an obvious error. >>>> >>>> Ross >>> //Peter >>> >> Paulo Neves > //Peter >
diff --git a/meta-selftest/recipes-test/wrapper/cmdline-shebang-wrapper-test.bb b/meta-selftest/recipes-test/wrapper/cmdline-shebang-wrapper-test.bb new file mode 100644 index 0000000000..c4126a41fc --- /dev/null +++ b/meta-selftest/recipes-test/wrapper/cmdline-shebang-wrapper-test.bb @@ -0,0 +1,21 @@ +SUMMARY = "Check that create_cmdline_shebang works" +LICENSE = "MIT" +LIC_FILES_CHKSUM = "file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420" +INHIBIT_DEFAULT_DEPS = "1" + +SRC_URI += "file://test.awk" + +EXCLUDE_FROM_WORLD = "1" +do_install() { + install -d ${D}${bindir} + install -m 0755 ${WORKDIR}/test.awk ${D}${bindir}/test + sed -i -e 's|@AWK_BIN@|${bindir}/awk|g' ${D}${bindir}/test + create_cmdline_shebang_wrapper ${D}${bindir}/test + if [ $(${D}${bindir}/test) != "Don't Panic!" ]; then + bbfatal "Wrapper is broken" + else + bbnote "Wrapper is good" + fi +} + +BBCLASSEXTEND = "native" diff --git a/meta-selftest/recipes-test/wrapper/files/test.awk b/meta-selftest/recipes-test/wrapper/files/test.awk new file mode 100644 index 0000000000..91429197b1 --- /dev/null +++ b/meta-selftest/recipes-test/wrapper/files/test.awk @@ -0,0 +1,2 @@ +#! @AWK_BIN@ -f +BEGIN { print "Don't Panic!" } diff --git a/meta/classes/utils.bbclass b/meta/classes/utils.bbclass index b4eb3d38ab..b617632d9f 100644 --- a/meta/classes/utils.bbclass +++ b/meta/classes/utils.bbclass @@ -184,6 +184,40 @@ END chmod +x $cmd } +create_cmdline_shebang_wrapper () { + # Create a wrapper script where commandline options are needed + # + # These are useful to work around shebang relocation issues, where shebangs are too + # long or have arguments in them, thus preventing them from using the /usr/bin/env + # shebang + # + # Usage: create_cmdline_wrapper FILENAME <extra-options> + + cmd=$1 + shift + + echo "Generating wrapper script for $cmd" + + # Strip #! and get remaining interpreter + arg + argument="$(basename "$(head -n1 $cmd | sed -e 's|#![ ]*||g' )")" + # strip the shebang from the real script as we do not want it to be usable anyway + tail -n +2 $cmd > $cmd.real + cmdname=$(basename $cmd) + dirname=$(dirname $cmd) + cmdoptions=$@ + if [ "${base_prefix}" != "" ]; then + relpath=`python3 -c "import os; print(os.path.relpath('${D}${base_prefix}', '$dirname'))"` + cmdoptions=`echo $@ | sed -e "s:${base_prefix}:\\$realdir/$relpath:g"` + fi + cat <<END >$cmd +#!/usr/bin/env bash +realpath=\`readlink -fn \$0\` +realdir=\`dirname \$realpath\` +exec -a \$realdir/$cmdname $argument \$realdir/$cmdname.real $cmdoptions "\$@" +END + chmod +x $cmd +} + create_wrapper () { # Create a wrapper script where extra environment variables are needed # diff --git a/meta/lib/oeqa/selftest/cases/wrapper.py b/meta/lib/oeqa/selftest/cases/wrapper.py new file mode 100644 index 0000000000..6de63310c0 --- /dev/null +++ b/meta/lib/oeqa/selftest/cases/wrapper.py @@ -0,0 +1,11 @@ +from oeqa.selftest.case import OESelftestTestCase +from oeqa.utils.commands import bitbake + +class WrapperTests(OESelftestTestCase): + def test_shebang_wrapper(self): + """ + Summary: Build a recipe which will fail if the cmdline_shebang_wrapper function is defective. + Expected: Exit status to be 0. + Author: Paulo Neves <ptsneves@gmail.com> + """ + res = bitbake("cmdline-shebang-wrapper-test -c install", ignore_status=False)