Message ID | 20220806090714.2001241-1-Martin.Jansa@gmail.com |
---|---|
State | New |
Headers | show |
Series | [RFC,DO,NOT,MERGE] insane: check for 'unsafe for cross-compilation' in log.do_compile and log.do_install again | expand |
Hi Martin, Martin Jansa <Martin.Jansa@gmail.com> escreveu no dia sábado, 6/08/2022 à(s) 10:07: > * only a RFC to discuss what might prevent > --enable-poison-system-directories=true in gcc-cross to work or to > test this on AB to see if there are other places where these issues > sneaked in without being detected > > * it was removed in: > > https://git.openembedded.org/openembedded-core/commit/?id=a67e9ebfd5b8002fd4a7d8d27ff0d997817f76e1 > but looks like the "follow-up patches": > d039d6fbfc gcc-cross: make use of the system include directories fatal > 1fcaa5f452 gcc: add an option for --enable-poison-system-directories > to be fatal > don't work anymore? I was checking if e.g. -Wno-error is used in > webkitgtk to disable them, but it's not there, so something else went > wrong, because in webkit-gtk log.do_compile I've just noticed: > > # grep unsafe > poky/build/tmp/work/core2-64-poky-linux/webkitgtk/2.36.4-r0/temp/log.do_compile.3901410 > | sort | uniq -c > 6 cc1: warning: include location "/usr/include/gstreamer-1.0" is > unsafe for cross-compilation [-Wpoison-system-directories] > 1267 cc1plus: warning: include location "/usr/include/gstreamer-1.0" is > unsafe for cross-compilation [-Wpoison-system-directories] > > * and with this it confirms the same in webkitgtk: > > ERROR: webkitgtk-2.36.4-r0 do_compile: QA Issue: This > /OE/build/oe-core/tmp-glibc/work/core2-64-oe-linux/webkitgtk/2.36.4-r0/temp/log.do_compile > log indicates 'unsafe for cross-compilation' or > '-Wpoison-system-directories' errors, it looked at host include and/or > library paths [compile-unsafe] > > and this was the only error (when building webkitgtk from scratch) > > Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> > --- > meta/classes/insane.bbclass | 38 ++++++++++++++++++++++++++++++++----- > 1 file changed, 33 insertions(+), 5 deletions(-) > > diff --git a/meta/classes/insane.bbclass b/meta/classes/insane.bbclass > index c8b434bb54..9ce5ae0f6f 100644 > --- a/meta/classes/insane.bbclass > +++ b/meta/classes/insane.bbclass > @@ -35,8 +35,8 @@ ERROR_QA ?= "dev-so debug-deps dev-deps debug-files arch > pkgconfig la \ > version-going-backwards expanded-d invalid-chars \ > license-checksum dev-elf file-rdeps configure-unsafe \ > configure-gettext perllocalpod shebang-size \ > - already-stripped installed-vs-shipped ldflags > compile-host-path \ > - install-host-path pn-overrides unknown-configure-option \ > + already-stripped installed-vs-shipped ldflags compile-unsafe \ > + install-unsafe pn-overrides unknown-configure-option \ > useless-rpaths rpaths staticdev empty-dirs \ > " > # Add usrmerge QA check based on distro feature > @@ -1231,6 +1231,31 @@ python do_qa_patch() { > bb.error("Missing Upstream-Status in patch\n%s\nPlease > add according to %s ." % (fullpath, guidelines)) > } > > +def check_poisoned_system_directories(file): > + import subprocess > + > + statement = "grep -q -F -e 'is unsafe for cross-compilation' %s" % > file > + if subprocess.call(statement, shell=True) == 0: > + return """This %s log indicates 'unsafe for cross-compilation' > or '-Wpoison-system-directories' errors, it looked at host include and/or > library paths""" % file > + > +python do_qa_compile() { > + log = os.path.join(d.getVar('T'),"log.do_compile") > + skip = (d.getVar('INSANE_SKIP') or "").split() > + if 'compile-unsafe' not in skip and os.path.exists(log): > + error_msg = check_poisoned_system_directories(log) > + if error_msg: > + oe.qa.handle_error("compile-unsafe", error_msg, d) > +} > + > +python do_qa_install() { > + log = os.path.join(d.getVar('T'),"log.do_install") > + skip = (d.getVar('INSANE_SKIP') or "").split() > + if 'install-unsafe' not in skip and os.path.exists(log): > + error_msg = check_poisoned_system_directories(log) > + if error_msg: > + oe.qa.handle_error("install-unsafe", error_msg, d) > +} > + > python do_qa_configure() { > import subprocess > > @@ -1253,9 +1278,8 @@ python do_qa_configure() { > statement = "grep -q -F -e 'is unsafe for cross-compilation' > %s" % \ > os.path.join(root,"config.log") > if "config.log" in files: > - if subprocess.call(statement, shell=True) == 0: > - error_msg = """This autoconf log indicates errors, it > looked at host include and/or library paths while determining system > capabilities. > -Rerun configure task after fixing this.""" > + error_msg = > check_poisoned_system_directories(os.path.join(root,"config.log")) > + if error_msg: > oe.qa.handle_error("configure-unsafe", error_msg, d) > > if "configure.ac" in files: > @@ -1356,6 +1380,10 @@ do_patch[postfuncs] += "do_qa_patch " > #addtask qa_configure after do_configure before do_compile > do_configure[postfuncs] += "do_qa_configure " > > +# Check poison-system-directories in log.do_compile and log.do_install > +do_compile[postfuncs] += "do_qa_compile" > +do_compile[postfuncs] += "do_qa_install" > I think this is a typo and the second do_compile is do_install Jose + > # Check does S exist. > do_unpack[postfuncs] += "do_qa_unpack" > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#168969): > https://lists.openembedded.org/g/openembedded-core/message/168969 > Mute This Topic: https://lists.openembedded.org/mt/92851436/5052612 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ > quaresma.jose@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- > >
FWIW: boost can also trigger this issue now: "x86_64-webos-linux-g++" "-m64" "-march=core2" "-mtune=core2" "-msse3" "-mfpmath=sse" "-Wl,-O1" "-Wl,--hash-style=gnu" "-Wl,--as-needed" "-fmacro-prefix-map=/OE/build/luneos-langdale/webos-ports/tmp-glibc/work/core2-64-webos-linux/boost/1.80.0-r0/boost_1_80_0=/usr/src/debug/boost/1.80.0-r0" "-fdebug-prefix-map=/OE/build/luneos-langdale/webos-ports/tmp-glibc/work/core2-64-webos-linux/boost/1.80.0-r0/boost_1_80_0=/usr/src/debug/boost/1.80.0-r0" "-fmacro-prefix-map=/OE/build/luneos-langdale/webos-ports/tmp-glibc/work/core2-64-webos-linux/boost/1.80.0-r0/build=/usr/src/debug/boost/1.80.0-r0" "-fdebug-prefix-map=/OE/build/luneos-langdale/webos-ports/tmp-glibc/work/core2-64-webos-linux/boost/1.80.0-r0/build=/usr/src/debug/boost/1.80.0-r0" "-fdebug-prefix-map=/OE/build/luneos-langdale/webos-ports/tmp-glibc/work/core2-64-webos-linux/boost/1.80.0-r0/recipe-sysroot=" "-fmacro-prefix-map=/OE/build/luneos-langdale/webos-ports/tmp-glibc/work/core2-64-webos-linux/boost/1.80.0-r0/recipe-sysroot=" "-fdebug-prefix-map=/OE/build/luneos-langdale/webos-ports/tmp-glibc/work/core2-64-webos-linux/boost/1.80.0-r0/recipe-sysroot-native=" "--sysroot=/OE/build/luneos-langdale/webos-ports/tmp-glibc/work/core2-64-webos-linux/boost/1.80.0-r0/recipe-sysroot" -fvisibility-inlines-hidden -O2 -pipe -g -feliminate-unused-debug-types -fmacro-prefix-map=/OE/build/luneos-langdale/webos-ports/tmp-glibc/work/core2-64-webos-linux/boost/1.80.0-r0/boost_1_80_0=/usr/src/debug/boost/1.80.0-r0 -fdebug-prefix-map=/OE/build/luneos-langdale/webos-ports/tmp-glibc/work/core2-64-webos-linux/boost/1.80.0-r0/boost_1_80_0=/usr/src/debug/boost/1.80.0-r0 -fmacro-prefix-map=/OE/build/luneos-langdale/webos-ports/tmp-glibc/work/core2-64-webos-linux/boost/1.80.0-r0/build=/usr/src/debug/boost/1.80.0-r0 -fdebug-prefix-map=/OE/build/luneos-langdale/webos-ports/tmp-glibc/work/core2-64-webos-linux/boost/1.80.0-r0/build=/usr/src/debug/boost/1.80.0-r0 -fdebug-prefix-map=/OE/build/luneos-langdale/webos-ports/tmp-glibc/work/core2-64-webos-linux/boost/1.80.0-r0/recipe-sysroot= -fmacro-prefix-map=/OE/build/luneos-langdale/webos-ports/tmp-glibc/work/core2-64-webos-linux/boost/1.80.0-r0/recipe-sysroot= -fdebug-prefix-map=/OE/build/luneos-langdale/webos-ports/tmp-glibc/work/core2-64-webos-linux/boost/1.80.0-r0/recipe-sysroot-native= -fvisibility-inlines-hidden -m64 -pthread -O3 -finline-functions -Wno-inline -Wall -fvisibility=hidden -Wno-long-long -Wno-deprecated-declarations -DBOOST_ALL_NO_LIB=1 -DBOOST_CHRONO_STATIC_LINK=1 -DBOOST_LOCALE_NO_WINAPI_BACKEND=1 -DBOOST_LOCALE_WITH_ICONV=1 -DBOOST_LOCALE_WITH_ICU=1 -DBOOST_THREAD_BUILD_LIB=1 -DBOOST_THREAD_NO_LIB=1 -DBOOST_THREAD_POSIX -DBOOST_THREAD_USE_LIB=1 -DNDEBUG -I"." -I"/OE/build/luneos-langdale/webos-ports/tmp-glibc/work/core2-64-webos-linux/boost/1.80.0-r0/boost_1_80_0/libs/locale/src" -I"/usr/include" -c -o "/OE/build/luneos-langdale/webos-ports/tmp-glibc/work/core2-64-webos-linux/boost/1.80.0-r0/build/boost/bin.v2/libs/locale/build/f2e0de3f303a0b3f8e80823b216d5db4/posix/converter.o" "/OE/build/luneos-langdale/webos-ports/tmp-glibc/work/core2-64-webos-linux/boost/1.80.0-r0/boost_1_80_0/libs/locale/src/boost/locale/posix/converter.cpp" cc1plus: warning: include location "/usr/include" is unsafe for cross-compilation [-Wpoison-system-directories] On Sat, Aug 6, 2022 at 2:55 PM Martin Jansa via lists.openembedded.org <Martin.Jansa=gmail.com@lists.openembedded.org> wrote: > * it was removed in: > > https://git.openembedded.org/openembedded-core/commit/?id=a67e9ebfd5b8002fd4a7d8d27ff0d997817f76e1 > but looks like the "follow-up patches": > d039d6fbfc gcc-cross: make use of the system include directories fatal > 1fcaa5f452 gcc: add an option for --enable-poison-system-directories > to be fatal > don't work anymore? I was checking if e.g. -Wno-error is used in > webkitgtk to disable them, but it's not there, so something else went > wrong, because in webkit-gtk log.do_compile I've just noticed: > > # grep unsafe > poky/build/tmp/work/core2-64-poky-linux/webkitgtk/2.36.4-r0/temp/log.do_compile.3901410 > | sort | uniq -c > 6 cc1: warning: include location "/usr/include/gstreamer-1.0" is > unsafe for cross-compilation [-Wpoison-system-directories] > 1267 cc1plus: warning: include location "/usr/include/gstreamer-1.0" is > unsafe for cross-compilation [-Wpoison-system-directories] > > Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> > --- > > v2: fixed copy&paste issue in postfuncs reported by Jose > > meta/classes/insane.bbclass | 38 ++++++++++++++++++++++++++++++++----- > 1 file changed, 33 insertions(+), 5 deletions(-) > > diff --git a/meta/classes/insane.bbclass b/meta/classes/insane.bbclass > index c8b434bb54..2cb730111b 100644 > --- a/meta/classes/insane.bbclass > +++ b/meta/classes/insane.bbclass > @@ -35,8 +35,8 @@ ERROR_QA ?= "dev-so debug-deps dev-deps debug-files arch > pkgconfig la \ > version-going-backwards expanded-d invalid-chars \ > license-checksum dev-elf file-rdeps configure-unsafe \ > configure-gettext perllocalpod shebang-size \ > - already-stripped installed-vs-shipped ldflags > compile-host-path \ > - install-host-path pn-overrides unknown-configure-option \ > + already-stripped installed-vs-shipped ldflags compile-unsafe \ > + install-unsafe pn-overrides unknown-configure-option \ > useless-rpaths rpaths staticdev empty-dirs \ > " > # Add usrmerge QA check based on distro feature > @@ -1231,6 +1231,31 @@ python do_qa_patch() { > bb.error("Missing Upstream-Status in patch\n%s\nPlease > add according to %s ." % (fullpath, guidelines)) > } > > +def check_poisoned_system_directories(file): > + import subprocess > + > + statement = "grep -q -F -e 'is unsafe for cross-compilation' %s" % > file > + if subprocess.call(statement, shell=True) == 0: > + return """This %s log indicates 'unsafe for cross-compilation' > or '-Wpoison-system-directories' errors, it looked at host include and/or > library paths""" % file > + > +python do_qa_compile() { > + log = os.path.join(d.getVar('T'),"log.do_compile") > + skip = (d.getVar('INSANE_SKIP') or "").split() > + if 'compile-unsafe' not in skip and os.path.exists(log): > + error_msg = check_poisoned_system_directories(log) > + if error_msg: > + oe.qa.handle_error("compile-unsafe", error_msg, d) > +} > + > +python do_qa_install() { > + log = os.path.join(d.getVar('T'),"log.do_install") > + skip = (d.getVar('INSANE_SKIP') or "").split() > + if 'install-unsafe' not in skip and os.path.exists(log): > + error_msg = check_poisoned_system_directories(log) > + if error_msg: > + oe.qa.handle_error("install-unsafe", error_msg, d) > +} > + > python do_qa_configure() { > import subprocess > > @@ -1253,9 +1278,8 @@ python do_qa_configure() { > statement = "grep -q -F -e 'is unsafe for cross-compilation' > %s" % \ > os.path.join(root,"config.log") > if "config.log" in files: > - if subprocess.call(statement, shell=True) == 0: > - error_msg = """This autoconf log indicates errors, it > looked at host include and/or library paths while determining system > capabilities. > -Rerun configure task after fixing this.""" > + error_msg = > check_poisoned_system_directories(os.path.join(root,"config.log")) > + if error_msg: > oe.qa.handle_error("configure-unsafe", error_msg, d) > > if "configure.ac" in files: > @@ -1356,6 +1380,10 @@ do_patch[postfuncs] += "do_qa_patch " > #addtask qa_configure after do_configure before do_compile > do_configure[postfuncs] += "do_qa_configure " > > +# Check poison-system-directories in log.do_compile and log.do_install > +do_compile[postfuncs] += "do_qa_compile" > +do_install[postfuncs] += "do_qa_install" > + > # Check does S exist. > do_unpack[postfuncs] += "do_qa_unpack" > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#168973): > https://lists.openembedded.org/g/openembedded-core/message/168973 > Mute This Topic: https://lists.openembedded.org/mt/92853487/3617156 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ > Martin.Jansa@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- > >
On 6 Aug 2022, at 10:07, Martin Jansa <martin.jansa@gmail.com> wrote: > > * only a RFC to discuss what might prevent > --enable-poison-system-directories=true in gcc-cross to work or to > test this on AB to see if there are other places where these issues > sneaked in without being detected This is… odd. There’s a test case for this in oe-selftest, specifically the poison recipe which does a compile with -I/usr/include, and it fails: cc1: error: include location "/usr/include" is unsafe for cross-compilation [-Werror=poison-system-directories] cc1: some warnings being treated as errors WARNING: exit code 1 from a shell command. ERROR: Task (/home/ross/Yocto/poky/meta-selftest/recipes-test/poison/poison.bb:do_compile) failed with exit code ‘1' Can you confirm that your gcc-cross is being passed --enable-poison-system-directories=error when configured? =true means just warnings, you need =error. Ross
diff --git a/meta/classes/insane.bbclass b/meta/classes/insane.bbclass index c8b434bb54..9ce5ae0f6f 100644 --- a/meta/classes/insane.bbclass +++ b/meta/classes/insane.bbclass @@ -35,8 +35,8 @@ ERROR_QA ?= "dev-so debug-deps dev-deps debug-files arch pkgconfig la \ version-going-backwards expanded-d invalid-chars \ license-checksum dev-elf file-rdeps configure-unsafe \ configure-gettext perllocalpod shebang-size \ - already-stripped installed-vs-shipped ldflags compile-host-path \ - install-host-path pn-overrides unknown-configure-option \ + already-stripped installed-vs-shipped ldflags compile-unsafe \ + install-unsafe pn-overrides unknown-configure-option \ useless-rpaths rpaths staticdev empty-dirs \ " # Add usrmerge QA check based on distro feature @@ -1231,6 +1231,31 @@ python do_qa_patch() { bb.error("Missing Upstream-Status in patch\n%s\nPlease add according to %s ." % (fullpath, guidelines)) } +def check_poisoned_system_directories(file): + import subprocess + + statement = "grep -q -F -e 'is unsafe for cross-compilation' %s" % file + if subprocess.call(statement, shell=True) == 0: + return """This %s log indicates 'unsafe for cross-compilation' or '-Wpoison-system-directories' errors, it looked at host include and/or library paths""" % file + +python do_qa_compile() { + log = os.path.join(d.getVar('T'),"log.do_compile") + skip = (d.getVar('INSANE_SKIP') or "").split() + if 'compile-unsafe' not in skip and os.path.exists(log): + error_msg = check_poisoned_system_directories(log) + if error_msg: + oe.qa.handle_error("compile-unsafe", error_msg, d) +} + +python do_qa_install() { + log = os.path.join(d.getVar('T'),"log.do_install") + skip = (d.getVar('INSANE_SKIP') or "").split() + if 'install-unsafe' not in skip and os.path.exists(log): + error_msg = check_poisoned_system_directories(log) + if error_msg: + oe.qa.handle_error("install-unsafe", error_msg, d) +} + python do_qa_configure() { import subprocess @@ -1253,9 +1278,8 @@ python do_qa_configure() { statement = "grep -q -F -e 'is unsafe for cross-compilation' %s" % \ os.path.join(root,"config.log") if "config.log" in files: - if subprocess.call(statement, shell=True) == 0: - error_msg = """This autoconf log indicates errors, it looked at host include and/or library paths while determining system capabilities. -Rerun configure task after fixing this.""" + error_msg = check_poisoned_system_directories(os.path.join(root,"config.log")) + if error_msg: oe.qa.handle_error("configure-unsafe", error_msg, d) if "configure.ac" in files: @@ -1356,6 +1380,10 @@ do_patch[postfuncs] += "do_qa_patch " #addtask qa_configure after do_configure before do_compile do_configure[postfuncs] += "do_qa_configure " +# Check poison-system-directories in log.do_compile and log.do_install +do_compile[postfuncs] += "do_qa_compile" +do_compile[postfuncs] += "do_qa_install" + # Check does S exist. do_unpack[postfuncs] += "do_qa_unpack"
* only a RFC to discuss what might prevent --enable-poison-system-directories=true in gcc-cross to work or to test this on AB to see if there are other places where these issues sneaked in without being detected * it was removed in: https://git.openembedded.org/openembedded-core/commit/?id=a67e9ebfd5b8002fd4a7d8d27ff0d997817f76e1 but looks like the "follow-up patches": d039d6fbfc gcc-cross: make use of the system include directories fatal 1fcaa5f452 gcc: add an option for --enable-poison-system-directories to be fatal don't work anymore? I was checking if e.g. -Wno-error is used in webkitgtk to disable them, but it's not there, so something else went wrong, because in webkit-gtk log.do_compile I've just noticed: # grep unsafe poky/build/tmp/work/core2-64-poky-linux/webkitgtk/2.36.4-r0/temp/log.do_compile.3901410 | sort | uniq -c 6 cc1: warning: include location "/usr/include/gstreamer-1.0" is unsafe for cross-compilation [-Wpoison-system-directories] 1267 cc1plus: warning: include location "/usr/include/gstreamer-1.0" is unsafe for cross-compilation [-Wpoison-system-directories] * and with this it confirms the same in webkitgtk: ERROR: webkitgtk-2.36.4-r0 do_compile: QA Issue: This /OE/build/oe-core/tmp-glibc/work/core2-64-oe-linux/webkitgtk/2.36.4-r0/temp/log.do_compile log indicates 'unsafe for cross-compilation' or '-Wpoison-system-directories' errors, it looked at host include and/or library paths [compile-unsafe] and this was the only error (when building webkitgtk from scratch) Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> --- meta/classes/insane.bbclass | 38 ++++++++++++++++++++++++++++++++----- 1 file changed, 33 insertions(+), 5 deletions(-)