diff mbox series

[1/2] python3: add wrapper for native

Message ID PR2PR09MB310005D5AE7E68AD1153DBC2A8719@PR2PR09MB3100.eurprd09.prod.outlook.com
State New
Headers show
Series [1/2] python3: add wrapper for native | expand

Commit Message

Konrad Weihmann Aug. 22, 2022, 7:57 p.m. UTC 
which makes sure to set PYTHONNOUSERSITE to prevent
dynamically loading site packages from the host sided
package dirs.
In before when a module was using pluggy and/or
importlib_metadata the host sided path ~./local/... was still
part of the sys.path search tree and had priority over the
recipe-sysroot-native, leading to host/workspace contamination

Signed-off-by: Konrad Weihmann <kweihmann@outlook.com>
---
 meta/recipes-devtools/python/python3/nativepython3 | 2 ++
 meta/recipes-devtools/python/python3_3.10.6.bb     | 9 ++++-----
 2 files changed, 6 insertions(+), 5 deletions(-)
 create mode 100644 meta/recipes-devtools/python/python3/nativepython3

Comments

Alexander Kanavin Aug. 22, 2022, 8:03 p.m. UTC | #1 
This needs to be applied to python3-native/python3, not just to the
nativepython3 link to it. You can use 'create_wrapper' helper that was
written exactly for the purpose (see meta/ for examples).

Also, can the lookup in ~ be simply disabled at python3-native build time?

Alex

On Mon, 22 Aug 2022 at 21:57, Konrad Weihmann <kweihmann@outlook.com> wrote:
>
> which makes sure to set PYTHONNOUSERSITE to prevent
> dynamically loading site packages from the host sided
> package dirs.
> In before when a module was using pluggy and/or
> importlib_metadata the host sided path ~./local/... was still
> part of the sys.path search tree and had priority over the
> recipe-sysroot-native, leading to host/workspace contamination
>
> Signed-off-by: Konrad Weihmann <kweihmann@outlook.com>
> ---
>  meta/recipes-devtools/python/python3/nativepython3 | 2 ++
>  meta/recipes-devtools/python/python3_3.10.6.bb     | 9 ++++-----
>  2 files changed, 6 insertions(+), 5 deletions(-)
>  create mode 100644 meta/recipes-devtools/python/python3/nativepython3
>
> diff --git a/meta/recipes-devtools/python/python3/nativepython3 b/meta/recipes-devtools/python/python3/nativepython3
> new file mode 100644
> index 0000000000..0413e44e32
> --- /dev/null
> +++ b/meta/recipes-devtools/python/python3/nativepython3
> @@ -0,0 +1,2 @@
> +#!/bin/sh
> +PYTHONNOUSERSITE=1 python3 $@
> \ No newline at end of file
> diff --git a/meta/recipes-devtools/python/python3_3.10.6.bb b/meta/recipes-devtools/python/python3_3.10.6.bb
> index f19cd50273..1121c17887 100644
> --- a/meta/recipes-devtools/python/python3_3.10.6.bb
> +++ b/meta/recipes-devtools/python/python3_3.10.6.bb
> @@ -43,6 +43,7 @@ SRC_URI:append:class-native = " \
>             file://0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch \
>             file://12-distutils-prefix-is-inside-staging-area.patch \
>             file://0001-Don-t-search-system-for-headers-libraries.patch \
> +           file://nativepython3 \
>             "
>  SRC_URI[sha256sum] = "f795ff87d11d4b0c7c33bc8851b0c28648d8a4583aa2100a98c22b4326b6d3f3"
>
> @@ -153,11 +154,9 @@ do_install:append:class-native() {
>          for PYTHSCRIPT in `grep -rIl ${bindir}/${PN}/python ${D}${bindir}/${PN}`; do
>                  sed -i -e '1s|^#!.*|#!/usr/bin/env python3|' $PYTHSCRIPT
>          done
> -        # Add a symlink to the native Python so that scripts can just invoke
> -        # "nativepython" and get the right one without needing absolute paths
> -        # (these often end up too long for the #! parser in the kernel as the
> -        # buffer is 128 bytes long).
> -        ln -s python3-native/python3 ${D}${bindir}/nativepython3
> +        # Create a wrapper for python3 in native environment
> +        # make sure that no host sided modules are inherited
> +        install -m 755 ${WORKDIR}/nativepython3 ${D}${bindir}/nativepython3
>
>          # Remove the opt-1.pyc and opt-2.pyc files. There are over 3,000 of them
>          # and the overhead in each recipe-sysroot-native isn't worth it, particularly
> --
> 2.34.1
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#169681): https://lists.openembedded.org/g/openembedded-core/message/169681
> Mute This Topic: https://lists.openembedded.org/mt/93189857/1686489
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
Konrad Weihmann Aug. 23, 2022, 7:05 a.m. UTC | #2 
I read a more on this topic and found a less invasive way of disabling 
this feature globally for the native target.
Sadly there's no configure switch

Just send out a new patch, so this original series can be safely ignored

On 22.08.22 22:03, Alexander Kanavin wrote:
> This needs to be applied to python3-native/python3, not just to the
> nativepython3 link to it. You can use 'create_wrapper' helper that was
> written exactly for the purpose (see meta/ for examples).
> 
> Also, can the lookup in ~ be simply disabled at python3-native build time?
> 
> Alex
> 
> On Mon, 22 Aug 2022 at 21:57, Konrad Weihmann <kweihmann@outlook.com> wrote:
>>
>> which makes sure to set PYTHONNOUSERSITE to prevent
>> dynamically loading site packages from the host sided
>> package dirs.
>> In before when a module was using pluggy and/or
>> importlib_metadata the host sided path ~./local/... was still
>> part of the sys.path search tree and had priority over the
>> recipe-sysroot-native, leading to host/workspace contamination
>>
>> Signed-off-by: Konrad Weihmann <kweihmann@outlook.com>
>> ---
>>   meta/recipes-devtools/python/python3/nativepython3 | 2 ++
>>   meta/recipes-devtools/python/python3_3.10.6.bb     | 9 ++++-----
>>   2 files changed, 6 insertions(+), 5 deletions(-)
>>   create mode 100644 meta/recipes-devtools/python/python3/nativepython3
>>
>> diff --git a/meta/recipes-devtools/python/python3/nativepython3 b/meta/recipes-devtools/python/python3/nativepython3
>> new file mode 100644
>> index 0000000000..0413e44e32
>> --- /dev/null
>> +++ b/meta/recipes-devtools/python/python3/nativepython3
>> @@ -0,0 +1,2 @@
>> +#!/bin/sh
>> +PYTHONNOUSERSITE=1 python3 $@
>> \ No newline at end of file
>> diff --git a/meta/recipes-devtools/python/python3_3.10.6.bb b/meta/recipes-devtools/python/python3_3.10.6.bb
>> index f19cd50273..1121c17887 100644
>> --- a/meta/recipes-devtools/python/python3_3.10.6.bb
>> +++ b/meta/recipes-devtools/python/python3_3.10.6.bb
>> @@ -43,6 +43,7 @@ SRC_URI:append:class-native = " \
>>              file://0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch \
>>              file://12-distutils-prefix-is-inside-staging-area.patch \
>>              file://0001-Don-t-search-system-for-headers-libraries.patch \
>> +           file://nativepython3 \
>>              "
>>   SRC_URI[sha256sum] = "f795ff87d11d4b0c7c33bc8851b0c28648d8a4583aa2100a98c22b4326b6d3f3"
>>
>> @@ -153,11 +154,9 @@ do_install:append:class-native() {
>>           for PYTHSCRIPT in `grep -rIl ${bindir}/${PN}/python ${D}${bindir}/${PN}`; do
>>                   sed -i -e '1s|^#!.*|#!/usr/bin/env python3|' $PYTHSCRIPT
>>           done
>> -        # Add a symlink to the native Python so that scripts can just invoke
>> -        # "nativepython" and get the right one without needing absolute paths
>> -        # (these often end up too long for the #! parser in the kernel as the
>> -        # buffer is 128 bytes long).
>> -        ln -s python3-native/python3 ${D}${bindir}/nativepython3
>> +        # Create a wrapper for python3 in native environment
>> +        # make sure that no host sided modules are inherited
>> +        install -m 755 ${WORKDIR}/nativepython3 ${D}${bindir}/nativepython3
>>
>>           # Remove the opt-1.pyc and opt-2.pyc files. There are over 3,000 of them
>>           # and the overhead in each recipe-sysroot-native isn't worth it, particularly
>> --
>> 2.34.1
>>
>>
>> -=-=-=-=-=-=-=-=-=-=-=-
>> Links: You receive all messages sent to this group.
>> View/Reply Online (#169681): https://lists.openembedded.org/g/openembedded-core/message/169681
>> Mute This Topic: https://lists.openembedded.org/mt/93189857/1686489
>> Group Owner: openembedded-core+owner@lists.openembedded.org
>> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com]
>> -=-=-=-=-=-=-=-=-=-=-=-
>>
Alexander Kanavin Aug. 23, 2022, 7:26 a.m. UTC | #3 
The new patch is fine, thanks.

Alex

On Tue, 23 Aug 2022 at 09:05, Konrad Weihmann <kweihmann@outlook.com> wrote:
>
> I read a more on this topic and found a less invasive way of disabling
> this feature globally for the native target.
> Sadly there's no configure switch
>
> Just send out a new patch, so this original series can be safely ignored
>
> On 22.08.22 22:03, Alexander Kanavin wrote:
> > This needs to be applied to python3-native/python3, not just to the
> > nativepython3 link to it. You can use 'create_wrapper' helper that was
> > written exactly for the purpose (see meta/ for examples).
> >
> > Also, can the lookup in ~ be simply disabled at python3-native build time?
> >
> > Alex
> >
> > On Mon, 22 Aug 2022 at 21:57, Konrad Weihmann <kweihmann@outlook.com> wrote:
> >>
> >> which makes sure to set PYTHONNOUSERSITE to prevent
> >> dynamically loading site packages from the host sided
> >> package dirs.
> >> In before when a module was using pluggy and/or
> >> importlib_metadata the host sided path ~./local/... was still
> >> part of the sys.path search tree and had priority over the
> >> recipe-sysroot-native, leading to host/workspace contamination
> >>
> >> Signed-off-by: Konrad Weihmann <kweihmann@outlook.com>
> >> ---
> >>   meta/recipes-devtools/python/python3/nativepython3 | 2 ++
> >>   meta/recipes-devtools/python/python3_3.10.6.bb     | 9 ++++-----
> >>   2 files changed, 6 insertions(+), 5 deletions(-)
> >>   create mode 100644 meta/recipes-devtools/python/python3/nativepython3
> >>
> >> diff --git a/meta/recipes-devtools/python/python3/nativepython3 b/meta/recipes-devtools/python/python3/nativepython3
> >> new file mode 100644
> >> index 0000000000..0413e44e32
> >> --- /dev/null
> >> +++ b/meta/recipes-devtools/python/python3/nativepython3
> >> @@ -0,0 +1,2 @@
> >> +#!/bin/sh
> >> +PYTHONNOUSERSITE=1 python3 $@
> >> \ No newline at end of file
> >> diff --git a/meta/recipes-devtools/python/python3_3.10.6.bb b/meta/recipes-devtools/python/python3_3.10.6.bb
> >> index f19cd50273..1121c17887 100644
> >> --- a/meta/recipes-devtools/python/python3_3.10.6.bb
> >> +++ b/meta/recipes-devtools/python/python3_3.10.6.bb
> >> @@ -43,6 +43,7 @@ SRC_URI:append:class-native = " \
> >>              file://0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch \
> >>              file://12-distutils-prefix-is-inside-staging-area.patch \
> >>              file://0001-Don-t-search-system-for-headers-libraries.patch \
> >> +           file://nativepython3 \
> >>              "
> >>   SRC_URI[sha256sum] = "f795ff87d11d4b0c7c33bc8851b0c28648d8a4583aa2100a98c22b4326b6d3f3"
> >>
> >> @@ -153,11 +154,9 @@ do_install:append:class-native() {
> >>           for PYTHSCRIPT in `grep -rIl ${bindir}/${PN}/python ${D}${bindir}/${PN}`; do
> >>                   sed -i -e '1s|^#!.*|#!/usr/bin/env python3|' $PYTHSCRIPT
> >>           done
> >> -        # Add a symlink to the native Python so that scripts can just invoke
> >> -        # "nativepython" and get the right one without needing absolute paths
> >> -        # (these often end up too long for the #! parser in the kernel as the
> >> -        # buffer is 128 bytes long).
> >> -        ln -s python3-native/python3 ${D}${bindir}/nativepython3
> >> +        # Create a wrapper for python3 in native environment
> >> +        # make sure that no host sided modules are inherited
> >> +        install -m 755 ${WORKDIR}/nativepython3 ${D}${bindir}/nativepython3
> >>
> >>           # Remove the opt-1.pyc and opt-2.pyc files. There are over 3,000 of them
> >>           # and the overhead in each recipe-sysroot-native isn't worth it, particularly
> >> --
> >> 2.34.1
> >>
> >>
> >> -=-=-=-=-=-=-=-=-=-=-=-
> >> Links: You receive all messages sent to this group.
> >> View/Reply Online (#169681): https://lists.openembedded.org/g/openembedded-core/message/169681
> >> Mute This Topic: https://lists.openembedded.org/mt/93189857/1686489
> >> Group Owner: openembedded-core+owner@lists.openembedded.org
> >> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com]
> >> -=-=-=-=-=-=-=-=-=-=-=-
> >>
diff mbox series

Patch

diff --git a/meta/recipes-devtools/python/python3/nativepython3 b/meta/recipes-devtools/python/python3/nativepython3
new file mode 100644
index 0000000000..0413e44e32
--- /dev/null
+++ b/meta/recipes-devtools/python/python3/nativepython3
@@ -0,0 +1,2 @@ 
+#!/bin/sh
+PYTHONNOUSERSITE=1 python3 $@
\ No newline at end of file
diff --git a/meta/recipes-devtools/python/python3_3.10.6.bb b/meta/recipes-devtools/python/python3_3.10.6.bb
index f19cd50273..1121c17887 100644
--- a/meta/recipes-devtools/python/python3_3.10.6.bb
+++ b/meta/recipes-devtools/python/python3_3.10.6.bb
@@ -43,6 +43,7 @@  SRC_URI:append:class-native = " \
            file://0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch \
            file://12-distutils-prefix-is-inside-staging-area.patch \
            file://0001-Don-t-search-system-for-headers-libraries.patch \
+           file://nativepython3 \
            "
 SRC_URI[sha256sum] = "f795ff87d11d4b0c7c33bc8851b0c28648d8a4583aa2100a98c22b4326b6d3f3"
 
@@ -153,11 +154,9 @@  do_install:append:class-native() {
         for PYTHSCRIPT in `grep -rIl ${bindir}/${PN}/python ${D}${bindir}/${PN}`; do
                 sed -i -e '1s|^#!.*|#!/usr/bin/env python3|' $PYTHSCRIPT
         done
-        # Add a symlink to the native Python so that scripts can just invoke
-        # "nativepython" and get the right one without needing absolute paths
-        # (these often end up too long for the #! parser in the kernel as the
-        # buffer is 128 bytes long).
-        ln -s python3-native/python3 ${D}${bindir}/nativepython3
+        # Create a wrapper for python3 in native environment
+        # make sure that no host sided modules are inherited
+        install -m 755 ${WORKDIR}/nativepython3 ${D}${bindir}/nativepython3
 
         # Remove the opt-1.pyc and opt-2.pyc files. There are over 3,000 of them
         # and the overhead in each recipe-sysroot-native isn't worth it, particularly