diff mbox series

[3/3] pip_install_wheel: clean up

Message ID 20220304171408.2672507-3-ross.burton@arm.com
State Accepted, archived
Commit cb56166eef34ba4937dc487664c57815daeadbe8
Headers show
Series [1/3] asciidoc: update git repository | expand

Commit Message

Ross Burton March 4, 2022, 5:14 p.m. UTC 
There's been a lot of work in this class lately, so a little spring
cleaning is needed.

Remove redundant creation of PYTHON_SITEPACKAGES_DIR, pip will do that.

Remove redundant export of PYPA_WHEEL.

Simplyify recompile code using "realpath --relative-to".

Signed-off-by: Ross Burton <ross.burton@arm.com>
---
 meta/classes/pip_install_wheel.bbclass | 15 +++------------
 1 file changed, 3 insertions(+), 12 deletions(-)

Comments

Khem Raj March 5, 2022, 1:57 a.m. UTC | #1 
This is causing some install failures in meta-openembedded see
last five recipes in this batch

https://errors.yoctoproject.org/Errors/Details/651080/
https://errors.yoctoproject.org/Errors/Details/651082/
https://errors.yoctoproject.org/Errors/Details/651083/
https://errors.yoctoproject.org/Errors/Details/651084/
https://errors.yoctoproject.org/Errors/Details/651085/

These are the same failures across architectures and libcs.



On Fri, Mar 4, 2022 at 9:14 AM Ross Burton <ross@burtonini.com> wrote:
>
> There's been a lot of work in this class lately, so a little spring
> cleaning is needed.
>
> Remove redundant creation of PYTHON_SITEPACKAGES_DIR, pip will do that.
>
> Remove redundant export of PYPA_WHEEL.
>
> Simplyify recompile code using "realpath --relative-to".
>
> Signed-off-by: Ross Burton <ross.burton@arm.com>
> ---
>  meta/classes/pip_install_wheel.bbclass | 15 +++------------
>  1 file changed, 3 insertions(+), 12 deletions(-)
>
> diff --git a/meta/classes/pip_install_wheel.bbclass b/meta/classes/pip_install_wheel.bbclass
> index 3beff685bb..1870b916fe 100644
> --- a/meta/classes/pip_install_wheel.bbclass
> +++ b/meta/classes/pip_install_wheel.bbclass
> @@ -20,29 +20,20 @@ PIP_INSTALL_ARGS ?= "\
>      --prefix=${prefix} \
>  "
>
> -pip_install_wheel_do_install:prepend () {
> -    install -d ${D}${PYTHON_SITEPACKAGES_DIR}
> -}
> -
> -export PYPA_WHEEL
> -
>  PIP_INSTALL_PYTHON = "python3"
>  PIP_INSTALL_PYTHON:class-native = "nativepython3"
>
>  pip_install_wheel_do_install () {
>      nativepython3 -m pip install ${PIP_INSTALL_ARGS} ${PYPA_WHEEL} ||
> -    bbfatal_log "Failed to pip install wheel. Check the logs."
> +      bbfatal_log "Failed to pip install wheel. Check the logs."
>
> +    cd ${D}
>      for i in ${D}${bindir}/* ${D}${sbindir}/*; do
>          if [ -f "$i" ]; then
>              sed -i -e "1s,#!.*nativepython3,#!${USRBINPATH}/env ${PIP_INSTALL_PYTHON}," $i
>              sed -i -e "s:${PYTHON}:${USRBINPATH}/env\ ${PIP_INSTALL_PYTHON}:g" $i
>              sed -i -e "s:${STAGING_BINDIR_NATIVE}:${bindir}:g" $i
> -            # Recompile after modifying it
> -            cd ${D}
> -            file=`echo $i | sed 's:^${D}/::'`
> -            ${STAGING_BINDIR_NATIVE}/python3-native/python3 -c "from py_compile import compile; compile('$file')"
> -            cd -
> +            nativepython3 -mpy_compile $(realpath --relative-to=${D} $i)
>          fi
>      done
>  }
> --
> 2.25.1
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#162746): https://lists.openembedded.org/g/openembedded-core/message/162746
> Mute This Topic: https://lists.openembedded.org/mt/89553658/1997914
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [raj.khem@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
Richard Purdie March 5, 2022, 8:22 a.m. UTC | #2 
On Fri, 2022-03-04 at 17:14 +0000, Ross Burton wrote:
> There's been a lot of work in this class lately, so a little spring
> cleaning is needed.
> 
> Remove redundant creation of PYTHON_SITEPACKAGES_DIR, pip will do that.
> 
> Remove redundant export of PYPA_WHEEL.
> 
> Simplyify recompile code using "realpath --relative-to".
> 
> Signed-off-by: Ross Burton <ross.burton@arm.com>
> ---
>  meta/classes/pip_install_wheel.bbclass | 15 +++------------
>  1 file changed, 3 insertions(+), 12 deletions(-)
> 
> diff --git a/meta/classes/pip_install_wheel.bbclass b/meta/classes/pip_install_wheel.bbclass
> index 3beff685bb..1870b916fe 100644
> --- a/meta/classes/pip_install_wheel.bbclass
> +++ b/meta/classes/pip_install_wheel.bbclass
> @@ -20,29 +20,20 @@ PIP_INSTALL_ARGS ?= "\
>      --prefix=${prefix} \
>  "
>  
> -pip_install_wheel_do_install:prepend () {
> -    install -d ${D}${PYTHON_SITEPACKAGES_DIR}
> -}
> -
> -export PYPA_WHEEL
> -
>  PIP_INSTALL_PYTHON = "python3"
>  PIP_INSTALL_PYTHON:class-native = "nativepython3"
>  
>  pip_install_wheel_do_install () {
>      nativepython3 -m pip install ${PIP_INSTALL_ARGS} ${PYPA_WHEEL} ||
> -    bbfatal_log "Failed to pip install wheel. Check the logs."
> +      bbfatal_log "Failed to pip install wheel. Check the logs."
>  
> +    cd ${D}
>      for i in ${D}${bindir}/* ${D}${sbindir}/*; do
>          if [ -f "$i" ]; then
>              sed -i -e "1s,#!.*nativepython3,#!${USRBINPATH}/env ${PIP_INSTALL_PYTHON}," $i
>              sed -i -e "s:${PYTHON}:${USRBINPATH}/env\ ${PIP_INSTALL_PYTHON}:g" $i
>              sed -i -e "s:${STAGING_BINDIR_NATIVE}:${bindir}:g" $i
> -            # Recompile after modifying it
> -            cd ${D}
> -            file=`echo $i | sed 's:^${D}/::'`
> -            ${STAGING_BINDIR_NATIVE}/python3-native/python3 -c "from py_compile import compile; compile('$file')"
> -            cd -
> +            nativepython3 -mpy_compile $(realpath --relative-to=${D} $i)

This last bit likely doesn't work unless you are in ${D} when you run the
command :/.

Cheers,

Richard
Ross Burton March 5, 2022, 11:05 a.m. UTC | #3 
I think I found a bug in the dependency chasing code...

Ross

On Sat, 5 Mar 2022 at 08:22, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
>
> On Fri, 2022-03-04 at 17:14 +0000, Ross Burton wrote:
> > There's been a lot of work in this class lately, so a little spring
> > cleaning is needed.
> >
> > Remove redundant creation of PYTHON_SITEPACKAGES_DIR, pip will do that.
> >
> > Remove redundant export of PYPA_WHEEL.
> >
> > Simplyify recompile code using "realpath --relative-to".
> >
> > Signed-off-by: Ross Burton <ross.burton@arm.com>
> > ---
> >  meta/classes/pip_install_wheel.bbclass | 15 +++------------
> >  1 file changed, 3 insertions(+), 12 deletions(-)
> >
> > diff --git a/meta/classes/pip_install_wheel.bbclass b/meta/classes/pip_install_wheel.bbclass
> > index 3beff685bb..1870b916fe 100644
> > --- a/meta/classes/pip_install_wheel.bbclass
> > +++ b/meta/classes/pip_install_wheel.bbclass
> > @@ -20,29 +20,20 @@ PIP_INSTALL_ARGS ?= "\
> >      --prefix=${prefix} \
> >  "
> >
> > -pip_install_wheel_do_install:prepend () {
> > -    install -d ${D}${PYTHON_SITEPACKAGES_DIR}
> > -}
> > -
> > -export PYPA_WHEEL
> > -
> >  PIP_INSTALL_PYTHON = "python3"
> >  PIP_INSTALL_PYTHON:class-native = "nativepython3"
> >
> >  pip_install_wheel_do_install () {
> >      nativepython3 -m pip install ${PIP_INSTALL_ARGS} ${PYPA_WHEEL} ||
> > -    bbfatal_log "Failed to pip install wheel. Check the logs."
> > +      bbfatal_log "Failed to pip install wheel. Check the logs."
> >
> > +    cd ${D}
> >      for i in ${D}${bindir}/* ${D}${sbindir}/*; do
> >          if [ -f "$i" ]; then
> >              sed -i -e "1s,#!.*nativepython3,#!${USRBINPATH}/env ${PIP_INSTALL_PYTHON}," $i
> >              sed -i -e "s:${PYTHON}:${USRBINPATH}/env\ ${PIP_INSTALL_PYTHON}:g" $i
> >              sed -i -e "s:${STAGING_BINDIR_NATIVE}:${bindir}:g" $i
> > -            # Recompile after modifying it
> > -            cd ${D}
> > -            file=`echo $i | sed 's:^${D}/::'`
> > -            ${STAGING_BINDIR_NATIVE}/python3-native/python3 -c "from py_compile import compile; compile('$file')"
> > -            cd -
> > +            nativepython3 -mpy_compile $(realpath --relative-to=${D} $i)
>
> This last bit likely doesn't work unless you are in ${D} when you run the
> command :/.
>
> Cheers,
>
> Richard
>
>
>
Ross Burton March 7, 2022, 2:09 p.m. UTC | #4 
These appear to be because the new compile code doesn't ignore errors,
and the loop compiles everything installed in $bindir, even cmd.exe or
Bash scripts...

I'll make the loop stricter.

Ross

On Sat, 5 Mar 2022 at 01:57, Khem Raj <raj.khem@gmail.com> wrote:
>
> This is causing some install failures in meta-openembedded see
> last five recipes in this batch
>
> https://errors.yoctoproject.org/Errors/Details/651080/
> https://errors.yoctoproject.org/Errors/Details/651082/
> https://errors.yoctoproject.org/Errors/Details/651083/
> https://errors.yoctoproject.org/Errors/Details/651084/
> https://errors.yoctoproject.org/Errors/Details/651085/
>
> These are the same failures across architectures and libcs.
>
>
>
> On Fri, Mar 4, 2022 at 9:14 AM Ross Burton <ross@burtonini.com> wrote:
> >
> > There's been a lot of work in this class lately, so a little spring
> > cleaning is needed.
> >
> > Remove redundant creation of PYTHON_SITEPACKAGES_DIR, pip will do that.
> >
> > Remove redundant export of PYPA_WHEEL.
> >
> > Simplyify recompile code using "realpath --relative-to".
> >
> > Signed-off-by: Ross Burton <ross.burton@arm.com>
> > ---
> >  meta/classes/pip_install_wheel.bbclass | 15 +++------------
> >  1 file changed, 3 insertions(+), 12 deletions(-)
> >
> > diff --git a/meta/classes/pip_install_wheel.bbclass b/meta/classes/pip_install_wheel.bbclass
> > index 3beff685bb..1870b916fe 100644
> > --- a/meta/classes/pip_install_wheel.bbclass
> > +++ b/meta/classes/pip_install_wheel.bbclass
> > @@ -20,29 +20,20 @@ PIP_INSTALL_ARGS ?= "\
> >      --prefix=${prefix} \
> >  "
> >
> > -pip_install_wheel_do_install:prepend () {
> > -    install -d ${D}${PYTHON_SITEPACKAGES_DIR}
> > -}
> > -
> > -export PYPA_WHEEL
> > -
> >  PIP_INSTALL_PYTHON = "python3"
> >  PIP_INSTALL_PYTHON:class-native = "nativepython3"
> >
> >  pip_install_wheel_do_install () {
> >      nativepython3 -m pip install ${PIP_INSTALL_ARGS} ${PYPA_WHEEL} ||
> > -    bbfatal_log "Failed to pip install wheel. Check the logs."
> > +      bbfatal_log "Failed to pip install wheel. Check the logs."
> >
> > +    cd ${D}
> >      for i in ${D}${bindir}/* ${D}${sbindir}/*; do
> >          if [ -f "$i" ]; then
> >              sed -i -e "1s,#!.*nativepython3,#!${USRBINPATH}/env ${PIP_INSTALL_PYTHON}," $i
> >              sed -i -e "s:${PYTHON}:${USRBINPATH}/env\ ${PIP_INSTALL_PYTHON}:g" $i
> >              sed -i -e "s:${STAGING_BINDIR_NATIVE}:${bindir}:g" $i
> > -            # Recompile after modifying it
> > -            cd ${D}
> > -            file=`echo $i | sed 's:^${D}/::'`
> > -            ${STAGING_BINDIR_NATIVE}/python3-native/python3 -c "from py_compile import compile; compile('$file')"
> > -            cd -
> > +            nativepython3 -mpy_compile $(realpath --relative-to=${D} $i)
> >          fi
> >      done
> >  }
> > --
> > 2.25.1
> >
> >
> > -=-=-=-=-=-=-=-=-=-=-=-
> > Links: You receive all messages sent to this group.
> > View/Reply Online (#162746): https://lists.openembedded.org/g/openembedded-core/message/162746
> > Mute This Topic: https://lists.openembedded.org/mt/89553658/1997914
> > Group Owner: openembedded-core+owner@lists.openembedded.org
> > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [raj.khem@gmail.com]
> > -=-=-=-=-=-=-=-=-=-=-=-
> >
diff mbox series

Patch

diff --git a/meta/classes/pip_install_wheel.bbclass b/meta/classes/pip_install_wheel.bbclass
index 3beff685bb..1870b916fe 100644
--- a/meta/classes/pip_install_wheel.bbclass
+++ b/meta/classes/pip_install_wheel.bbclass
@@ -20,29 +20,20 @@  PIP_INSTALL_ARGS ?= "\
     --prefix=${prefix} \
 "
 
-pip_install_wheel_do_install:prepend () {
-    install -d ${D}${PYTHON_SITEPACKAGES_DIR}
-}
-
-export PYPA_WHEEL
-
 PIP_INSTALL_PYTHON = "python3"
 PIP_INSTALL_PYTHON:class-native = "nativepython3"
 
 pip_install_wheel_do_install () {
     nativepython3 -m pip install ${PIP_INSTALL_ARGS} ${PYPA_WHEEL} ||
-    bbfatal_log "Failed to pip install wheel. Check the logs."
+      bbfatal_log "Failed to pip install wheel. Check the logs."
 
+    cd ${D}
     for i in ${D}${bindir}/* ${D}${sbindir}/*; do
         if [ -f "$i" ]; then
             sed -i -e "1s,#!.*nativepython3,#!${USRBINPATH}/env ${PIP_INSTALL_PYTHON}," $i
             sed -i -e "s:${PYTHON}:${USRBINPATH}/env\ ${PIP_INSTALL_PYTHON}:g" $i
             sed -i -e "s:${STAGING_BINDIR_NATIVE}:${bindir}:g" $i
-            # Recompile after modifying it
-            cd ${D}
-            file=`echo $i | sed 's:^${D}/::'`
-            ${STAGING_BINDIR_NATIVE}/python3-native/python3 -c "from py_compile import compile; compile('$file')"
-            cd -
+            nativepython3 -mpy_compile $(realpath --relative-to=${D} $i)
         fi
     done
 }