Patchwork distutils/steuptools: Fix files layout and unbreak builds

login
mail settings
Submitter Richard Purdie
Date Aug. 24, 2012, 4:15 p.m.
Message ID <1345824924.14369.82.camel@ted>
Download mbox | patch
Permalink /patch/35319/
State Accepted
Commit bed18d5df7915e4127a538be9c7550e185c8c850
Headers show

Comments

Richard Purdie - Aug. 24, 2012, 4:15 p.m.
The last two distutils changes progressively broke the builds. Firstly they
moved things from the site_packages directory to being higher up the tree
which introduced package QA warnings as a side effect. Secondly, it interacts
badly with setuptools which passes in --root=${D} itself.

This patch restores the original directory layout, hence fixing the QA
warnings and also passes extra options to setuptools to deal with the
--root option it passes.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
McClintock Matthew-B29882 - Aug. 24, 2012, 5:45 p.m.
On Fri, Aug 24, 2012 at 11:15 AM, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> The last two distutils changes progressively broke the builds. Firstly they
> moved things from the site_packages directory to being higher up the tree
> which introduced package QA warnings as a side effect. Secondly, it interacts
> badly with setuptools which passes in --root=${D} itself.
>
> This patch restores the original directory layout, hence fixing the QA
> warnings and also passes extra options to setuptools to deal with the
> --root option it passes.
>
> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> ---
> diff --git a/meta/classes/distutils.bbclass b/meta/classes/distutils.bbclass
> index 52a1aa8..c73b24f 100644
> --- a/meta/classes/distutils.bbclass
> +++ b/meta/classes/distutils.bbclass
> @@ -38,7 +38,7 @@ distutils_do_install() {
>          STAGING_LIBDIR=${STAGING_LIBDIR} \
>          PYTHONPATH=${D}/${PYTHON_SITEPACKAGES_DIR} \
>          BUILD_SYS=${BUILD_SYS} HOST_SYS=${HOST_SYS} \
> -        ${STAGING_BINDIR_NATIVE}/python-native/python setup.py install --install-lib=${D}${libdir}/${PYTHON_DIR} ${DISTUTILS_INSTALL_ARGS} || \
> +        ${STAGING_BINDIR_NATIVE}/python-native/python setup.py install --install-lib=${D}/${PYTHON_SITEPACKAGES_DIR} ${DISTUTILS_INSTALL_ARGS} || \
>          bbfatal "python setup.py install execution failed."
>
>          for i in `find ${D} -name "*.py"` ; do \
> diff --git a/meta/classes/setuptools.bbclass b/meta/classes/setuptools.bbclass
> index ced9509..ba9cf13 100644
> --- a/meta/classes/setuptools.bbclass
> +++ b/meta/classes/setuptools.bbclass
> @@ -5,4 +5,5 @@ DEPENDS += "python-setuptools-native"
>  DISTUTILS_INSTALL_ARGS = "--root=${D} \
>      --single-version-externally-managed \
>      --prefix=${prefix} \
> +    --install-lib=${PYTHON_SITEPACKAGES_DIR} \
>      --install-data=${datadir}"

Looks good... stuff is packaged properly for the libdir = /usr/lib64

Thanks,
M

>
>
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
McClintock Matthew-B29882 - Aug. 24, 2012, 8:48 p.m.
On Fri, Aug 24, 2012 at 11:15 AM, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> The last two distutils changes progressively broke the builds. Firstly they
> moved things from the site_packages directory to being higher up the tree
> which introduced package QA warnings as a side effect. Secondly, it interacts
> badly with setuptools which passes in --root=${D} itself.
>
> This patch restores the original directory layout, hence fixing the QA
> warnings and also passes extra options to setuptools to deal with the
> --root option it passes.

Hmm, I think you don't need certain bits anymore

>
> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> ---
> diff --git a/meta/classes/distutils.bbclass b/meta/classes/distutils.bbclass
> index 52a1aa8..c73b24f 100644
> --- a/meta/classes/distutils.bbclass
> +++ b/meta/classes/distutils.bbclass
> @@ -38,7 +38,7 @@ distutils_do_install() {
>          STAGING_LIBDIR=${STAGING_LIBDIR} \
>          PYTHONPATH=${D}/${PYTHON_SITEPACKAGES_DIR} \
>          BUILD_SYS=${BUILD_SYS} HOST_SYS=${HOST_SYS} \
> -        ${STAGING_BINDIR_NATIVE}/python-native/python setup.py install --install-lib=${D}${libdir}/${PYTHON_DIR} ${DISTUTILS_INSTALL_ARGS} || \
> +        ${STAGING_BINDIR_NATIVE}/python-native/python setup.py install --install-lib=${D}/${PYTHON_SITEPACKAGES_DIR} ${DISTUTILS_INSTALL_ARGS} || \

This change should be handled by the bit below? So you could

s/--install-lib=${D}/${PYTHON_SITEPACKAGES_DIR}//

?

>          bbfatal "python setup.py install execution failed."
>
>          for i in `find ${D} -name "*.py"` ; do \
> diff --git a/meta/classes/setuptools.bbclass b/meta/classes/setuptools.bbclass
> index ced9509..ba9cf13 100644
> --- a/meta/classes/setuptools.bbclass
> +++ b/meta/classes/setuptools.bbclass
> @@ -5,4 +5,5 @@ DEPENDS += "python-setuptools-native"
>  DISTUTILS_INSTALL_ARGS = "--root=${D} \
>      --single-version-externally-managed \
>      --prefix=${prefix} \
> +    --install-lib=${PYTHON_SITEPACKAGES_DIR} \
>      --install-data=${datadir}"
>
>
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Andrei Gherzan - Aug. 24, 2012, 8:51 p.m.
On Fri, Aug 24, 2012 at 11:48 PM, McClintock Matthew-B29882 <
B29882@freescale.com> wrote:

> On Fri, Aug 24, 2012 at 11:15 AM, Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> > The last two distutils changes progressively broke the builds. Firstly
> they
> > moved things from the site_packages directory to being higher up the tree
> > which introduced package QA warnings as a side effect. Secondly, it
> interacts
> > badly with setuptools which passes in --root=${D} itself.
> >
> > This patch restores the original directory layout, hence fixing the QA
> > warnings and also passes extra options to setuptools to deal with the
> > --root option it passes.
>
>
Exactly what i was working on...


> Hmm, I think you don't need certain bits anymore
>
> >
> > Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> > ---
> > diff --git a/meta/classes/distutils.bbclass
> b/meta/classes/distutils.bbclass
> > index 52a1aa8..c73b24f 100644
> > --- a/meta/classes/distutils.bbclass
> > +++ b/meta/classes/distutils.bbclass
> > @@ -38,7 +38,7 @@ distutils_do_install() {
> >          STAGING_LIBDIR=${STAGING_LIBDIR} \
> >          PYTHONPATH=${D}/${PYTHON_SITEPACKAGES_DIR} \
> >          BUILD_SYS=${BUILD_SYS} HOST_SYS=${HOST_SYS} \
> > -        ${STAGING_BINDIR_NATIVE}/python-native/python setup.py install
> --install-lib=${D}${libdir}/${PYTHON_DIR} ${DISTUTILS_INSTALL_ARGS} || \
> > +        ${STAGING_BINDIR_NATIVE}/python-native/python setup.py install
> --install-lib=${D}/${PYTHON_SITEPACKAGES_DIR} ${DISTUTILS_INSTALL_ARGS} || \
>
> This change should be handled by the bit below? So you could
>
> s/--install-lib=${D}/${PYTHON_SITEPACKAGES_DIR}//
>
> ?
>
> >          bbfatal "python setup.py install execution failed."
> >
> >          for i in `find ${D} -name "*.py"` ; do \
> > diff --git a/meta/classes/setuptools.bbclass
> b/meta/classes/setuptools.bbclass
> > index ced9509..ba9cf13 100644
> > --- a/meta/classes/setuptools.bbclass
> > +++ b/meta/classes/setuptools.bbclass
> > @@ -5,4 +5,5 @@ DEPENDS += "python-setuptools-native"
> >  DISTUTILS_INSTALL_ARGS = "--root=${D} \
> >      --single-version-externally-managed \
> >      --prefix=${prefix} \
> > +    --install-lib=${PYTHON_SITEPACKAGES_DIR} \
> >      --install-data=${datadir}"
>

Exactly my fix. Please merge.

Andrei
McClintock Matthew-B29882 - Aug. 24, 2012, 8:56 p.m.
On Fri, Aug 24, 2012 at 3:51 PM, Andrei Gherzan <andrei@gherzan.ro> wrote:
> Exactly my fix. Please merge.

It's already been merged.

-M
Andrei Gherzan - Aug. 24, 2012, 8:58 p.m.
On Fri, Aug 24, 2012 at 11:56 PM, McClintock Matthew-B29882 <
B29882@freescale.com> wrote:

> On Fri, Aug 24, 2012 at 3:51 PM, Andrei Gherzan <andrei@gherzan.ro> wrote:
> > Exactly my fix. Please merge.
>
> It's already been merged.


Ding! Thanks.

ag
Richard Purdie - Aug. 24, 2012, 10:01 p.m.
On Fri, 2012-08-24 at 20:48 +0000, McClintock Matthew-B29882 wrote:
> On Fri, Aug 24, 2012 at 11:15 AM, Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> > The last two distutils changes progressively broke the builds. Firstly they
> > moved things from the site_packages directory to being higher up the tree
> > which introduced package QA warnings as a side effect. Secondly, it interacts
> > badly with setuptools which passes in --root=${D} itself.
> >
> > This patch restores the original directory layout, hence fixing the QA
> > warnings and also passes extra options to setuptools to deal with the
> > --root option it passes.
> 
> Hmm, I think you don't need certain bits anymore
> 
> >
> > Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> > ---
> > diff --git a/meta/classes/distutils.bbclass b/meta/classes/distutils.bbclass
> > index 52a1aa8..c73b24f 100644
> > --- a/meta/classes/distutils.bbclass
> > +++ b/meta/classes/distutils.bbclass
> > @@ -38,7 +38,7 @@ distutils_do_install() {
> >          STAGING_LIBDIR=${STAGING_LIBDIR} \
> >          PYTHONPATH=${D}/${PYTHON_SITEPACKAGES_DIR} \
> >          BUILD_SYS=${BUILD_SYS} HOST_SYS=${HOST_SYS} \
> > -        ${STAGING_BINDIR_NATIVE}/python-native/python setup.py install --install-lib=${D}${libdir}/${PYTHON_DIR} ${DISTUTILS_INSTALL_ARGS} || \
> > +        ${STAGING_BINDIR_NATIVE}/python-native/python setup.py install --install-lib=${D}/${PYTHON_SITEPACKAGES_DIR} ${DISTUTILS_INSTALL_ARGS} || \
> 
> This change should be handled by the bit below? So you could
> 
> s/--install-lib=${D}/${PYTHON_SITEPACKAGES_DIR}//
> 
> ?

Not everything uses setuptools, there are things using just disutils.
I'm open to further cleanup of this but we need to consider both
cases...

Cheers,

Richard

> >          bbfatal "python setup.py install execution failed."
> >
> >          for i in `find ${D} -name "*.py"` ; do \
> > diff --git a/meta/classes/setuptools.bbclass b/meta/classes/setuptools.bbclass
> > index ced9509..ba9cf13 100644
> > --- a/meta/classes/setuptools.bbclass
> > +++ b/meta/classes/setuptools.bbclass
> > @@ -5,4 +5,5 @@ DEPENDS += "python-setuptools-native"
> >  DISTUTILS_INSTALL_ARGS = "--root=${D} \
> >      --single-version-externally-managed \
> >      --prefix=${prefix} \
> > +    --install-lib=${PYTHON_SITEPACKAGES_DIR} \
> >      --install-data=${datadir}"
> >
> >
> >
> > _______________________________________________
> > Openembedded-core mailing list
> > Openembedded-core@lists.openembedded.org
> > http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>
McClintock Matthew-B29882 - Aug. 27, 2012, 7:23 p.m.
On Fri, Aug 24, 2012 at 5:01 PM, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> On Fri, 2012-08-24 at 20:48 +0000, McClintock Matthew-B29882 wrote:
>> On Fri, Aug 24, 2012 at 11:15 AM, Richard Purdie
>> <richard.purdie@linuxfoundation.org> wrote:
>> > The last two distutils changes progressively broke the builds. Firstly they
>> > moved things from the site_packages directory to being higher up the tree
>> > which introduced package QA warnings as a side effect. Secondly, it interacts
>> > badly with setuptools which passes in --root=${D} itself.
>> >
>> > This patch restores the original directory layout, hence fixing the QA
>> > warnings and also passes extra options to setuptools to deal with the
>> > --root option it passes.
>>
>> Hmm, I think you don't need certain bits anymore
>>
>> >
>> > Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
>> > ---
>> > diff --git a/meta/classes/distutils.bbclass b/meta/classes/distutils.bbclass
>> > index 52a1aa8..c73b24f 100644
>> > --- a/meta/classes/distutils.bbclass
>> > +++ b/meta/classes/distutils.bbclass
>> > @@ -38,7 +38,7 @@ distutils_do_install() {
>> >          STAGING_LIBDIR=${STAGING_LIBDIR} \
>> >          PYTHONPATH=${D}/${PYTHON_SITEPACKAGES_DIR} \
>> >          BUILD_SYS=${BUILD_SYS} HOST_SYS=${HOST_SYS} \
>> > -        ${STAGING_BINDIR_NATIVE}/python-native/python setup.py install --install-lib=${D}${libdir}/${PYTHON_DIR} ${DISTUTILS_INSTALL_ARGS} || \
>> > +        ${STAGING_BINDIR_NATIVE}/python-native/python setup.py install --install-lib=${D}/${PYTHON_SITEPACKAGES_DIR} ${DISTUTILS_INSTALL_ARGS} || \
>>
>> This change should be handled by the bit below? So you could
>>
>> s/--install-lib=${D}/${PYTHON_SITEPACKAGES_DIR}//
>>
>> ?
>
> Not everything uses setuptools, there are things using just disutils.
> I'm open to further cleanup of this but we need to consider both
> cases...

Ah right, I incorrectly assumed both of these always applied.

Thanks for looking at this.

-M

>
> Cheers,
>
> Richard
>
>> >          bbfatal "python setup.py install execution failed."
>> >
>> >          for i in `find ${D} -name "*.py"` ; do \
>> > diff --git a/meta/classes/setuptools.bbclass b/meta/classes/setuptools.bbclass
>> > index ced9509..ba9cf13 100644
>> > --- a/meta/classes/setuptools.bbclass
>> > +++ b/meta/classes/setuptools.bbclass
>> > @@ -5,4 +5,5 @@ DEPENDS += "python-setuptools-native"
>> >  DISTUTILS_INSTALL_ARGS = "--root=${D} \
>> >      --single-version-externally-managed \
>> >      --prefix=${prefix} \
>> > +    --install-lib=${PYTHON_SITEPACKAGES_DIR} \
>> >      --install-data=${datadir}"
>> >
>> >
>> >
>> > _______________________________________________
>> > Openembedded-core mailing list
>> > Openembedded-core@lists.openembedded.org
>> > http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>>
>
>
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core

Patch

diff --git a/meta/classes/distutils.bbclass b/meta/classes/distutils.bbclass
index 52a1aa8..c73b24f 100644
--- a/meta/classes/distutils.bbclass
+++ b/meta/classes/distutils.bbclass
@@ -38,7 +38,7 @@  distutils_do_install() {
         STAGING_LIBDIR=${STAGING_LIBDIR} \
         PYTHONPATH=${D}/${PYTHON_SITEPACKAGES_DIR} \
         BUILD_SYS=${BUILD_SYS} HOST_SYS=${HOST_SYS} \
-        ${STAGING_BINDIR_NATIVE}/python-native/python setup.py install --install-lib=${D}${libdir}/${PYTHON_DIR} ${DISTUTILS_INSTALL_ARGS} || \
+        ${STAGING_BINDIR_NATIVE}/python-native/python setup.py install --install-lib=${D}/${PYTHON_SITEPACKAGES_DIR} ${DISTUTILS_INSTALL_ARGS} || \
         bbfatal "python setup.py install execution failed."
 
         for i in `find ${D} -name "*.py"` ; do \
diff --git a/meta/classes/setuptools.bbclass b/meta/classes/setuptools.bbclass
index ced9509..ba9cf13 100644
--- a/meta/classes/setuptools.bbclass
+++ b/meta/classes/setuptools.bbclass
@@ -5,4 +5,5 @@  DEPENDS += "python-setuptools-native"
 DISTUTILS_INSTALL_ARGS = "--root=${D} \
     --single-version-externally-managed \
     --prefix=${prefix} \
+    --install-lib=${PYTHON_SITEPACKAGES_DIR} \
     --install-data=${datadir}"