Patchwork [1/2] easyinstall: class to create and extend the easyinstall.pth file

login
register
mail settings
Submitter Lukas Bulwahn
Date Dec. 18, 2012, 1:58 p.m.
Message ID <1355839120-12142-2-git-send-email-lukas.bulwahn@oss.bmw-carit.de>
Download mbox | patch
Permalink /patch/41271/
State New
Headers show

Comments

Lukas Bulwahn - Dec. 18, 2012, 1:58 p.m.
From: Lukas Bulwahn <lukas.bulwahn@bmw-carit.de>

When installing python packages that are installed with python's easyinstall,
the installation creates easyinstall.pth files. This file must exist in the
root filesystem so that python 2.7 can locate and import the installed modules
in the host system.

Due to the import of the changeset aaaac4c545623d8818de2d3fa71d9d63b717c575
from the Classic OpenEmbedded Development Tree in changeset
733dc27ecf485a0ec5318534dfbbc9491c08df8f, the easyinstall.pth file is removed.
The original changeset claims that python 2.5 can import packages without the
easyinstall.pth file. However in my experience, python 2.7 cannot import
packages without this file. Furthermore, the easyinstall.pth file must
accumulate entries when multiple packages are installed with easyinstall.

The easyinstall class provides a simple postinst script that adds the content
of the variable EASYINSTALL_${PN} to the easyinstall.pth file and initializes
the file if necessary.
---
 meta/classes/easyinstall.bbclass |   11 +++++++++++
 1 file changed, 11 insertions(+)
 create mode 100644 meta/classes/easyinstall.bbclass
Otavio Salvador - Dec. 18, 2012, 2:27 p.m.
On Tue, Dec 18, 2012 at 11:58 AM, Lukas Bulwahn
<lukas.bulwahn@oss.bmw-carit.de> wrote:
> From: Lukas Bulwahn <lukas.bulwahn@bmw-carit.de>
>
> When installing python packages that are installed with python's easyinstall,
> the installation creates easyinstall.pth files. This file must exist in the
> root filesystem so that python 2.7 can locate and import the installed modules
> in the host system.
>
> Due to the import of the changeset aaaac4c545623d8818de2d3fa71d9d63b717c575
> from the Classic OpenEmbedded Development Tree in changeset
> 733dc27ecf485a0ec5318534dfbbc9491c08df8f, the easyinstall.pth file is removed.
> The original changeset claims that python 2.5 can import packages without the
> easyinstall.pth file. However in my experience, python 2.7 cannot import
> packages without this file. Furthermore, the easyinstall.pth file must
> accumulate entries when multiple packages are installed with easyinstall.
>
> The easyinstall class provides a simple postinst script that adds the content
> of the variable EASYINSTALL_${PN} to the easyinstall.pth file and initializes
> the file if necessary.
> ---
>  meta/classes/easyinstall.bbclass |   11 +++++++++++
>  1 file changed, 11 insertions(+)
>  create mode 100644 meta/classes/easyinstall.bbclass
>
> diff --git a/meta/classes/easyinstall.bbclass b/meta/classes/easyinstall.bbclass
> new file mode 100644
> index 0000000..8ae975d
> --- /dev/null
> +++ b/meta/classes/easyinstall.bbclass
> @@ -0,0 +1,11 @@
> +
> +pkg_postinst_${PN} () {
> +       if test ! -e ${PYTHON_SITEPACKAGES_DIR}/easy-install.pth; then
> +            cat > ${PYTHON_SITEPACKAGES_DIR}/easy-install.pth << EOF
> +import sys; sys.__plen = len(sys.path)
> +import sys; new=sys.path[sys.__plen:]; del sys.path[sys.__plen:]; p=getattr(sys,'__egginsert',0); sys.path[p:p]=new; sys.__egginsert = p+len(new)

All this need to be in a single line? Seems ugly and hard to read for me.

> +EOF
> +        fi
> +        sed -i '$ i ${EASYINSTALL_${PN}}' ${PYTHON_SITEPACKAGES_DIR}/easy-install.pth
> +}
> +
> --
> 1.7.9.5
>
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core



--
Otavio Salvador                             O.S. Systems
E-mail: otavio@ossystems.com.br  http://www.ossystems.com.br
Mobile: +55 53 9981-7854              http://projetos.ossystems.com.br
Ross Burton - Dec. 18, 2012, 2:55 p.m.
On 18 December 2012 13:58, Lukas Bulwahn <lukas.bulwahn@oss.bmw-carit.de> wrote:
> When installing python packages that are installed with python's easyinstall,
> the installation creates easyinstall.pth files. This file must exist in the
> root filesystem so that python 2.7 can locate and import the installed modules
> in the host system.

Is this the logic that where packages get installed as Eggs, so into
e.g. /usr/lib/python2.7/site-packages/Foo-1.0/ instead of
../packages/foo)?  In that case we're basically having to deal with a
lot of complication.   I've been battling this a bit with python-mako
and this section of the setuptools document applies to us:

"However, packaging tools that build binary distributions by running
setup.py install on the command line or as a subprocess will require
modification to work with setuptools. They should use the
--single-version-externally-managed option to the install command,
combined with the standard --root or --record options. See the install
command documentation below for more details."

http://peak.telecommunity.com/DevCenter/setuptools#what-your-users-should-know
http://peak.telecommunity.com/DevCenter/setuptools#install-command

I'm all in favour of a setuptools class, but I think it should follow
what the documentation says and use --s-v-e-m.   My recent python-mako
patch to oe-devel uses these options to do a "old-school" install
which doesn't mean dealing with .pth files, as we're a perfectly good
package management system already.

Ross
Lukas Bulwahn - Dec. 19, 2012, 12:18 p.m.
On 12/18/2012 03:55 PM, Burton, Ross wrote:
> On 18 December 2012 13:58, Lukas Bulwahn <lukas.bulwahn@oss.bmw-carit.de> wrote:
>> When installing python packages that are installed with python's easyinstall,
>> the installation creates easyinstall.pth files. This file must exist in the
>> root filesystem so that python 2.7 can locate and import the installed modules
>> in the host system.
> Is this the logic that where packages get installed as Eggs, so into
> e.g. /usr/lib/python2.7/site-packages/Foo-1.0/ instead of
> ../packages/foo)?  In that case we're basically having to deal with a
> lot of complication.   I've been battling this a bit with python-mako
> and this section of the setuptools document applies to us:
>
> "However, packaging tools that build binary distributions by running
> setup.py install on the command line or as a subprocess will require
> modification to work with setuptools. They should use the
> --single-version-externally-managed option to the install command,
> combined with the standard --root or --record options. See the install
> command documentation below for more details."
>
> http://peak.telecommunity.com/DevCenter/setuptools#what-your-users-should-know
> http://peak.telecommunity.com/DevCenter/setuptools#install-command
I think your solution is a simpler and better one that I did not think of.

> I'm all in favour of a setuptools class, but I think it should follow
> what the documentation says and use --s-v-e-m.   My recent python-mako
> patch to oe-devel uses these options to do a "old-school" install
> which doesn't mean dealing with .pth files, as we're a perfectly good
> package management system already.
I am also in favor of that.

 From your description I could not find the version of python-mako that 
uses the --s-v-e-m option.
Where can I find this version of the file? Maybe I was just looking at 
the wrong repository...

Lukas
Ross Burton - Dec. 19, 2012, 12:24 p.m.
Hi Lukas,

On 19 December 2012 12:18, Lukas Bulwahn <lukas.bulwahn@oss.bmw-carit.de> wrote:
> From your description I could not find the version of python-mako that uses
> the --s-v-e-m option.
> Where can I find this version of the file? Maybe I was just looking at the
> wrong repository...

The patch isn't in meta-oe yet, it's in the list archives somewhere
but it's also on my branch here:

http://git.openembedded.org/meta-openembedded-contrib/commit/?h=ross/piglit&id=2d6ae0a328c2e3d24e43b2858c70a5f404796d21

Ross
Ross Burton - Dec. 20, 2012, 11:34 a.m.
On 19 December 2012 12:24, Burton, Ross <ross.burton@intel.com> wrote:
>> From your description I could not find the version of python-mako that uses
>> the --s-v-e-m option.
>> Where can I find this version of the file? Maybe I was just looking at the
>> wrong repository...
>
> The patch isn't in meta-oe yet, it's in the list archives somewhere
> but it's also on my branch here:
>
> http://git.openembedded.org/meta-openembedded-contrib/commit/?h=ross/piglit&id=2d6ae0a328c2e3d24e43b2858c70a5f404796d21

Not sure how I failed to notice this, but there's a setuptools.bbclass
in oe-core:

inherit distutils

DEPENDS += "python-setuptools-native"

DISTUTILS_INSTALL_ARGS = "--root=${D} \
    --single-version-externally-managed \
    --prefix=${prefix} \
    --install-lib=${PYTHON_SITEPACKAGES_DIR} \
    --install-data=${datadir}"

Yeah, that.  I'll fix python-mako and this should be suitable for you too.

Ross
Lukas Bulwahn - Jan. 9, 2013, 8:21 a.m.
On 12/20/2012 12:34 PM, Burton, Ross wrote:
> Not sure how I failed to notice this, but there's a setuptools.bbclass
> in oe-core:
>
> inherit distutils
>
> DEPENDS += "python-setuptools-native"
>
> DISTUTILS_INSTALL_ARGS = "--root=${D} \
>      --single-version-externally-managed \
>      --prefix=${prefix} \
>      --install-lib=${PYTHON_SITEPACKAGES_DIR} \
>      --install-data=${datadir}"
>
> Yeah, that.  I'll fix python-mako and this should be suitable for you too.
>
> Ross
Hi Ross,

the setuptools class does great work. Writing recipes and installing 
python packages now works like a charm.
As you probably noticed as well, to add a package, one must add recipes 
for the transitive closure of all required packages defined in the 
setup.py file with the lines install_requires=[.*] and setup_requires=[.*].

The previous patches are now completely obsolete.

Lukas

Patch

diff --git a/meta/classes/easyinstall.bbclass b/meta/classes/easyinstall.bbclass
new file mode 100644
index 0000000..8ae975d
--- /dev/null
+++ b/meta/classes/easyinstall.bbclass
@@ -0,0 +1,11 @@ 
+
+pkg_postinst_${PN} () {
+       if test ! -e ${PYTHON_SITEPACKAGES_DIR}/easy-install.pth; then
+            cat > ${PYTHON_SITEPACKAGES_DIR}/easy-install.pth << EOF
+import sys; sys.__plen = len(sys.path)
+import sys; new=sys.path[sys.__plen:]; del sys.path[sys.__plen:]; p=getattr(sys,'__egginsert',0); sys.path[p:p]=new; sys.__egginsert = p+len(new)
+EOF
+        fi
+        sed -i '$ i ${EASYINSTALL_${PN}}' ${PYTHON_SITEPACKAGES_DIR}/easy-install.pth
+}
+