| Message ID | 20220310114035.2226442-1-ross.burton@arm.com |
|---|---|
| State | Accepted, archived |
| Commit | d5d702a2cd06f863340f8e4cdce0904c9d86384d |
| Headers | show |
| Series | [1/3] pip_install_wheel: add a generic do_install for bootstrapping | expand |
On 10.03.22 12:40, Ross Burton wrote: > Several recipes are duplicating the same bootstrap logic for installing > a wheel without using any tools. Add an implementation to > pip_install_wheel to centralise the code, and remove the duplicated code > from the following recipes: > > - python3-flit-core > - python3-pip > - python3-setuptools > - python3-wheel > > Signed-off-by: Ross Burton <ross.burton@arm.com> > --- > meta/classes/pip_install_wheel.bbclass | 6 ++++++ > meta/recipes-devtools/python/python3-flit-core_3.7.1.bb | 3 +-- > meta/recipes-devtools/python/python3-pip_22.0.3.bb | 4 +--- > meta/recipes-devtools/python/python3-setuptools_59.5.0.bb | 4 +--- > meta/recipes-devtools/python/python3-wheel_0.37.1.bb | 4 +--- > 5 files changed, 10 insertions(+), 11 deletions(-) > > diff --git a/meta/classes/pip_install_wheel.bbclass b/meta/classes/pip_install_wheel.bbclass > index 3861dae75a..29cd544aa3 100644 > --- a/meta/classes/pip_install_wheel.bbclass > +++ b/meta/classes/pip_install_wheel.bbclass > @@ -40,4 +40,10 @@ pip_install_wheel_do_install () { > done > } > > +# A manual do_install that just uses unzip for bootstrapping purposes. Callers should DEPEND on unzip-native. Minor nitpick - wouldn't it make sense to add a hard DEPENDS on unzip-native right here into the class - it should do no harm when done like DEPENDS:append:class-native = " unzip-native" > +pip_install_wheel_do_bootstrap_install () { > + install -d ${D}${PYTHON_SITEPACKAGES_DIR} > + unzip -d ${D}${PYTHON_SITEPACKAGES_DIR} ${PIP_INSTALL_DIST_PATH}/*.whl > +} I think these two lines could be moved into a generic do_install:native - and then removing all the override code from the recipes below. Or is there some special case I'm not seeing here that would require having this extra function. Thanks for the patches - after all these rounds code looks good and should provide a seamless transition for most of the layers > + > EXPORT_FUNCTIONS do_install > diff --git a/meta/recipes-devtools/python/python3-flit-core_3.7.1.bb b/meta/recipes-devtools/python/python3-flit-core_3.7.1.bb > index 196e391a8d..1f14345d50 100644 > --- a/meta/recipes-devtools/python/python3-flit-core_3.7.1.bb > +++ b/meta/recipes-devtools/python/python3-flit-core_3.7.1.bb > @@ -21,8 +21,7 @@ PYPI_PACKAGE = "flit" > PEP517_SOURCE_PATH = "${S}/flit_core" > > do_install:class-native () { > - install -d ${D}${PYTHON_SITEPACKAGES_DIR} > - unzip -d ${D}${PYTHON_SITEPACKAGES_DIR} ${PIP_INSTALL_DIST_PATH}/flit_core*.whl > + pip_install_wheel_do_bootstrap_install > } > > PACKAGES =+ "${PN}-tests" > diff --git a/meta/recipes-devtools/python/python3-pip_22.0.3.bb b/meta/recipes-devtools/python/python3-pip_22.0.3.bb > index d7000b891a..f7fcf47c06 100644 > --- a/meta/recipes-devtools/python/python3-pip_22.0.3.bb > +++ b/meta/recipes-devtools/python/python3-pip_22.0.3.bb > @@ -19,9 +19,7 @@ SRC_URI += "file://reproducible.patch" > SRC_URI[sha256sum] = "f29d589df8c8ab99c060e68ad294c4a9ed896624f6368c5349d70aa581b333d0" > > do_install:class-native() { > - # Bootstrap to prevent dependency loop in python3-pip-native > - install -d ${D}${PYTHON_SITEPACKAGES_DIR} > - unzip -d ${D}${PYTHON_SITEPACKAGES_DIR} ${PIP_INSTALL_DIST_PATH}/*.whl > + pip_install_wheel_do_bootstrap_install > > # pip install would normally generate [console_scripts] in ${bindir} > install -d ${D}/${bindir} > diff --git a/meta/recipes-devtools/python/python3-setuptools_59.5.0.bb b/meta/recipes-devtools/python/python3-setuptools_59.5.0.bb > index 9d2c2fd744..7cd67b85f9 100644 > --- a/meta/recipes-devtools/python/python3-setuptools_59.5.0.bb > +++ b/meta/recipes-devtools/python/python3-setuptools_59.5.0.bb > @@ -22,9 +22,7 @@ DEPENDS:remove:class-native = "python3-pip-native python3-setuptools-native" > DEPENDS:append:class-native = " unzip-native" > > do_install:class-native() { > - # Bootstrap to prevent dependency loop in python3-pip-native > - install -d ${D}${PYTHON_SITEPACKAGES_DIR} > - unzip -d ${D}${PYTHON_SITEPACKAGES_DIR} ${PIP_INSTALL_DIST_PATH}/*.whl > + pip_install_wheel_do_bootstrap_install > } > > RDEPENDS:${PN} = "\ > diff --git a/meta/recipes-devtools/python/python3-wheel_0.37.1.bb b/meta/recipes-devtools/python/python3-wheel_0.37.1.bb > index 83a7c986a8..cf0d8191b4 100644 > --- a/meta/recipes-devtools/python/python3-wheel_0.37.1.bb > +++ b/meta/recipes-devtools/python/python3-wheel_0.37.1.bb > @@ -13,9 +13,7 @@ SRC_URI += " file://0001-Backport-pyproject.toml-from-flit-backend-branch.patch" > DEPENDS:remove:class-native = "python3-pip-native" > > do_install:class-native () { > - # We need to bootstrap python3-wheel-native > - install -d ${D}${PYTHON_SITEPACKAGES_DIR} > - unzip -d ${D}${PYTHON_SITEPACKAGES_DIR} ${PIP_INSTALL_DIST_PATH}/*.whl > + pip_install_wheel_do_bootstrap_install > > # pip install would normally generate [project.scripts] in ${bindir} > install -d ${D}/${bindir} > > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#163014): https://lists.openembedded.org/g/openembedded-core/message/163014 > Mute This Topic: https://lists.openembedded.org/mt/89684105/3647476 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [kweihmann@outlook.com] > -=-=-=-=-=-=-=-=-=-=-=- >
On Thu, 10 Mar 2022 at 11:48, Konrad Weihmann <kweihmann@outlook.com> wrote: > > +pip_install_wheel_do_bootstrap_install () { > > + install -d ${D}${PYTHON_SITEPACKAGES_DIR} > > + unzip -d ${D}${PYTHON_SITEPACKAGES_DIR} ${PIP_INSTALL_DIST_PATH}/*.whl > > +} > > I think these two lines could be moved into a generic do_install:native > - and then removing all the override code from the recipes below. There's only a few recipes which need this bootstrap, 99% of native recipes can use pip: it's just flit_core/pip/setuptools/wheel which can't as they're needed to build pip itself. (and once installer is used instead of pip, it's just installer and flit_core) I may send a v2 to this, as we should be byte-compiling the bootstrap code too. Ross
On Thu, 10 Mar 2022 at 14:01, Ross Burton <ross@burtonini.com> wrote: > > On Thu, 10 Mar 2022 at 11:48, Konrad Weihmann <kweihmann@outlook.com> wrote: > > > +pip_install_wheel_do_bootstrap_install () { > > > + install -d ${D}${PYTHON_SITEPACKAGES_DIR} > > > + unzip -d ${D}${PYTHON_SITEPACKAGES_DIR} ${PIP_INSTALL_DIST_PATH}/*.whl > > > +} > > > > I think these two lines could be moved into a generic do_install:native > > - and then removing all the override code from the recipes below. > > There's only a few recipes which need this bootstrap, 99% of native > recipes can use pip: it's just flit_core/pip/setuptools/wheel which > can't as they're needed to build pip itself. > > (and once installer is used instead of pip, it's just installer and flit_core) > > I may send a v2 to this, as we should be byte-compiling the bootstrap code too. I won't send a v2 as this causes interesting problems which are not easy to fix... Ross
diff --git a/meta/classes/pip_install_wheel.bbclass b/meta/classes/pip_install_wheel.bbclass index 3861dae75a..29cd544aa3 100644 --- a/meta/classes/pip_install_wheel.bbclass +++ b/meta/classes/pip_install_wheel.bbclass @@ -40,4 +40,10 @@ pip_install_wheel_do_install () { done } +# A manual do_install that just uses unzip for bootstrapping purposes. Callers should DEPEND on unzip-native. +pip_install_wheel_do_bootstrap_install () { + install -d ${D}${PYTHON_SITEPACKAGES_DIR} + unzip -d ${D}${PYTHON_SITEPACKAGES_DIR} ${PIP_INSTALL_DIST_PATH}/*.whl +} + EXPORT_FUNCTIONS do_install diff --git a/meta/recipes-devtools/python/python3-flit-core_3.7.1.bb b/meta/recipes-devtools/python/python3-flit-core_3.7.1.bb index 196e391a8d..1f14345d50 100644 --- a/meta/recipes-devtools/python/python3-flit-core_3.7.1.bb +++ b/meta/recipes-devtools/python/python3-flit-core_3.7.1.bb @@ -21,8 +21,7 @@ PYPI_PACKAGE = "flit" PEP517_SOURCE_PATH = "${S}/flit_core" do_install:class-native () { - install -d ${D}${PYTHON_SITEPACKAGES_DIR} - unzip -d ${D}${PYTHON_SITEPACKAGES_DIR} ${PIP_INSTALL_DIST_PATH}/flit_core*.whl + pip_install_wheel_do_bootstrap_install } PACKAGES =+ "${PN}-tests" diff --git a/meta/recipes-devtools/python/python3-pip_22.0.3.bb b/meta/recipes-devtools/python/python3-pip_22.0.3.bb index d7000b891a..f7fcf47c06 100644 --- a/meta/recipes-devtools/python/python3-pip_22.0.3.bb +++ b/meta/recipes-devtools/python/python3-pip_22.0.3.bb @@ -19,9 +19,7 @@ SRC_URI += "file://reproducible.patch" SRC_URI[sha256sum] = "f29d589df8c8ab99c060e68ad294c4a9ed896624f6368c5349d70aa581b333d0" do_install:class-native() { - # Bootstrap to prevent dependency loop in python3-pip-native - install -d ${D}${PYTHON_SITEPACKAGES_DIR} - unzip -d ${D}${PYTHON_SITEPACKAGES_DIR} ${PIP_INSTALL_DIST_PATH}/*.whl + pip_install_wheel_do_bootstrap_install # pip install would normally generate [console_scripts] in ${bindir} install -d ${D}/${bindir} diff --git a/meta/recipes-devtools/python/python3-setuptools_59.5.0.bb b/meta/recipes-devtools/python/python3-setuptools_59.5.0.bb index 9d2c2fd744..7cd67b85f9 100644 --- a/meta/recipes-devtools/python/python3-setuptools_59.5.0.bb +++ b/meta/recipes-devtools/python/python3-setuptools_59.5.0.bb @@ -22,9 +22,7 @@ DEPENDS:remove:class-native = "python3-pip-native python3-setuptools-native" DEPENDS:append:class-native = " unzip-native" do_install:class-native() { - # Bootstrap to prevent dependency loop in python3-pip-native - install -d ${D}${PYTHON_SITEPACKAGES_DIR} - unzip -d ${D}${PYTHON_SITEPACKAGES_DIR} ${PIP_INSTALL_DIST_PATH}/*.whl + pip_install_wheel_do_bootstrap_install } RDEPENDS:${PN} = "\ diff --git a/meta/recipes-devtools/python/python3-wheel_0.37.1.bb b/meta/recipes-devtools/python/python3-wheel_0.37.1.bb index 83a7c986a8..cf0d8191b4 100644 --- a/meta/recipes-devtools/python/python3-wheel_0.37.1.bb +++ b/meta/recipes-devtools/python/python3-wheel_0.37.1.bb @@ -13,9 +13,7 @@ SRC_URI += " file://0001-Backport-pyproject.toml-from-flit-backend-branch.patch" DEPENDS:remove:class-native = "python3-pip-native" do_install:class-native () { - # We need to bootstrap python3-wheel-native - install -d ${D}${PYTHON_SITEPACKAGES_DIR} - unzip -d ${D}${PYTHON_SITEPACKAGES_DIR} ${PIP_INSTALL_DIST_PATH}/*.whl + pip_install_wheel_do_bootstrap_install # pip install would normally generate [project.scripts] in ${bindir} install -d ${D}/${bindir}
Several recipes are duplicating the same bootstrap logic for installing a wheel without using any tools. Add an implementation to pip_install_wheel to centralise the code, and remove the duplicated code from the following recipes: - python3-flit-core - python3-pip - python3-setuptools - python3-wheel Signed-off-by: Ross Burton <ross.burton@arm.com> --- meta/classes/pip_install_wheel.bbclass | 6 ++++++ meta/recipes-devtools/python/python3-flit-core_3.7.1.bb | 3 +-- meta/recipes-devtools/python/python3-pip_22.0.3.bb | 4 +--- meta/recipes-devtools/python/python3-setuptools_59.5.0.bb | 4 +--- meta/recipes-devtools/python/python3-wheel_0.37.1.bb | 4 +--- 5 files changed, 10 insertions(+), 11 deletions(-)