diff mbox series

[kirkstone,07/13] python3: Ensure stale empty python module directories don't break the build

Message ID cdb6879b49d4f6dc2bea8ff064a5b1e62db34781.1655848177.git.steve@sakoman.com
State Accepted, archived
Commit cdb6879b49d4f6dc2bea8ff064a5b1e62db34781
Headers show
Series [kirkstone,01/13] cve-check: add support for Ignored CVEs | expand

Commit Message

Steve Sakoman June 21, 2022, 11:27 p.m. UTC 
From: Richard Purdie <richard.purdie@linuxfoundation.org>

There are two issues inside importlib. Firstly, the modules are accessed in
on disk order. This means behaviour seen on one system might not reproduce
on another and is a real headache.

Secondly, empty directories left behind by previous modules might be looked
at. This has caused a long string of different issues for us.

As a result, patch this to a behaviour which works for us. Upstream discussion
can follow later, this is breaking builds for too many people to leave unpatched.

[YOCTO #14816]

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit e5944a38db513e033c3a3e9313267055f7254be7)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 .../python3/deterministic_imports.patch       | 32 +++++++++++++++++++
 .../recipes-devtools/python/python3_3.10.4.bb |  1 +
 2 files changed, 33 insertions(+)
 create mode 100644 meta/recipes-devtools/python/python3/deterministic_imports.patch
diff mbox series

Patch

diff --git a/meta/recipes-devtools/python/python3/deterministic_imports.patch b/meta/recipes-devtools/python/python3/deterministic_imports.patch
new file mode 100644
index 0000000000..296b413cab
--- /dev/null
+++ b/meta/recipes-devtools/python/python3/deterministic_imports.patch
@@ -0,0 +1,32 @@ 
+There are two issues here. Firstly, the modules are accessed in on disk order. This
+means behaviour seen on one system might not reproduce on another and is a real headache.
+
+Secondly, empty directories left behind by previous modules might be looked at. This
+has caused a long string of different issues for us.
+
+As a result, patch this to a behaviour which works for us.
+
+Upstream-Status: Pending [need to talk to upstream to see if they'll take one or both fixes]
+Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
+
+
+Index: Python-3.10.4/Lib/importlib/metadata/__init__.py
+===================================================================
+--- Python-3.10.4.orig/Lib/importlib/metadata/__init__.py
++++ Python-3.10.4/Lib/importlib/metadata/__init__.py
+@@ -819,7 +819,14 @@ class Lookup:
+         self.infos = FreezableDefaultDict(list)
+         self.eggs = FreezableDefaultDict(list)
+ 
+-        for child in path.children():
++        for child in sorted(path.children()):
++            childpath = pathlib.Path(path.root, child)
++            try:
++                if childpath.is_dir() and not any(childpath.iterdir()):
++                    # Empty directories aren't interesting
++                    continue
++            except PermissionError:
++                continue
+             low = child.lower()
+             if low.endswith((".dist-info", ".egg-info")):
+                 # rpartition is faster than splitext and suitable for this purpose.
diff --git a/meta/recipes-devtools/python/python3_3.10.4.bb b/meta/recipes-devtools/python/python3_3.10.4.bb
index 9f1b96b519..6bd3a6aba8 100644
--- a/meta/recipes-devtools/python/python3_3.10.4.bb
+++ b/meta/recipes-devtools/python/python3_3.10.4.bb
@@ -33,6 +33,7 @@  SRC_URI = "http://www.python.org/ftp/python/${PV}/Python-${PV}.tar.xz \
            file://0001-sysconfig.py-use-platlibdir-also-for-purelib.patch \
            file://0001-Lib-pty.py-handle-stdin-I-O-errors-same-way-as-maste.patch \
            file://0001-setup.py-Do-not-detect-multiarch-paths-when-cross-co.patch \
+           file://deterministic_imports.patch \
            "
 
 SRC_URI:append:class-native = " \