diff mbox series

[2/2] python3: Ensure stale empty python module directories don't break the build

Message ID 20220527160825.461505-2-richard.purdie@linuxfoundation.org
State Accepted, archived
Commit e5944a38db513e033c3a3e9313267055f7254be7
Headers show
Series [1/2] python3: Remove problematic paths from sysroot files | expand

Commit Message

Richard Purdie May 27, 2022, 4:08 p.m. UTC 
There are two issues inside importlib. Firstly, the modules are accessed in
on disk order. This means behaviour seen on one system might not reproduce
on another and is a real headache.

Secondly, empty directories left behind by previous modules might be looked
at. This has caused a long string of different issues for us.

As a result, patch this to a behaviour which works for us. Upstream discussion
can follow later, this is breaking builds for too many people to leave unpatched.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
 .../python3/deterministic_imports.patch       | 29 +++++++++++++++++++
 .../recipes-devtools/python/python3_3.10.4.bb |  1 +
 2 files changed, 30 insertions(+)
 create mode 100644 meta/recipes-devtools/python/python3/deterministic_imports.patch
diff mbox series

Patch

diff --git a/meta/recipes-devtools/python/python3/deterministic_imports.patch b/meta/recipes-devtools/python/python3/deterministic_imports.patch
new file mode 100644
index 00000000000..53cd9fac91b
--- /dev/null
+++ b/meta/recipes-devtools/python/python3/deterministic_imports.patch
@@ -0,0 +1,29 @@ 
+There are two issues here. Firstly, the modules are accessed in on disk order. This
+means behaviour seen on one system might not reproduce on another and is a real headache.
+
+Secondly, empty directories left behind by previous modules might be looked at. This
+has caused a long string of different issues for us.
+
+As a result, patch this to a behaviour which works for us.
+
+Upstream-Status: Pending [need to talk to upstream to see if they'll take one or both fixes]
+Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
+
+
+Index: Python-3.10.4/Lib/importlib/metadata/__init__.py
+===================================================================
+--- Python-3.10.4.orig/Lib/importlib/metadata/__init__.py
++++ Python-3.10.4/Lib/importlib/metadata/__init__.py
+@@ -819,7 +819,11 @@ class Lookup:
+         self.infos = FreezableDefaultDict(list)
+         self.eggs = FreezableDefaultDict(list)
+ 
+-        for child in path.children():
++        for child in sorted(path.children()):
++            childpath = pathlib.Path(path.root, child)
++            if childpath.is_dir() and not any(childpath.iterdir()):
++                # Empty directories aren't interesting
++                continue
+             low = child.lower()
+             if low.endswith((".dist-info", ".egg-info")):
+                 # rpartition is faster than splitext and suitable for this purpose.
diff --git a/meta/recipes-devtools/python/python3_3.10.4.bb b/meta/recipes-devtools/python/python3_3.10.4.bb
index 9f1b96b5192..6bd3a6aba84 100644
--- a/meta/recipes-devtools/python/python3_3.10.4.bb
+++ b/meta/recipes-devtools/python/python3_3.10.4.bb
@@ -33,6 +33,7 @@  SRC_URI = "http://www.python.org/ftp/python/${PV}/Python-${PV}.tar.xz \
            file://0001-sysconfig.py-use-platlibdir-also-for-purelib.patch \
            file://0001-Lib-pty.py-handle-stdin-I-O-errors-same-way-as-maste.patch \
            file://0001-setup.py-Do-not-detect-multiarch-paths-when-cross-co.patch \
+           file://deterministic_imports.patch \
            "
 
 SRC_URI:append:class-native = " \