[bitbake-devel,1/1] cooker: fix watching directories with Python 3.6+

Submitted by Paul Eggleton on Oct. 19, 2017, 3:32 a.m. | Patch ID: 145092


Message ID 0ee582a4068cf1d54d247d8da274f933055bc700.1508383917.git.paul.eggleton@linux.intel.com
State Accepted
Commit d02e90db32e7ee341c2ba3be79b0627d8796bdd6
Headers show

Commit Message

Paul Eggleton Oct. 19, 2017, 3:32 a.m.
In Python 3.6, glob.glob() was reimplemented to use os.scandir() (which
itself appeared in Python 3.5), thus our monkey patching of os.listdir()
here was no longer effective. The end result was not only that bitbake
wouldn't notice added recipes or bbappends with BB_SERVER_TIMEOUT set
when being run with Python 3.6 (the shipped Python version on Fedora 26
and some other distribution versions), it also broke devtool modify,
devtool upgrade and devtool extract since they rely on the ability to
create a bbappend on the fly and have bitbake pick it up.

To fix it, do the same monkey patching for os.scandir(), which needs to
be conditional upon that actually existing since we have to support
Python 3.4 that doesn't have it. Long term we should probably look for a
better way to handle this that doesn't involve monkey patching Python
library code.

Fixes [YOCTO #12185].

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
 lib/bb/cooker.py | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

Patch hide | download patch | download mbox

diff --git a/lib/bb/cooker.py b/lib/bb/cooker.py
index 73a2fef..c7fdd72 100644
--- a/lib/bb/cooker.py
+++ b/lib/bb/cooker.py
@@ -1686,15 +1686,23 @@  class CookerCollectFiles(object):
         # We need to track where we look so that we can add inotify watches. There
         # is no nice way to do this, this is horrid. We intercept the os.listdir()
-        # calls while we run glob().
+        # (or os.scandir() for python 3.6+) calls while we run glob().
         origlistdir = os.listdir
+        if hasattr(os, 'scandir'):
+            origscandir = os.scandir
         searchdirs = []
         def ourlistdir(d):
             return origlistdir(d)
+        def ourscandir(d):
+            searchdirs.append(d)
+            return origscandir(d)
         os.listdir = ourlistdir
+        if hasattr(os, 'scandir'):
+            os.scandir = ourscandir
             # Can't use set here as order is important
             newfiles = []
@@ -1714,6 +1722,8 @@  class CookerCollectFiles(object):
             os.listdir = origlistdir
+            if hasattr(os, 'scandir'):
+                os.scandir = origscandir
         bbmask = config.getVar('BBMASK')