cooker: Fix inotify watches causing memory resident bitbake corruption

Message ID 20220325150919.1361415-1-richard.purdie@linuxfoundation.org
State Accepted, archived
Commit 2c414f659d793d732041614caedd773959eb4f27
Headers show
Series cooker: Fix inotify watches causing memory resident bitbake corruption | expand

Commit Message

Richard Purdie March 25, 2022, 3:09 p.m. UTC
Thanks to great debugging from pavel@zhukoff.net we had a simpler reproducer
for the corruption see in oe-selftest when using BB_SERVER_TIMEOUT=60, i.e.
with bitbake in memory resident mode. This was effectively:

oe-selftest -r devtool.DevtoolUpgradeTests.test_devtool_upgrade devtool.DevtoolUpgradeTests.test_devtool_upgrade_git -j 1 -K

The issue is that if directories are removed (such as workspace), if
they are added again, we don't have the watches in place any more. This
patch adds some slightly paranoid checks to ensure we do the correct things
for directory additions and removals (we track directories, not files
specifically to avoid running out of watches).

[YOCTO #14023]

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
 lib/bb/cooker.py | 10 ++++++++++
 1 file changed, 10 insertions(+)

Patch

diff --git a/lib/bb/cooker.py b/lib/bb/cooker.py
index c0a7a2fd79..eac956aa97 100644
--- a/lib/bb/cooker.py
+++ b/lib/bb/cooker.py
@@ -253,6 +253,11 @@  class BBCooker:
             return
         if not event.pathname in self.configwatcher.bbwatchedfiles:
             return
+        if "IN_ISDIR" in event.maskname:
+            if "IN_CREATE" in event.maskname:
+                self.add_filewatch([[event.pathname]], watcher=self.configwatcher, dirs=True)
+            elif "IN_DELETE" in event.maskname and event.pathname in self.watcher.bbseen:
+                self.configwatcher.bbseen.remove(event.pathname)
         if not event.pathname in self.inotify_modified_files:
             self.inotify_modified_files.append(event.pathname)
         self.baseconfig_valid = False
@@ -266,6 +271,11 @@  class BBCooker:
         if event.pathname.endswith("bitbake-cookerdaemon.log") \
                 or event.pathname.endswith("bitbake.lock"):
             return
+        if "IN_ISDIR" in event.maskname:
+            if "IN_CREATE" in event.maskname:
+                self.add_filewatch([[event.pathname]], dirs=True)
+            elif "IN_DELETE" in event.maskname and event.pathname in self.watcher.bbseen:
+                self.watcher.bbseen.remove(event.pathname)
         if not event.pathname in self.inotify_modified_files:
             self.inotify_modified_files.append(event.pathname)
         self.parsecache_valid = False