diff mbox series

[1.50,7/8] cooker: Handle parsing results queue race

Message ID d0fe9d4efa81bc67e6b0d64c9fe58a4115b0c4d8.1637547674.git.anuj.mittal@intel.com
State Accepted, archived
Commit 4a06b2fa37648204cab41a5cbfb6fa217aa32686
Headers show
Series [1.50,1/8] bitbake: correct the collections vs collections.abc deprecation | expand

Commit Message

Mittal, Anuj Nov. 22, 2021, 2:24 a.m. UTC 
From: Richard Purdie <richard.purdie@linuxfoundation.org>

The previous fix introduced a race where the queue might not be empty
but all the parser processes have exited. Handle this correctly to avoid
occasional errors.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 8e7f2b6500e26610f52d128b48ca0a09bf6fb2cb)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
---
 lib/bb/cooker.py | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)
diff mbox series

Patch

diff --git a/lib/bb/cooker.py b/lib/bb/cooker.py
index c952c574..e2a5dc43 100644
--- a/lib/bb/cooker.py
+++ b/lib/bb/cooker.py
@@ -2214,24 +2214,28 @@  class CookerParser(object):
             yield not cached, mc, infos
 
     def parse_generator(self):
-        while self.processes:
+        empty = False
+        while self.processes or not empty:
+            for process in self.processes.copy():
+                if not process.is_alive():
+                    process.join()
+                    self.processes.remove(process)
+
             if self.parsed >= self.toparse:
                 break
 
             try:
                 result = self.result_queue.get(timeout=0.25)
             except queue.Empty:
+                empty = True
                 pass
             else:
+                empty = False
                 value = result[1]
                 if isinstance(value, BaseException):
                     raise value
                 else:
                     yield result
-            for process in self.processes.copy():
-                if not process.is_alive():
-                    process.join()
-                    self.processes.remove(process)
 
         if not (self.parsed >= self.toparse):
             raise bb.parse.ParseError("Not all recipes parsed, parser thread killed/died? Exiting.", None)