diff mbox series

[1.52,3/4] cooker: Handle parsing results queue race

Message ID 8eaddb92a5fd14de6b5995aa92a6eed03b90a252.1637079892.git.anuj.mittal@intel.com
State Accepted, archived
Commit 8eaddb92a5fd14de6b5995aa92a6eed03b90a252
Headers show
Series [1.52,1/4] cooker: Handle parse threads disappearing to avoid hangs | expand

Commit Message

Mittal, Anuj Nov. 16, 2021, 4:26 p.m. UTC 
From: Richard Purdie <richard.purdie@linuxfoundation.org>

The previous fix introduced a race where the queue might not be empty
but all the parser processes have exited. Handle this correctly to avoid
occasional errors.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 8e7f2b6500e26610f52d128b48ca0a09bf6fb2cb)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
---
 lib/bb/cooker.py | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)
diff mbox series

Patch

diff --git a/lib/bb/cooker.py b/lib/bb/cooker.py
index 9be6603f..c032762f 100644
--- a/lib/bb/cooker.py
+++ b/lib/bb/cooker.py
@@ -2218,24 +2218,28 @@  class CookerParser(object):
             yield not cached, mc, infos
 
     def parse_generator(self):
-        while self.processes:
+        empty = False
+        while self.processes or not empty:
+            for process in self.processes.copy():
+                if not process.is_alive():
+                    process.join()
+                    self.processes.remove(process)
+
             if self.parsed >= self.toparse:
                 break
 
             try:
                 result = self.result_queue.get(timeout=0.25)
             except queue.Empty:
+                empty = True
                 pass
             else:
+                empty = False
                 value = result[1]
                 if isinstance(value, BaseException):
                     raise value
                 else:
                     yield result
-            for process in self.processes.copy():
-                if not process.is_alive():
-                    process.join()
-                    self.processes.remove(process)
 
         if not (self.parsed >= self.toparse):
             raise bb.parse.ParseError("Not all recipes parsed, parser thread killed/died? Exiting.", None)