[bitbake-devel,3/3] cooker: Avoid parser deadlocks
Submitted by Richard Purdie on Sept. 3, 2020, 11:24 a.m.
|
Patch ID: 175998
Details
Commit Message
@@ -1991,7 +1991,8 @@ class Parser(multiprocessing.Process):
except queue.Empty:
pass
else:
- self.results.cancel_join_thread()
+ self.results.close()
+ self.results.join_thread()
break
if pending:
@@ -2000,6 +2001,8 @@ class Parser(multiprocessing.Process):
try:
job = self.jobs.pop()
except IndexError:
+ self.results.close()
+ self.results.join_thread()
break
result = self.parse(*job)
# Clear the siggen cache after parsing to control memory usage, its huge
@@ -2119,10 +2122,11 @@ class CookerParser(object):
bb.event.fire(event, self.cfgdata)
- # Allow data left in the cancel queue to be discarded
- self.parser_quit.cancel_join_thread()
for process in self.processes:
self.parser_quit.put(None)
+ self.parser_quit.close()
+ # Allow data left in the cancel queue to be discarded
+ self.parser_quit.cancel_join_thread()
# Cleanup the queue before call process.join(), otherwise there might be
# deadlocks.
If you make parsing fail (e.g. add something like: X := "${@d.getVar('MCMACHINES').split()[1]}" to meson.bbclass, then run "while true; do bitbake -g bash; done" it will eventually hang. It appears the cancel_join_thread() call the parsing failure triggers, breaks the results_queue badly enough that it sits in read() indefintely (called from self.result_queue.get(timeout=0.25)). The timeout only applies to lock aquisition, not the read call. I've tried various other approaches such as using cancel_join_thread() in other places but the only way things don't lock up is to avoid cancel_join_thread() entirely for results_queue. I do have a concern that this may adversely affect Ctrl+C handling but equally, its broken now already and this appears to improve things. [YOCTO #14034] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> --- lib/bb/cooker.py | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-)