[bitbake-devel,dunfell,1.46,8/8] server/process: Ensure UI-less servers don't sit in infinite loops

Submitted by Steve Sakoman on July 15, 2020, 2:26 p.m. | Patch ID: 174458

Details

Message ID 81cad9b8c4df15218d1a419c6b8e3ac73b54600c.1594822992.git.steve@sakoman.com
State New
Headers show

Commit Message

Steve Sakoman July 15, 2020, 2:26 p.m.
From: Richard Purdie <richard.purdie@linuxfoundation.org>

If server startup is broken for some reason (e.g. lockfile issues)
and no UI connection is made, the server will just sit inifinitely
waiting.

Add a timeout upon startup in the non-memory resident case so that
such infinite waits are avoided. In the memory resident case, the
server wouldn't have shut down in the first place or will timeout
according to configuration.

Since any race may mean the socket file is no longer present, ensure
the unlink doesn't fault upon exit, thus ensuring any hashequiv or
PRServ is removed from memory, allowing all processes to exit
cleanly in such scenarios.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 39888b750df12478e8bdea6727cca112dce1df85)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 lib/bb/server/process.py | 19 +++++++++++++++----
 1 file changed, 15 insertions(+), 4 deletions(-)

Patch hide | download patch | download mbox

diff --git a/lib/bb/server/process.py b/lib/bb/server/process.py
index 475931a7..d4beac43 100644
--- a/lib/bb/server/process.py
+++ b/lib/bb/server/process.py
@@ -48,7 +48,7 @@  class ProcessServer(multiprocessing.Process):
 
         self.event_handle = None
         self.haveui = False
-        self.lastui = False
+        self.maxuiwait = 30
         self.xmlrpc = False
 
         self._idlefuns = {}
@@ -155,6 +155,7 @@  class ProcessServer(multiprocessing.Process):
                 print("No timeout, exiting.")
                 self.quit = True
 
+        self.lastui = time.time()
         while not self.quit:
             if self.sock in ready:
                 while select.select([self.sock],[],[],0)[0]:
@@ -191,11 +192,18 @@  class ProcessServer(multiprocessing.Process):
                 except (EOFError, OSError):
                     disconnect_client(self, fds)
 
-            if not self.timeout == -1.0 and not self.haveui and self.lastui and self.timeout and \
+            if not self.timeout == -1.0 and not self.haveui and self.timeout and \
                     (self.lastui + self.timeout) < time.time():
                 print("Server timeout, exiting.")
                 self.quit = True
 
+            # If we don't see a UI connection within maxuiwait, its unlikely we're going to see
+            # one. We have had issue with processes hanging indefinitely so timing out UI-less
+            # servers is useful.
+            if not self.haveui and not self.timeout and (self.lastui + self.maxuiwait) < time.time():
+                print("No UI connection within max timeout, exiting to avoid infinite loop.")
+                self.quit = True
+
             if self.command_channel in ready:
                 try:
                     command = self.command_channel.get()
@@ -220,10 +228,13 @@  class ProcessServer(multiprocessing.Process):
 
         print("Exiting")
         # Remove the socket file so we don't get any more connections to avoid races
-        os.unlink(self.sockname)
+        try:
+            os.unlink(self.sockname)
+        except:
+            pass
         self.sock.close()
 
-        try: 
+        try:
             self.cooker.shutdown(True)
             self.cooker.notifier.stop()
             self.cooker.confignotifier.stop()