Patchwork [bitbake-devel] ui/knotty: Add a footer to the build output for interactive terminals

login
register
mail settings
Submitter Richard Purdie
Date March 15, 2012, 10:56 p.m.
Message ID <1331852164.18586.166.camel@ted>
Download mbox | patch
Permalink /patch/23475/
State New
Headers show

Comments

Richard Purdie - March 15, 2012, 10:56 p.m.
I've sent this before, this is an updated version with several bugfixes
and improvements. Its a major change but one I think people will like
overall as I hear a lot of complaints about the verbose console
messages. v0 never got complaints so I will merge this fairly quickly
unless there are objections.

---
On terminals which support it, add summary information to the end of the
build output about the number of tasks currently running and how many tasks
we've run so far.

This provides a summary at a glace of what the current state of the build is
and what the build is currently doing which is lacking in the current UI.

Also disable echo of characters on stdin since this corrupts the disable,
particularly Crtl+C.

The "waiting for X tasks" code can be merged into this code too since
that is only useful on interactive terminals and this improves the
readability of that output too.

Improvements since v0:

* The tasks are ordered in execution order.
* The display is only updated when the list of tasks changes or there
  is output above the footer.
* Errors early in the init process don't corrupt the terminal
* Running task x of y and package messages are suppressed from the console

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
Koen Kooi - March 19, 2012, 11:48 a.m.
Op 15 mrt. 2012, om 23:56 heeft Richard Purdie het volgende geschreven:

> I've sent this before, this is an updated version with several bugfixes
> and improvements. Its a major change but one I think people will like
> overall as I hear a lot of complaints about the verbose console
> messages. v0 never got complaints so I will merge this fairly quickly
> unless there are objections.
> 
> ---
> On terminals which support it, add summary information to the end of the
> build output about the number of tasks currently running and how many tasks
> we've run so far.
> 
> This provides a summary at a glace of what the current state of the build is
> and what the build is currently doing which is lacking in the current UI.
> 
> Also disable echo of characters on stdin since this corrupts the disable,
> particularly Crtl+C.
> 
> The "waiting for X tasks" code can be merged into this code too since
> that is only useful on interactive terminals and this improves the
> readability of that output too.
> 
> Improvements since v0:
> 
> * The tasks are ordered in execution order.
> * The display is only updated when the list of tasks changes or there
>  is output above the footer.
> * Errors early in the init process don't corrupt the terminal
> * Running task x of y and package messages are suppressed from the console
> 
> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> ---
> diff --git a/bitbake/lib/bb/ui/knotty.py b/bitbake/lib/bb/ui/knotty.py

This doesn't apply for obvious reasons.
Koen Kooi - March 20, 2012, 12:24 p.m.
Op 15 mrt. 2012, om 23:56 heeft Richard Purdie het volgende geschreven:

> I've sent this before, this is an updated version with several bugfixes
> and improvements. Its a major change but one I think people will like
> overall as I hear a lot of complaints about the verbose console
> messages. v0 never got complaints so I will merge this fairly quickly
> unless there are objections.
> 
> ---
> On terminals which support it, add summary information to the end of the
> build output about the number of tasks currently running and how many tasks
> we've run so far.
> 
> This provides a summary at a glace of what the current state of the build is
> and what the build is currently doing which is lacking in the current UI.
> 
> Also disable echo of characters on stdin since this corrupts the disable,
> particularly Crtl+C.
> 
> The "waiting for X tasks" code can be merged into this code too since
> that is only useful on interactive terminals and this improves the
> readability of that output too.
> 
> Improvements since v0:
> 
> * The tasks are ordered in execution order.
> * The display is only updated when the list of tasks changes or there
>  is output above the footer.
> * Errors early in the init process don't corrupt the terminal
> * Running task x of y and package messages are suppressed from the console

I like it a lot, but there is one missing thing. Have a look at the following output:

-------------------------------------------
Loading cache: 100% |####################################################################################################################################################################################################################################################################################################################################################################################################| ETA:  00:00:00
Loaded 2642 entries from dependency cache.

OE Build Configuration:
BB_VERSION        = "1.15.1"
TARGET_ARCH       = "arm"
TARGET_OS         = "linux-gnueabi"
MACHINE           = "beaglebone"
DISTRO            = "angstrom"
DISTRO_VERSION    = "v2012.03-core"
TUNE_FEATURES     = "armv7a vfp neon cortexa8"
TARGET_FPU        = "vfp-neon"
meta-angstrom     = "master:51ba0dd081af1d7de56ffc9fb318044bc3c7c47e"
meta-oe           
meta-efl          
meta-gpe          
meta-gnome        
meta-xfce         
meta-initramfs    = "master:b0a7f2e761f79e8967b1d3f7c67177283805bf75"
meta-opie         = "master:efa3892b20a4ef80274e56e5633ebd62c16f9731"
meta-java         = "master:e90110b85733f9cdda26861bc29588257173c11c"
meta-ti           = "master:8f1fc028be8567c57ed7d40b1800ece37128b9ac"
meta-efikamx      = "master:2c09a3a780b23448e8a6ca964256ff7f5ccba65d"
meta-nslu2        = "master:3d9fc951b05b4df476374b6fc3085ebac7f293ee"
meta-htc          
meta-nokia        
meta-openmoko     
meta-palm         = "master:57cd9600e285f394873b943644879b166a256769"
meta-handheld     = "master:f2904e00c9c709d1f40e4a93f0f552f882bacf4f"
meta-intel        
meta-sugarbay     
meta-crownbay     
meta-emenlow      
meta-fishriver    
meta-jasperforest 
meta-n450         = "master:9124870a6bcd99ebaeb0342f4fedf14dc1afebd8"
meta              = "master:c981fa4fad8a457882293157b7bdca05aad5a778"

NOTE: Resolving any missing task queue dependencies
NOTE: Preparing runqueue
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
NOTE: Tasks Summary: Attempted 3179 tasks of which 3165 didn't need to be rerun and all succeeded.
-------------------------------------------

What did I just build? A mention of the '<foo>' in 'bitbake <foo>' would be nice :)

regards,

Koen

Patch

diff --git a/bitbake/lib/bb/ui/knotty.py b/bitbake/lib/bb/ui/knotty.py
index 14989d4..4a7e6b9 100644
--- a/bitbake/lib/bb/ui/knotty.py
+++ b/bitbake/lib/bb/ui/knotty.py
@@ -72,6 +72,26 @@  def pluralise(singular, plural, qty):
 
 def main(server, eventHandler):
 
+    cuu = None
+    stdinbackup = None
+
+    if interactive:
+        import curses
+        import termios
+        import copy
+        try:
+            fd = sys.stdin.fileno()
+            stdinbackup = termios.tcgetattr(fd)
+            new = copy.deepcopy(stdinbackup)
+            new[3] = new[3] & ~termios.ECHO
+            termios.tcsetattr(fd, termios.TCSADRAIN, new)
+            curses.setupterm()
+            ed = curses.tigetstr("ed")
+            if ed:
+                cuu = curses.tigetstr("cuu")
+        except:
+            cuu = None
+
     # Get values of variables which control our output
     includelogs = server.runCommand(["getVariable", "BBINCLUDELOGS"])
     loglines = server.runCommand(["getVariable", "BBINCLUDELOGS_LINES"])
@@ -115,8 +135,56 @@  def main(server, eventHandler):
     errors = 0
     warnings = 0
     taskfailures = []
+    main.footer_present = False
+    main.lastpids = []
+
+    def updateFooter():
+        if not cuu:
+            return
+        activetasks = helper.running_tasks
+        failedtasks = helper.failed_tasks
+        runningpids = helper.running_pids
+        if main.footer_present and (main.lastpids == runningpids):
+            return
+        if main.footer_present:
+            clearFooter()
+        if not activetasks:
+            return
+        lines = 1
+        tasks = []
+        for t in runningpids:
+            tasks.append("%s (pid %s)" % (activetasks[t]["title"], t))
+
+        if shutdown:
+            print("Waiting for %s running tasks to finish:" % len(activetasks))
+        else:
+            print("Currently %s running tasks (%s of %s):" % (len(activetasks), helper.tasknumber_current, helper.tasknumber_total))
+        for tasknum, task in enumerate(tasks):
+            print("%s: %s" % (tasknum, task))
+            lines = lines + 1
+        main.footer_present = lines
+        main.lastpids = runningpids[:]
+
+    def clearFooter():
+        if main.footer_present:
+            lines = main.footer_present
+            sys.stdout.write(curses.tparm(cuu, lines))
+            sys.stdout.write(curses.tparm(ed))
+        main.footer_present = False
+
+    class InteractConsoleLogFilter(logging.Filter):
+        def filter(self, record):
+            if record.levelno == format.NOTE and (record.msg.startswith("Running") or record.msg.startswith("package ")):
+                return False
+            clearFooter()
+            return True
+
+    if interactive:
+        console.addFilter(InteractConsoleLogFilter())
+
     while True:
         try:
+            updateFooter()
             event = eventHandler.waitEvent(0.25)
             if event is None:
                 if shutdown > 1:
@@ -126,12 +194,6 @@  def main(server, eventHandler):
             if isinstance(event, bb.runqueue.runQueueExitWait):
                 if not shutdown:
                     shutdown = 1
-            if shutdown and helper.needUpdate:
-                activetasks, failedtasks = helper.getTasks()
-                if activetasks:
-                    print("Waiting for %s active tasks to finish:" % len(activetasks))
-                    for tasknum, task in enumerate(activetasks):
-                        print("%s: %s (pid %s)" % (tasknum, activetasks[task]["title"], task))
 
             if isinstance(event, logging.LogRecord):
                 if event.levelno >= format.ERROR:
@@ -151,6 +213,7 @@  def main(server, eventHandler):
                 return_value = 1
                 logfile = event.logfile
                 if logfile and os.path.exists(logfile):
+                    clearFooter()
                     print("ERROR: Logfile of failure stored in: %s" % logfile)
                     if includelogs and not event.errprinted:
                         print("Log data follows:")
@@ -281,10 +344,12 @@  def main(server, eventHandler):
             logger.error("Unknown event: %s", event)
 
         except EnvironmentError as ioerror:
+            clearFooter()
             # ignore interrupted io
             if ioerror.args[0] == 4:
                 pass
         except KeyboardInterrupt:
+            clearFooter()
             if shutdown == 1:
                 print("\nSecond Keyboard Interrupt, stopping...\n")
                 server.runCommand(["stateStop"])
@@ -315,4 +380,6 @@  def main(server, eventHandler):
         if return_value == 0:
             return_value = 1
 
+    if stdinbackup:
+        termios.tcsetattr(fd, termios.TCSADRAIN, stdinbackup)
     return return_value
diff --git a/bitbake/lib/bb/ui/uihelper.py b/bitbake/lib/bb/ui/uihelper.py
index c96f381..2c78695 100644
--- a/bitbake/lib/bb/ui/uihelper.py
+++ b/bitbake/lib/bb/ui/uihelper.py
@@ -23,6 +23,8 @@  class BBUIHelper:
     def __init__(self):
         self.needUpdate = False
         self.running_tasks = {}
+        # Running PIDs preserves the order tasks were executed in
+        self.running_pids = []
         self.failed_tasks = []
         self.tasknumber_current = 0
         self.tasknumber_total = 0
@@ -30,16 +32,20 @@  class BBUIHelper:
     def eventHandler(self, event):
         if isinstance(event, bb.build.TaskStarted):
             self.running_tasks[event.pid] = { 'title' : "%s %s" % (event._package, event._task) }
+            self.running_pids.append(event.pid)
             self.needUpdate = True
         if isinstance(event, bb.build.TaskSucceeded):
             del self.running_tasks[event.pid]
+            self.running_pids.remove(event.pid)
             self.needUpdate = True
         if isinstance(event, bb.build.TaskFailedSilent):
             del self.running_tasks[event.pid]
+            self.running_pids.remove(event.pid)
             # Don't add to the failed tasks list since this is e.g. a setscene task failure
             self.needUpdate = True
         if isinstance(event, bb.build.TaskFailed):
             del self.running_tasks[event.pid]
+            self.running_pids.remove(event.pid)
             self.failed_tasks.append( { 'title' : "%s %s" % (event._package, event._task)})
             self.needUpdate = True
         if isinstance(event, bb.runqueue.runQueueTaskStarted):