Patchwork [bitbake-devel,1/8] bitbake: Unbuffer stdout for log files

login
register
mail settings
Submitter Jason Wessel
Date Sept. 17, 2012, 10:43 p.m.
Message ID <1347921818-60562-2-git-send-email-jason.wessel@windriver.com>
Download mbox | patch
Permalink /patch/36665/
State Accepted
Commit c6a367bc3224adafca698a4ffc5414ad83842c16
Headers show

Comments

Jason Wessel - Sept. 17, 2012, 10:43 p.m.
It is possible to lose critical log data when python exits in an
unorderly fashion via segmentation fault or certain types of crashes.
This is because the buffer characteristics are inherited from the top
level stdout, which should be set to unbuffered, for the purpose of
all the forked children.

This pushes the buffering to the OS, instead of having python managing
the buffers in its stream handler class.

This change is also to provide the ability to tail logs written from
processes in "real time" because they would be written in an orderly
fashion depending upon the OS characteristics for the file I/O.

(LOCAL REV ONLY: NOT UPSTREAM)

Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
---
 bin/bitbake |    8 ++++++++
 1 files changed, 8 insertions(+), 0 deletions(-)
Chris Larson - Sept. 18, 2012, 12:46 a.m.
On Mon, Sep 17, 2012 at 3:43 PM, Jason Wessel
<jason.wessel@windriver.com> wrote:
> It is possible to lose critical log data when python exits in an
> unorderly fashion via segmentation fault or certain types of crashes.
> This is because the buffer characteristics are inherited from the top
> level stdout, which should be set to unbuffered, for the purpose of
> all the forked children.
>
> This pushes the buffering to the OS, instead of having python managing
> the buffers in its stream handler class.
>
> This change is also to provide the ability to tail logs written from
> processes in "real time" because they would be written in an orderly
> fashion depending upon the OS characteristics for the file I/O.
>
> (LOCAL REV ONLY: NOT UPSTREAM)
>
> Signed-off-by: Jason Wessel <jason.wessel@windriver.com>


I'm assuming you forgot to rebase -i and remove the LOCAL REV ONLY? :)
Jason Wessel - Sept. 18, 2012, 1:15 a.m.
On 09/17/2012 07:46 PM, Chris Larson wrote:
> On Mon, Sep 17, 2012 at 3:43 PM, Jason Wessel
> <jason.wessel@windriver.com> wrote:
>> It is possible to lose critical log data when python exits in an
>> unorderly fashion via segmentation fault or certain types of crashes.
>> This is because the buffer characteristics are inherited from the top
>> level stdout, which should be set to unbuffered, for the purpose of
>> all the forked children.
>>
>> This pushes the buffering to the OS, instead of having python managing
>> the buffers in its stream handler class.
>>
>> This change is also to provide the ability to tail logs written from
>> processes in "real time" because they would be written in an orderly
>> fashion depending upon the OS characteristics for the file I/O.
>>
>> (LOCAL REV ONLY: NOT UPSTREAM)
>>
>> Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
>
> I'm assuming you forgot to rebase -i and remove the LOCAL REV ONLY? :)


Oops :-)

I can resend the patches, but I did re-base against the head but for some reason the filter was not working.

Jason.

Patch

diff --git a/bin/bitbake b/bin/bitbake
index ed2ff06..c000d98 100755
--- a/bin/bitbake
+++ b/bin/bitbake
@@ -43,6 +43,14 @@  from bb import server
 __version__ = "1.15.3"
 logger = logging.getLogger("BitBake")
 
+# Unbuffer stdout to avoid log truncation in the event
+# of an unorderly exit as well as to provide timely
+# updates to log files for use with tail
+try:
+    if sys.stdout.name == '<stdout>':
+        sys.stdout = os.fdopen(sys.stdout.fileno(), 'w', 0)
+except:
+    pass
 
 class BBConfiguration(object):
     """