[bitbake-devel,master|dunfell|zeus] knotty: ignore pipe errors

Submitted by Mikko Rapeli on July 23, 2020, 2:58 p.m. | Patch ID: 174652

Details

Message ID 20200723145812.14029-1-mikko.rapeli@bmw.de
State New
Headers show

Commit Message

Mikko Rapeli July 23, 2020, 2:58 p.m.
Piping bitbake output to another process like less currently
results in an ugly error if the other end of pipe exits
before bitbake has written all output to it.

For example with "bitbake -e linux-yocto | less" and
pressing q to quit less results in:

Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "/home/mcfrisk/src/yocto/poky/bitbake/lib/bb/event.py", line 133, in print_ui_queue
    sys.stdout.flush()
BrokenPipeError: [Errno 32] Broken pipe
Exception ignored in: <_io.TextIOWrapper name='<stdout>' mode='w' encoding='UTF-8'>
BrokenPipeError: [Errno 32] Broken pipe

Fix is to use default signal handler for pipe signal. It's
perfectly normal that on Unix systems the other end of pipe
gets killed before all data is written.

In some cases there seems to also be a loop which results in
a large number of "[Errno 32] Broken pipe" messages printed to
console until knotty process is killed. I think this also
fixes those cases.

This patch applies to master, dunfell and zeus branches.

Signed-off-by: Mikko Rapeli <mikko.rapeli@bmw.de>
---
 bitbake/lib/bb/ui/knotty.py | 3 +++
 1 file changed, 3 insertions(+)

Patch hide | download patch | download mbox

diff --git a/bitbake/lib/bb/ui/knotty.py b/bitbake/lib/bb/ui/knotty.py
index 87e873d644..56ac41d0f9 100644
--- a/bitbake/lib/bb/ui/knotty.py
+++ b/bitbake/lib/bb/ui/knotty.py
@@ -29,6 +29,9 @@  featureSet = [bb.cooker.CookerFeatures.SEND_SANITYEVENTS]
 logger = logging.getLogger("BitBake")
 interactive = sys.stdout.isatty()
 
+# ignore pipe errors since they are normal
+signal.signal(signal.SIGPIPE, signal.SIG_DFL)
+
 class BBProgress(progressbar.ProgressBar):
     def __init__(self, msg, maxval, widgets=None, extrapos=-1, resize_handler=None):
         self.msg = msg

Comments

Richard Purdie July 24, 2020, 6:01 a.m.
On Thu, 2020-07-23 at 17:58 +0300, Mikko Rapeli wrote:
> Piping bitbake output to another process like less currently
> results in an ugly error if the other end of pipe exits
> before bitbake has written all output to it.
> 
> For example with "bitbake -e linux-yocto | less" and
> pressing q to quit less results in:
> 
> Error in atexit._run_exitfuncs:
> Traceback (most recent call last):
>   File "/home/mcfrisk/src/yocto/poky/bitbake/lib/bb/event.py", line
> 133, in print_ui_queue
>     sys.stdout.flush()
> BrokenPipeError: [Errno 32] Broken pipe
> Exception ignored in: <_io.TextIOWrapper name='<stdout>' mode='w'
> encoding='UTF-8'>
> BrokenPipeError: [Errno 32] Broken pipe
> 
> Fix is to use default signal handler for pipe signal. It's
> perfectly normal that on Unix systems the other end of pipe
> gets killed before all data is written.
> 
> In some cases there seems to also be a loop which results in
> a large number of "[Errno 32] Broken pipe" messages printed to
> console until knotty process is killed. I think this also
> fixes those cases.
> 
> This patch applies to master, dunfell and zeus branches.
> 
> Signed-off-by: Mikko Rapeli <mikko.rapeli@bmw.de>
> ---
>  bitbake/lib/bb/ui/knotty.py | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/bitbake/lib/bb/ui/knotty.py
> b/bitbake/lib/bb/ui/knotty.py
> index 87e873d644..56ac41d0f9 100644
> --- a/bitbake/lib/bb/ui/knotty.py
> +++ b/bitbake/lib/bb/ui/knotty.py
> @@ -29,6 +29,9 @@ featureSet =
> [bb.cooker.CookerFeatures.SEND_SANITYEVENTS]
>  logger = logging.getLogger("BitBake")
>  interactive = sys.stdout.isatty()
>  
> +# ignore pipe errors since they are normal
> +signal.signal(signal.SIGPIPE, signal.SIG_DFL)
> +
>  class BBProgress(progressbar.ProgressBar):
>      def __init__(self, msg, maxval, widgets=None, extrapos=-1,
> resize_handler=None):
>          self.msg = msg

I think but have not 100% confirmed this caused the failures in:

https://autobuilder.yoctoproject.org/typhoon/#/builders/83/builds/1200

(new build running to check)

Cheers,

Richard
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#11513): https://lists.openembedded.org/g/bitbake-devel/message/11513
Mute This Topic: https://lists.openembedded.org/mt/75747491/3617530
Group Owner: bitbake-devel+owner@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub  [oe-patchwork@oe-patch.openembedded.org]
-=-=-=-=-=-=-=-=-=-=-=-
Mikko Rapeli July 24, 2020, 7:58 a.m.
On Fri, Jul 24, 2020 at 07:01:46AM +0100, Richard Purdie wrote:
> On Thu, 2020-07-23 at 17:58 +0300, Mikko Rapeli wrote:
> > Piping bitbake output to another process like less currently
> > results in an ugly error if the other end of pipe exits
> > before bitbake has written all output to it.
> > 
> > For example with "bitbake -e linux-yocto | less" and
> > pressing q to quit less results in:
> > 
> > Error in atexit._run_exitfuncs:
> > Traceback (most recent call last):
> >   File "/home/mcfrisk/src/yocto/poky/bitbake/lib/bb/event.py", line
> > 133, in print_ui_queue
> >     sys.stdout.flush()
> > BrokenPipeError: [Errno 32] Broken pipe
> > Exception ignored in: <_io.TextIOWrapper name='<stdout>' mode='w'
> > encoding='UTF-8'>
> > BrokenPipeError: [Errno 32] Broken pipe
> > 
> > Fix is to use default signal handler for pipe signal. It's
> > perfectly normal that on Unix systems the other end of pipe
> > gets killed before all data is written.
> > 
> > In some cases there seems to also be a loop which results in
> > a large number of "[Errno 32] Broken pipe" messages printed to
> > console until knotty process is killed. I think this also
> > fixes those cases.
> > 
> > This patch applies to master, dunfell and zeus branches.
> > 
> > Signed-off-by: Mikko Rapeli <mikko.rapeli@bmw.de>
> > ---
> >  bitbake/lib/bb/ui/knotty.py | 3 +++
> >  1 file changed, 3 insertions(+)
> > 
> > diff --git a/bitbake/lib/bb/ui/knotty.py
> > b/bitbake/lib/bb/ui/knotty.py
> > index 87e873d644..56ac41d0f9 100644
> > --- a/bitbake/lib/bb/ui/knotty.py
> > +++ b/bitbake/lib/bb/ui/knotty.py
> > @@ -29,6 +29,9 @@ featureSet =
> > [bb.cooker.CookerFeatures.SEND_SANITYEVENTS]
> >  logger = logging.getLogger("BitBake")
> >  interactive = sys.stdout.isatty()
> >  
> > +# ignore pipe errors since they are normal
> > +signal.signal(signal.SIGPIPE, signal.SIG_DFL)
> > +
> >  class BBProgress(progressbar.ProgressBar):
> >      def __init__(self, msg, maxval, widgets=None, extrapos=-1,
> > resize_handler=None):
> >          self.msg = msg
> 
> I think but have not 100% confirmed this caused the failures in:
> 
> https://autobuilder.yoctoproject.org/typhoon/#/builders/83/builds/1200

Hmm, I can't see the connection based on logs though I don't know the test
setups at all.

> (new build running to check)

Looking forward to these results...

-Mikko

> Cheers,
> 
> Richard
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#11514): https://lists.openembedded.org/g/bitbake-devel/message/11514
Mute This Topic: https://lists.openembedded.org/mt/75747491/3617530
Group Owner: bitbake-devel+owner@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub  [oe-patchwork@oe-patch.openembedded.org]
-=-=-=-=-=-=-=-=-=-=-=-
Richard Purdie July 25, 2020, 12:58 p.m.
On Fri, 2020-07-24 at 07:58 +0000, Mikko.Rapeli@bmw.de wrote:
> On Fri, Jul 24, 2020 at 07:01:46AM +0100, Richard Purdie wrote:
> > On Thu, 2020-07-23 at 17:58 +0300, Mikko Rapeli wrote:
> > > Piping bitbake output to another process like less currently
> > > results in an ugly error if the other end of pipe exits
> > > before bitbake has written all output to it.
> > > 
> > > For example with "bitbake -e linux-yocto | less" and
> > > pressing q to quit less results in:
> > > 
> > > Error in atexit._run_exitfuncs:
> > > Traceback (most recent call last):
> > >   File "/home/mcfrisk/src/yocto/poky/bitbake/lib/bb/event.py",
> > > line
> > > 133, in print_ui_queue
> > >     sys.stdout.flush()
> > > BrokenPipeError: [Errno 32] Broken pipe
> > > Exception ignored in: <_io.TextIOWrapper name='<stdout>' mode='w'
> > > encoding='UTF-8'>
> > > BrokenPipeError: [Errno 32] Broken pipe
> > > 
> > > Fix is to use default signal handler for pipe signal. It's
> > > perfectly normal that on Unix systems the other end of pipe
> > > gets killed before all data is written.
> > > 
> > > In some cases there seems to also be a loop which results in
> > > a large number of "[Errno 32] Broken pipe" messages printed to
> > > console until knotty process is killed. I think this also
> > > fixes those cases.
> > > 
> > > This patch applies to master, dunfell and zeus branches.
> > > 
> > > Signed-off-by: Mikko Rapeli <mikko.rapeli@bmw.de>
> > > ---
> > >  bitbake/lib/bb/ui/knotty.py | 3 +++
> > >  1 file changed, 3 insertions(+)
> > > 
> > > diff --git a/bitbake/lib/bb/ui/knotty.py
> > > b/bitbake/lib/bb/ui/knotty.py
> > > index 87e873d644..56ac41d0f9 100644
> > > --- a/bitbake/lib/bb/ui/knotty.py
> > > +++ b/bitbake/lib/bb/ui/knotty.py
> > > @@ -29,6 +29,9 @@ featureSet =
> > > [bb.cooker.CookerFeatures.SEND_SANITYEVENTS]
> > >  logger = logging.getLogger("BitBake")
> > >  interactive = sys.stdout.isatty()
> > >  
> > > +# ignore pipe errors since they are normal
> > > +signal.signal(signal.SIGPIPE, signal.SIG_DFL)
> > > +
> > >  class BBProgress(progressbar.ProgressBar):
> > >      def __init__(self, msg, maxval, widgets=None, extrapos=-1,
> > > resize_handler=None):
> > >          self.msg = msg
> > 
> > I think but have not 100% confirmed this caused the failures in:
> > 
> > https://autobuilder.yoctoproject.org/typhoon/#/builders/83/builds/1200
> 
> Hmm, I can't see the connection based on logs though I don't know the
> test
> setups at all.
> 
> > (new build running to check)
> 
> Looking forward to these results...

It confirmed this patch is at fault. Looks like devtool/eSDK/tinfoil
are the problem areas at a glance at the failures.

Cheers,

Richard
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#11515): https://lists.openembedded.org/g/bitbake-devel/message/11515
Mute This Topic: https://lists.openembedded.org/mt/75747491/3617530
Group Owner: bitbake-devel+owner@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub  [oe-patchwork@oe-patch.openembedded.org]
-=-=-=-=-=-=-=-=-=-=-=-