Patchwork [bitbake-devel] bitbake/lib/bb/fetch2: Fix stderr handling when running commands

login
register
mail settings
Submitter Adrien Bustany
Date Jan. 19, 2012, 3:49 p.m.
Message ID <20120119164907.5BA773E8BA@mail.mymadcat.com>
Download mbox | patch
Permalink /patch/19789/
State New
Headers show

Comments

Adrien Bustany - Jan. 19, 2012, 3:49 p.m.
stderr would previously be appended to stdout, corrupting the result
when something was outputed to stderr but exit code was still 0
(non-fatal warning messages). This commit makes the code parse only
stdout, but output stderr if an error happened.

Signed-off-by: Adrien Bustany <adrien.bustany@nokia.com>
---
 bitbake/lib/bb/fetch2/__init__.py |   35 ++++++++++++++++-------------------
 1 files changed, 16 insertions(+), 19 deletions(-)
Richard Purdie - Jan. 27, 2012, 4:37 p.m.
On Thu, 2012-01-19 at 17:49 +0200, Adrien Bustany wrote:
> stderr would previously be appended to stdout, corrupting the result
> when something was outputed to stderr but exit code was still 0
> (non-fatal warning messages). This commit makes the code parse only
> stdout, but output stderr if an error happened.
> 
> Signed-off-by: Adrien Bustany <adrien.bustany@nokia.com>
> ---
>  bitbake/lib/bb/fetch2/__init__.py |   35 ++++++++++++++++-------------------
>  1 files changed, 16 insertions(+), 19 deletions(-)

Just for reference, I tried to apply this but couldn't as it conflicts
with other changes in that area.

If you have time to rebase it, I'll likely apply it sooner, else I'll
need to find time to do that.

Cheers,

Richard

Patch

diff --git a/bitbake/lib/bb/fetch2/__init__.py b/bitbake/lib/bb/fetch2/__init__.py
index 374ac9b..c3b22d7 100644
--- a/bitbake/lib/bb/fetch2/__init__.py
+++ b/bitbake/lib/bb/fetch2/__init__.py
@@ -378,6 +378,9 @@  def runfetchcmd(cmd, d, quiet = False, cleanup = []):
     Optionally remove the files/directories listed in cleanup upon failure
     """
 
+    import bb.process
+    import subprocess
+
     # Need to export PATH as binary could be in metadata paths
     # rather than host provided
     # Also include some other variables.
@@ -394,33 +397,27 @@  def runfetchcmd(cmd, d, quiet = False, cleanup = []):
 
     logger.debug(1, "Running %s", cmd)
 
-    # redirect stderr to stdout
-    stdout_handle = os.popen(cmd + " 2>&1", "r")
-    output = ""
-
-    while True:
-        line = stdout_handle.readline()
-        if not line:
-            break
-        if not quiet:
-            print(line, end=' ')
-        output += line
+    success = False
+    error_message = ""
 
-    status = stdout_handle.close() or 0
-    signal = status >> 8
-    exitstatus = status & 0xff
+    try:
+        (output, errors) = bb.process.run(cmd, shell=True, stderr=subprocess.PIPE)
+        success = True
+    except bb.process.NotFoundError as e:
+        error_message = "Fetch command %s" % (e.command)
+    except bb.process.CmdError as e:
+        error_message = "Fetch command %s could not be run:\n%s" % (e.command, e.msg)
+    except bb.process.ExecutionError as e:
+        error_message = "Fetch command %s failed with exit code %s, output:\n%s" % (e.command, e.exitcode, e.stderr)
 
-    if (signal or status != 0):
+    if not success:
         for f in cleanup:
             try:
                 bb.utils.remove(f, True)
             except OSError:
                 pass
 
-        if signal:
-            raise FetchError("Fetch command %s failed with signal %s, output:\n%s" % (cmd, signal, output))
-        elif status != 0:
-            raise FetchError("Fetch command %s failed with exit code %s, output:\n%s" % (cmd, status, output))
+        raise FetchError(error_message)
 
     return output