Patchwork [bitbake-devel,0/2] V4 replace os.system and os.popen with subbprocess module

login
register
mail settings
Submitter Robert Yang
Date May 20, 2012, 12:36 p.m.
Message ID <cover.1337515887.git.liezhi.yang@windriver.com>
Download mbox
Permalink /patch/28001/
State New
Headers show

Pull-request

git://git.pokylinux.org/poky-contrib robert/subprocess

Comments

Robert Yang - May 20, 2012, 12:36 p.m.
* Changes of V4:
  Fix use readline() for the return value of bb.process.run(), the
  return value is already a string, we can't (or don't have to) use
  readline() for it, the string is what we need.

* Changes of V3:
  Use bb.process.run() rather than bb.process.Popen() to replace os.popen()

* Changes of V2:

  - Remove the 2>/dev/null since we dont' need it.
  - Use the wrapped Popen from bb.process, which is simpler than
    subprocess.Popen(....).

* Original message of V1:
Replace os.popen and os.system with subprocess.Popen and
subprocess.call, since the older functions would fail (more or less) if
the executed program cannot be found, this would cause potential errors
since we don't know whether the problem executed well or not.

For the performance issue, I've done the testing before the patches and
after with the oe-core layer (also the oe-core have applied the similar
patches):

# The sources are on local disk

1) Before applied these pacthes to bitbake and similar patches to oe-core:
$ time bitbake core-image-sato
real    177m50.723s
user    436m1.551s
sys     71m29.588s

2) After applied the pathes:
$ time bitbake core-image-sato
real    176m26.194s
user    436m7.931s
sys     71m1.994s

After applied these patches, the time has reduced 84 seconds, this is
very slight, I think that we can assume this is just a deviation, it
doesn't cause performance problems.

// Robert

The following changes since commit b4c8c74a45e386f99344cf9799eb5294ad6c9e3e:

  hob: update required pygtk to 2.22.0 and gtk+ to 2.20.0 (2012-05-20 09:24:26 +0100)

are available in the git repository at:
  git://git.pokylinux.org/poky-contrib robert/subprocess
  http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=robert/subprocess

Robert Yang (2):
  replace os.system with subprocess.call
  replace os.popen with subprocess.Popen

 bitbake/lib/bb/fetch2/perforce.py            |   14 ++++++++------
 bitbake/lib/bb/fetch2/svk.py                 |    4 ++--
 bitbake/lib/bb/shell.py                      |    6 +++---
 bitbake/lib/bb/ui/crumbs/builddetailspage.py |    5 +++--
 bitbake/lib/bb/ui/crumbs/hig.py              |    7 ++++---
 bitbake/lib/bb/ui/crumbs/imagedetailspage.py |    3 ++-
 bitbake/lib/bb/ui/ncurses.py                 |    4 ++--
 7 files changed, 24 insertions(+), 19 deletions(-)
Chris Larson - May 20, 2012, 6:16 p.m.
On Sun, May 20, 2012 at 5:36 AM, Robert Yang <liezhi.yang@windriver.com> wrote:
> * Changes of V4:
>  Fix use readline() for the return value of bb.process.run(), the
>  return value is already a string, we can't (or don't have to) use
>  readline() for it, the string is what we need.
>
> * Changes of V3:
>  Use bb.process.run() rather than bb.process.Popen() to replace os.popen()
>
> * Changes of V2:
>
>  - Remove the 2>/dev/null since we dont' need it.
>  - Use the wrapped Popen from bb.process, which is simpler than
>    subprocess.Popen(....).
>
> * Original message of V1:
> Replace os.popen and os.system with subprocess.Popen and
> subprocess.call, since the older functions would fail (more or less) if
> the executed program cannot be found, this would cause potential errors
> since we don't know whether the problem executed well or not.
>
> For the performance issue, I've done the testing before the patches and
> after with the oe-core layer (also the oe-core have applied the similar
> patches):
>
> # The sources are on local disk
>
> 1) Before applied these pacthes to bitbake and similar patches to oe-core:
> $ time bitbake core-image-sato
> real    177m50.723s
> user    436m1.551s
> sys     71m29.588s
>
> 2) After applied the pathes:
> $ time bitbake core-image-sato
> real    176m26.194s
> user    436m7.931s
> sys     71m1.994s
>
> After applied these patches, the time has reduced 84 seconds, this is
> very slight, I think that we can assume this is just a deviation, it
> doesn't cause performance problems.

This version looks good to me.
Richard Purdie - May 23, 2012, 10:30 a.m.
On Sun, 2012-05-20 at 20:36 +0800, Robert Yang wrote:
> * Changes of V4:
>   Fix use readline() for the return value of bb.process.run(), the
>   return value is already a string, we can't (or don't have to) use
>   readline() for it, the string is what we need.
> 
> * Changes of V3:
>   Use bb.process.run() rather than bb.process.Popen() to replace os.popen()
> 
> * Changes of V2:
> 
>   - Remove the 2>/dev/null since we dont' need it.
>   - Use the wrapped Popen from bb.process, which is simpler than
>     subprocess.Popen(....).
> 
> * Original message of V1:
> Replace os.popen and os.system with subprocess.Popen and
> subprocess.call, since the older functions would fail (more or less) if
> the executed program cannot be found, this would cause potential errors
> since we don't know whether the problem executed well or not.
> 
> For the performance issue, I've done the testing before the patches and
> after with the oe-core layer (also the oe-core have applied the similar
> patches):
> 
> # The sources are on local disk
> 
> 1) Before applied these pacthes to bitbake and similar patches to oe-core:
> $ time bitbake core-image-sato
> real    177m50.723s
> user    436m1.551s
> sys     71m29.588s
> 
> 2) After applied the pathes:
> $ time bitbake core-image-sato
> real    176m26.194s
> user    436m7.931s
> sys     71m1.994s
> 
> After applied these patches, the time has reduced 84 seconds, this is
> very slight, I think that we can assume this is just a deviation, it
> doesn't cause performance problems.
> 
> // Robert
> 
> The following changes since commit b4c8c74a45e386f99344cf9799eb5294ad6c9e3e:
> 
>   hob: update required pygtk to 2.22.0 and gtk+ to 2.20.0 (2012-05-20 09:24:26 +0100)
> 
> are available in the git repository at:
>   git://git.pokylinux.org/poky-contrib robert/subprocess
>   http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=robert/subprocess
> 
> Robert Yang (2):
>   replace os.system with subprocess.call
>   replace os.popen with subprocess.Popen

Merged to master, thanks.

Richard