Patchwork [bitbake-devel] data: fix performance regression

login
register
mail settings
Submitter Paul Eggleton
Date April 15, 2013, 2:27 p.m.
Message ID <1366036054-12799-1-git-send-email-paul.eggleton@linux.intel.com>
Download mbox | patch
Permalink /patch/48245/
State Accepted
Commit 389a4ab300465f7e461eff52bc80646246c359d5
Headers show

Comments

Paul Eggleton - April 15, 2013, 2:27 p.m.
BitBake commit 7c568132c54a21161de28907159f902462f1e2bb resulted in a
fairly serious performance regression during parsing, almost doubling
the time taken to do a full parse and almost certainly impacting
performance during building. The expandKeys function is called
frequently, and if we avoid using keys() and instead just use the normal
variable lookup mechanism, performance is restored.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
 lib/bb/data.py |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
Chris Larson - April 15, 2013, 3:49 p.m.
On Mon, Apr 15, 2013 at 7:27 AM, Paul Eggleton <
paul.eggleton@linux.intel.com> wrote:

> BitBake commit 7c568132c54a21161de28907159f902462f1e2bb resulted in a
> fairly serious performance regression during parsing, almost doubling
> the time taken to do a full parse and almost certainly impacting
> performance during building. The expandKeys function is called
> frequently, and if we avoid using keys() and instead just use the normal
> variable lookup mechanism, performance is restored.
>
> Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
>

I would suggest using the existing __contains__ mechanism provided by
DataSmart.

    if newval in alterdata:

Should be sufficient, as MutableMapping from collections provides
__contains__ from Mapping, which in turn uses __getitem__.

Patch

diff --git a/lib/bb/data.py b/lib/bb/data.py
index 110666c..abf210a 100644
--- a/lib/bb/data.py
+++ b/lib/bb/data.py
@@ -158,9 +158,9 @@  def expandKeys(alterdata, readdata = None):
 
     for key in todolist:
         ekey = todolist[key]
-        if ekey in keys(alterdata):
+        newval = alterdata.getVar(ekey, 0)
+        if newval:
             val = alterdata.getVar(key, 0)
-            newval = alterdata.getVar(ekey, 0)
             if val is not None and newval is not None:
                 bb.warn("Variable key %s (%s) replaces original key %s (%s)." % (key, val, ekey, newval))
         alterdata.renameVar(key, ekey)