Patchwork [bitbake-devel] bb/ui: store_dependency_information optimization

login
register
mail settings
Submitter Marius Avram
Date Feb. 18, 2014, 2:39 p.m.
Message ID <1392734364-8388-1-git-send-email-marius.avram@intel.com>
Download mbox | patch
Permalink /patch/66931/
State New
Headers show

Comments

Marius Avram - Feb. 18, 2014, 2:39 p.m.
This optimization is in support of the bug #5485. The function called
at the beginning of every build: store_dependency_information was taking
approximately 20sec and it was delaying the arrival of events from the
event queue. The change minimizes the calls to _save_a_task(),
reducing the time to half.

Signed-off-by: Marius Avram <marius.avram@intel.com>
---
 bitbake/lib/bb/ui/buildinfohelper.py |   16 +++++++++++++---
 1 file changed, 13 insertions(+), 3 deletions(-)
Alex Damian - Feb. 18, 2014, 5:42 p.m.
+1, works nice.


On Tue, Feb 18, 2014 at 2:39 PM, Marius Avram <marius.avram@intel.com>wrote:

> This optimization is in support of the bug #5485. The function called
> at the beginning of every build: store_dependency_information was taking
> approximately 20sec and it was delaying the arrival of events from the
> event queue. The change minimizes the calls to _save_a_task(),
> reducing the time to half.
>
> Signed-off-by: Marius Avram <marius.avram@intel.com>
> ---
>  bitbake/lib/bb/ui/buildinfohelper.py |   16 +++++++++++++---
>  1 file changed, 13 insertions(+), 3 deletions(-)
>
> diff --git a/bitbake/lib/bb/ui/buildinfohelper.py
> b/bitbake/lib/bb/ui/buildinfohelper.py
> index 0a8073f..da777f1 100644
> --- a/bitbake/lib/bb/ui/buildinfohelper.py
> +++ b/bitbake/lib/bb/ui/buildinfohelper.py
> @@ -605,11 +605,21 @@ class BuildInfoHelper(object):
>              task_info['task_name'] = taskname
>              task_obj = self.orm_wrapper.get_update_task_object(task_info)
>              return task_obj
> +
> +        # create tasks
> +        tasks = {}
> +        for taskdesc in event._depgraph['tdepends']:
> +            tasks[taskdesc] = _save_a_task(taskdesc)
>
> +        # create dependencies between tasks
>          for taskdesc in event._depgraph['tdepends']:
> -            target = _save_a_task(taskdesc)
> -            for taskdesc1 in event._depgraph['tdepends'][taskdesc]:
> -                dep = _save_a_task(taskdesc1)
> +            target = tasks[taskdesc]
> +            for taskdep in event._depgraph['tdepends'][taskdesc]:
> +                if taskdep not in tasks:
> +                    # Fetch tasks info is not collected previously
> +                    dep = _save_a_task(taskdep)
> +                else:
> +                    dep = tasks[taskdep]
>                  Task_Dependency.objects.get_or_create( task = target,
> depends_on = dep )
>
>      def store_build_package_information(self, event):
> --
> 1.7.9.5
>
> _______________________________________________
> bitbake-devel mailing list
> bitbake-devel@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/bitbake-devel
>

Patch

diff --git a/bitbake/lib/bb/ui/buildinfohelper.py b/bitbake/lib/bb/ui/buildinfohelper.py
index 0a8073f..da777f1 100644
--- a/bitbake/lib/bb/ui/buildinfohelper.py
+++ b/bitbake/lib/bb/ui/buildinfohelper.py
@@ -605,11 +605,21 @@  class BuildInfoHelper(object):
             task_info['task_name'] = taskname
             task_obj = self.orm_wrapper.get_update_task_object(task_info)
             return task_obj
+ 
+        # create tasks
+        tasks = {}
+        for taskdesc in event._depgraph['tdepends']:
+            tasks[taskdesc] = _save_a_task(taskdesc)
 
+        # create dependencies between tasks
         for taskdesc in event._depgraph['tdepends']:
-            target = _save_a_task(taskdesc)
-            for taskdesc1 in event._depgraph['tdepends'][taskdesc]:
-                dep = _save_a_task(taskdesc1)
+            target = tasks[taskdesc]
+            for taskdep in event._depgraph['tdepends'][taskdesc]:
+                if taskdep not in tasks:
+                    # Fetch tasks info is not collected previously
+                    dep = _save_a_task(taskdep)
+                else:
+                    dep = tasks[taskdep]
                 Task_Dependency.objects.get_or_create( task = target, depends_on = dep )
 
     def store_build_package_information(self, event):