toaster: buildinfohelper Handle regex pattern paths

Submitted by Michael Wood on Oct. 20, 2016, 2:36 p.m. | Patch ID: 133095

Details

Message ID 1476974166-8004-1-git-send-email-michael.g.wood@intel.com
State New
Headers show

Commit Message

Michael Wood Oct. 20, 2016, 2:36 p.m.
From: brian avery <brian.avery@intel.com>

We were presuming that all the layer dependency information was of the
form "^/path/to/layer" to we were just stripping the leading "^" off of
the layer information when we were matching the layer priorities to the
toaster database.  This patch splits out the priorities layer match which
gets a  regex from the task/recipe match which is gets a path.

Signed-off-by: brian avery <brian.avery@intel.com>
---
 bitbake/lib/bb/ui/buildinfohelper.py | 23 ++++++++++++++++++++++-
 1 file changed, 22 insertions(+), 1 deletion(-)

Patch hide | download patch | download mbox

diff --git a/bitbake/lib/bb/ui/buildinfohelper.py b/bitbake/lib/bb/ui/buildinfohelper.py
index 5b69660..30adfc8 100644
--- a/bitbake/lib/bb/ui/buildinfohelper.py
+++ b/bitbake/lib/bb/ui/buildinfohelper.py
@@ -982,6 +982,27 @@  class BuildInfoHelper(object):
             pass
         return task_information
 
+    def _get_layer_version_for_dependency(self, pathRE):
+        self._ensure_build()
+
+        def _slkey_interactive(layer_version):
+            assert isinstance(layer_version, Layer_Version)
+            return len(layer_version.local_path)
+
+        # we don't care if we match the trailing slashes
+        p = re.compile(re.sub("/[^/]*?$","",pathRE))
+        # Heuristics: we always match recipe to the deepest layer path in the discovered layers
+        for lvo in sorted(self.orm_wrapper.layer_version_objects, reverse=True, key=_slkey_interactive):
+            if p.fullmatch(lvo.local_path):
+                return lvo
+            if lvo.layer.local_source_dir:
+                if p.fullmatch(lvo.layer.local_source_dir):
+                    return lvo
+        #if we get here, we didn't read layers correctly; dump whatever information we have on the error log
+        logger.warning("Could not match layer dependency for path %s : %s", path, self.orm_wrapper.layer_version_objects)
+
+
+
     def _get_layer_version_for_path(self, path):
         self._ensure_build()
 
@@ -1372,7 +1393,7 @@  class BuildInfoHelper(object):
         if 'layer-priorities' in event._depgraph.keys():
             for lv in event._depgraph['layer-priorities']:
                 (_, path, _, priority) = lv
-                layer_version_obj = self._get_layer_version_for_path(path[1:]) # paths start with a ^
+                layer_version_obj = self._get_layer_version_for_dependency(path)
                 assert layer_version_obj is not None
                 layer_version_obj.priority = priority
                 layer_version_obj.save()

Comments

Michael Wood Oct. 20, 2016, 4:57 p.m.
On 20/10/16 15:36, Michael Wood wrote:
> From: brian avery <brian.avery@intel.com>
>
> We were presuming that all the layer dependency information was of the
> form "^/path/to/layer" to we were just stripping the leading "^" off of
> the layer information when we were matching the layer priorities to the
> toaster database.  This patch splits out the priorities layer match which
> gets a  regex from the task/recipe match which is gets a path.
>
> Signed-off-by: brian avery <brian.avery@intel.com>
> ---
>   bitbake/lib/bb/ui/buildinfohelper.py | 23 ++++++++++++++++++++++-
>   1 file changed, 22 insertions(+), 1 deletion(-)
>
> diff --git a/bitbake/lib/bb/ui/buildinfohelper.py b/bitbake/lib/bb/ui/buildinfohelper.py
> index 5b69660..30adfc8 100644
> --- a/bitbake/lib/bb/ui/buildinfohelper.py
> +++ b/bitbake/lib/bb/ui/buildinfohelper.py
> @@ -982,6 +982,27 @@ class BuildInfoHelper(object):
>               pass
>           return task_information
>   
> +    def _get_layer_version_for_dependency(self, pathRE):

As this is quite an obscure issue to the toaster developer could you add 
a doc string into the new function to explain what a pathRE is

> +        self._ensure_build()
> +
> +        def _slkey_interactive(layer_version):
> +            assert isinstance(layer_version, Layer_Version)
> +            return len(layer_version.local_path)

I know this naming is just from the _get_layer_version_for_path but 
while we're there we may as well make the function name make sense, 
something like _sort_longest_path?


> +
> +        # we don't care if we match the trailing slashes
> +        p = re.compile(re.sub("/[^/]*?$","",pathRE))
> +        # Heuristics: we always match recipe to the deepest layer path in the discovered layers
> +        for lvo in sorted(self.orm_wrapper.layer_version_objects, reverse=True, key=_slkey_interactive):
> +            if p.fullmatch(lvo.local_path):
> +                return lvo
> +            if lvo.layer.local_source_dir:
> +                if p.fullmatch(lvo.layer.local_source_dir):
> +                    return lvo
> +        #if we get here, we didn't read layers correctly; dump whatever information we have on the error log
> +        logger.warning("Could not match layer dependency for path %s : %s", path, self.orm_wrapper.layer_version_objects)
> +
> +
> +
>       def _get_layer_version_for_path(self, path):
>           self._ensure_build()
>   
> @@ -1372,7 +1393,7 @@ class BuildInfoHelper(object):
>           if 'layer-priorities' in event._depgraph.keys():
>               for lv in event._depgraph['layer-priorities']:
>                   (_, path, _, priority) = lv
> -                layer_version_obj = self._get_layer_version_for_path(path[1:]) # paths start with a ^
> +                layer_version_obj = self._get_layer_version_for_dependency(path)
>                   assert layer_version_obj is not None
>                   layer_version_obj.priority = priority
>                   layer_version_obj.save()

I've tested this and it works for me, my test setup was having meta-mel 
in the project's layers and then building core-image-minimal, with this 
patch I no longer see 'unidentified layer' and the correct recipe 
information gets logged.

Thanks,

Michael