Patchwork [RFC,1/2] packagedata: show error when there are multiple pkgdata directories

login
register
mail settings
Submitter Martin Jansa
Date April 3, 2013, 10:14 a.m.
Message ID <1364984074-3636-1-git-send-email-Martin.Jansa@gmail.com>
Download mbox | patch
Permalink /patch/47327/
State Accepted, archived
Headers show

Comments

Martin Jansa - April 3, 2013, 10:14 a.m.
* when PACKAGE_ARCH is changed e.g. from MACHINE_ARCH to TUNE_PKGARCH
  get_subpkgedata_fn is still reading old MACHINE_ARCH directory instead of newer with TUNE_PKGARCH
---
 meta/lib/oe/packagedata.py | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)
Martin Jansa - April 4, 2013, 3:39 p.m.
On Wed, Apr 03, 2013 at 12:14:33PM +0200, Martin Jansa wrote:
> * when PACKAGE_ARCH is changed e.g. from MACHINE_ARCH to TUNE_PKGARCH
>   get_subpkgedata_fn is still reading old MACHINE_ARCH directory instead of newer with TUNE_PKGARCH

This is too strict in some cases, e.g.:

ERROR: More then one pkgdata dir found for pkg 'opkg-collateral'
(
'/OE/shr-core/tmp-eglibc/pkgdata/cortexa8-vfp-neon-oe-linux-gnueabi/runtime/opkg-collateral',
'/OE/shr-core/tmp-eglibc/pkgdata/armv5te-oe-linux-gnueabi/runtime/opkg-collateral'),
returning first

Why do we need to look in all pkgdatadirs? Why not just PACKAGE_ARCH?

> ---
>  meta/lib/oe/packagedata.py | 13 +++++++++++--
>  1 file changed, 11 insertions(+), 2 deletions(-)
> 
> diff --git a/meta/lib/oe/packagedata.py b/meta/lib/oe/packagedata.py
> index 62fd718..2533700 100644
> --- a/meta/lib/oe/packagedata.py
> +++ b/meta/lib/oe/packagedata.py
> @@ -34,11 +34,18 @@ def get_subpkgedata_fn(pkg, d):
>      dirs = all_pkgdatadirs(d)
>  
>      pkgdata = d.expand('${TMPDIR}/pkgdata/')
> +    found = None
>      for dir in dirs:
>          fn = pkgdata + dir + pkg
>          if os.path.exists(fn):
> -            return fn
> -    return d.expand('${PKGDATA_DIR}/runtime/%s' % pkg)
> +	    if found:
> +                bb.error("More then one pkgdata dir found for pkg '%s' ('%s', '%s'), returning first" % (pkg, found, fn))
> +                continue
> +            found = fn
> +    if found:
> +        return found
> +    else:
> +        return d.expand('${PKGDATA_DIR}/runtime/%s' % pkg)
>  
>  def has_subpkgdata(pkg, d):
>      return os.access(get_subpkgedata_fn(pkg, d), os.R_OK)
> @@ -68,6 +75,7 @@ def read_subpkgdata_dict(pkg, d):
>      return ret
>  
>  def _pkgmap(d):
> +    import bb
>      """Return a dictionary mapping package to recipe name."""
>  
>      target_os = d.getVar("TARGET_OS", True)
> @@ -85,6 +93,7 @@ def _pkgmap(d):
>              continue
>  
>          for pn in filter(lambda f: not os.path.isdir(os.path.join(pkgdatadir, f)), files):
> +            bb.warn("Trying path '%s'" % os.path.join(pkgdatadir, pn))
>              try:
>                  pkgdata = read_pkgdatafile(os.path.join(pkgdatadir, pn))
>              except OSError:
> -- 
> 1.8.1.5
>
Paul Eggleton - April 4, 2013, 3:48 p.m.
On Thursday 04 April 2013 17:39:30 Martin Jansa wrote:
> On Wed, Apr 03, 2013 at 12:14:33PM +0200, Martin Jansa wrote:
> > * when PACKAGE_ARCH is changed e.g. from MACHINE_ARCH to TUNE_PKGARCH
> > 
> >   get_subpkgedata_fn is still reading old MACHINE_ARCH directory instead
> >   of newer with TUNE_PKGARCH
> This is too strict in some cases, e.g.:
> 
> ERROR: More then one pkgdata dir found for pkg 'opkg-collateral'
> (
> '/OE/shr-core/tmp-eglibc/pkgdata/cortexa8-vfp-neon-oe-linux-gnueabi/runtime/
> opkg-collateral',
> '/OE/shr-core/tmp-eglibc/pkgdata/armv5te-oe-linux-gnueabi/runtime/opkg-coll
> ateral'), returning first
> 
> Why do we need to look in all pkgdatadirs? Why not just PACKAGE_ARCH?

So this code may well not be doing the right thing, but PACKAGE_ARCH won't 
work either because this is called to get a list of all packages not just one, 
and we therefore have no context to know PACKAGE_ARCH.

Cheers,
Paul
Richard Purdie - April 4, 2013, 3:58 p.m.
On Thu, 2013-04-04 at 17:39 +0200, Martin Jansa wrote:
> On Wed, Apr 03, 2013 at 12:14:33PM +0200, Martin Jansa wrote:
> > * when PACKAGE_ARCH is changed e.g. from MACHINE_ARCH to TUNE_PKGARCH
> >   get_subpkgedata_fn is still reading old MACHINE_ARCH directory instead of newer with TUNE_PKGARCH
> 
> This is too strict in some cases, e.g.:
> 
> ERROR: More then one pkgdata dir found for pkg 'opkg-collateral'
> (
> '/OE/shr-core/tmp-eglibc/pkgdata/cortexa8-vfp-neon-oe-linux-gnueabi/runtime/opkg-collateral',
> '/OE/shr-core/tmp-eglibc/pkgdata/armv5te-oe-linux-gnueabi/runtime/opkg-collateral'),
> returning first
> 
> Why do we need to look in all pkgdatadirs? Why not just PACKAGE_ARCH?

It needs to do resolution, so for example you might have a machine
specific library you'd expect the code to find it first, then
architecture specific and finally all arch only if the others don't work
out.

Having said that, I've had thoughts about making this just look at
machine, package_arch and allarch, not everything. It also gets more
complicated with multilibs.

It may be the different callers might need to specify more about what
exactly they're after as they may want different things.

Cheers,

Richard

Patch

diff --git a/meta/lib/oe/packagedata.py b/meta/lib/oe/packagedata.py
index 62fd718..2533700 100644
--- a/meta/lib/oe/packagedata.py
+++ b/meta/lib/oe/packagedata.py
@@ -34,11 +34,18 @@  def get_subpkgedata_fn(pkg, d):
     dirs = all_pkgdatadirs(d)
 
     pkgdata = d.expand('${TMPDIR}/pkgdata/')
+    found = None
     for dir in dirs:
         fn = pkgdata + dir + pkg
         if os.path.exists(fn):
-            return fn
-    return d.expand('${PKGDATA_DIR}/runtime/%s' % pkg)
+	    if found:
+                bb.error("More then one pkgdata dir found for pkg '%s' ('%s', '%s'), returning first" % (pkg, found, fn))
+                continue
+            found = fn
+    if found:
+        return found
+    else:
+        return d.expand('${PKGDATA_DIR}/runtime/%s' % pkg)
 
 def has_subpkgdata(pkg, d):
     return os.access(get_subpkgedata_fn(pkg, d), os.R_OK)
@@ -68,6 +75,7 @@  def read_subpkgdata_dict(pkg, d):
     return ret
 
 def _pkgmap(d):
+    import bb
     """Return a dictionary mapping package to recipe name."""
 
     target_os = d.getVar("TARGET_OS", True)
@@ -85,6 +93,7 @@  def _pkgmap(d):
             continue
 
         for pn in filter(lambda f: not os.path.isdir(os.path.join(pkgdatadir, f)), files):
+            bb.warn("Trying path '%s'" % os.path.join(pkgdatadir, pn))
             try:
                 pkgdata = read_pkgdatafile(os.path.join(pkgdatadir, pn))
             except OSError: