Patchwork [1/1] package.bbclass: Allow overriding of debugedit starting path

login
register
mail settings
Submitter Peter Seebach
Date July 11, 2012, 4 p.m.
Message ID <22e5e5adc39fb855badc6d1260fbd4b30d966530.1342022120.git.peter.seebach@windriver.com>
Download mbox | patch
Permalink /patch/31773/
State New
Headers show

Comments

Peter Seebach - July 11, 2012, 4 p.m.
In the usual case, the computed path used for debugedit to fix
up path references for the target filesystem is correct. However,
prebuilt binary components, such as prebuilt toolchains, can
have debug paths that do not reflect paths within the local
build directory. Providing an override lets us continue to use
the standard debugedit logic in package.bbclass.

Signed-off-by: Peter Seebach <peter.seebach@windriver.com>
---
 meta/classes/package.bbclass |    3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)
Richard Purdie - July 11, 2012, 4:12 p.m.
On Wed, 2012-07-11 at 11:00 -0500, Peter Seebach wrote:
> In the usual case, the computed path used for debugedit to fix
> up path references for the target filesystem is correct. However,
> prebuilt binary components, such as prebuilt toolchains, can
> have debug paths that do not reflect paths within the local
> build directory. Providing an override lets us continue to use
> the standard debugedit logic in package.bbclass.
> 
> Signed-off-by: Peter Seebach <peter.seebach@windriver.com>
> ---
>  meta/classes/package.bbclass |    3 ++-
>  1 files changed, 2 insertions(+), 1 deletions(-)
> 
> diff --git a/meta/classes/package.bbclass b/meta/classes/package.bbclass
> index e6856cd..1a8c7f8 100644
> --- a/meta/classes/package.bbclass
> +++ b/meta/classes/package.bbclass
> @@ -191,6 +191,7 @@ def splitfile(file, debugfile, debugsrcdir, d):
>      debugedit = d.expand("${STAGING_LIBDIR_NATIVE}/rpm/bin/debugedit")
>      workdir = d.getVar("WORKDIR", True)
>      workparentdir = os.path.dirname(workdir)
> +    overridedir = d.getVar("DEBUGSRC_OVERRIDE_PATH", True) or ""
>      sourcefile = d.expand("${WORKDIR}/debugsources.list")

I think I at least would find this slightly less confusing as:

workparentdir = d.getVar("DEBUGSRC_OVERRIDE_PATH", True) or os.path.dirname(workdir)

Cheers,

Richard
Peter Seebach - July 11, 2012, 4:33 p.m.
On Wed, 11 Jul 2012 17:12:29 +0100
Richard Purdie <richard.purdie@linuxfoundation.org> wrote:

> I think I at least would find this slightly less confusing as:
> 
> workparentdir = d.getVar("DEBUGSRC_OVERRIDE_PATH", True) or
> os.path.dirname(workdir)

Wait, LESS confusing?

I appear to have tragically misunderstood the design goals of
package.bbclass.  :P

But yes, that's a good improvement. Applied locally.

Speaking of confusing: If purely hypothetically I wanted to
submit a patch which standardized the indentation in package.bbclass,
would anyone be interested in that?  I ask only because while I can
accept either 8-space or 4-space indentation, I find it comforting when
any given file full of Python source uses one or the other.

And while there's currently only a couple of blocks of 4-space
indentation in the file, we *normally* use 4-space, that's the
quasi-official Python community norm, and a LOT of the "too long" lines
in that file would be much more readable at 4 spaces.

(This would be a totally separate patch, and I'm not super happy about
the idea of a patch which updates half or more of the lines in the
file, but it's not as though it'll be less painful to fix later.)

-s
Richard Purdie - July 11, 2012, 5:36 p.m.
On Wed, 2012-07-11 at 11:33 -0500, Peter Seebach wrote:
> On Wed, 11 Jul 2012 17:12:29 +0100
> Richard Purdie <richard.purdie@linuxfoundation.org> wrote:
> 
> > I think I at least would find this slightly less confusing as:
> > 
> > workparentdir = d.getVar("DEBUGSRC_OVERRIDE_PATH", True) or
> > os.path.dirname(workdir)
> 
> Wait, LESS confusing?
> 
> I appear to have tragically misunderstood the design goals of
> package.bbclass.  :P

Well, we are trying over time... :)

> But yes, that's a good improvement. Applied locally.
> 
> Speaking of confusing: If purely hypothetically I wanted to
> submit a patch which standardized the indentation in package.bbclass,
> would anyone be interested in that?  I ask only because while I can
> accept either 8-space or 4-space indentation, I find it comforting when
> any given file full of Python source uses one or the other.

It should all use 4 space for python functions. There is however a twist
which is due to the way we handle _prepend and _append. Those prepends
and appends have whitespace too and I seem to remember issues with
whitespace matching.

Yes, this is horrible. This is why that file hasn't been touched for
whitespace though.

> And while there's currently only a couple of blocks of 4-space
> indentation in the file, we *normally* use 4-space, that's the
> quasi-official Python community norm, and a LOT of the "too long" lines
> in that file would be much more readable at 4 spaces.
> 
> (This would be a totally separate patch, and I'm not super happy about
> the idea of a patch which updates half or more of the lines in the
> file, but it's not as though it'll be less painful to fix later.)

I'd be interested to see how much breakage we get from changing it. In
fact I just tried it, the result:

http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=rpurdie/t14&id=3db917bfb2455715a3a3a542ea831d05ca1cf9f7

the particularly nasty bit:

python populate_packages () {
	# Whitespace is deliberately a tab here due to the number of packages which
	# prepend this fuction :(
	populate_packages_core(d)
}

other than that it does seem to be working as long as I tweaked the
busybox recipe and update-alternatives too. We could go through and
change all the populate_packages_prepend functions but its the ones
outside OE-Core I worry about. I also worry there are some _append
functions now silently failing though :(.

Cheers,

Richard
Ross Burton - July 11, 2012, 5:56 p.m.
Incredibly sorry for top-posting, but a build history diff should show any delta and assuming none will give a lot more confidence in the changes being complete. In theory a simple change of indentation shouldn't result in any changes to the image, right?

Ross
Peter Seebach - July 11, 2012, 6:54 p.m.
On Wed, 11 Jul 2012 18:36:50 +0100
Richard Purdie <richard.purdie@linuxfoundation.org> wrote:

> It should all use 4 space for python functions. There is however a
> twist which is due to the way we handle _prepend and _append. Those
> prepends and appends have whitespace too and I seem to remember
> issues with whitespace matching.
> 
> Yes, this is horrible. This is why that file hasn't been touched for
> whitespace though.

noooooooo

That hurts so much.

Okay, crazy thought.  Imagine that we were to make a modified parser,
with the following behavior:

Whenever reading a python function, count the indentation of the first
line (N), then replace every N spaces of indentation with 4 spaces.

If this worked, it would make it possible to start fixing stuff.

DISCLAIMER:  Due to Reasons, I have slept about 15 minutes out of the
last day, also I'm sick. When I say "crazy thought", I do not kid.

Anyway:  Anyone have other thoughts/comments on the possible starting
path fixup? I am pretty sure that it is sufficient for all real cases,
with the one exception being prebuilts where the debug path names are
unreasonably short.  (I theorize that a more advanced debugedit could
be written which could handle those by rewriting files.  I nominate
someone else to try to actually do it.)

My initial theory that I'd need to be able to specify the destination
path evaporates with the realization that we can always move source
files to whatever path we want them in; it's just that it's nicer for
everyone if the debug source files are always in the standard locations,
give or take, no matter how they were built.

-s
Chris Larson - July 11, 2012, 7:01 p.m.
On Wed, Jul 11, 2012 at 11:54 AM, Peter Seebach
<peter.seebach@windriver.com> wrote:
> Okay, crazy thought.  Imagine that we were to make a modified parser,
> with the following behavior:
>
> Whenever reading a python function, count the indentation of the first
> line (N), then replace every N spaces of indentation with 4 spaces.
>
> If this worked, it would make it possible to start fixing stuff.

That's one possibility. The other would be to treat _append/_prepended
functions as independent entities. Rather than concatenating
functions, it'd add them to the prefuncs/postfuncs or equivalent.
Without concatenation, there'd be no issues with mismatches.
Ross Burton - July 11, 2012, 7:54 p.m.
On 11 July 2012 20:01, Chris Larson <clarson@kergoth.com> wrote:
> That's one possibility. The other would be to treat _append/_prepended
> functions as independent entities. Rather than concatenating
> functions, it'd add them to the prefuncs/postfuncs or equivalent.
> Without concatenation, there'd be no issues with mismatches.

I always thought it would be best to give each append/prepend chunk a
unique name and then call them sequentially instead of gluing them
together.

Ross
Richard Purdie - July 11, 2012, 8:20 p.m.
On Wed, 2012-07-11 at 20:54 +0100, Burton, Ross wrote:
> On 11 July 2012 20:01, Chris Larson <clarson@kergoth.com> wrote:
> > That's one possibility. The other would be to treat _append/_prepended
> > functions as independent entities. Rather than concatenating
> > functions, it'd add them to the prefuncs/postfuncs or equivalent.
> > Without concatenation, there'd be no issues with mismatches.
> 
> I always thought it would be best to give each append/prepend chunk a
> unique name and then call them sequentially instead of gluing them
> together.

Some of them even rely on variables set in other fragments. I think I
killed the one example I saw but there still could be something
potentially relying on this :(

I agree we need to fix this. I'm tempted to have the python parser just
error on finding tab indentation.

Cheers,

Richard

Patch

diff --git a/meta/classes/package.bbclass b/meta/classes/package.bbclass
index e6856cd..1a8c7f8 100644
--- a/meta/classes/package.bbclass
+++ b/meta/classes/package.bbclass
@@ -191,6 +191,7 @@  def splitfile(file, debugfile, debugsrcdir, d):
     debugedit = d.expand("${STAGING_LIBDIR_NATIVE}/rpm/bin/debugedit")
     workdir = d.getVar("WORKDIR", True)
     workparentdir = os.path.dirname(workdir)
+    overridedir = d.getVar("DEBUGSRC_OVERRIDE_PATH", True) or ""
     sourcefile = d.expand("${WORKDIR}/debugsources.list")
 
     # We ignore kernel modules, we don't generate debug info files.
@@ -205,7 +206,7 @@  def splitfile(file, debugfile, debugsrcdir, d):
 
     # We need to extract the debug src information here...
     if debugsrcdir:
-	subprocess.call("%s'%s' -b '%s' -d '%s' -i -l '%s' '%s'" % (pathprefix, debugedit, workparentdir, debugsrcdir, sourcefile, file), shell=True)
+	subprocess.call("%s'%s' -b '%s' -d '%s' -i -l '%s' '%s'" % (pathprefix, debugedit, overridedir or workparentdir, debugsrcdir, sourcefile, file), shell=True)
 
     bb.mkdirhier(os.path.dirname(debugfile))