Patchwork [1/1] utils.bbclass: make FILESEXTRAPATHS colon delimited

login
register
mail settings
Submitter Darren Hart
Date May 25, 2011, 11:05 p.m.
Message ID <f2eb88161222a6df60739367d2f44342d3bfbbf2.1306364294.git.dvhart@linux.intel.com>
Download mbox | patch
Permalink /patch/4777/
State New, archived
Headers show

Comments

Darren Hart - May 25, 2011, 11:05 p.m.
Fixes [YOCTO 1102]

Path variables are typically : delimited. White space is allowed in paths, so
is not a good choice for separating paths. Currently utils.bbclass performs the
following:

    extrapaths = (bb.data.getVar("FILESEXTRAPATHS", d, True) or "").split()

This splits FILESEXTRAPATHS on whitespace. It later splits overrides on : and
reassembles them all together as : delimited.

There is only one user of FILESEXTRAPATHS in oe-core (qt4-tools-native, which
uses : anyway) and none in oe.

Change the split() in utils.bbclass to split on : instead of whitespace. When
splitting on a defined string (":") we must be careful to handle the empty
string case which returns [''] instead of [].

Tested building qt4-tools-native and core-image-minimal for surgarbay from
meta-intel with a couple extra layers with FILESEXTRAPATHS modifications added.

Signed-off-by: Darren Hart <dvhart@linux.intel.com>
Cc: Phil Blundell <pb@pbcl.net>
---
 meta/classes/utils.bbclass |    6 ++++--
 1 files changed, 4 insertions(+), 2 deletions(-)
Richard Purdie - May 26, 2011, 11:46 p.m.
On Wed, 2011-05-25 at 16:05 -0700, Darren Hart wrote:
> Fixes [YOCTO 1102]
> 
> Path variables are typically : delimited. White space is allowed in paths, so
> is not a good choice for separating paths. Currently utils.bbclass performs the
> following:
> 
>     extrapaths = (bb.data.getVar("FILESEXTRAPATHS", d, True) or "").split()
> 
> This splits FILESEXTRAPATHS on whitespace. It later splits overrides on : and
> reassembles them all together as : delimited.
> 
> There is only one user of FILESEXTRAPATHS in oe-core (qt4-tools-native, which
> uses : anyway) and none in oe.
> 
> Change the split() in utils.bbclass to split on : instead of whitespace. When
> splitting on a defined string (":") we must be careful to handle the empty
> string case which returns [''] instead of [].
> 
> Tested building qt4-tools-native and core-image-minimal for surgarbay from
> meta-intel with a couple extra layers with FILESEXTRAPATHS modifications added.
> 
> Signed-off-by: Darren Hart <dvhart@linux.intel.com>
> Cc: Phil Blundell <pb@pbcl.net>
> ---
>  meta/classes/utils.bbclass |    6 ++++--
>  1 files changed, 4 insertions(+), 2 deletions(-)
> 
> diff --git a/meta/classes/utils.bbclass b/meta/classes/utils.bbclass
> index e103351..9930a24 100644
> --- a/meta/classes/utils.bbclass
> +++ b/meta/classes/utils.bbclass
> @@ -331,8 +331,10 @@ def explode_deps(s):
>  
>  def base_set_filespath(path, d):
>  	filespath = []
> -	extrapaths = (bb.data.getVar("FILESEXTRAPATHS", d, True) or "").split()
> -	path = extrapaths + path
> +	extrapaths = (bb.data.getVar("FILESEXTRAPATHS", d, True) or "")
> +	# Don't prepend empty strings to the path list
> +	if extrapaths != "":
> +		path = extrapaths.split(":") + path
>  	# The ":" ensures we have an 'empty' override
>  	overrides = (bb.data.getVar("OVERRIDES", d, 1) or "") + ":"
>  	for p in path:

This is being a little picky but I find the above hard to read and can't
we just do:

-	extrapaths = (bb.data.getVar("FILESEXTRAPATHS", d, True) or "").split()
+	extrapaths = (bb.data.getVar("FILESEXTRAPATHS", d, True) or "").split(":")

?

Cheers,

Richard
Darren Hart - May 27, 2011, 3:16 a.m.
On 05/26/2011 04:46 PM, Richard Purdie wrote:
> On Wed, 2011-05-25 at 16:05 -0700, Darren Hart wrote:
>> Fixes [YOCTO 1102]
>>
>> Path variables are typically : delimited. White space is allowed in paths, so
>> is not a good choice for separating paths. Currently utils.bbclass performs the
>> following:
>>
>>     extrapaths = (bb.data.getVar("FILESEXTRAPATHS", d, True) or "").split()
>>
>> This splits FILESEXTRAPATHS on whitespace. It later splits overrides on : and
>> reassembles them all together as : delimited.
>>
>> There is only one user of FILESEXTRAPATHS in oe-core (qt4-tools-native, which
>> uses : anyway) and none in oe.
>>
>> Change the split() in utils.bbclass to split on : instead of whitespace. When
>> splitting on a defined string (":") we must be careful to handle the empty
>> string case which returns [''] instead of [].
>>
>> Tested building qt4-tools-native and core-image-minimal for surgarbay from
>> meta-intel with a couple extra layers with FILESEXTRAPATHS modifications added.
>>
>> Signed-off-by: Darren Hart <dvhart@linux.intel.com>
>> Cc: Phil Blundell <pb@pbcl.net>
>> ---
>>  meta/classes/utils.bbclass |    6 ++++--
>>  1 files changed, 4 insertions(+), 2 deletions(-)
>>
>> diff --git a/meta/classes/utils.bbclass b/meta/classes/utils.bbclass
>> index e103351..9930a24 100644
>> --- a/meta/classes/utils.bbclass
>> +++ b/meta/classes/utils.bbclass
>> @@ -331,8 +331,10 @@ def explode_deps(s):
>>  
>>  def base_set_filespath(path, d):
>>  	filespath = []
>> -	extrapaths = (bb.data.getVar("FILESEXTRAPATHS", d, True) or "").split()
>> -	path = extrapaths + path
>> +	extrapaths = (bb.data.getVar("FILESEXTRAPATHS", d, True) or "")
>> +	# Don't prepend empty strings to the path list
>> +	if extrapaths != "":
>> +		path = extrapaths.split(":") + path
>>  	# The ":" ensures we have an 'empty' override
>>  	overrides = (bb.data.getVar("OVERRIDES", d, 1) or "") + ":"
>>  	for p in path:
> 
> This is being a little picky but I find the above hard to read and can't
> we just do:
> 
> -	extrapaths = (bb.data.getVar("FILESEXTRAPATHS", d, True) or "").split()
> +	extrapaths = (bb.data.getVar("FILESEXTRAPATHS", d, True) or "").split(":")

Unfortunately not (this is what I tried initially), python split() has a
rather annoying "feature", see:
http://docs.python.org/library/string.html

Where it states:
"
The behavior of split on an empty string depends on the value of sep. If
sep is not specified, or specified as None, the result will be an empty
list. If sep is specified as any string, the result will be a list
containing one element which is an empty string.
"

ie:
"".split() -> []
"".split(":") -> [""]

So by changing from split() to split(":") we change the behavior of
split when operating on empty strings, requiring us to special case the
output. Rather obnoxious wouldn't you say?
Phil Blundell - May 27, 2011, 9:37 a.m.
On Thu, 2011-05-26 at 20:16 -0700, Darren Hart wrote:
> So by changing from split() to split(":") we change the behavior of
> split when operating on empty strings, requiring us to special case the
> output. Rather obnoxious wouldn't you say?

Yes, that is an annoying misfeature.  The other related thing to watch
out for is this:

>>> print " a b ".split()
['a', 'b']
>>> print ":a:b:".split(":")
['', 'a', 'b', '']
>>> 

... in other words, if you make the separator be anything other than a
space, you need to take special care to strip off any null elements that
might have been accidentally generated at the start and end.

p.
Richard Purdie - May 27, 2011, 3:45 p.m.
On Thu, 2011-05-26 at 20:16 -0700, Darren Hart wrote:
> 
> On 05/26/2011 04:46 PM, Richard Purdie wrote:
> > On Wed, 2011-05-25 at 16:05 -0700, Darren Hart wrote:
> >> Fixes [YOCTO 1102]
> >>
> >> Path variables are typically : delimited. White space is allowed in paths, so
> >> is not a good choice for separating paths. Currently utils.bbclass performs the
> >> following:
> >>
> >>     extrapaths = (bb.data.getVar("FILESEXTRAPATHS", d, True) or "").split()
> >>
> >> This splits FILESEXTRAPATHS on whitespace. It later splits overrides on : and
> >> reassembles them all together as : delimited.
> >>
> >> There is only one user of FILESEXTRAPATHS in oe-core (qt4-tools-native, which
> >> uses : anyway) and none in oe.
> >>
> >> Change the split() in utils.bbclass to split on : instead of whitespace. When
> >> splitting on a defined string (":") we must be careful to handle the empty
> >> string case which returns [''] instead of [].
> >>
> >> Tested building qt4-tools-native and core-image-minimal for surgarbay from
> >> meta-intel with a couple extra layers with FILESEXTRAPATHS modifications added.
> >>
> >> Signed-off-by: Darren Hart <dvhart@linux.intel.com>
> >> Cc: Phil Blundell <pb@pbcl.net>
> >> ---
> >>  meta/classes/utils.bbclass |    6 ++++--
> >>  1 files changed, 4 insertions(+), 2 deletions(-)
> >>
> >> diff --git a/meta/classes/utils.bbclass b/meta/classes/utils.bbclass
> >> index e103351..9930a24 100644
> >> --- a/meta/classes/utils.bbclass
> >> +++ b/meta/classes/utils.bbclass
> >> @@ -331,8 +331,10 @@ def explode_deps(s):
> >>  
> >>  def base_set_filespath(path, d):
> >>  	filespath = []
> >> -	extrapaths = (bb.data.getVar("FILESEXTRAPATHS", d, True) or "").split()
> >> -	path = extrapaths + path
> >> +	extrapaths = (bb.data.getVar("FILESEXTRAPATHS", d, True) or "")
> >> +	# Don't prepend empty strings to the path list
> >> +	if extrapaths != "":
> >> +		path = extrapaths.split(":") + path
> >>  	# The ":" ensures we have an 'empty' override
> >>  	overrides = (bb.data.getVar("OVERRIDES", d, 1) or "") + ":"
> >>  	for p in path:
> > 
> > This is being a little picky but I find the above hard to read and can't
> > we just do:
> > 
> > -	extrapaths = (bb.data.getVar("FILESEXTRAPATHS", d, True) or "").split()
> > +	extrapaths = (bb.data.getVar("FILESEXTRAPATHS", d, True) or "").split(":")
> 
> Unfortunately not (this is what I tried initially), python split() has a
> rather annoying "feature", see:
> http://docs.python.org/library/string.html
> 
> Where it states:
> "
> The behavior of split on an empty string depends on the value of sep. If
> sep is not specified, or specified as None, the result will be an empty
> list. If sep is specified as any string, the result will be a list
> containing one element which is an empty string.
> "
> 
> ie:
> "".split() -> []
> "".split(":") -> [""]
> 
> So by changing from split() to split(":") we change the behavior of
> split when operating on empty strings, requiring us to special case the
> output. Rather obnoxious wouldn't you say?

Horrible but merged to master ;-).

Cheers,

Richard

Patch

diff --git a/meta/classes/utils.bbclass b/meta/classes/utils.bbclass
index e103351..9930a24 100644
--- a/meta/classes/utils.bbclass
+++ b/meta/classes/utils.bbclass
@@ -331,8 +331,10 @@  def explode_deps(s):
 
 def base_set_filespath(path, d):
 	filespath = []
-	extrapaths = (bb.data.getVar("FILESEXTRAPATHS", d, True) or "").split()
-	path = extrapaths + path
+	extrapaths = (bb.data.getVar("FILESEXTRAPATHS", d, True) or "")
+	# Don't prepend empty strings to the path list
+	if extrapaths != "":
+		path = extrapaths.split(":") + path
 	# The ":" ensures we have an 'empty' override
 	overrides = (bb.data.getVar("OVERRIDES", d, 1) or "") + ":"
 	for p in path: