Patchwork [RFC,v5,1/2] base.bbclass: Add support to EXTRA_DISTRO_FEATURES

login
register
mail settings
Submitter Otavio Salvador
Date Aug. 23, 2013, 2:48 p.m.
Message ID <1377269339-6139-2-git-send-email-otavio@ossystems.com.br>
Download mbox | patch
Permalink /patch/56361/
State New
Headers show

Comments

Otavio Salvador - Aug. 23, 2013, 2:48 p.m.
This allow the addition and removal of distro features easily. To add
a feature, use:

 EXTRA_DISTRO_FEATURES += "wayland"

and to remove, use '~' prefix, as:

 EXTRA_DISTRO_FEATURES += "~x11"

This code has been mostly copied from Mentor Graphics public layer but
changed the variable name for a more descriptive name. The original
code can be seen at user_feature.bbclass at:

 http://git.yoctoproject.org/cgit/cgit.cgi/meta-mentor

Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
---
Changes in v5:
- Resend

Changes in v4:
- Drop _prepend/_append hack and remove/add DISTRO_FEATURES (thanks to
  RP for the tip).

Changes in v3:
- Drop _prepend/_append flags from DISTRO_FEATURES to avoid the re-add

Changes in v2:
- Use data copy to fix wrong prepend/append expanding.

 meta/classes/base.bbclass |  4 ++++
 meta/lib/oe/utils.py      | 24 ++++++++++++++++++++++++
 2 files changed, 28 insertions(+)
Richard Purdie - Aug. 23, 2013, 2:59 p.m.
On Fri, 2013-08-23 at 11:48 -0300, Otavio Salvador wrote:
> This allow the addition and removal of distro features easily. To add
> a feature, use:
> 
>  EXTRA_DISTRO_FEATURES += "wayland"
> 
> and to remove, use '~' prefix, as:
> 
>  EXTRA_DISTRO_FEATURES += "~x11"
> 
> This code has been mostly copied from Mentor Graphics public layer but
> changed the variable name for a more descriptive name. The original
> code can be seen at user_feature.bbclass at:
> 
>  http://git.yoctoproject.org/cgit/cgit.cgi/meta-mentor
> 
> Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>

I discussed this with Otavio. My personal view is that we should hold
off this right now. The removal issues he ran into with poky have been
addressed in other ways so there is less of an immediate issue.

The better fix for this kind of problem is -=/=- or more likely _remove
override support in bitbake. I did propose a _remove patch a while back
but Chris has some valid concerns about it, I don't remember the details
offhand. I'd much rather we focused on fixing that and getting that
merged than trying to add this functionality to every variable.

Cheers,

Richard



> ---
> Changes in v5:
> - Resend
> 
> Changes in v4:
> - Drop _prepend/_append hack and remove/add DISTRO_FEATURES (thanks to
>   RP for the tip).
> 
> Changes in v3:
> - Drop _prepend/_append flags from DISTRO_FEATURES to avoid the re-add
> 
> Changes in v2:
> - Use data copy to fix wrong prepend/append expanding.
> 
>  meta/classes/base.bbclass |  4 ++++
>  meta/lib/oe/utils.py      | 24 ++++++++++++++++++++++++
>  2 files changed, 28 insertions(+)
> 
> diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
> index 98b823e..f30f4db 100644
> --- a/meta/classes/base.bbclass
> +++ b/meta/classes/base.bbclass
> @@ -288,6 +288,9 @@ def buildcfg_neededvars(d):
>      if pesteruser:
>          bb.fatal('The following variable(s) were not set: %s\nPlease set them directly, or choose a MACHINE or DISTRO that sets them.' % ', '.join(pesteruser))
>  
> +EXTRA_DISTRO_FEATURES ?= ""
> +EXTRA_DISTRO_FEATURES[type] = "list"
> +
>  addhandler base_eventhandler
>  base_eventhandler[eventmask] = "bb.event.ConfigParsed bb.event.BuildStarted"
>  python base_eventhandler() {
> @@ -297,6 +300,7 @@ python base_eventhandler() {
>          preferred_ml_updates(e.data)
>          oe.utils.features_backfill("DISTRO_FEATURES", e.data)
>          oe.utils.features_backfill("MACHINE_FEATURES", e.data)
> +        oe.utils.extra_distro_features(e.data)
>  
>      if isinstance(e, bb.event.BuildStarted):
>          localdata = bb.data.createCopy(e.data)
> diff --git a/meta/lib/oe/utils.py b/meta/lib/oe/utils.py
> index 82987e8..0c9dde0 100644
> --- a/meta/lib/oe/utils.py
> +++ b/meta/lib/oe/utils.py
> @@ -117,6 +117,30 @@ def features_backfill(var,d):
>      if addfeatures:
>          d.appendVar(var, " " + " ".join(addfeatures))
>  
> +def extra_distro_features(d):
> +    import oe.data
> +
> +    l = d.createCopy()
> +    l.finalize()
> +
> +    extra_features = oe.data.typed_value('EXTRA_DISTRO_FEATURES', l)
> +    if not extra_features:
> +        return
> +
> +    distro_features = l.getVar('DISTRO_FEATURES', True).split()
> +    for feature in extra_features:
> +        if feature.startswith('~'):
> +            feature = feature[1:]
> +            if feature in distro_features:
> +                distro_features.remove(feature)
> +        else:
> +            if feature not in distro_features:
> +                distro_features.append(feature)
> +
> +    # Avoid readding the removed features later
> +    d.delVar("DISTRO_FEATURES")
> +
> +    d.setVar('DISTRO_FEATURES', ' '.join(distro_features))
>  
>  def packages_filter_out_system(d):
>      """
Otavio Salvador - Aug. 23, 2013, 3:11 p.m.
On Fri, Aug 23, 2013 at 11:59 AM, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> On Fri, 2013-08-23 at 11:48 -0300, Otavio Salvador wrote:
>> This allow the addition and removal of distro features easily. To add
>> a feature, use:
>>
>>  EXTRA_DISTRO_FEATURES += "wayland"
>>
>> and to remove, use '~' prefix, as:
>>
>>  EXTRA_DISTRO_FEATURES += "~x11"
>>
>> This code has been mostly copied from Mentor Graphics public layer but
>> changed the variable name for a more descriptive name. The original
>> code can be seen at user_feature.bbclass at:
>>
>>  http://git.yoctoproject.org/cgit/cgit.cgi/meta-mentor
>>
>> Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
>
> I discussed this with Otavio. My personal view is that we should hold
> off this right now. The removal issues he ran into with poky have been
> addressed in other ways so there is less of an immediate issue.
>
> The better fix for this kind of problem is -=/=- or more likely _remove
> override support in bitbake. I did propose a _remove patch a while back
> but Chris has some valid concerns about it, I don't remember the details
> offhand. I'd much rather we focused on fixing that and getting that
> merged than trying to add this functionality to every variable.

Chris, do you know the problems Richard's patch has? I can take a look
and try to fix it so we can get this in for 1.5.
Richard Purdie - Aug. 23, 2013, 3:29 p.m.
On Fri, 2013-08-23 at 12:11 -0300, Otavio Salvador wrote:
> On Fri, Aug 23, 2013 at 11:59 AM, Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> > On Fri, 2013-08-23 at 11:48 -0300, Otavio Salvador wrote:
> >> This allow the addition and removal of distro features easily. To add
> >> a feature, use:
> >>
> >>  EXTRA_DISTRO_FEATURES += "wayland"
> >>
> >> and to remove, use '~' prefix, as:
> >>
> >>  EXTRA_DISTRO_FEATURES += "~x11"
> >>
> >> This code has been mostly copied from Mentor Graphics public layer but
> >> changed the variable name for a more descriptive name. The original
> >> code can be seen at user_feature.bbclass at:
> >>
> >>  http://git.yoctoproject.org/cgit/cgit.cgi/meta-mentor
> >>
> >> Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
> >
> > I discussed this with Otavio. My personal view is that we should hold
> > off this right now. The removal issues he ran into with poky have been
> > addressed in other ways so there is less of an immediate issue.
> >
> > The better fix for this kind of problem is -=/=- or more likely _remove
> > override support in bitbake. I did propose a _remove patch a while back
> > but Chris has some valid concerns about it, I don't remember the details
> > offhand. I'd much rather we focused on fixing that and getting that
> > merged than trying to add this functionality to every variable.
> 
> Chris, do you know the problems Richard's patch has? I can take a look
> and try to fix it so we can get this in for 1.5.

http://lists.openembedded.org/pipermail/bitbake-devel/2013-March/003319.html

We don't want to rush into adding type support to bitbake, that is
something which needs a lot of thought. It might be good enough to have
_remove assume space delimited for now and we can then have the type
flag alter the behaviour later.

So the thing that needs fixing is the naive str.replace() :)

(x.startswith(y) or x.endswith(y) or " " + y + " " in x) kind of thing.

Cheers,

Richard
Richard Purdie - Aug. 23, 2013, 3:31 p.m.
On Fri, 2013-08-23 at 12:11 -0300, Otavio Salvador wrote:
> On Fri, Aug 23, 2013 at 11:59 AM, Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> > On Fri, 2013-08-23 at 11:48 -0300, Otavio Salvador wrote:
> >> This allow the addition and removal of distro features easily. To add
> >> a feature, use:
> >>
> >>  EXTRA_DISTRO_FEATURES += "wayland"
> >>
> >> and to remove, use '~' prefix, as:
> >>
> >>  EXTRA_DISTRO_FEATURES += "~x11"
> >>
> >> This code has been mostly copied from Mentor Graphics public layer but
> >> changed the variable name for a more descriptive name. The original
> >> code can be seen at user_feature.bbclass at:
> >>
> >>  http://git.yoctoproject.org/cgit/cgit.cgi/meta-mentor
> >>
> >> Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
> >
> > I discussed this with Otavio. My personal view is that we should hold
> > off this right now. The removal issues he ran into with poky have been
> > addressed in other ways so there is less of an immediate issue.
> >
> > The better fix for this kind of problem is -=/=- or more likely _remove
> > override support in bitbake. I did propose a _remove patch a while back
> > but Chris has some valid concerns about it, I don't remember the details
> > offhand. I'd much rather we focused on fixing that and getting that
> > merged than trying to add this functionality to every variable.
> 
> Chris, do you know the problems Richard's patch has? I can take a look
> and try to fix it so we can get this in for 1.5.

One more thing, we need to grep the existing metadata and change any
existing _remove in variable names. I seem to remember finding one
somewhere.

Cheers,

Richard

Patch

diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
index 98b823e..f30f4db 100644
--- a/meta/classes/base.bbclass
+++ b/meta/classes/base.bbclass
@@ -288,6 +288,9 @@  def buildcfg_neededvars(d):
     if pesteruser:
         bb.fatal('The following variable(s) were not set: %s\nPlease set them directly, or choose a MACHINE or DISTRO that sets them.' % ', '.join(pesteruser))
 
+EXTRA_DISTRO_FEATURES ?= ""
+EXTRA_DISTRO_FEATURES[type] = "list"
+
 addhandler base_eventhandler
 base_eventhandler[eventmask] = "bb.event.ConfigParsed bb.event.BuildStarted"
 python base_eventhandler() {
@@ -297,6 +300,7 @@  python base_eventhandler() {
         preferred_ml_updates(e.data)
         oe.utils.features_backfill("DISTRO_FEATURES", e.data)
         oe.utils.features_backfill("MACHINE_FEATURES", e.data)
+        oe.utils.extra_distro_features(e.data)
 
     if isinstance(e, bb.event.BuildStarted):
         localdata = bb.data.createCopy(e.data)
diff --git a/meta/lib/oe/utils.py b/meta/lib/oe/utils.py
index 82987e8..0c9dde0 100644
--- a/meta/lib/oe/utils.py
+++ b/meta/lib/oe/utils.py
@@ -117,6 +117,30 @@  def features_backfill(var,d):
     if addfeatures:
         d.appendVar(var, " " + " ".join(addfeatures))
 
+def extra_distro_features(d):
+    import oe.data
+
+    l = d.createCopy()
+    l.finalize()
+
+    extra_features = oe.data.typed_value('EXTRA_DISTRO_FEATURES', l)
+    if not extra_features:
+        return
+
+    distro_features = l.getVar('DISTRO_FEATURES', True).split()
+    for feature in extra_features:
+        if feature.startswith('~'):
+            feature = feature[1:]
+            if feature in distro_features:
+                distro_features.remove(feature)
+        else:
+            if feature not in distro_features:
+                distro_features.append(feature)
+
+    # Avoid readding the removed features later
+    d.delVar("DISTRO_FEATURES")
+
+    d.setVar('DISTRO_FEATURES', ' '.join(distro_features))
 
 def packages_filter_out_system(d):
     """