Patchwork license.bbclass base.bbclass: support for 'or' operand in LICENSE and for SPDX license names

login
register
mail settings
Submitter Andrei Gherzan
Date Jan. 10, 2012, 3:17 p.m.
Message ID <1326208678-17244-1-git-send-email-andrei@gherzan.ro>
Download mbox | patch
Permalink /patch/18931/
State Accepted
Commit 28456593be0b7e15bb51595d547d7e5347cce24b
Headers show

Comments

Andrei Gherzan - Jan. 10, 2012, 3:17 p.m.
From: Andrei Gherzan <andrei.gherzan@windriver.com>

A new function was defined in license.bbclass in order to correctly exclude packages where OE-Style licence naming
is used. In this way licenses as GPL-3, GPLv3, GPLv3.0 etc will be excluded from a non-GPLv3 build. This function
takes into consideration if 'or' operand is used.
The function defined in license.bbclass is called in base.bbclass where packages are excluded based on
INCOMPATIBLE_LICENSE variable.

[YOCTO #1884]
[YOCTO #1844]

Signed-off-by: Andrei Gherzan <andrei at gherzan.ro>
---
 meta/classes/base.bbclass    |    3 +-
 meta/classes/license.bbclass |   45 ++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 46 insertions(+), 2 deletions(-)
Chris Larson - Jan. 10, 2012, 3:34 p.m.
On Tue, Jan 10, 2012 at 8:17 AM, Andrei Gherzan <andrei@gherzan.ro> wrote:
> From: Andrei Gherzan <andrei.gherzan@windriver.com>
>
> A new function was defined in license.bbclass in order to correctly exclude packages where OE-Style licence naming
> is used. In this way licenses as GPL-3, GPLv3, GPLv3.0 etc will be excluded from a non-GPLv3 build. This function
> takes into consideration if 'or' operand is used.
> The function defined in license.bbclass is called in base.bbclass where packages are excluded based on
> INCOMPATIBLE_LICENSE variable.
>
> [YOCTO #1884]
> [YOCTO #1844]
>
> Signed-off-by: Andrei Gherzan <andrei at gherzan.ro>


Out of curiosity, why are we using regex for this when we already have
license parsing in the oe.license module which can handle the OR case?
Andrei Gherzan - Jan. 10, 2012, 4:07 p.m.
On 01/10/2012 05:34 PM, Chris Larson wrote:
> On Tue, Jan 10, 2012 at 8:17 AM, Andrei Gherzan<andrei@gherzan.ro>  wrote:
>> From: Andrei Gherzan<andrei.gherzan@windriver.com>
>>
>> A new function was defined in license.bbclass in order to correctly exclude packages where OE-Style licence naming
>> is used. In this way licenses as GPL-3, GPLv3, GPLv3.0 etc will be excluded from a non-GPLv3 build. This function
>> takes into consideration if 'or' operand is used.
>> The function defined in license.bbclass is called in base.bbclass where packages are excluded based on
>> INCOMPATIBLE_LICENSE variable.
>>
>> [YOCTO #1884]
>> [YOCTO #1844]
>>
>> Signed-off-by: Andrei Gherzan<andrei at gherzan.ro>
>
> Out of curiosity, why are we using regex for this when we already have
> license parsing in the oe.license module which can handle the OR case?

I'm using regex only for excluding '+' characters from LICENSE. The rest 
of the work is done in oe.license.
If X is defined as INCOMPATIBLE_LICENSE, X+ should be exluded as well.
Chris Larson - Jan. 10, 2012, 4:10 p.m.
On Tue, Jan 10, 2012 at 9:07 AM, Andrei Gherzan <andrei@gherzan.ro> wrote:
> On 01/10/2012 05:34 PM, Chris Larson wrote:
>>
>> On Tue, Jan 10, 2012 at 8:17 AM, Andrei Gherzan<andrei@gherzan.ro>  wrote:
>>>
>>> From: Andrei Gherzan<andrei.gherzan@windriver.com>
>>>
>>> A new function was defined in license.bbclass in order to correctly
>>> exclude packages where OE-Style licence naming
>>> is used. In this way licenses as GPL-3, GPLv3, GPLv3.0 etc will be
>>> excluded from a non-GPLv3 build. This function
>>> takes into consideration if 'or' operand is used.
>>> The function defined in license.bbclass is called in base.bbclass where
>>> packages are excluded based on
>>> INCOMPATIBLE_LICENSE variable.
>>>
>>> [YOCTO #1884]
>>> [YOCTO #1844]
>>>
>>> Signed-off-by: Andrei Gherzan<andrei at gherzan.ro>
>>
>>
>> Out of curiosity, why are we using regex for this when we already have
>> license parsing in the oe.license module which can handle the OR case?
>
>
> I'm using regex only for excluding '+' characters from LICENSE. The rest of
> the work is done in oe.license.
> If X is defined as INCOMPATIBLE_LICENSE, X+ should be exluded as well.

Ah, right, apparently I'm blind :) We should really think about a
generic mechanism for handling + somehow. Hmm.
Andrei Gherzan - Jan. 10, 2012, 4:12 p.m.
On 01/10/2012 06:10 PM, Chris Larson wrote:
> On Tue, Jan 10, 2012 at 9:07 AM, Andrei Gherzan<andrei@gherzan.ro>  wrote:
>> On 01/10/2012 05:34 PM, Chris Larson wrote:
>>> On Tue, Jan 10, 2012 at 8:17 AM, Andrei Gherzan<andrei@gherzan.ro>    wrote:
>>>> From: Andrei Gherzan<andrei.gherzan@windriver.com>
>>>>
>>>> A new function was defined in license.bbclass in order to correctly
>>>> exclude packages where OE-Style licence naming
>>>> is used. In this way licenses as GPL-3, GPLv3, GPLv3.0 etc will be
>>>> excluded from a non-GPLv3 build. This function
>>>> takes into consideration if 'or' operand is used.
>>>> The function defined in license.bbclass is called in base.bbclass where
>>>> packages are excluded based on
>>>> INCOMPATIBLE_LICENSE variable.
>>>>
>>>> [YOCTO #1884]
>>>> [YOCTO #1844]
>>>>
>>>> Signed-off-by: Andrei Gherzan<andrei at gherzan.ro>
>>>
>>> Out of curiosity, why are we using regex for this when we already have
>>> license parsing in the oe.license module which can handle the OR case?
>>
>> I'm using regex only for excluding '+' characters from LICENSE. The rest of
>> the work is done in oe.license.
>> If X is defined as INCOMPATIBLE_LICENSE, X+ should be exluded as well.
> Ah, right, apparently I'm blind :) We should really think about a
> generic mechanism for handling + somehow. Hmm.
No worries. Anyway, you are right about the "+" mechanism but right now 
this is a fast and complete solution.

ag
Chris Larson - Jan. 10, 2012, 4:34 p.m.
On Tue, Jan 10, 2012 at 9:12 AM, Andrei Gherzan <andrei@gherzan.ro> wrote:
> No worries. Anyway, you are right about the "+" mechanism but right now this
> is a fast and complete solution.

Indeed, nice work on this.
Elizabeth Flanagan - Jan. 10, 2012, 4:55 p.m.
On Tue, Jan 10, 2012 at 8:34 AM, Chris Larson <clarson@kergoth.com> wrote:
> On Tue, Jan 10, 2012 at 9:12 AM, Andrei Gherzan <andrei@gherzan.ro> wrote:
>> No worries. Anyway, you are right about the "+" mechanism but right now this
>> is a fast and complete solution.
>
> Indeed, nice work on this.

Agreed! One note about "+". There is still a discussion about "or
greater" licensing within the SPDX community about how "or greater"
should be dealt with.

http://www.spdx.org/wiki/proposal-2010-11-16-1-or-later-version-licensing
https://bugs.linuxfoundation.org/show_bug.cgi?id=591

I'm partial to, as a stop gap, re-including L/GPL-x.0+ license files
into common-licenses until they hash it out and treating "+" as just
part of the license name. It's not a perfect solution but it should
solve most "or greater" use cases.

-b

> --
> Christopher Larson
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Chris Larson - Jan. 10, 2012, 5:05 p.m.
On Tue, Jan 10, 2012 at 9:55 AM, Flanagan, Elizabeth
<elizabeth.flanagan@intel.com> wrote:
> Agreed! One note about "+". There is still a discussion about "or
> greater" licensing within the SPDX community about how "or greater"
> should be dealt with.
>
> http://www.spdx.org/wiki/proposal-2010-11-16-1-or-later-version-licensing
> https://bugs.linuxfoundation.org/show_bug.cgi?id=591

Interesting.

> I'm partial to, as a stop gap, re-including L/GPL-x.0+ license files
> into common-licenses until they hash it out and treating "+" as just
> part of the license name. It's not a perfect solution but it should
> solve most "or greater" use cases.

I've been thinking about something like this.

LICENSE_EXPANSIONS += "GPL-2.0+:GPL-2.0,GPL-3.0"

Then process the license, turning GPL-2.0+ into an OR operation.
Replace with (GPL-2.0 | GPL-3.0).

Perhaps coupled with something like https://gist.github.com/1590028 to
express distributor preferences, we then get a sanitized, SDPX, flat
string with the licenses we're going to be using for this. Then we run
this against each binary package's LICENSE (if different than recipe)
and ensure our emitted binary packages match up with our license
choices.
Andrei Gherzan - Jan. 10, 2012, 6:13 p.m.
On Tue, Jan 10, 2012 at 18:34, Chris Larson <clarson@kergoth.com> wrote:

> On Tue, Jan 10, 2012 at 9:12 AM, Andrei Gherzan <andrei@gherzan.ro> wrote:
> > No worries. Anyway, you are right about the "+" mechanism but right now
> this
> > is a fast and complete solution.
>
> Indeed, nice work on this.
> --
> Christopher Larson
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>

On Tue, Jan 10, 2012 at 19:05, Chris Larson <clarson@kergoth.com> wrote:

> On Tue, Jan 10, 2012 at 9:55 AM, Flanagan, Elizabeth
> <elizabeth.flanagan@intel.com> wrote:
> > Agreed!
>

Thank you.



On Tue, Jan 10, 2012 at 19:05, Chris Larson <clarson@kergoth.com> wrote:

> On Tue, Jan 10, 2012 at 9:55 AM, Flanagan, Elizabeth
> <elizabeth.flanagan@intel.com> wrote:
> > Agreed! One note about "+". There is still a discussion about "or
> > greater" licensing within the SPDX community about how "or greater"
> > should be dealt with.
> >
> >
> http://www.spdx.org/wiki/proposal-2010-11-16-1-or-later-version-licensing
> > https://bugs.linuxfoundation.org/show_bug.cgi?id=591
>
> Interesting.
>
> > I'm partial to, as a stop gap, re-including L/GPL-x.0+ license files
> > into common-licenses until they hash it out and treating "+" as just
> > part of the license name. It's not a perfect solution but it should
> > solve most "or greater" use cases.
>
> I've been thinking about something like this.
>
> LICENSE_EXPANSIONS += "GPL-2.0+:GPL-2.0,GPL-3.0"
>
> Then process the license, turning GPL-2.0+ into an OR operation.
> Replace with (GPL-2.0 | GPL-3.0).
>
> Perhaps coupled with something like https://gist.github.com/1590028 to
> express distributor preferences, we then get a sanitized, SDPX, flat
> string with the licenses we're going to be using for this. Then we run
> this against each binary package's LICENSE (if different than recipe)
> and ensure our emitted binary packages match up with our license
> choices.
> --
> Christopher Larson
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>
>
Christopher's solution crossed my mind as well. Actually i tried something
like this but i didn't want to modify more from this patch and i didn't
really know how to define those expansions.
Another problem here that i faced and dropped the idea for, was the the
scenario where a developer wants to exclude only GPLv3+. This case needed a
another couple of "if"s and so on.

So my solution seemed to deal most of the situations even if it looks a
little... childish.
Chris Larson - Jan. 10, 2012, 6:54 p.m.
On Tue, Jan 10, 2012 at 11:13 AM, Andrei Gherzan <andrei@gherzan.ro> wrote:
> Christopher's solution crossed my mind as well. Actually i tried something
> like this but i didn't want to modify more from this patch and i didn't
> really know how to define those expansions.
> Another problem here that i faced and dropped the idea for, was the the
> scenario where a developer wants to exclude only GPLv3+. This case needed a
> another couple of "if"s and so on.
>
> So my solution seemed to deal most of the situations even if it looks a
> little... childish.

I wouldn't put it that way. It's just something we may want to improve
in the longer term to cover more cases.
Saul Wold - Jan. 12, 2012, 6:44 a.m.
On 01/10/2012 07:17 AM, Andrei Gherzan wrote:
> From: Andrei Gherzan<andrei.gherzan@windriver.com>
>
> A new function was defined in license.bbclass in order to correctly exclude packages where OE-Style licence naming
> is used. In this way licenses as GPL-3, GPLv3, GPLv3.0 etc will be excluded from a non-GPLv3 build. This function
> takes into consideration if 'or' operand is used.
> The function defined in license.bbclass is called in base.bbclass where packages are excluded based on
> INCOMPATIBLE_LICENSE variable.
>
> [YOCTO #1884]
> [YOCTO #1844]
>
> Signed-off-by: Andrei Gherzan<andrei at gherzan.ro>
> ---
>   meta/classes/base.bbclass    |    3 +-
>   meta/classes/license.bbclass |   45 ++++++++++++++++++++++++++++++++++++++++++
>   2 files changed, 46 insertions(+), 2 deletions(-)
>
> diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
> index e65a722..f0c358e 100644
> --- a/meta/classes/base.bbclass
> +++ b/meta/classes/base.bbclass
> @@ -398,9 +398,8 @@ python () {
>               dont_want_whitelist = (d.getVar('WHITELIST_%s' % dont_want_license, 1) or "").split()
>               if pn not in hosttools_whitelist and pn not in lgplv2_whitelist and pn not in dont_want_whitelist:
>
> -                import re
>                   this_license = d.getVar('LICENSE', 1)
> -                if this_license and re.search(dont_want_license, this_license):
> +                if incompatible_license(d,dont_want_license):
>                       bb.note("SKIPPING %s because it's %s" % (pn, this_license))
>                       raise bb.parse.SkipPackage("incompatible with license %s" % this_license)
>
> diff --git a/meta/classes/license.bbclass b/meta/classes/license.bbclass
> index d351b5a..4b98e29 100644
> --- a/meta/classes/license.bbclass
> +++ b/meta/classes/license.bbclass
> @@ -237,6 +237,51 @@ python do_populate_lic() {
>
>   }
>
> +def incompatible_license(d,dont_want_license):
> +    """
> +    This function checks if a package has only incompatible licenses. It also take into consideration 'or'
> +    operand.
> +    """
> +    import re
> +    import oe.license
> +    from fnmatch import fnmatchcase as fnmatch
> +
> +    dont_want_licenses = []
> +    dont_want_licenses.append(d.getVar('INCOMPATIBLE_LICENSE', 1))
> +    if d.getVarFlag('SPDXLICENSEMAP', dont_want_license):
> +	dont_want_licenses.append(d.getVarFlag('SPDXLICENSEMAP', dont_want_license))
> +
> +    def include_license(license):
> +	if any(fnmatch(license, pattern) for pattern in dont_want_licenses):
> +	    return False
> +	else:
> +	    spdx_license = d.getVarFlag('SPDXLICENSEMAP', license)
> +	    if spdx_license and any(fnmatch(spdx_license, pattern) for pattern in dont_want_licenses):
> +		return False
> +	    else:
> +		return True
> +
> +    def choose_licenses(a, b):
> +        if all(include_license(lic) for lic in a):
> +		return a
> +        else:
> +		return b
> +
> +    """
> +    If you want to exlude license named generically 'X', we surely want to exlude 'X+' as well.
> +    In consequence, we will exclude the '+' character from LICENSE in case INCOMPATIBLE_LICENSE
> +    is not a 'X+' license.
> +    """
> +    if not re.search(r'[+]',dont_want_license):
> +	licenses=oe.license.flattened_licenses(re.sub(r'[+]', '', d.getVar('LICENSE', True)), choose_licenses)
> +    else:
> +	licenses=oe.license.flattened_licenses(d.getVar('LICENSE', True), choose_licenses)
> +
> +    for onelicense in licenses:
> +	if not include_license(onelicense):
> +		return True
> +    return False
> +
>   SSTATETASKS += "do_populate_lic"
>   do_populate_lic[sstate-name] = "populate-lic"
>   do_populate_lic[sstate-inputdirs] = "${LICSSTATEDIR}"

Andrei, thanks for your persistence with this one, there are clearly 
some issues in this area that need work, this is a great start down that 
line.

Merged into OE-Core

Sau!

Patch

diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
index e65a722..f0c358e 100644
--- a/meta/classes/base.bbclass
+++ b/meta/classes/base.bbclass
@@ -398,9 +398,8 @@  python () {
             dont_want_whitelist = (d.getVar('WHITELIST_%s' % dont_want_license, 1) or "").split()
             if pn not in hosttools_whitelist and pn not in lgplv2_whitelist and pn not in dont_want_whitelist:
 
-                import re
                 this_license = d.getVar('LICENSE', 1)
-                if this_license and re.search(dont_want_license, this_license):
+                if incompatible_license(d,dont_want_license):
                     bb.note("SKIPPING %s because it's %s" % (pn, this_license))
                     raise bb.parse.SkipPackage("incompatible with license %s" % this_license)
 
diff --git a/meta/classes/license.bbclass b/meta/classes/license.bbclass
index d351b5a..4b98e29 100644
--- a/meta/classes/license.bbclass
+++ b/meta/classes/license.bbclass
@@ -237,6 +237,51 @@  python do_populate_lic() {
 
 }
 
+def incompatible_license(d,dont_want_license):
+    """
+    This function checks if a package has only incompatible licenses. It also take into consideration 'or'
+    operand.
+    """
+    import re
+    import oe.license
+    from fnmatch import fnmatchcase as fnmatch
+
+    dont_want_licenses = []
+    dont_want_licenses.append(d.getVar('INCOMPATIBLE_LICENSE', 1))
+    if d.getVarFlag('SPDXLICENSEMAP', dont_want_license):
+	dont_want_licenses.append(d.getVarFlag('SPDXLICENSEMAP', dont_want_license))
+
+    def include_license(license):
+	if any(fnmatch(license, pattern) for pattern in dont_want_licenses):
+	    return False
+	else:
+	    spdx_license = d.getVarFlag('SPDXLICENSEMAP', license)
+	    if spdx_license and any(fnmatch(spdx_license, pattern) for pattern in dont_want_licenses):
+		return False
+	    else:
+		return True
+
+    def choose_licenses(a, b):
+        if all(include_license(lic) for lic in a):
+		return a
+        else:
+		return b
+
+    """
+    If you want to exlude license named generically 'X', we surely want to exlude 'X+' as well.
+    In consequence, we will exclude the '+' character from LICENSE in case INCOMPATIBLE_LICENSE
+    is not a 'X+' license.
+    """
+    if not re.search(r'[+]',dont_want_license):
+	licenses=oe.license.flattened_licenses(re.sub(r'[+]', '', d.getVar('LICENSE', True)), choose_licenses)
+    else:
+	licenses=oe.license.flattened_licenses(d.getVar('LICENSE', True), choose_licenses)
+
+    for onelicense in licenses:
+	if not include_license(onelicense):
+		return True
+    return False
+
 SSTATETASKS += "do_populate_lic"
 do_populate_lic[sstate-name] = "populate-lic"
 do_populate_lic[sstate-inputdirs] = "${LICSSTATEDIR}"