[RFC,0/7] SPDX3 Proof-of-Concept

This patch-set adds a proof-of-concept implementation of the upcoming
SPDX3 standard to the SBOM generation of the Yocto Project/OpenEmbedded.

The current code delivers an equivalent of what is produced for SPDX2.2.
The standard has not been released yet, and there is some specification
work in progress still. Our questions and open points are available
in the README.SPDX3 file.

Also, this first RFC delivery will be followed by another one with
SPDX assembly and the Licensing profile.

Louis Rannou (5):
  create-spdx-3.0: copy 2.2 class
  oe/spdx: extend spdx.py objects
  oe/sbom: change the write_doc to prepare for spdx3
  create-spdx-3.0: SPDX3 objects as classes
  oe/sbom: search into json

Marta Rybczynska (1):
  README.SPDX3: add file

Samantha Jalabert (1):
  create-spdx-3.0: support for recipe spdx creation

 README.SPDX3                         |  42 ++
 meta/classes/create-spdx-3.0.bbclass | 878 +++++++++++++++++++++++++++
 meta/classes/create-spdx.bbclass     |   2 +-
 meta/lib/oe/sbom.py                  |  37 +-
 meta/lib/oe/spdx.py                  |  30 +-
 meta/lib/oe/spdx3.py                 | 385 ++++++++++++
 6 files changed, 1364 insertions(+), 10 deletions(-)
 create mode 100644 README.SPDX3
 create mode 100644 meta/classes/create-spdx-3.0.bbclass
 create mode 100644 meta/lib/oe/spdx3.py

Hello Marta,

On Thu, 26 Oct 2023 12:48:40 +0200
"Marta Rybczynska" <rybczynska@gmail.com> wrote:

> This patch-set adds a proof-of-concept implementation of the upcoming
> SPDX3 standard to the SBOM generation of the Yocto Project/OpenEmbedded.
> 
> The current code delivers an equivalent of what is produced for SPDX2.2.
> The standard has not been released yet, and there is some specification
> work in progress still. Our questions and open points are available
> in the README.SPDX3 file.
> 
> Also, this first RFC delivery will be followed by another one with
> SPDX assembly and the Licensing profile.

I ran this series on the autobuilders and it's triggering:

2023-10-26 22:30:42,532 - oe-selftest - INFO - 2: 27/33 214/548 (53.63s) (0 failed) (spdx.SPDXCheck.test_spdx_base_files)
2023-10-26 22:30:42,532 - oe-selftest - INFO - testtools.testresult.real._StringException: Traceback (most recent call last):
  File "/home/pokybuild/yocto-worker/oe-selftest-fedora/build/meta/lib/oeqa/selftest/cases/spdx.py", line 54, in test_spdx_base_files
    self.check_recipe_spdx("packages", "base-files.spdx.json", "base-files")
  File "/home/pokybuild/yocto-worker/oe-selftest-fedora/build/meta/lib/oeqa/selftest/cases/spdx.py", line 51, in check_recipe_spdx
    result = check_spdx_json(full_file_path)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pokybuild/yocto-worker/oe-selftest-fedora/build/meta/lib/oeqa/selftest/cases/spdx.py", line 44, in check_spdx_json
    self.assertNotEqual(report["SPDXID"], None)
                        ~~~~~~^^^^^^^^^^
KeyError: 'SPDXID'

https://autobuilder.yoctoproject.org/typhoon/#/builders/86/builds/5969/steps/14/logs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/5969/steps/14/logs/stdio

Luca

On Fri, 27 Oct 2023 09:37:28 +0200
"Luca Ceresoli via lists.openembedded.org"
<luca.ceresoli=bootlin.com@lists.openembedded.org> wrote:

> Hello Marta,
> 
> On Thu, 26 Oct 2023 12:48:40 +0200
> "Marta Rybczynska" <rybczynska@gmail.com> wrote:
> 
> > This patch-set adds a proof-of-concept implementation of the upcoming
> > SPDX3 standard to the SBOM generation of the Yocto Project/OpenEmbedded.
> > 
> > The current code delivers an equivalent of what is produced for SPDX2.2.
> > The standard has not been released yet, and there is some specification
> > work in progress still. Our questions and open points are available
> > in the README.SPDX3 file.
> > 
> > Also, this first RFC delivery will be followed by another one with
> > SPDX assembly and the Licensing profile.  
> 
> I ran this series on the autobuilders and it's triggering:
> 
> 2023-10-26 22:30:42,532 - oe-selftest - INFO - 2: 27/33 214/548 (53.63s) (0 failed) (spdx.SPDXCheck.test_spdx_base_files)
> 2023-10-26 22:30:42,532 - oe-selftest - INFO - testtools.testresult.real._StringException: Traceback (most recent call last):
>   File "/home/pokybuild/yocto-worker/oe-selftest-fedora/build/meta/lib/oeqa/selftest/cases/spdx.py", line 54, in test_spdx_base_files
>     self.check_recipe_spdx("packages", "base-files.spdx.json", "base-files")
>   File "/home/pokybuild/yocto-worker/oe-selftest-fedora/build/meta/lib/oeqa/selftest/cases/spdx.py", line 51, in check_recipe_spdx
>     result = check_spdx_json(full_file_path)
>              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>   File "/home/pokybuild/yocto-worker/oe-selftest-fedora/build/meta/lib/oeqa/selftest/cases/spdx.py", line 44, in check_spdx_json
>     self.assertNotEqual(report["SPDXID"], None)
>                         ~~~~~~^^^^^^^^^^
> KeyError: 'SPDXID'
> 
> https://autobuilder.yoctoproject.org/typhoon/#/builders/86/builds/5969/steps/14/logs/stdio
> https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/5969/steps/14/logs/stdio

And also, noticed after sending the previous e-mail:

ERROR: libtool-native-2.4.7-r0 do_create_spdx: Recipe libtool-native is trying to create package libltdl which was already written by recipe libtool. This will cause corruption, please resolve this and only provide the package from one recipe or the other or only build one of the recipes.
ERROR: quilt-native-0.67-r0 do_create_spdx: Recipe quilt-native is trying to create package guards which was already written by recipe quilt. This will cause corruption, please resolve this and only provide the package from one recipe or the other or only build one of the recipes.

https://autobuilder.yoctoproject.org/typhoon/#/builders/97/builds/7390/steps/16/logs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/5639/steps/12/logs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/96/builds/5527/steps/12/logs/stdio

Luca