mbox series

[v2,0/2] testimage: enable artifacts retrieval for failed tests

Message ID 20240203092538.5491-1-alexis.lothore@bootlin.com
Headers show
Series testimage: enable artifacts retrieval for failed tests | expand

Message

Alexis Lothoré Feb. 3, 2024, 9:25 a.m. UTC
Hello,
this small series is a new version of failed ptests artifacts retrieval
series.
The first version can be found in [1].

This new version moves default TESTIMAGE_FAILED_QA_ARTIFACTS
in core-image class to make sure all CI images get this default value. With
this setup:
- "standard" images running tests leads to default artifact list to be
  stored
- if the image has ptest feature enabled, failed ptests directories will be
  retrieved
- artifacts lists can be customized: TESTIMAGE_FAILED_QA_ARTIFACTS:append =
  " <file1> <file2>..."

The feature has been tested with core-image-sato and core-image-ptest-fast
(running only a subtests of configured tests)

[1] https://lore.kernel.org/openembedded-core/20240124142930.1038759-1-alexis.lothore@bootlin.com/
[2] https://lore.kernel.org/openembedded-core/20230609064802.11777-5-alexis.lothore@bootlin.com/

Alexis Lothoré (2):
  testimage: move TESTIMAGE_FAILED_QA_ARTIFACTS default value to
    core-image-minimal
  core-image-ptest: retrieve ptests directory when ptests fail

 meta/classes-recipe/core-image.bbclass       | 8 ++++++++
 meta/classes-recipe/testimage.bbclass        | 9 ---------
 meta/recipes-core/images/core-image-ptest.bb | 2 ++
 3 files changed, 10 insertions(+), 9 deletions(-)

Comments

Alexis Lothoré Feb. 3, 2024, 9:43 a.m. UTC | #1
On 2/3/24 10:25, Alexis Lothoré via lists.openembedded.org wrote:
> From: Alexis Lothoré <alexis.lothore@bootlin.com>
> 
> TESTIMAGE_FAILED_QA_ARTIFACTS currently sets a default list of files to be
> saved whenever some tests fail. Unfortunately, this default value is very
> easily discarded, because TESTIMAGE_FAILED_QA_ARTIFACTS is expected to be
> enriched by some core recipes (example: ptest images) which are often
> parsed before testimage.bbclass. Those core recipes could still manage to
> get the default value AND add some files by using override syntax, but then
> it makes it difficult for downstream recipes or bbappend files to tune this
> variable.
> 
> Allow to set this default value and make sure it is not overwritten by
> recipes wanting to tune it, by setting the default value in
> core-image.
> While doing so, set it as a default value instead of a weak default value.

And I realize that I forgot to remove "minimal" from the commit message, which
is not true anymore for this version. I'll wait a bit before resending, it looks
like I have some formatting issues detected by patchtest too