Patchwork ptest-runner: return 1 instead of zero if any test fails

login
register
mail settings
Submitter Laszlo Papp
Date Aug. 22, 2014, 2:05 p.m.
Message ID <1408716303-32264-1-git-send-email-root@pol-lx-007.polatis.com>
Download mbox | patch
Permalink /patch/78825/
State Accepted
Commit 73618768454a633c9966dff93611bf5693df0d99
Headers show

Comments

Laszlo Papp - Aug. 22, 2014, 2:05 p.m.
From: Laszlo Papp <lpapp@kde.org>

It is better to know this information from the exit code rather than parsing
test specific outputs.

Signed-off-by: Laszlo Papp <lpapp@kde.org>
---
 meta/recipes-support/ptest-runner/files/ptest-runner | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
Corneliu Stoicescu - Aug. 25, 2014, 6:42 a.m.
Hello,

From my experience with ptest, there will always be a few tests failing, so we will always get exit code 1. Should this better return exit code 0 if the individual tests performed correctly(weather they passed or failed) and 1 if there were any errors during the running of the tests?

Regards,
Corneliu
Yocto QA

> -----Original Message-----
> From: openembedded-core-bounces@lists.openembedded.org
> [mailto:openembedded-core-bounces@lists.openembedded.org] On Behalf
> Of root@myhostname
> Sent: Friday, August 22, 2014 5:05 PM
> To: openembedded-core@lists.openembedded.org
> Subject: [OE-core] [PATCH] ptest-runner: return 1 instead of zero if any test
> fails
> 
> From: Laszlo Papp <lpapp@kde.org>
> 
> It is better to know this information from the exit code rather than parsing
> test specific outputs.
> 
> Signed-off-by: Laszlo Papp <lpapp@kde.org>
> ---
>  meta/recipes-support/ptest-runner/files/ptest-runner | 8 ++++++--
>  1 file changed, 6 insertions(+), 2 deletions(-)
> 
> diff --git a/meta/recipes-support/ptest-runner/files/ptest-runner
> b/meta/recipes-support/ptest-runner/files/ptest-runner
> index 3342413..c618f11 100644
> --- a/meta/recipes-support/ptest-runner/files/ptest-runner
> +++ b/meta/recipes-support/ptest-runner/files/ptest-runner
> @@ -1,5 +1,5 @@
>  #!/bin/sh
> -
> +ANYFAILED=no
>  echo "START: $0"
> 
>  for libdir in /usr/lib*
> @@ -15,9 +15,13 @@ do
>          date "+%Y-%m-%dT%H:%M"
>          echo "BEGIN: $x"
>          cd "$x"
> -        ./run-ptest
> +        ./run-ptest || ANYFAILED=yes
>          echo "END: $x"
>          date "+%Y-%m-%dT%H:%M"
>      done
>  done
>  echo "STOP: $0"
> +if [ "$ANYFAILED" = "yes"  ]; then
> +    exit 1
> +fi
> +exit 0
> --
> 2.0.4
> 
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
Laszlo Papp - Aug. 29, 2014, 10:54 a.m.
On Mon, Aug 25, 2014 at 7:42 AM, Stoicescu, CorneliuX
<corneliux.stoicescu@intel.com> wrote:
> Hello,
>
> From my experience with ptest, there will always be a few tests failing, so we will always get exit code 1.

Ptest is not only about meta-core, you need to remember that. My use
case is actually quite different: running ptest only on my binaries,
and I will trust the rest of the system. For me, it is critical to see
if there is any issues in my system. This patch resolves that use
case, but even for Yocto, you really oughta aim for fixing your tests,
either with "known failure" or correct means.

> Should this better return exit code 0 if the individual tests performed correctly(weather they passed or failed) and 1 if there were any errors during the running of the tests?

I do not understand this question. This is what the current change tries to do.

Patch

diff --git a/meta/recipes-support/ptest-runner/files/ptest-runner b/meta/recipes-support/ptest-runner/files/ptest-runner
index 3342413..c618f11 100644
--- a/meta/recipes-support/ptest-runner/files/ptest-runner
+++ b/meta/recipes-support/ptest-runner/files/ptest-runner
@@ -1,5 +1,5 @@ 
 #!/bin/sh
-
+ANYFAILED=no
 echo "START: $0"
 
 for libdir in /usr/lib*
@@ -15,9 +15,13 @@  do
         date "+%Y-%m-%dT%H:%M"
         echo "BEGIN: $x"
         cd "$x"
-        ./run-ptest
+        ./run-ptest || ANYFAILED=yes
         echo "END: $x"
         date "+%Y-%m-%dT%H:%M"
     done
 done
 echo "STOP: $0"
+if [ "$ANYFAILED" = "yes"  ]; then
+    exit 1
+fi
+exit 0