diff mbox series

[2/2] patchtest-send-results: properly parse test status

Message ID 20240205155730.115895-3-alexis.lothore@bootlin.com
State Accepted, archived
Commit 3567c21af8ed65448f9325ee3fe85b8be839e1b5
Headers show
Series patchtest-send-results: prevent false positives due to commit subject | expand

Commit Message

Alexis Lothoré Feb. 5, 2024, 3:57 p.m. UTC 
From: Alexis Lothoré <alexis.lothore@bootlin.com>

patchtest-send-results currently search the word "FAIL" in the whole
testresults file to decide whether it should send a report to patch
submitter or not. This global search can lead to false positives, for
example if the commit subject contains the word "FAIL" (as observed in
[1])

Prevent those false positives by explicitely parsing the test status from
each line. Each test result line, generated by the patchtest script, is
expected to have the following format:
<STATUS>: <some info, depending on the status>

[1] https://lore.kernel.org/openembedded-core/0101018d79bfe020-06f2ce89-ea19-456b-92e7-66ee1c710fd1-000000@us-west-2.amazonses.com/

Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
---
 scripts/patchtest-send-results | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
diff mbox series

Patch

diff --git a/scripts/patchtest-send-results b/scripts/patchtest-send-results
index 024be003ceb7..075b60d8f7d5 100755
--- a/scripts/patchtest-send-results
+++ b/scripts/patchtest-send-results
@@ -33,6 +33,9 @@  under 'Yocto Project Subprojects'). For more information on specific
 failures, see: https://wiki.yoctoproject.org/wiki/Patchtest. Thank
 you!"""
 
+def has_a_failed_test(raw_results):
+    return any(raw_result.split(':')[0] == "FAIL" for raw_result in raw_results.splitlines())
+
 parser = argparse.ArgumentParser(description="Send patchtest results to a submitter for a given patch")
 parser.add_argument("-p", "--patch", dest="patch", required=True, help="The patch file to summarize")
 args = parser.parse_args()
@@ -69,7 +72,7 @@  from_address = "patchtest@automation.yoctoproject.org"
 # mailing list to CC
 cc_address = "openembedded-core@lists.openembedded.org"
 
-if "FAIL" in testresult:
+if has_a_failed_test(testresult):
     reply_contents = None
     if len(max(open(result_file, 'r'), key=len)) > 220:
         warning = "Tests failed for the patch, but the results log could not be processed due to excessive result line length."