diff mbox series

gstreamer1.0: skip a test that is known to be flaky

Message ID 20240228133104.608403-1-ross.burton@arm.com
State Accepted, archived
Commit c2c9cbc107e5428122ad26b5c478602f0c8c0fbe
Headers show
Series gstreamer1.0: skip a test that is known to be flaky | expand

Commit Message

Ross Burton Feb. 28, 2024, 1:31 p.m. UTC
From: Ross Burton <ross.burton@arm.com>

The aggregator testcase test_infinite_seek_50_src_live is known upstream
to be flaky[1] and when this fails in their CI they just ignore it.

It's failing often on our autobuilder, so disable the test case for now
until upstream have resolved this issue.

[ YOCTO #15054 ]

[1] https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/410

Signed-off-by: Ross Burton <ross.burton@arm.com>
---
 .../gstreamer1.0/skip-aggregator-test.patch   | 35 +++++++++++++++++++
 .../gstreamer/gstreamer1.0_1.22.10.bb         |  1 +
 2 files changed, 36 insertions(+)
 create mode 100644 meta/recipes-multimedia/gstreamer/gstreamer1.0/skip-aggregator-test.patch

Comments

patchtest@automation.yoctoproject.org Feb. 28, 2024, 1:48 p.m. UTC | #1
Thank you for your submission. Patchtest identified one
or more issues with the patch. Please see the log below for
more information:

---
Testing patch /home/patchtest/share/mboxes/gstreamer1.0-skip-a-test-that-is-known-to-be-flaky.patch

FAIL: test Upstream-Status presence: Upstream-Status is Inappropriate, but no reason was provided (test_patch.TestPatch.test_upstream_status_presence_format)

PASS: pretest src uri left files (test_metadata.TestMetadata.pretest_src_uri_left_files)
PASS: test CVE check ignore (test_metadata.TestMetadata.test_cve_check_ignore)
PASS: test CVE tag format (test_patch.TestPatch.test_cve_tag_format)
PASS: test Signed-off-by presence (test_mbox.TestMbox.test_signed_off_by_presence)
PASS: test Signed-off-by presence (test_patch.TestPatch.test_signed_off_by_presence)
PASS: test author valid (test_mbox.TestMbox.test_author_valid)
PASS: test bugzilla entry format (test_mbox.TestMbox.test_bugzilla_entry_format)
PASS: test commit message presence (test_mbox.TestMbox.test_commit_message_presence)
PASS: test lic files chksum modified not mentioned (test_metadata.TestMetadata.test_lic_files_chksum_modified_not_mentioned)
PASS: test max line length (test_metadata.TestMetadata.test_max_line_length)
PASS: test mbox format (test_mbox.TestMbox.test_mbox_format)
PASS: test non-AUH upgrade (test_mbox.TestMbox.test_non_auh_upgrade)
PASS: test shortlog format (test_mbox.TestMbox.test_shortlog_format)
PASS: test shortlog length (test_mbox.TestMbox.test_shortlog_length)
PASS: test src uri left files (test_metadata.TestMetadata.test_src_uri_left_files)

SKIP: pretest pylint: No python related patches, skipping test (test_python_pylint.PyLint.pretest_pylint)
SKIP: test lic files chksum presence: No added recipes, skipping test (test_metadata.TestMetadata.test_lic_files_chksum_presence)
SKIP: test license presence: No added recipes, skipping test (test_metadata.TestMetadata.test_license_presence)
SKIP: test pylint: No python related patches, skipping test (test_python_pylint.PyLint.test_pylint)
SKIP: test series merge on head: Merge test is disabled for now (test_mbox.TestMbox.test_series_merge_on_head)
SKIP: test summary presence: No added recipes, skipping test (test_metadata.TestMetadata.test_summary_presence)
SKIP: test target mailing list: Series merged, no reason to check other mailing lists (test_mbox.TestMbox.test_target_mailing_list)

---

Please address the issues identified and
submit a new revision of the patch, or alternatively, reply to this
email with an explanation of why the patch should be accepted. If you
believe these results are due to an error in patchtest, please submit a
bug at https://bugzilla.yoctoproject.org/ (use the 'Patchtest' category
under 'Yocto Project Subprojects'). For more information on specific
failures, see: https://wiki.yoctoproject.org/wiki/Patchtest. Thank
you!
diff mbox series

Patch

diff --git a/meta/recipes-multimedia/gstreamer/gstreamer1.0/skip-aggregator-test.patch b/meta/recipes-multimedia/gstreamer/gstreamer1.0/skip-aggregator-test.patch
new file mode 100644
index 00000000000..81337512fdd
--- /dev/null
+++ b/meta/recipes-multimedia/gstreamer/gstreamer1.0/skip-aggregator-test.patch
@@ -0,0 +1,35 @@ 
+From 9b72aa7cdbc2a81cffc6f855933afe90c81046d5 Mon Sep 17 00:00:00 2001
+From: Ross Burton <ross.burton@arm.com>
+Date: Wed, 28 Feb 2024 12:40:34 +0000
+Subject: [PATCH] Skip aggregator test
+
+This test case is known to be flaky upstream[1] and often fails on the
+autobuilder[2], so skip it until this has been resolved upstream.
+
+[1] https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/410
+[2] https://bugzilla.yoctoproject.org/show_bug.cgi?id=15054
+
+Upstream-Status: Inappropriate
+Signed-off-by: Ross Burton <ross.burton@arm.com>
+---
+ tests/check/libs/aggregator.c | 4 +++-
+ 1 file changed, 3 insertions(+), 1 deletion(-)
+
+diff --git a/tests/check/libs/aggregator.c b/tests/check/libs/aggregator.c
+index 1f2c5b4..27b3ac7 100644
+--- a/tests/check/libs/aggregator.c
++++ b/tests/check/libs/aggregator.c
+@@ -1475,7 +1475,9 @@ gst_aggregator_suite (void)
+   tcase_add_test (general, test_flushing_seek);
+   tcase_add_test (general, test_infinite_seek);
+   tcase_add_test (general, test_infinite_seek_50_src);
+-  tcase_add_test (general, test_infinite_seek_50_src_live);
++  // This test case is known to be flaky, remove it until resolved:
++  // https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/410
++  // tcase_add_test (general, test_infinite_seek_50_src_live);
+   tcase_add_test (general, test_linear_pipeline);
+   tcase_add_test (general, test_two_src_pipeline);
+   tcase_add_test (general, test_timeout_pipeline);
+-- 
+2.34.1
+
diff --git a/meta/recipes-multimedia/gstreamer/gstreamer1.0_1.22.10.bb b/meta/recipes-multimedia/gstreamer/gstreamer1.0_1.22.10.bb
index 0c944e9e110..c16f0de3b72 100644
--- a/meta/recipes-multimedia/gstreamer/gstreamer1.0_1.22.10.bb
+++ b/meta/recipes-multimedia/gstreamer/gstreamer1.0_1.22.10.bb
@@ -21,6 +21,7 @@  SRC_URI = "https://gstreamer.freedesktop.org/src/gstreamer/gstreamer-${PV}.tar.x
            file://0002-tests-add-support-for-install-the-tests.patch \
            file://0003-tests-use-a-dictionaries-for-environment.patch \
            file://0004-tests-add-helper-script-to-run-the-installed_tests.patch \
+           file://skip-aggregator-test.patch \
            "
 SRC_URI[sha256sum] = "969aaef396f252ce925132a4be2aa004e0320f5c1baf0acaaae09c544a6759f4"