diff mbox series

python3: add additional timing-related test skips

Message ID 20230803174724.1107094-1-tgamblin@baylibre.com
State Accepted, archived
Commit 4da3b653e0fad52e7af97328b49e40d5179ddfe5
Headers show
Series python3: add additional timing-related test skips | expand

Commit Message

Trevor Gamblin Aug. 3, 2023, 5:47 p.m. UTC 
The following tests have been seen failing intermittently on the
autobuilder:

test.test_multiprocessing_forkserver.WithManagerTestBarrier.test_timeout
test.test_time.TimeTestCase.test_thread_time

Since these could be affected by AB systems under heavy load, disable
them. New test outputs:

test_timeout (test.test_multiprocessing_fork.WithProcessesTestBarrier.test_timeout)
SKIP: Test wait(timeout) 'timing related test, dependent on load'
...
SKIP: test_thread_time (test.test_time.TimeTestCase.test_thread_time) 'timing related test, dependent on load'

[YOCTO # 15131]
[YOCTO # 15177]

Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
---
 ...sts-due-to-load-variability-on-YP-AB.patch | 43 +++++++++++++++----
 1 file changed, 34 insertions(+), 9 deletions(-)

Comments

Alexander Kanavin Aug. 3, 2023, 5:57 p.m. UTC | #1 
On Thu, 3 Aug 2023 at 19:47, Trevor Gamblin <tgamblin@baylibre.com> wrote:
>
> The following tests have been seen failing intermittently on the
> autobuilder:
>
> test.test_multiprocessing_forkserver.WithManagerTestBarrier.test_timeout
> test.test_time.TimeTestCase.test_thread_time

But have they, or have we seen them fail once? Both tickets have only
one reference each to a failure. I'm not sure we need to disable the
tests this quickly.

https://bugzilla.yoctoproject.org/show_bug.cgi?id=15177
https://bugzilla.yoctoproject.org/show_bug.cgi?id=15131

Alex
Alexander Kanavin Aug. 3, 2023, 5:58 p.m. UTC | #2 
On Thu, 3 Aug 2023 at 19:57, Alexander Kanavin <alex.kanavin@gmail.com> wrote:
> But have they, or have we seen them fail once? Both tickets have only
> one reference each to a failure. I'm not sure we need to disable the
> tests this quickly.
>
> https://bugzilla.yoctoproject.org/show_bug.cgi?id=15177
> https://bugzilla.yoctoproject.org/show_bug.cgi?id=15131

Also, disabling tests still needs upstream tickets to be filed.

Alex
Trevor Gamblin Aug. 3, 2023, 6:33 p.m. UTC | #3 
On 2023-08-03 13:58, Alexander Kanavin wrote:
> On Thu, 3 Aug 2023 at 19:57, Alexander Kanavin <alex.kanavin@gmail.com> wrote:
>> But have they, or have we seen them fail once? Both tickets have only
>> one reference each to a failure. I'm not sure we need to disable the
>> tests this quickly.
>>
>> https://bugzilla.yoctoproject.org/show_bug.cgi?id=15177
>> https://bugzilla.yoctoproject.org/show_bug.cgi?id=15131
> Also, disabling tests still needs upstream tickets to be filed.
>
> Alex

We had discussed disabling both of these tests during the YP Bug Triage 
as they're similar to some other timing/load-dependent tests we've 
disabled before.

I'll keep an eye on the issues and resubmit if/when the failures have 
happened a few more times.
diff mbox series

Patch

diff --git a/meta/recipes-devtools/python/python3/0001-Skip-failing-tests-due-to-load-variability-on-YP-AB.patch b/meta/recipes-devtools/python/python3/0001-Skip-failing-tests-due-to-load-variability-on-YP-AB.patch
index 61dd4a5bc0..a7d37266ef 100644
--- a/meta/recipes-devtools/python/python3/0001-Skip-failing-tests-due-to-load-variability-on-YP-AB.patch
+++ b/meta/recipes-devtools/python/python3/0001-Skip-failing-tests-due-to-load-variability-on-YP-AB.patch
@@ -1,4 +1,4 @@ 
-From 01e02fb4720ecbbc44c694ee1b7fb6d5d95b5fe2 Mon Sep 17 00:00:00 2001
+From 0c8002d12149fdfbbcd9a9215876d474f157eb9c Mon Sep 17 00:00:00 2001
 From: Yi Fan Yu <yifan.yu@windriver.com>
 Date: Thu, 1 Apr 2021 13:08:37 -0700
 Subject: [PATCH] Skip failing tests due to load variability on YP AB
@@ -11,16 +11,22 @@  Upstream-Status: Inappropriate [OE-Specific]
 
 Signed-off-by: Yi Fan Yu <yifan.yu@windriver.com>
 
+Skip two additional tests due to suspected load variability failures.
+
+[YOCTO #15131]
+[YOCTO #15177]
+
+Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
 ---
- Lib/test/_test_multiprocessing.py | 2 ++
- Lib/test/test_time.py             | 1 +
- 2 files changed, 3 insertions(+)
+ Lib/test/_test_multiprocessing.py | 3 +++
+ Lib/test/test_time.py             | 2 ++
+ 2 files changed, 5 insertions(+)
 
 diff --git a/Lib/test/_test_multiprocessing.py b/Lib/test/_test_multiprocessing.py
-index 599c3f2..23328be 100644
+index b50a154320..b8b2c4204d 100644
 --- a/Lib/test/_test_multiprocessing.py
 +++ b/Lib/test/_test_multiprocessing.py
-@@ -618,6 +618,7 @@ class _TestProcess(BaseTestCase):
+@@ -618,6 +618,7 @@ def test_close(self):
  
          close_queue(q)
  
@@ -28,7 +34,15 @@  index 599c3f2..23328be 100644
      def test_many_processes(self):
          if self.TYPE == 'threads':
              self.skipTest('test not appropriate for {}'.format(self.TYPE))
-@@ -4890,6 +4891,7 @@ class TestWait(unittest.TestCase):
+@@ -2003,6 +2004,7 @@ def _test_timeout_f(cls, barrier, results):
+         except threading.BrokenBarrierError:
+             results.append(True)
+ 
++    @unittest.skip('timing related test, dependent on load')
+     def test_timeout(self):
+         """
+         Test wait(timeout)
+@@ -4890,6 +4892,7 @@ def signal_and_sleep(cls, sem, period):
          sem.release()
          time.sleep(period)
  
@@ -37,10 +51,10 @@  index 599c3f2..23328be 100644
          from multiprocessing.connection import wait
  
 diff --git a/Lib/test/test_time.py b/Lib/test/test_time.py
-index 884b142..542e980 100644
+index 884b14231f..8f0638c0d7 100644
 --- a/Lib/test/test_time.py
 +++ b/Lib/test/test_time.py
-@@ -492,6 +492,7 @@ class TimeTestCase(unittest.TestCase):
+@@ -492,6 +492,7 @@ def test_perf_counter(self):
      @unittest.skipIf(
          support.is_wasi, "process_time not available on WASI"
      )
@@ -48,3 +62,14 @@  index 884b142..542e980 100644
      def test_process_time(self):
          # process_time() should not include time spend during a sleep
          start = time.process_time()
+@@ -505,6 +506,7 @@ def test_process_time(self):
+         self.assertTrue(info.monotonic)
+         self.assertFalse(info.adjustable)
+ 
++    @unittest.skip('timing related test, dependent on load')
+     def test_thread_time(self):
+         if not hasattr(time, 'thread_time'):
+             if sys.platform.startswith(('linux', 'win')):
+-- 
+2.41.0
+