diff mbox series

[v2,3/3] buildstats.bbclass: correct sampling of system stats

Message ID 20220621161413.2779759-3-aryaman.gupta@windriver.com
State Accepted, archived
Commit 0e2df45ab066bb4ad2c4f8622ee9c1a8ecdea9cb
Headers show
Series [v2,1/3] buildstats.py: enable collection of /proc/pressure data | expand

Commit Message

Aryaman Gupta June 21, 2022, 4:14 p.m. UTC 
The last time of sampling would be updated within the SystemStats class
but not re-recorded into the datastore, leading to multiple samples being
collected in the same second in the sample function of buildstats.py due
to the runQueueTaskStarted events. Fix this to collect and store only one
sample per second, as originally intended, by only recording samples
on instances of the Heartbeat and BuildCompleted events.

This fix elimates the spikiness of sampled data, in cases where the difference
between the current and the last sample is taken. Previously, since many
samples per second were recorded, certain types of data would result in a
very small elapsed time and hence a small numerical difference. For example,
the CPU usage from /proc/stat is a running total of usage and taking the
difference between data collected 0.1 seconds apart would result in usage
appearing lower than it actually was.

Signed-off-by: Aryaman Gupta <aryaman.gupta@windriver.com>
Signed-off-by: Randy MacLeod <randy.macleod@windriver.com>
---
 meta/classes/buildstats.bbclass | 3 ++-
 meta/lib/buildstats.py          | 7 ++++++-
 2 files changed, 8 insertions(+), 2 deletions(-)
diff mbox series

Patch

diff --git a/meta/classes/buildstats.bbclass b/meta/classes/buildstats.bbclass
index 0de605200a..9e25a42a5c 100644
--- a/meta/classes/buildstats.bbclass
+++ b/meta/classes/buildstats.bbclass
@@ -282,10 +282,11 @@  python runqueue_stats () {
     if not system_stats and isinstance(e, (bb.runqueue.sceneQueueTaskStarted, bb.runqueue.runQueueTaskStarted)):
         system_stats = buildstats.SystemStats(d)
         d.setVar('_buildstats_system_stats', system_stats)
-    if system_stats:
+    if system_stats and isinstance(e, (bb.event.HeartbeatEvent, bb.event.BuildCompleted)):
         # Ensure that we sample at important events.
         done = isinstance(e, bb.event.BuildCompleted)
         system_stats.sample(e, force=done)
+        d.setVar('_buildstats_system_stats', system_stats)
         if done:
             system_stats.close()
             d.delVar('_buildstats_system_stats')
diff --git a/meta/lib/buildstats.py b/meta/lib/buildstats.py
index 64ad3ef40e..9829a0ff65 100644
--- a/meta/lib/buildstats.py
+++ b/meta/lib/buildstats.py
@@ -55,7 +55,12 @@  class SystemStats:
         # becames relevant when we get called very often while many
         # short tasks get started. Sampling during quiet periods
         # depends on the heartbeat event, which fires less often.
-        self.min_seconds = 1
+        # The Heartbeat events occur roughly once every second but the actual time
+        # between these events deviates by a few milliseconds, in most cases. Hence
+        # pick a somewhat arbitary threshold such that we sample a large majority
+        # of the Heartbeat events. This ignores rare events that fall outside the minimum
+        # but allows for fairly consistent intervals between samples.
+        self.min_seconds = 0.990
 
         self.meminfo_regex = re.compile(rb'^(MemTotal|MemFree|Buffers|Cached|SwapTotal|SwapFree):\s*(\d+)')
         self.diskstats_regex = re.compile(rb'^([hsv]d.|mtdblock\d|mmcblk\d|cciss/c\d+d\d+.*)$')