| Message ID | 20240415144129.92641-1-ninette@thehoodiefirm.com |
|---|---|
| Headers | show |
| Series | Improvements for performance test report view | expand |
Hi Ninette, Firstly, thanks for these patches, it is great to be able to try and improve these and make the data more usable and interactive. To help others visualise the new output and test the patches I've made a test run on the autobuilder and the results are: http://autobuilder.yocto.io/pub/non-release/20240413-21/testresults/buildperf-alma8/perf-alma8_master-next_20240413135138_904bb385c3.html and http://autobuilder.yocto.io/pub/non-release/20240413-19/testresults/buildperf-debian11/perf-debian11_master-next_20240413064005_7116cd908d.html In general I love the wider commit range of the new chart and that you can interact with them, this is great. The is one big problem which is the change to "start_time" which is problematic. We need the commit numbers on the graph to be able to decode which change a given point in time represents. Is there a way we could show both the commit number and date of the build together? I think if we could fix that issue these would be good to merge from my perspective. I have shared links to the examples above in our weekly status report for others to review as well though. Cheers, Richard On Mon, 2024-04-15 at 16:41 +0200, Ninette Adhikari via lists.openembedded.org wrote: > This work is done according to "Milestone 9: Build performance test > report view" as stated in the Scope of Work with Sovereign Tech Fund > (STF) (https://www.sovereigntechfund.de/). > The current report can be accessed here: > Performance test report HTML > (https://autobuilder.yocto.io/pub/non-release/20240117-15/testresults > /buildperf-alma8/perf-alma8_master_20240117090048_663f180574.html) > The report is created using the `oe-build-perf-report` script in the > poky repository. This script generates a performance test report in > HTML format using the data from the yocto-buildstats > (https://git.yoctoproject.org/yocto-buildstats/) repository. > The report displays the performance test results in line chart > format. The chart x-axis represents the commit numbers, and the y- > axis represents the test duration in minutes. > The report also includes a table that displays the measurement > statistics data for each test. The report is interactive and allows > users to zoom in on specific sections of the line chart. > > The current report format required some updates to make it more > interactive and user-friendly. And this patch addresses such > improvements: > > - Add Apache echart (https://echarts.apache.org/en/index.html) > library to create oe build performance report charts and make them > interactive. > - Restructure data to time and value array format to be used by > echarts. It also converts test duration to minutes and adds zoom to > the line charts. > - Update measurement statistics data to include `start_time` so that > time can be displayed instead of commit numbers on the chart. It also > updates default commit history length to 300. > - Add styling updates including page margin, labels for x and y axis, > tooltip, and section descriptions. > > Updated report screenshots: > https://github.com/neighbourhoodie/poky/assets/13760198/65a1890c-fd2a-40d4-ac90-f13055735e53 > https://github.com/neighbourhoodie/poky/assets/13760198/1ed43876-73a9-487e-aed3-ca0edf97514c > > For local setup, you can do the following: > > 1. Clone the yocto-buildstats > (https://git.yoctoproject.org/yocto-buildstats/) and the poky > repository (https://git.yoctoproject.org/poky/) > > 2. In the poky repository run the following to build the report HTML: > ```bash > ./scripts/oe-build-perf-report -r "LOCAL_PATH_TO_YOCTO_BUILDSTATS" -- > branch "master" --commit "663f1805742ff6fb6955719d0ab7846a425debcf" - > -branch2 "master" --html > test.html > ``` > Note: > - Add your local path to the yocto-buildstats repo > - The above command builds the report in a file called `test.html`. > You can access it in the root directory in poky. > - This exmaple report uses the commit > `663f1805742ff6fb6955719d0ab7846a425debcf` from `master` branch.
On 2024-04-15 10:41 a.m., Ninette Adhikari via lists.openembedded.org wrote: > This work is done according to "Milestone 9: Build performance test report view" as stated in the Scope of Work with Sovereign Tech Fund (STF) (https://www.sovereigntechfund.de/). > The current report can be accessed here: > Performance test report HTML (https://autobuilder.yocto.io/pub/non-release/20240117-15/testresults/buildperf-alma8/perf-alma8_master_20240117090048_663f180574.html) > The report is created using the `oe-build-perf-report` script in the poky repository. This script generates a performance test report in HTML format using the data from the yocto-buildstats (https://git.yoctoproject.org/yocto-buildstats/) repository. > The report displays the performance test results in line chart format. The chart x-axis represents the commit numbers, and the y-axis represents the test duration in minutes. > The report also includes a table that displays the measurement statistics data for each test. The report is interactive and allows users to zoom in on specific sections of the line chart. > > The current report format required some updates to make it more interactive and user-friendly. And this patch addresses such improvements: > > - Add Apache echart (https://echarts.apache.org/en/index.html) library to create oe build performance report charts and make them interactive. > - Restructure data to time and value array format to be used by echarts. It also converts test duration to minutes and adds zoom to the line charts. > - Update measurement statistics data to include `start_time` so that time can be displayed instead of commit numbers on the chart. It also updates default commit history length to 300. > - Add styling updates including page margin, labels for x and y axis, tooltip, and section descriptions. > > Updated report screenshots: > https://github.com/neighbourhoodie/poky/assets/13760198/65a1890c-fd2a-40d4-ac90-f13055735e53 > https://github.com/neighbourhoodie/poky/assets/13760198/1ed43876-73a9-487e-aed3-ca0edf97514c Thanks for the nice, dynamic UI web graphs! Overall, I find them useful and well-designed. Would it be possible to present the data in a more discrete manner since the interpolated view presented now, while nice to look at, does not always reflect the underlying data trend? For example, if I have this data plotted as 1) stair-steps or 2) line segments (I didn't take the time to do a spline fit), I find that the line style for 1) is usually less misleading about what we really know about the data. 1) stair-step 2) straight lines: and I'd like to see this data: plotted in the stair-step style. From: http://autobuilder.yocto.io/pub/non-release/20240413-19/testresults/buildperf-debian11/perf-debian11_master-next_20240413064005_7116cd908d.html > For local setup, you can do the following: > > 1. Clone the yocto-buildstats (https://git.yoctoproject.org/yocto-buildstats/) and the poky repository (https://git.yoctoproject.org/poky/) > > 2. In the poky repository run the following to build the report HTML: > ```bash > ./scripts/oe-build-perf-report -r "LOCAL_PATH_TO_YOCTO_BUILDSTATS" --branch "master" --commit "663f1805742ff6fb6955719d0ab7846a425debcf" --branch2 "master" --html > test.html > ``` > Note: > - Add your local path to the yocto-buildstats repo > - The above command builds the report in a file called `test.html`. You can access it in the root directory in poky. > - This exmaple report uses the commit `663f1805742ff6fb6955719d0ab7846a425debcf` from `master` branch. I tried that out: ❯ cd .../distro/yocto/yocto-bs-html ❯ ../poky.git/scripts/oe-build-perf-report -r "/media/rmacleod/gitter/rmacleod/src/distro/yocto/yocto-bs-html/../yocto-buildstats.git" --branch "master" --commit "663f1805742ff6fb6955719d0ab7846a425debcf" --branch2 "master" --html > test.html ERROR: No buildstats found, please try running 'git fetch origin refs/notes/buildstats/perf-alma8/master/qemux86:refs/notes/buildstats/perf-alma8/master/qemux86' to fetch them from the remote so I did that. It was > 3 GB of data... !! Receiving objects: 89% (10568/11747), 2.03 GiB | 3.28 MiB/s6 MiB/s ❯ du -sh . 3.2G . Also, one shouldn't have to call bash just to run your script since you shell does that already if you have the right #!/path/to/foo and you have made the script executable: ❯ file ../poky.git/scripts/oe-build-perf-report ../poky.git/scripts/oe-build-perf-report: Python script, ASCII text executable ❯ head -1 ../poky.git/scripts/oe-build-perf-report #!/usr/bin/env python3 Anyway, that did generate a test.html file in ~ 10 seconds for me on my Pop!OS (Ubuntu-based) distro and it rendered nicely in Firefox 124.0.1 (64-bit) so that's nice to see. Thanks again, ../Randy > > > Ninette Adhikari (3): > oe-build-perf-report: Add apache echarts to make report interactive > oe-build-perf-report: Display more than 300 commits and date instead > of commit number > oe-build-perf-report: Improve report styling and add descriptions > > .../build_perf/html/measurement_chart.html | 116 +++++++++++------- > scripts/lib/build_perf/html/report.html | 96 ++++++++++----- > scripts/lib/build_perf/report.py | 4 +- > scripts/oe-build-perf-report | 6 +- > 4 files changed, 143 insertions(+), 79 deletions(-) > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#198239):https://lists.openembedded.org/g/openembedded-core/message/198239 > Mute This Topic:https://lists.openembedded.org/mt/105537213/3616765 > Group Owner:openembedded-core+owner@lists.openembedded.org > Unsubscribe:https://lists.openembedded.org/g/openembedded-core/unsub [randy.macleod@windriver.com] > -=-=-=-=-=-=-=-=-=-=-=- >
This work is done according to "Milestone 9: Build performance test report view" as stated in the Scope of Work with Sovereign Tech Fund (STF) (https://www.sovereigntechfund.de/). The current report can be accessed here: Performance test report HTML (https://autobuilder.yocto.io/pub/non-release/20240117-15/testresults/buildperf-alma8/perf-alma8_master_20240117090048_663f180574.html) The report is created using the `oe-build-perf-report` script in the poky repository. This script generates a performance test report in HTML format using the data from the yocto-buildstats (https://git.yoctoproject.org/yocto-buildstats/) repository. The report displays the performance test results in line chart format. The chart x-axis represents the commit numbers, and the y-axis represents the test duration in minutes. The report also includes a table that displays the measurement statistics data for each test. The report is interactive and allows users to zoom in on specific sections of the line chart. The current report format required some updates to make it more interactive and user-friendly. And this patch addresses such improvements: - Add Apache echart (https://echarts.apache.org/en/index.html) library to create oe build performance report charts and make them interactive. - Restructure data to time and value array format to be used by echarts. It also converts test duration to minutes and adds zoom to the line charts. - Update measurement statistics data to include `start_time` so that time can be displayed instead of commit numbers on the chart. It also updates default commit history length to 300. - Add styling updates including page margin, labels for x and y axis, tooltip, and section descriptions. Updated report screenshots: https://github.com/neighbourhoodie/poky/assets/13760198/65a1890c-fd2a-40d4-ac90-f13055735e53 https://github.com/neighbourhoodie/poky/assets/13760198/1ed43876-73a9-487e-aed3-ca0edf97514c For local setup, you can do the following: 1. Clone the yocto-buildstats (https://git.yoctoproject.org/yocto-buildstats/) and the poky repository (https://git.yoctoproject.org/poky/) 2. In the poky repository run the following to build the report HTML: ```bash ./scripts/oe-build-perf-report -r "LOCAL_PATH_TO_YOCTO_BUILDSTATS" --branch "master" --commit "663f1805742ff6fb6955719d0ab7846a425debcf" --branch2 "master" --html > test.html ``` Note: - Add your local path to the yocto-buildstats repo - The above command builds the report in a file called `test.html`. You can access it in the root directory in poky. - This exmaple report uses the commit `663f1805742ff6fb6955719d0ab7846a425debcf` from `master` branch. Ninette Adhikari (3): oe-build-perf-report: Add apache echarts to make report interactive oe-build-perf-report: Display more than 300 commits and date instead of commit number oe-build-perf-report: Improve report styling and add descriptions .../build_perf/html/measurement_chart.html | 116 +++++++++++------- scripts/lib/build_perf/html/report.html | 96 ++++++++++----- scripts/lib/build_perf/report.py | 4 +- scripts/oe-build-perf-report | 6 +- 4 files changed, 143 insertions(+), 79 deletions(-)