bitbakery: automate build testing across all MACHINEs/LIBCs

Submitted by Trevor Gamblin on Jan. 20, 2020, 1:27 p.m. | Patch ID: 169224

Details

Message ID 20200120132743.2178420-1-trevor.gamblin@windriver.com
State Superseded
Headers show

Commit Message

Trevor Gamblin Jan. 20, 2020, 1:27 p.m.
bitbakery simplifies the process of build testing an upgraded or
patched recipe. It cycles through all qemu targets for glibc and musl,
and then generates a log file in the format "<recipe>-bitbakery.log",
containing the results of each build along with some basic system info
(time, hostname, OS, kernel version). Additionally, it retains the
previous run's log for comparison, renamed to <recipe>-bitbakery.log.old
when a new set of builds is triggered.

Sample log output:

BITBAKERY LOG FOR aspell
START TIME: 2020-01-17_08:39:05
HOSTNAME: yow-tgamblin-fedora1
OS: Fedora 31 (Server Edition)
KERNEL: 5.4.7-200.fc31.x86_64

Patch hide | download patch | download mbox

===============
BUILD RESULTS:
[glibc]
FAIL: qemuarm64
PASS: qemuarm
PASS: qemuarmv5
PASS: qemumips64
PASS: qemumips
PASS: qemuppc
PASS: qemuriscv64
FAIL: qemux86-64
FAIL: qemux86
[musl]
PASS: qemuarm64
PASS: qemuarm
PASS: qemuarmv5
PASS: qemumips64
PASS: qemumips
PASS: qemuppc
PASS: qemuriscv64
PASS: qemux86-64
PASS: qemux86
===============
PASSED: 15
FAILED: 3
---
 scripts/bitbakery | 47 +++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 47 insertions(+)
 create mode 100755 scripts/bitbakery

diff --git a/scripts/bitbakery b/scripts/bitbakery
new file mode 100755
index 0000000000..cb2957a7bd
--- /dev/null
+++ b/scripts/bitbakery
@@ -0,0 +1,47 @@ 
+#!/bin/sh
+#  Copyright (c) 2020 Wind River Systems, Inc.
+#
+# SPDX-License-Identifier: GPL-2.0-only
+#
+
+# Get path to oe-core directory
+OE_CORE_PATH=$( cd - || exit )
+
+# Get target list and host machine information
+TARGET_LIST=$(ls "$OE_CORE_PATH"/meta/conf/machine | grep qemu | sed -e 's/\.conf//')
+LIBC_LIST="glibc musl"
+START_TIME=$(date "+%Y-%m-%d_%H:%M:%S")
+LOG_FILE="$1-bitbakery.log"
+OS_INFO=$(cat /etc/os-release | grep "PRETTY_NAME=" | awk -F "=" '{print $2}' | sed -e 's/^"//' -e 's/"$//')
+
+# Append an existing log file for this build with .old if one exists
+if [ -f "${LOG_FILE}" ]; then
+   mv "${LOG_FILE}" "${LOG_FILE}.old"
+else
+       touch "${LOG_FILE}"
+fi
+
+# Fill the log file with build and host info
+echo "BITBAKERY LOG FOR $1" >> "${LOG_FILE}"
+echo "START TIME: ${START_TIME}" >> "${LOG_FILE}"
+echo "HOSTNAME: $(uname -n)" >> "${LOG_FILE}"
+echo "OS: ${OS_INFO}" >> "${LOG_FILE}"
+echo "KERNEL: $(uname -r)" >> "${LOG_FILE}"
+echo "===============" >> "${LOG_FILE}"
+echo "BUILD RESULTS:" >> "${LOG_FILE}"
+
+for j in ${LIBC_LIST}; do
+    echo "[$j]" >> "${LOG_FILE}"
+    for i in ${TARGET_LIST}; do
+    echo "$i" "$j"; \
+        TCLIBC=$j MACHINE=$i bitbake "$1" && echo "PASS: $i" >> "${LOG_FILE}" || echo "FAIL: $i" >> "${LOG_FILE}";
+    done;
+done
+
+# Get pass/fail totals and add them to the end of the log
+PASSED=$(grep "PASS:" "${LOG_FILE}" | wc -l)
+FAILED=$(grep "FAIL:" "${LOG_FILE}" | wc -l)
+
+echo "===============" >> "${LOG_FILE}"
+echo "PASSED: ${PASSED}" >> "${LOG_FILE}"
+echo "FAILED: ${FAILED}" >> "${LOG_FILE}"

Comments

Trevor Gamblin Jan. 20, 2020, 1:29 p.m.
On 1/20/20 8:27 AM, Trevor Gamblin wrote:
> bitbakery simplifies the process of build testing an upgraded or
> patched recipe. It cycles through all qemu targets for glibc and musl,
> and then generates a log file in the format "<recipe>-bitbakery.log",
> containing the results of each build along with some basic system info
> (time, hostname, OS, kernel version). Additionally, it retains the
> previous run's log for comparison, renamed to <recipe>-bitbakery.log.old
> when a new set of builds is triggered.
>
> Sample log output:
>
> BITBAKERY LOG FOR aspell
> START TIME: 2020-01-17_08:39:05
> HOSTNAME: yow-tgamblin-fedora1
> OS: Fedora 31 (Server Edition)
> KERNEL: 5.4.7-200.fc31.x86_64
> ===============
> BUILD RESULTS:
> [glibc]
> FAIL: qemuarm64
> PASS: qemuarm
> PASS: qemuarmv5
> PASS: qemumips64
> PASS: qemumips
> PASS: qemuppc
> PASS: qemuriscv64
> FAIL: qemux86-64
> FAIL: qemux86
> [musl]
> PASS: qemuarm64
> PASS: qemuarm
> PASS: qemuarmv5
> PASS: qemumips64
> PASS: qemumips
> PASS: qemuppc
> PASS: qemuriscv64
> PASS: qemux86-64
> PASS: qemux86
> ===============
> PASSED: 15
> FAILED: 3
> ---
>   scripts/bitbakery | 47 +++++++++++++++++++++++++++++++++++++++++++++++
>   1 file changed, 47 insertions(+)
>   create mode 100755 scripts/bitbakery
>
> diff --git a/scripts/bitbakery b/scripts/bitbakery
> new file mode 100755
> index 0000000000..cb2957a7bd
> --- /dev/null
> +++ b/scripts/bitbakery
> @@ -0,0 +1,47 @@
> +#!/bin/sh
> +#  Copyright (c) 2020 Wind River Systems, Inc.
> +#
> +# SPDX-License-Identifier: GPL-2.0-only
> +#
> +
> +# Get path to oe-core directory
> +OE_CORE_PATH=$( cd - || exit )
> +
> +# Get target list and host machine information
> +TARGET_LIST=$(ls "$OE_CORE_PATH"/meta/conf/machine | grep qemu | sed -e 's/\.conf//')
> +LIBC_LIST="glibc musl"
> +START_TIME=$(date "+%Y-%m-%d_%H:%M:%S")
> +LOG_FILE="$1-bitbakery.log"
> +OS_INFO=$(cat /etc/os-release | grep "PRETTY_NAME=" | awk -F "=" '{print $2}' | sed -e 's/^"//' -e 's/"$//')
> +
> +# Append an existing log file for this build with .old if one exists
> +if [ -f "${LOG_FILE}" ]; then
> +   mv "${LOG_FILE}" "${LOG_FILE}.old"
> +else
> +       touch "${LOG_FILE}"
> +fi
> +
> +# Fill the log file with build and host info
> +echo "BITBAKERY LOG FOR $1" >> "${LOG_FILE}"
> +echo "START TIME: ${START_TIME}" >> "${LOG_FILE}"
> +echo "HOSTNAME: $(uname -n)" >> "${LOG_FILE}"
> +echo "OS: ${OS_INFO}" >> "${LOG_FILE}"
> +echo "KERNEL: $(uname -r)" >> "${LOG_FILE}"
> +echo "===============" >> "${LOG_FILE}"
> +echo "BUILD RESULTS:" >> "${LOG_FILE}"
> +
> +for j in ${LIBC_LIST}; do
> +    echo "[$j]" >> "${LOG_FILE}"
> +    for i in ${TARGET_LIST}; do
> +    echo "$i" "$j"; \
> +        TCLIBC=$j MACHINE=$i bitbake "$1" && echo "PASS: $i" >> "${LOG_FILE}" || echo "FAIL: $i" >> "${LOG_FILE}";
> +    done;
> +done
> +
> +# Get pass/fail totals and add them to the end of the log
> +PASSED=$(grep "PASS:" "${LOG_FILE}" | wc -l)
> +FAILED=$(grep "FAIL:" "${LOG_FILE}" | wc -l)
> +
> +echo "===============" >> "${LOG_FILE}"
> +echo "PASSED: ${PASSED}" >> "${LOG_FILE}"
> +echo "FAILED: ${FAILED}" >> "${LOG_FILE}"
Sending a v2 - didn't sign off.
Trevor Gamblin Jan. 21, 2020, 12:30 a.m.
This is a system generated Comment: Patch 169224 was automatically marked as superseded by patch 169225.