| Message ID | 20240124090402.3126423-1-chi.xu@windriver.com |
|---|---|
| State | Accepted, archived |
| Headers | show |
| Series | xz: Add ptest support | expand |
On Wed, 24 Jan 2024 at 10:04, Chi Xu <chi.xu@eng.windriver.com> wrote: > +Subject: [PATCH] Update test scripts: replace ../src/* with /usr/bin > + > +In order to run scripts on target, replace ../src/* with /usr/bin > + > +Upstream-Status: Inappropriate [embedded specific] > +-if test -x ../src/xz/xz ; then > ++if test -x /usr/bin/xz ; then It's better to bundle the xz* executables into the ptest package where the script expects them, and avoid making a non-upstreamable patch altogether. > +ptestdir=$(dirname "$(readlink -f "$0")") > +cd "$ptestdir"/tests || exit > + > +# executables test cases > +tests_exec=$(file $(ls test_*) | grep "ELF" | awk -F: '{print $1}') > +for i in ${tests_exec}; do > + ./$i > +done > + > +# POSIX shell script test cases > +export srcdir=${ptestdir}/tests > +test_shell="test_files.sh test_scripts.sh test_compress.sh" > +for j in ${test_shell}; do > + if [ $j == "test_compress.sh" ]; then > + ./$j create_compress_files > + else > + ./$j > + fi > + if [ $? -eq 0 ]; then > + echo "PASS: $j" > + else > + echo "FAIL: $j" > + fi I still don't understand this part. If these scripts are executed as a part of standard in-tree unit testing, what prints PASS/FAIL in that context? Shouldn't we reuse that, to replicate the in-tree testing sequence as much as possible? Alex
On Wed, 2024-01-24 at 10:41 +0100, Alexander Kanavin wrote: > On Wed, 24 Jan 2024 at 10:04, Chi Xu <chi.xu@eng.windriver.com> wrote: > > +Subject: [PATCH] Update test scripts: replace ../src/* with /usr/bin > > + > > +In order to run scripts on target, replace ../src/* with /usr/bin > > + > > +Upstream-Status: Inappropriate [embedded specific] > > +-if test -x ../src/xz/xz ; then > > ++if test -x /usr/bin/xz ; then > > It's better to bundle the xz* executables into the ptest package where > the script expects them, and avoid making a non-upstreamable patch > altogether. I'd put links in place there to redirect to binares, that way we test the actual binaries and don't have copies. Cheers, Richard
> -----Original Message----- > From: Richard Purdie <richard.purdie@linuxfoundation.org> > Sent: Wednesday, January 24, 2024 5:47 PM > To: Alexander Kanavin <alex.kanavin@gmail.com>; Xu, Chi > <Chi.Xu@windriver.com> > Cc: openembedded-core@lists.openembedded.org > Subject: Re: [OE-core] [PATCH] xz: Add ptest support > > CAUTION: This email comes from a non Wind River email account! > Do not click links or open attachments unless you recognize the sender and > know the content is safe. > > On Wed, 2024-01-24 at 10:41 +0100, Alexander Kanavin wrote: > > On Wed, 24 Jan 2024 at 10:04, Chi Xu <chi.xu@eng.windriver.com> wrote: > > > +Subject: [PATCH] Update test scripts: replace ../src/* with > > > +/usr/bin > > > + > > > +In order to run scripts on target, replace ../src/* with /usr/bin > > > + > > > +Upstream-Status: Inappropriate [embedded specific] -if test -x > > > +../src/xz/xz ; then > > > ++if test -x /usr/bin/xz ; then > > > > It's better to bundle the xz* executables into the ptest package where > > the script expects them, and avoid making a non-upstreamable patch > > altogether. > > I'd put links in place there to redirect to binares, that way we test the actual > binaries and don't have copies. > Thanks Richard, this is a good alternative if we want to avoid non-upstreamable patch. Thanks, Chi > Cheers, > > Richard
> -----Original Message----- > From: Alexander Kanavin <alex.kanavin@gmail.com> > Sent: Wednesday, January 24, 2024 5:42 PM > To: Xu, Chi <Chi.Xu@windriver.com> > Cc: openembedded-core@lists.openembedded.org > Subject: Re: [OE-core] [PATCH] xz: Add ptest support > > CAUTION: This email comes from a non Wind River email account! > Do not click links or open attachments unless you recognize the sender and > know the content is safe. > > On Wed, 24 Jan 2024 at 10:04, Chi Xu <chi.xu@eng.windriver.com> wrote: > > +Subject: [PATCH] Update test scripts: replace ../src/* with /usr/bin > > + > > +In order to run scripts on target, replace ../src/* with /usr/bin > > + > > +Upstream-Status: Inappropriate [embedded specific] -if test -x > > +../src/xz/xz ; then > > ++if test -x /usr/bin/xz ; then > > It's better to bundle the xz* executables into the ptest package where the > script expects them, and avoid making a non-upstreamable patch altogether. > > > +ptestdir=$(dirname "$(readlink -f "$0")") cd "$ptestdir"/tests || > > +exit > > + > > +# executables test cases > > +tests_exec=$(file $(ls test_*) | grep "ELF" | awk -F: '{print $1}') > > +for i in ${tests_exec}; do > > + ./$i > > +done > > + > > +# POSIX shell script test cases > > +export srcdir=${ptestdir}/tests > > +test_shell="test_files.sh test_scripts.sh test_compress.sh" > > +for j in ${test_shell}; do > > + if [ $j == "test_compress.sh" ]; then > > + ./$j create_compress_files > > + else > > + ./$j > > + fi > > + if [ $? -eq 0 ]; then > > + echo "PASS: $j" > > + else > > + echo "FAIL: $j" > > + fi > > I still don't understand this part. If these scripts are executed as a part of > standard in-tree unit testing, what prints PASS/FAIL in that context? Shouldn't > we reuse that, to replicate the in-tree testing sequence as much as possible? > According to the description on section 4.2 in 'INSTALL' file of xz, "The shell script tests can be copied from the source directory to the target machine to execute." So I copy them to target and execute them in ptest. They don't have out put like "PASS/FAIL", just return zero or non-zero, so in run-ptest, I print "PASS" or "FAIL" according to the return value. ============== 4.2. Cross compiling The tests can be built without running them: make check TESTS= The TESTS variable is the list of tests you wish to run. Leaving it empty will compile the tests without running any. If the tests are copied to a target machine to execute, the test data files in the directory tests/files must also be copied. The tests search for the data files using the environment variable $srcdir, expecting to find the data files under $srcdir/files/. If $srcdir isn't set then it defaults to the current directory. The shell script tests can be copied from the source directory to the target machine to execute. In addition to the test files, these tests will expect the following relative file paths to execute properly: ============== Out put of shell script cases: root@qemux86-64:/usr/lib/xz/ptest/tests# export srcdir=/usr/lib/xz/ptest/tests root@qemux86-64:/usr/lib/xz/ptest/tests# ./test_files.sh root@qemux86-64:/usr/lib/xz/ptest/tests# echo $? 0 root@qemux86-64:/usr/lib/xz/ptest/tests# ./test_scripts.sh root@qemux86-64:/usr/lib/xz/ptest/tests# echo $? 0 root@qemux86-64:/usr/lib/xz/ptest/tests# ./test_compress.sh create_compress_files root@qemux86-64:/usr/lib/xz/ptest/tests# echo $? 0 Thanks, Chi > Alex
On Thu, 25 Jan 2024 at 03:55, Xu, Chi <Chi.Xu@windriver.com> wrote: > > I still don't understand this part. If these scripts are executed as a part of > > standard in-tree unit testing, what prints PASS/FAIL in that context? Shouldn't > > we reuse that, to replicate the in-tree testing sequence as much as possible? > > > > According to the description on section 4.2 in 'INSTALL' file of xz, > "The shell script tests can be copied from the source directory to the target machine to execute." > So I copy them to target and execute them in ptest. > They don't have out put like "PASS/FAIL", just return zero or non-zero, so in run-ptest, I print > "PASS" or "FAIL" according to the return value. Thanks, I confirmed from the source code that binaries print PASS/FAIL directly. Shell scripts only indicate the test outcome with a return code. Then Makefile (produced with autotools macros) prints PASS/FAIL for all of them. So run-ptest is fine, nothing needs to be changed. Alex
diff --git a/meta/conf/distro/include/ptest-packagelists.inc b/meta/conf/distro/include/ptest-packagelists.inc index ef6f471e7b..cddc84195f 100644 --- a/meta/conf/distro/include/ptest-packagelists.inc +++ b/meta/conf/distro/include/ptest-packagelists.inc @@ -77,6 +77,7 @@ PTESTS_FAST = "\ sed \ slang \ wayland \ + xz \ zlib \ libexif \ " diff --git a/meta/recipes-extended/xz/xz/0001-Update-test-scripts.patch b/meta/recipes-extended/xz/xz/0001-Update-test-scripts.patch new file mode 100644 index 0000000000..81bff29b3d --- /dev/null +++ b/meta/recipes-extended/xz/xz/0001-Update-test-scripts.patch @@ -0,0 +1,80 @@ +From c3921adf93c578412f9ce520ed48b42d60c64f77 Mon Sep 17 00:00:00 2001 +From: "Xu, Chi" <chi.xu@windriver.com> +Date: Wed, 24 Jan 2024 14:28:13 +0800 +Subject: [PATCH] Update test scripts: replace ../src/* with /usr/bin + +In order to run scripts on target, replace ../src/* with /usr/bin + +Upstream-Status: Inappropriate [embedded specific] + +Signed-off-by: Xu, Chi <chi.xu@windriver.com> +--- + tests/test_compress.sh | 8 ++++---- + tests/test_files.sh | 4 ++-- + tests/test_scripts.sh | 6 +++--- + 3 files changed, 9 insertions(+), 9 deletions(-) + +diff --git a/tests/test_compress.sh b/tests/test_compress.sh +index ff13cef..6169672 100755 +--- a/tests/test_compress.sh ++++ b/tests/test_compress.sh +@@ -10,7 +10,7 @@ + ############################################################################### + + # If xz wasn't built, this test is skipped. +-if test -x ../src/xz/xz ; then ++if test -x /usr/bin/xz ; then + : + else + exit 77 +@@ -76,12 +76,12 @@ test_xz() { + fi + } + +-XZ="../src/xz/xz --memlimit-compress=48MiB --memlimit-decompress=5MiB \ ++XZ="/usr/bin/xz --memlimit-compress=48MiB --memlimit-decompress=5MiB \ + --no-adjust --threads=1 --check=crc32" + grep "define HAVE_CHECK_CRC64" ../config.h > /dev/null \ + && XZ="$XZ --check=crc64" +-XZDEC="../src/xzdec/xzdec" # No memory usage limiter available +-test -x ../src/xzdec/xzdec || XZDEC= ++XZDEC="/usr/bin/xzdec" # No memory usage limiter available ++test -x /usr/bin/xzdec || XZDEC= + + # Create the required input file if needed. + # +diff --git a/tests/test_files.sh b/tests/test_files.sh +index 60402e7..37dcf4a 100755 +--- a/tests/test_files.sh ++++ b/tests/test_files.sh +@@ -10,8 +10,8 @@ + ############################################################################### + + # If both xz and xzdec were not build, skip this test. +-XZ=../src/xz/xz +-XZDEC=../src/xzdec/xzdec ++XZ=/usr/bin/xz ++XZDEC=/usr/bin/xzdec + test -x "$XZ" || XZ= + test -x "$XZDEC" || XZDEC= + if test -z "$XZ$XZDEC"; then +diff --git a/tests/test_scripts.sh b/tests/test_scripts.sh +index ee82361..f91c5d8 100755 +--- a/tests/test_scripts.sh ++++ b/tests/test_scripts.sh +@@ -10,9 +10,9 @@ + ############################################################################### + + # If scripts weren't built, this test is skipped. +-XZ=../src/xz/xz +-XZDIFF=../src/scripts/xzdiff +-XZGREP=../src/scripts/xzgrep ++XZ=/usr/bin/xz ++XZDIFF=/usr/bin/xzdiff ++XZGREP=/usr/bin/xzgrep + + for i in XZ XZDIFF XZGREP; do + eval test -x "\$$i" && continue +-- +2.25.1 + diff --git a/meta/recipes-extended/xz/xz/run-ptest b/meta/recipes-extended/xz/xz/run-ptest new file mode 100644 index 0000000000..cc8ba92016 --- /dev/null +++ b/meta/recipes-extended/xz/xz/run-ptest @@ -0,0 +1,26 @@ +#!/bin/bash + +ptestdir=$(dirname "$(readlink -f "$0")") +cd "$ptestdir"/tests || exit + +# executables test cases +tests_exec=$(file $(ls test_*) | grep "ELF" | awk -F: '{print $1}') +for i in ${tests_exec}; do + ./$i +done + +# POSIX shell script test cases +export srcdir=${ptestdir}/tests +test_shell="test_files.sh test_scripts.sh test_compress.sh" +for j in ${test_shell}; do + if [ $j == "test_compress.sh" ]; then + ./$j create_compress_files + else + ./$j + fi + if [ $? -eq 0 ]; then + echo "PASS: $j" + else + echo "FAIL: $j" + fi +done diff --git a/meta/recipes-extended/xz/xz_5.4.5.bb b/meta/recipes-extended/xz/xz_5.4.5.bb index 09eade8be0..e3a99465b2 100644 --- a/meta/recipes-extended/xz/xz_5.4.5.bb +++ b/meta/recipes-extended/xz/xz_5.4.5.bb @@ -24,13 +24,15 @@ LIC_FILES_CHKSUM = "file://COPYING;md5=c8ea84ebe7b93cce676b54355dc6b2c0 \ file://lib/getopt.c;endline=23;md5=2069b0ee710572c03bb3114e4532cd84 \ " -SRC_URI = "https://tukaani.org/xz/xz-${PV}.tar.gz" +SRC_URI = "https://tukaani.org/xz/xz-${PV}.tar.gz \ + ${@bb.utils.contains('PTEST_ENABLED', '1', 'file://run-ptest file://0001-Update-test-scripts.patch', '', d)} \ + " SRC_URI[sha256sum] = "135c90b934aee8fbc0d467de87a05cb70d627da36abe518c357a873709e5b7d6" UPSTREAM_CHECK_REGEX = "xz-(?P<pver>\d+(\.\d+)+)\.tar" CACHED_CONFIGUREVARS += "gl_cv_posix_shell=/bin/sh" -inherit autotools gettext +inherit autotools gettext ptest PACKAGES =+ "liblzma" @@ -42,3 +44,18 @@ ALTERNATIVE:${PN} = "xz xzcat unxz \ lzma lzcat unlzma" BBCLASSEXTEND = "native nativesdk" + +RDEPENDS:${PN}-ptest += "bash file" + +do_compile_ptest() { + oe_runmake check TESTS= +} + +do_install_ptest () { + install -d ${D}/${PTEST_PATH}/tests + find ${B}/tests/.libs -type f -executable -exec cp {} ${D}${PTEST_PATH}/tests \; + cp ${B}/config.h ${D}${PTEST_PATH} + for i in files xzgrep_expected_output test_files.sh test_scripts.sh test_compress.sh; do + cp -r ${S}/tests/$i ${D}${PTEST_PATH}/tests + done +}
There are two types of cases: executables and POSIX shell scripts. The output of them are different, so run them with different method in run-ptest. 74 test cases in all and all test cases PASS. Add xz-ptest to PTESTS_FAST because test duration is 19s on qemux86-64. Test results on qemux86-64: root@qemux86-64:~# ptest-runner xz START: ptest-runner 2024-01-24T06:45 BEGIN: /usr/lib/xz/ptest === test_bcj_exact_size.c === PASS: test_exact_size PASS: test_empty_block --- === END === ... ... === test_vli.c === PASS: test_lzma_vli_size PASS: test_lzma_vli_encode PASS: test_lzma_vli_decode --- === END === PASS: test_files.sh PASS: test_scripts.sh PASS: test_compress.sh DURATION: 19 END: /usr/lib/xz/ptest 2024-01-24T06:45 STOP: ptest-runner TOTAL: 1 FAIL: 0 Signed-off-by: Xu, Chi <chi.xu@windriver.com> --- .../distro/include/ptest-packagelists.inc | 1 + .../xz/xz/0001-Update-test-scripts.patch | 80 +++++++++++++++++++ meta/recipes-extended/xz/xz/run-ptest | 26 ++++++ meta/recipes-extended/xz/xz_5.4.5.bb | 21 ++++- 4 files changed, 126 insertions(+), 2 deletions(-) create mode 100644 meta/recipes-extended/xz/xz/0001-Update-test-scripts.patch create mode 100644 meta/recipes-extended/xz/xz/run-ptest