| Message ID | 20220704071832.3400931-1-sundeep.kokkonda@gmail.com |
|---|---|
| State | New, archived |
| Headers | show |
| Series | [kirkstone] Binutils, Glibc: Updated with latest commits | expand |
The regression test for Binutils and Glibc is performed. 1) Binutils test shows no regression after update. 2) Glibc test result: Test summary *before* glibc update on Kirkstone: 204 FAIL 4320 PASS 43 UNSUPPORTED 12 XFAIL 6 XPASS Test summary *after* glibc update on Kirkstone: 204 FAIL 4339 PASS 41 UNSUPPORTED 12 XFAIL 6 XPASS Test results analysis: Total 4585 test cases before Glibc update & Total 4602 test cases after Glibc update. 17 new test cases added (out of those 16 PASS & 1 FAIL) 12 new FAIL test cases after glibc update 14 new PASS test cases after glibc update Refer the attached test report 'TestSummary.xlsx' for detailed analysis.
On Mon, 2022-07-04 at 00:51 -0700, Sundeep KOKKONDA wrote: > The regression test for Binutils and Glibc is performed. > > 1) Binutils test shows no regression after update. > > 2) Glibc test result: > Test summary before glibc update on Kirkstone: > 204 FAIL > 4320 PASS > 43 UNSUPPORTED > 12 XFAIL > 6 XPASS > > Test summary after glibc update on Kirkstone: > 204 FAIL > 4339 PASS > 41 UNSUPPORTED > 12 XFAIL > 6 XPASS > > Test results analysis: > Total 4585 test cases before Glibc update & Total 4602 test cases > after Glibc update. > 17 new test cases added (out of those 16 PASS & 1 FAIL) > 12 new FAIL test cases after glibc update > 14 new PASS test cases after glibc update Do we know why some of the failing test cass now pass and why some of the passing test cases now fail? Is this some issue with the way we're testing things? I'm wondering if this is "normal" noise between test runs and if there is some underlying instability or glibc changes do cause changes in pass/fails of tests like this in general? Cheers, Richard
Hello Richard, > -----Original Message----- > From: Richard Purdie <richard.purdie@linuxfoundation.org> > Sent: 04 July 2022 14:20 > To: Sundeep KOKKONDA <sundeep.kokkonda@gmail.com>; openembedded- > core@lists.openembedded.org > Subject: Re: [OE-core] [kirkstone] Binutils, Glibc: Updated with latest commits > > On Mon, 2022-07-04 at 00:51 -0700, Sundeep KOKKONDA wrote: > > The regression test for Binutils and Glibc is performed. > > > > 1) Binutils test shows no regression after update. > > > > 2) Glibc test result: > > Test summary before glibc update on Kirkstone: > > 204 FAIL > > 4320 PASS > > 43 UNSUPPORTED > > 12 XFAIL > > 6 XPASS > > > > Test summary after glibc update on Kirkstone: > > 204 FAIL > > 4339 PASS > > 41 UNSUPPORTED > > 12 XFAIL > > 6 XPASS > > > > Test results analysis: > > Total 4585 test cases before Glibc update & Total 4602 test cases > > after Glibc update. > > 17 new test cases added (out of those 16 PASS & 1 FAIL) > > 12 new FAIL test cases after glibc update > > 14 new PASS test cases after glibc update > > Do we know why some of the failing test cass now pass and why some of the > passing test cases now fail? Is this some issue with the way we're testing things? > I'm wondering if this is "normal" noise between test runs and if there is some > underlying instability or glibc changes do cause changes in pass/fails of tests like > this in general? Hello Richard, The test results are impacted by Execution procedure OR by Changes in glibc. Execution procedure: I tested the glibc with the cross-compiler method given in https://sourceware.org/glibc/wiki/Testing/Testsuite for the aarch64 architecture on qemu. However, based on our past test experience, we're following a few additional steps like copying the 'glibc-locale/2.28-r0/image/usr/*' to booted qemu image /usr/ directory (This step is not mentioned the glibc wiki) etc. Also, the glibc test is aborted when the -Werror=format-security flag is used and to get the test done we've removed this error flag and compiled glibc sources. Since the glibc test not performed very frequently, I am not sure the additional steps what we followed are obsolete or not. I am now working on adhering to the test procedure mentioned in the glibc test wiki and will see the results. Changes in glibc: To comment on the code change impact on the test failures I've to look into each failed test case by case and what are the changes in glibc and how it is impacting the test. I'll keep you posted on this and since the glibc test execution is taking a lot of time my response may be delayed. Thanks, Sundeep K. > > Cheers, > > Richard > > >
On 7/5/22 6:18 AM, Sundeep KOKKONDA wrote: > Hello Richard, > >> -----Original Message----- >> From: Richard Purdie <richard.purdie@linuxfoundation.org> >> Sent: 04 July 2022 14:20 >> To: Sundeep KOKKONDA <sundeep.kokkonda@gmail.com>; openembedded- >> core@lists.openembedded.org >> Subject: Re: [OE-core] [kirkstone] Binutils, Glibc: Updated with latest commits >> >> On Mon, 2022-07-04 at 00:51 -0700, Sundeep KOKKONDA wrote: >>> The regression test for Binutils and Glibc is performed. >>> >>> 1) Binutils test shows no regression after update. >>> >>> 2) Glibc test result: >>> Test summary before glibc update on Kirkstone: >>> 204 FAIL >>> 4320 PASS >>> 43 UNSUPPORTED >>> 12 XFAIL >>> 6 XPASS >>> >>> Test summary after glibc update on Kirkstone: >>> 204 FAIL >>> 4339 PASS >>> 41 UNSUPPORTED >>> 12 XFAIL >>> 6 XPASS >>> >>> Test results analysis: >>> Total 4585 test cases before Glibc update & Total 4602 test cases >>> after Glibc update. >>> 17 new test cases added (out of those 16 PASS & 1 FAIL) >>> 12 new FAIL test cases after glibc update >>> 14 new PASS test cases after glibc update >> Do we know why some of the failing test cass now pass and why some of the >> passing test cases now fail? Is this some issue with the way we're testing things? >> I'm wondering if this is "normal" noise between test runs and if there is some >> underlying instability or glibc changes do cause changes in pass/fails of tests like >> this in general? > > Hello Richard, > > The test results are impacted by Execution procedure OR by Changes in glibc. > > Execution procedure: > I tested the glibc with the cross-compiler method given in https://sourceware.org/glibc/wiki/Testing/Testsuite for the aarch64 architecture on qemu. > However, based on our past test experience, we're following a few additional steps like copying the 'glibc-locale/2.28-r0/image/usr/*' to booted qemu image /usr/ directory (This step is not mentioned the glibc wiki) etc. > Also, the glibc test is aborted when the -Werror=format-security flag is used and to get the test done we've removed this error flag and compiled glibc sources. > > Since the glibc test not performed very frequently, I am not sure the additional steps what we followed are obsolete or not. I am now working on adhering to the test procedure mentioned in the glibc test wiki and will see the results. Perhaps run few iterations on unmodified setup and see if the results vary. If they do then we know that there are some jitter to look into. otherwise, its perhaps a combination of updates which would also be good to know. Failing one's are more important to look into as they may cause unwanted regressions. > Changes in glibc: > To comment on the code change impact on the test failures I've to look into each failed test case by case and what are the changes in glibc and how it is impacting the test. > > > I'll keep you posted on this and since the glibc test execution is taking a lot of time my response may be delayed. > > > > Thanks, > Sundeep K. > > >> Cheers, >> >> Richard >> >> >> > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#167648): https://lists.openembedded.org/g/openembedded-core/message/167648 > Mute This Topic: https://lists.openembedded.org/mt/92160650/1997914 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [raj.khem@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- >
diff --git a/meta/recipes-core/glibc/glibc-version.inc b/meta/recipes-core/glibc/glibc-version.inc index 080e905b6e..99017ce1d4 100644 --- a/meta/recipes-core/glibc/glibc-version.inc +++ b/meta/recipes-core/glibc/glibc-version.inc @@ -1,6 +1,6 @@ SRCBRANCH ?= "release/2.35/master" PV = "2.35" -SRCREV_glibc ?= "24962427071fa532c3c48c918e9d64d719cc8a6c" +SRCREV_glibc ?= "b6aade18a7e5719c942aa2da6cf3157aca993fa4" SRCREV_localedef ?= "794da69788cbf9bf57b59a852f9f11307663fa87" GLIBC_GIT_URI ?= "git://sourceware.org/git/glibc.git" diff --git a/meta/recipes-devtools/binutils/binutils-2.38.inc b/meta/recipes-devtools/binutils/binutils-2.38.inc index dc0a2a4054..c2502e0901 100644 --- a/meta/recipes-devtools/binutils/binutils-2.38.inc +++ b/meta/recipes-devtools/binutils/binutils-2.38.inc @@ -18,7 +18,7 @@ SRCBRANCH ?= "binutils-2_38-branch" UPSTREAM_CHECK_GITTAGREGEX = "binutils-(?P<pver>\d+_(\d_?)*)" -SRCREV ?= "134f17ef688ba4c72a6c4e57af7382882cc1a705" +SRCREV ?= "eed56ee299b9ef8754bb4e53f2e9cf2a7c28c04d" BINUTILS_GIT_URI ?= "git://sourceware.org/git/binutils-gdb.git;branch=${SRCBRANCH};protocol=git" SRC_URI = "\ ${BINUTILS_GIT_URI} \