[V4] dbus: fix ptest failure

Submitted by changqing.li@windriver.com on April 17, 2019, 8:38 a.m. | Patch ID: 160497

Details

Message ID 1555490306-29097-1-git-send-email-changqing.li@windriver.com
State Master Next
Commit bd7f7226e3b19d59a0de823bfec1410df2988c27
Headers show

Commit Message

changqing.li@windriver.com April 17, 2019, 8:38 a.m.
From: Changqing Li <changqing.li@windriver.com>

1. since one bug in run-ptest, testcase test-bus have never been
actually run (althrough it's result is PASS).

After commit 0828850, test-bus can actually run but it
did not install:
  test-service, test-shell-service, test-segfault, and
  dbus-daemon-launch-helper-test
Add the configure flag:
  --enable-embedded-tests
to generate binary dbus-daemon-launch-helper-test, then install
them so that test-bus will now pass.

2. fix testcase test-dbus-daemon failed
we enable --enable-verbose-mode in recipe dbus-test, and don't
enable it in recipe dbus. This will make below test code get
unexpect result of have_verbose and assert.
disable --enable-verbose-mode for recipe dbus-test to fix it.

 #ifdef DBUS_ENABLE_STATS
  g_assert_true (have_stats);
 #else
  g_assert_false (have_stats);
 #endif

Signed-off-by: Changqing Li <changqing.li@windriver.com>
---
 meta/recipes-core/dbus/dbus-test_1.12.12.bb | 13 ++++++++++---
 meta/recipes-core/dbus/dbus/run-ptest       | 16 ++++++++++++----
 2 files changed, 22 insertions(+), 7 deletions(-)

Patch hide | download patch | download mbox

diff --git a/meta/recipes-core/dbus/dbus-test_1.12.12.bb b/meta/recipes-core/dbus/dbus-test_1.12.12.bb
index f413192..0502c44 100644
--- a/meta/recipes-core/dbus/dbus-test_1.12.12.bb
+++ b/meta/recipes-core/dbus/dbus-test_1.12.12.bb
@@ -33,13 +33,14 @@  EXTRA_OECONF = "--enable-tests \
                 --enable-installed-tests \
                 --enable-checks \
                 --enable-asserts \
-                --enable-verbose-mode \
                 --enable-largefile \
                 --disable-xml-docs \
                 --disable-doxygen-docs \
                 --disable-libaudit \
                 --with-dbus-test-dir=${PTEST_PATH} \
-                ${EXTRA_OECONF_X}"
+                ${EXTRA_OECONF_X} \
+                --enable-embedded-tests \
+             "
 
 EXTRA_OECONF_append_class-target = " SYSTEMCTL=${base_bindir}/systemctl"
 
@@ -50,6 +51,7 @@  PACKAGECONFIG_class-nativesdk = ""
 PACKAGECONFIG[systemd] = "--enable-systemd --with-systemdsystemunitdir=${systemd_system_unitdir},--disable-systemd --without-systemdsystemunitdir,systemd"
 PACKAGECONFIG[x11] = "--with-x --enable-x11-autolaunch,--without-x --disable-x11-autolaunch, virtual/libx11 libsm"
 PACKAGECONFIG[user-session] = "--enable-user-session --with-systemduserunitdir=${systemd_user_unitdir},--disable-user-session"
+PACKAGECONFIG[verbose-mode] = "--enable-verbose-mode,,,"
 
 do_install() {
     :
@@ -58,12 +60,17 @@  do_install() {
 do_install_ptest() {
 	install -d ${D}${PTEST_PATH}/test
 	l="shell printf refs syslog marshal syntax corrupt dbus-daemon dbus-daemon-eavesdrop loopback relay \
-	   variant uid-permissions syntax spawn sd-activation names monitor message fdpass "
+	   variant uid-permissions syntax spawn sd-activation names monitor message fdpass service shell-service"
 	for i in $l; do install ${B}/test/.libs/test-$i ${D}${PTEST_PATH}/test; done
 
 	l="bus bus-system bus-launch-helper"
 	for i in $l; do install ${B}/bus/.libs/test-$i ${D}${PTEST_PATH}/test; done
 
+	install -d ${D}${PTEST_PATH}/bus
+	install ${B}/bus/.libs/dbus-daemon-launch-helper-test ${D}${PTEST_PATH}/bus
+
+	install ${B}/test/test-segfault ${D}${PTEST_PATH}/test
+
 	cp -r ${B}/test/data ${D}${PTEST_PATH}/test
 	install ${B}/dbus/.libs/test-dbus ${D}${PTEST_PATH}/test
 
diff --git a/meta/recipes-core/dbus/dbus/run-ptest b/meta/recipes-core/dbus/dbus/run-ptest
index 353ba1e..cf2e68f 100755
--- a/meta/recipes-core/dbus/dbus/run-ptest
+++ b/meta/recipes-core/dbus/dbus/run-ptest
@@ -17,8 +17,16 @@  export LD_LIBRARY_PATH=@PTEST_PATH@/test/.libs
 files=`ls test/test-*`
 
 for i in $files
-	do
-		./$i ./test/data >/dev/null
-		output
-	done
+do
+     #these programs are used by testcase test-bus, don't run here
+     if [ $i = "test/test-service" ] \
+        || [ $i = "test/test-shell-service" ] \
+        || [ $i = "test/test-segfault" ]
+     then
+         continue
+     fi
+
+     ./$i ./test/data >/dev/null 2>&1
+     output
+done 
 

Comments

Richard Purdie April 17, 2019, 10:01 p.m.
On Wed, 2019-04-17 at 16:38 +0800, changqing.li@windriver.com wrote:
> From: Changqing Li <changqing.li@windriver.com>
> 
> 1. since one bug in run-ptest, testcase test-bus have never been
> actually run (althrough it's result is PASS).
> 
> After commit 0828850, test-bus can actually run but it
> did not install:
>   test-service, test-shell-service, test-segfault, and
>   dbus-daemon-launch-helper-test
> Add the configure flag:
>   --enable-embedded-tests
> to generate binary dbus-daemon-launch-helper-test, then install
> them so that test-bus will now pass.
> 
> 2. fix testcase test-dbus-daemon failed
> we enable --enable-verbose-mode in recipe dbus-test, and don't
> enable it in recipe dbus. This will make below test code get
> unexpect result of have_verbose and assert.
> disable --enable-verbose-mode for recipe dbus-test to fix it.
> 
>  #ifdef DBUS_ENABLE_STATS
>   g_assert_true (have_stats);
>  #else
>   g_assert_false (have_stats);
>  #endif
> 
> Signed-off-by: Changqing Li <changqing.li@windriver.com>
> ---
>  meta/recipes-core/dbus/dbus-test_1.12.12.bb | 13 ++++++++++---
>  meta/recipes-core/dbus/dbus/run-ptest       | 16 ++++++++++++----
>  2 files changed, 22 insertions(+), 7 deletions(-)

I'm wondering if some of these tests were intentionally not run due to
the length of time they take? It takes the dbus-ptest time from 26s to
250+s. Can you see which test that is and why its taking so long
please?

Cheers,

Richard
changqing.li@windriver.com April 18, 2019, 1:59 a.m.
On 4/18/19 6:01 AM, Richard Purdie wrote:
> On Wed, 2019-04-17 at 16:38 +0800, changqing.li@windriver.com wrote:
>> From: Changqing Li <changqing.li@windriver.com>
>>
>> 1. since one bug in run-ptest, testcase test-bus have never been
>> actually run (althrough it's result is PASS).
>>
>> After commit 0828850, test-bus can actually run but it
>> did not install:
>>    test-service, test-shell-service, test-segfault, and
>>    dbus-daemon-launch-helper-test
>> Add the configure flag:
>>    --enable-embedded-tests
>> to generate binary dbus-daemon-launch-helper-test, then install
>> them so that test-bus will now pass.
>>
>> 2. fix testcase test-dbus-daemon failed
>> we enable --enable-verbose-mode in recipe dbus-test, and don't
>> enable it in recipe dbus. This will make below test code get
>> unexpect result of have_verbose and assert.
>> disable --enable-verbose-mode for recipe dbus-test to fix it.
>>
>>   #ifdef DBUS_ENABLE_STATS
>>    g_assert_true (have_stats);
>>   #else
>>    g_assert_false (have_stats);
>>   #endif
>>
>> Signed-off-by: Changqing Li <changqing.li@windriver.com>
>> ---
>>   meta/recipes-core/dbus/dbus-test_1.12.12.bb | 13 ++++++++++---
>>   meta/recipes-core/dbus/dbus/run-ptest       | 16 ++++++++++++----
>>   2 files changed, 22 insertions(+), 7 deletions(-)
> I'm wondering if some of these tests were intentionally not run due to
> the length of time they take? It takes the dbus-ptest time from 26s to
> 250+s. Can you see which test that is and why its taking so long
> please?

test-bus take most of the time, it include several sub tests. My test 
result:

all test runed:

real    2m59.637s
user    0m59.494s
sys    0m41.952s

skipped test-bus:

real    0m13.125s
user    0m12.230s
sys    0m0.162s


Previously we don't intentionally skip test-bus,  just because we had a 
bug in run-ptest before,

so test-bus  accidentally not run,  just return PASS.

After below commit fix this bug, test-bus can be runned.

https://git.openembedded.org/openembedded-core/commit/?id=0828850fd09f738572ae8259384af07eeb81182b


-for i in `ls test/test-*`; do ./$i ./test/data DBUS_TEST_HOMEDIR=./test 
 >/dev/null; output; done


"DBUS_TEST_HOMEDIR=./test" will take as an argument, make no sub test is 
runned.



>
> Cheers,
>
> Richard
>
>
>
Richard Purdie April 24, 2019, 5:28 p.m.
On Thu, 2019-04-18 at 09:59 +0800, Changqing Li wrote:
> 
> On 4/18/19 6:01 AM, Richard Purdie wrote:
> > On Wed, 2019-04-17 at 16:38 +0800, changqing.li@windriver.com
> > wrote:
> > > From: Changqing Li <changqing.li@windriver.com>
> > > 
> > > 1. since one bug in run-ptest, testcase test-bus have never been
> > > actually run (althrough it's result is PASS).
> > > 
> > > After commit 0828850, test-bus can actually run but it
> > > did not install:
> > >   test-service, test-shell-service, test-segfault, and
> > >   dbus-daemon-launch-helper-test
> > > Add the configure flag:
> > >   --enable-embedded-tests
> > > to generate binary dbus-daemon-launch-helper-test, then install
> > > them so that test-bus will now pass.
> > > 
> > > 2. fix testcase test-dbus-daemon failed
> > > we enable --enable-verbose-mode in recipe dbus-test, and don't
> > > enable it in recipe dbus. This will make below test code get
> > > unexpect result of have_verbose and assert.
> > > disable --enable-verbose-mode for recipe dbus-test to fix it.
> > > 
> > >  #ifdef DBUS_ENABLE_STATS
> > >   g_assert_true (have_stats);
> > >  #else
> > >   g_assert_false (have_stats);
> > >  #endif
> > > 
> > > Signed-off-by: Changqing Li <changqing.li@windriver.com>
> > > ---
> > >  meta/recipes-core/dbus/dbus-test_1.12.12.bb | 13 ++++++++++---
> > >  meta/recipes-core/dbus/dbus/run-ptest       | 16 ++++++++++++---
> > > -
> > >  2 files changed, 22 insertions(+), 7 deletions(-)
> > 
> > I'm wondering if some of these tests were intentionally not run due
> > to
> > the length of time they take? It takes the dbus-ptest time from 26s
> > to
> > 250+s. Can you see which test that is and why its taking so long
> > please?
> 
> test-bus take most of the time, it include several sub tests. My test
> result:
> all test runed: 
> real    2m59.637s
> user    0m59.494s
> sys    0m41.952s
> skipped test-bus:
> real    0m13.125s
> user    0m12.230s
> sys    0m0.162s
> 
> Previously we don't intentionally skip test-bus,  just because we had
> a bug in run-ptest before,
> so test-bus  accidentally not run,  just return PASS. 
> After below commit fix this bug, test-bus can be runned.
> https://git.openembedded.org/openembedded-core/commit/?id=0828850fd09f738572ae8259384af07eeb81182b
> 
> -for i in `ls test/test-*`; do ./$i ./test/data
> DBUS_TEST_HOMEDIR=./test >/dev/null; output; done
> 
> "DBUS_TEST_HOMEDIR=./test" will take as an argument,  make no sub
> test is runned. 


Ross, do you remember if we intentionally skip this dbus test for
taking around 200s, taking the overall test time from 20s to 220s?

I'm strongly tempted to explicitly disable this test unless someone can
convince me it tests someting critical...

Cheers,

Richard
Ross Burton April 30, 2019, 1:35 p.m.
test-bus sounds like it actually tests the bus, which is useful.  If
it actually passes now then I'd say we keep it.

Ross

On Wed, 24 Apr 2019 at 18:28, <richard.purdie@linuxfoundation.org> wrote:
>
> On Thu, 2019-04-18 at 09:59 +0800, Changqing Li wrote:
> >
> > On 4/18/19 6:01 AM, Richard Purdie wrote:
> > > On Wed, 2019-04-17 at 16:38 +0800, changqing.li@windriver.com
> > > wrote:
> > > > From: Changqing Li <changqing.li@windriver.com>
> > > >
> > > > 1. since one bug in run-ptest, testcase test-bus have never been
> > > > actually run (althrough it's result is PASS).
> > > >
> > > > After commit 0828850, test-bus can actually run but it
> > > > did not install:
> > > >   test-service, test-shell-service, test-segfault, and
> > > >   dbus-daemon-launch-helper-test
> > > > Add the configure flag:
> > > >   --enable-embedded-tests
> > > > to generate binary dbus-daemon-launch-helper-test, then install
> > > > them so that test-bus will now pass.
> > > >
> > > > 2. fix testcase test-dbus-daemon failed
> > > > we enable --enable-verbose-mode in recipe dbus-test, and don't
> > > > enable it in recipe dbus. This will make below test code get
> > > > unexpect result of have_verbose and assert.
> > > > disable --enable-verbose-mode for recipe dbus-test to fix it.
> > > >
> > > >  #ifdef DBUS_ENABLE_STATS
> > > >   g_assert_true (have_stats);
> > > >  #else
> > > >   g_assert_false (have_stats);
> > > >  #endif
> > > >
> > > > Signed-off-by: Changqing Li <changqing.li@windriver.com>
> > > > ---
> > > >  meta/recipes-core/dbus/dbus-test_1.12.12.bb | 13 ++++++++++---
> > > >  meta/recipes-core/dbus/dbus/run-ptest       | 16 ++++++++++++---
> > > > -
> > > >  2 files changed, 22 insertions(+), 7 deletions(-)
> > >
> > > I'm wondering if some of these tests were intentionally not run due
> > > to
> > > the length of time they take? It takes the dbus-ptest time from 26s
> > > to
> > > 250+s. Can you see which test that is and why its taking so long
> > > please?
> >
> > test-bus take most of the time, it include several sub tests. My test
> > result:
> > all test runed:
> > real    2m59.637s
> > user    0m59.494s
> > sys    0m41.952s
> > skipped test-bus:
> > real    0m13.125s
> > user    0m12.230s
> > sys    0m0.162s
> >
> > Previously we don't intentionally skip test-bus,  just because we had
> > a bug in run-ptest before,
> > so test-bus  accidentally not run,  just return PASS.
> > After below commit fix this bug, test-bus can be runned.
> > https://git.openembedded.org/openembedded-core/commit/?id=0828850fd09f738572ae8259384af07eeb81182b
> >
> > -for i in `ls test/test-*`; do ./$i ./test/data
> > DBUS_TEST_HOMEDIR=./test >/dev/null; output; done
> >
> > "DBUS_TEST_HOMEDIR=./test" will take as an argument,  make no sub
> > test is runned.
>
>
> Ross, do you remember if we intentionally skip this dbus test for
> taking around 200s, taking the overall test time from 20s to 220s?
>
> I'm strongly tempted to explicitly disable this test unless someone can
> convince me it tests someting critical...
>
> Cheers,
>
> Richard
>