[v3,4/4] oe-selftest: add some tests for recipeutils module

Submitted by Paul Eggleton on Nov. 29, 2018, 9:21 a.m. | Patch ID: 156728

Details

Message ID 20181129092115.28502-1-paul.eggleton@linux.intel.com
State Master Next
Commit 444b81e826748f971ef79c88ba47ae5abc6a7afd
Headers show

Commit Message

Paul Eggleton Nov. 29, 2018, 9:21 a.m.
Add some tests for functions in meta/lib/oe/recipeutils.py, in
particular for a few issues I've just fixed. I haven't added tests for
all of the functions - some of them are already being tested via devtool
in any case.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
 .../python/python-async-test.inc              |  16 ++
 .../python/python3-async-test_0.6.2.bb        |   2 +
 .../recipeutils/recipeutils-test.inc          |   5 +
 .../recipeutils/recipeutils-test/anotherfile  |   0
 .../recipeutils/recipeutils-test/somefile     |   0
 .../recipeutils/recipeutils-test_1.2.bb       |  13 ++
 meta/lib/oeqa/selftest/cases/recipeutils.py   | 138 ++++++++++++++++++
 7 files changed, 174 insertions(+)
 create mode 100644 meta-selftest/recipes-devtools/python/python-async-test.inc
 create mode 100644 meta-selftest/recipes-devtools/python/python3-async-test_0.6.2.bb
 create mode 100644 meta-selftest/recipes-test/recipeutils/recipeutils-test.inc
 create mode 100644 meta-selftest/recipes-test/recipeutils/recipeutils-test/anotherfile
 create mode 100644 meta-selftest/recipes-test/recipeutils/recipeutils-test/somefile
 create mode 100644 meta-selftest/recipes-test/recipeutils/recipeutils-test_1.2.bb
 create mode 100644 meta/lib/oeqa/selftest/cases/recipeutils.py

Patch hide | download patch | download mbox

diff --git a/meta-selftest/recipes-devtools/python/python-async-test.inc b/meta-selftest/recipes-devtools/python/python-async-test.inc
new file mode 100644
index 00000000000..c9602e8e52d
--- /dev/null
+++ b/meta-selftest/recipes-devtools/python/python-async-test.inc
@@ -0,0 +1,16 @@ 
+SUMMARY = "Python framework to process interdependent tasks in a pool of workers"
+HOMEPAGE = "http://github.com/gitpython-developers/async"
+SECTION = "devel/python"
+LICENSE = "BSD"
+LIC_FILES_CHKSUM = "file://PKG-INFO;beginline=8;endline=8;md5=88df8e78b9edfd744953862179f2d14e"
+
+inherit pypi
+
+PYPI_PACKAGE = "async"
+
+SRC_URI[md5sum] = "9b06b5997de2154f3bc0273f80bcef6b"
+SRC_URI[sha256sum] = "ac6894d876e45878faae493b0cf61d0e28ec417334448ac0a6ea2229d8343051"
+
+RDEPENDS_${PN} += "${PYTHON_PN}-threading"
+
+BBCLASSEXTEND = "nativesdk"
diff --git a/meta-selftest/recipes-devtools/python/python3-async-test_0.6.2.bb b/meta-selftest/recipes-devtools/python/python3-async-test_0.6.2.bb
new file mode 100644
index 00000000000..22e241afb3c
--- /dev/null
+++ b/meta-selftest/recipes-devtools/python/python3-async-test_0.6.2.bb
@@ -0,0 +1,2 @@ 
+inherit setuptools3
+require python-async-test.inc
diff --git a/meta-selftest/recipes-test/recipeutils/recipeutils-test.inc b/meta-selftest/recipes-test/recipeutils/recipeutils-test.inc
new file mode 100644
index 00000000000..8490b902d75
--- /dev/null
+++ b/meta-selftest/recipes-test/recipeutils/recipeutils-test.inc
@@ -0,0 +1,5 @@ 
+SRC_URI = "http://xorg.freedesktop.org/releases/individual/lib/libxshmfence-${PV}.tar.bz2"
+
+SRC_URI[md5sum] = "66662e76899112c0f99e22f2fc775a7e"
+SRC_URI[sha256sum] = "d21b2d1fd78c1efbe1f2c16dae1cb23f8fd231dcf891465b8debe636a9054b0c"
+
diff --git a/meta-selftest/recipes-test/recipeutils/recipeutils-test/anotherfile b/meta-selftest/recipes-test/recipeutils/recipeutils-test/anotherfile
new file mode 100644
index 00000000000..e69de29bb2d
diff --git a/meta-selftest/recipes-test/recipeutils/recipeutils-test/somefile b/meta-selftest/recipes-test/recipeutils/recipeutils-test/somefile
new file mode 100644
index 00000000000..e69de29bb2d
diff --git a/meta-selftest/recipes-test/recipeutils/recipeutils-test_1.2.bb b/meta-selftest/recipes-test/recipeutils/recipeutils-test_1.2.bb
new file mode 100644
index 00000000000..f6da97b2d43
--- /dev/null
+++ b/meta-selftest/recipes-test/recipeutils/recipeutils-test_1.2.bb
@@ -0,0 +1,13 @@ 
+SUMMARY = "Test recipe for recipeutils.patch_recipe()"
+
+require recipeutils-test.inc
+
+LICENSE = "Proprietary"
+
+DEPENDS += "virtual/libx11"
+
+BBCLASSEXTEND = "native nativesdk"
+
+SRC_URI += "file://somefile"
+
+SRC_URI_append = " file://anotherfile"
diff --git a/meta/lib/oeqa/selftest/cases/recipeutils.py b/meta/lib/oeqa/selftest/cases/recipeutils.py
new file mode 100644
index 00000000000..b7cb55baae1
--- /dev/null
+++ b/meta/lib/oeqa/selftest/cases/recipeutils.py
@@ -0,0 +1,138 @@ 
+import os
+import re
+import time
+import logging
+import bb.tinfoil
+
+from oeqa.selftest.case import OESelftestTestCase
+from oeqa.utils.commands import runCmd
+from oeqa.core.decorator.oeid import OETestID
+
+
+def setUpModule():
+    global tinfoil
+    tinfoil = bb.tinfoil.Tinfoil(tracking=True)
+    tinfoil.prepare(config_only=False, quiet=2)
+
+
+def tearDownModule():
+    tinfoil.shutdown()
+
+
+class RecipeUtilsTests(OESelftestTestCase):
+    """ Tests for the recipeutils module functions """
+
+    def test_patch_recipe_varflag(self):
+        import oe.recipeutils
+        rd = tinfoil.parse_recipe('python3-async-test')
+        vals = {'SRC_URI[md5sum]': 'aaaaaa', 'LICENSE': 'something'}
+        corebase = rd.getVar('COREBASE')
+        patches = oe.recipeutils.patch_recipe(rd, rd.getVar('FILE'), vals, patch=True, relpath=corebase)
+
+        expected_patch = """
+--- a/meta-selftest/recipes-devtools/python/python-async-test.inc
++++ b/meta-selftest/recipes-devtools/python/python-async-test.inc
+@@ -1,14 +1,14 @@
+ SUMMARY = "Python framework to process interdependent tasks in a pool of workers"
+ HOMEPAGE = "http://github.com/gitpython-developers/async"
+ SECTION = "devel/python"
+-LICENSE = "BSD"
++LICENSE = "something"
+ LIC_FILES_CHKSUM = "file://PKG-INFO;beginline=8;endline=8;md5=88df8e78b9edfd744953862179f2d14e"
+ 
+ inherit pypi
+ 
+ PYPI_PACKAGE = "async"
+ 
+-SRC_URI[md5sum] = "9b06b5997de2154f3bc0273f80bcef6b"
++SRC_URI[md5sum] = "aaaaaa"
+ SRC_URI[sha256sum] = "ac6894d876e45878faae493b0cf61d0e28ec417334448ac0a6ea2229d8343051"
+ 
+ RDEPENDS_${PN} += "${PYTHON_PN}-threading"
+"""
+        patchlines = []
+        for f in patches:
+            for line in f:
+                patchlines.append(line)
+        self.maxDiff = None
+        self.assertEqual(''.join(patchlines).strip(), expected_patch.strip())
+
+
+    def test_patch_recipe_singleappend(self):
+        import oe.recipeutils
+        rd = tinfoil.parse_recipe('recipeutils-test')
+        val = rd.getVar('SRC_URI', False).split()
+        del val[1]
+        val = ' '.join(val)
+        vals = {'SRC_URI': val}
+        corebase = rd.getVar('COREBASE')
+        patches = oe.recipeutils.patch_recipe(rd, rd.getVar('FILE'), vals, patch=True, relpath=corebase)
+
+        expected_patch = """
+--- a/meta-selftest/recipes-test/recipeutils/recipeutils-test_1.2.bb
++++ b/meta-selftest/recipes-test/recipeutils/recipeutils-test_1.2.bb
+@@ -8,6 +8,4 @@
+ 
+ BBCLASSEXTEND = "native nativesdk"
+ 
+-SRC_URI += "file://somefile"
+-
+ SRC_URI_append = " file://anotherfile"
+"""
+        patchlines = []
+        for f in patches:
+            for line in f:
+                patchlines.append(line)
+        self.assertEqual(''.join(patchlines).strip(), expected_patch.strip())
+
+
+    def test_patch_recipe_appends(self):
+        import oe.recipeutils
+        rd = tinfoil.parse_recipe('recipeutils-test')
+        val = rd.getVar('SRC_URI', False).split()
+        vals = {'SRC_URI': val[0]}
+        corebase = rd.getVar('COREBASE')
+        patches = oe.recipeutils.patch_recipe(rd, rd.getVar('FILE'), vals, patch=True, relpath=corebase)
+
+        expected_patch = """
+--- a/meta-selftest/recipes-test/recipeutils/recipeutils-test_1.2.bb
++++ b/meta-selftest/recipes-test/recipeutils/recipeutils-test_1.2.bb
+@@ -8,6 +8,3 @@
+ 
+ BBCLASSEXTEND = "native nativesdk"
+ 
+-SRC_URI += "file://somefile"
+-
+-SRC_URI_append = " file://anotherfile"
+"""
+        patchlines = []
+        for f in patches:
+            for line in f:
+                patchlines.append(line)
+        self.assertEqual(''.join(patchlines).strip(), expected_patch.strip())
+
+
+    def test_validate_pn(self):
+        import oe.recipeutils
+        expected_results = {
+            'test': '',
+            'glib-2.0': '',
+            'gtk+': '',
+            'forcevariable': 'reserved',
+            'pn-something': 'reserved',
+            'test.bb': 'file',
+            'test_one': 'character',
+            'test!': 'character',
+        }
+
+        for pn, expected in expected_results.items():
+            result = oe.recipeutils.validate_pn(pn)
+            if expected:
+                self.assertIn(expected, result)
+            else:
+                self.assertEqual(result, '')
+
+    def test_split_var_value(self):
+        import oe.recipeutils
+        res = oe.recipeutils.split_var_value('test.1 test.2 ${@call_function("hi there world", false)} test.4')
+        self.assertEqual(res, ['test.1', 'test.2', '${@call_function("hi there world", false)}', 'test.4'])

Comments

Richard Purdie Nov. 29, 2018, 2:06 p.m.
On Thu, 2018-11-29 at 22:21 +1300, Paul Eggleton wrote:
> Add some tests for functions in meta/lib/oe/recipeutils.py, in
> particular for a few issues I've just fixed. I haven't added tests
> for
> all of the functions - some of them are already being tested via
> devtool
> in any case.

Hate to say it but this still isn't working:

https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/28/steps/7/logs/step2d

probably due to it not working with the -j option :/

Cheers,

Richard
Paul Eggleton Nov. 29, 2018, 7:35 p.m.
On Friday, 30 November 2018 3:06:32 AM NZDT Richard Purdie wrote:
> On Thu, 2018-11-29 at 22:21 +1300, Paul Eggleton wrote:
> > Add some tests for functions in meta/lib/oe/recipeutils.py, in
> > particular for a few issues I've just fixed. I haven't added tests
> > for
> > all of the functions - some of them are already being tested via
> > devtool
> > in any case.
> 
> Hate to say it but this still isn't working:
> 
> https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/28/steps/7/logs/step2d
> 
> probably due to it not working with the -j option :/

Right, so it seems that COREBASE isn't pointing where I'd expect it to 
here - instead of the copy of the core metadata that's made for the test, it's
pointing to the original layer. Is that a bug?

Cheers,
Paul
Richard Purdie Nov. 29, 2018, 11:09 p.m.
On Fri, 2018-11-30 at 08:35 +1300, Paul Eggleton wrote:
> On Friday, 30 November 2018 3:06:32 AM NZDT Richard Purdie wrote:
> > On Thu, 2018-11-29 at 22:21 +1300, Paul Eggleton wrote:
> > > Add some tests for functions in meta/lib/oe/recipeutils.py, in
> > > particular for a few issues I've just fixed. I haven't added
> > > tests
> > > for
> > > all of the functions - some of them are already being tested via
> > > devtool
> > > in any case.
> > 
> > Hate to say it but this still isn't working:
> > 
> > https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/28/steps/7/logs/step2d
> > 
> > probably due to it not working with the -j option :/
> 
> Right, so it seems that COREBASE isn't pointing where I'd expect it
> to 
> here - instead of the copy of the core metadata that's made for the
> test, it's
> pointing to the original layer. Is that a bug?

I put 

http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=rpurdie/t222&id=74b7df6ff73764e108bd81e05835c8a7d134c187

together to fix this. Its not a bug since the tests modify meta-
selftest and therefore when running in parallel, we create copies of
that layer per build directory.

I do think we should start using the separate build directory always
for oe-selftest for consistency though. It would also make ensuring
some things like rm_work disabled easier.

Cheers,

Richard
Paul Eggleton Nov. 29, 2018, 11:41 p.m.
On Friday, 30 November 2018 12:09:58 PM NZDT richard.purdie@linuxfoundation.org wrote:
> On Fri, 2018-11-30 at 08:35 +1300, Paul Eggleton wrote:
> > On Friday, 30 November 2018 3:06:32 AM NZDT Richard Purdie wrote:
> > > On Thu, 2018-11-29 at 22:21 +1300, Paul Eggleton wrote:
> > > > Add some tests for functions in meta/lib/oe/recipeutils.py, in
> > > > particular for a few issues I've just fixed. I haven't added
> > > > tests
> > > > for
> > > > all of the functions - some of them are already being tested via
> > > > devtool
> > > > in any case.
> > > 
> > > Hate to say it but this still isn't working:
> > > 
> > > https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/28/steps/7/logs/step2d
> > > 
> > > probably due to it not working with the -j option :/
> > 
> > Right, so it seems that COREBASE isn't pointing where I'd expect it
> > to 
> > here - instead of the copy of the core metadata that's made for the
> > test, it's
> > pointing to the original layer. Is that a bug?
> 
> I put 
> 
> http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=rpurdie/t222&id=74b7df6ff73764e108bd81e05835c8a7d134c187
> 
> together to fix this. Its not a bug since the tests modify meta-
> selftest and therefore when running in parallel, we create copies of
> that layer per build directory.

OK, I've sent a fix - I discussed with RP on IRC but for others' benefit
there's already a function in oeqa.utils.commands that gets the path
to meta-selftest from BBLAYERS so I used that in v4 that I just sent. 
I also tested this one with -j which I hadn't with the previous versions.

> I do think we should start using the separate build directory always
> for oe-selftest for consistency though. It would also make ensuring
> some things like rm_work disabled easier.

Probably since there are gotchas like this one.

Cheers,
Paul
Paul Eggleton Nov. 30, 2018, 10:38 a.m.
This is a system generated Comment: Patch 156728 was automatically marked as superseded by patch 156754.