mbox series

[0/4] QA, ptest: Add unimplemented-ptest checks

Message ID 20230929220503.3169745-1-yoann.congal@smile.fr
Headers show
Series QA, ptest: Add unimplemented-ptest checks | expand

Message

Yoann Congal Sept. 29, 2023, 10:04 p.m. UTC 
To increase ptest coverage we can check if the sources of a recipe looks like
it contains unittest and warn the user that a test may be implemented there.

This series provide the check infrastructure as a package QA check and some checks for :
python pytest, perl Test::, meson, cmake, autotools-based tests... as well as
the naive check of "Is there a test/ directory in the sources?" which work
surprisingly well.

Jérémy Rosen (4):
  QA, ptest: Add unimplemented-ptest infrastructure
  QA, ptest: Detect python and perl based tests
  QA, ptest: Detect build-system test harnesses
  QA, ptest: Add a naive heuristic to detect test subdirectories

 meta/classes-global/insane.bbclass | 54 ++++++++++++++++++++++++++++++
 1 file changed, 54 insertions(+)

Comments

Yoann Congal Oct. 3, 2023, 1:32 p.m. UTC | #1 
Hi,

Le sam. 30 sept. 2023 à 00:05, Yoann Congal <yoann.congal@smile.fr> a
écrit :

> To increase ptest coverage we can check if the sources of a recipe looks
> like
> it contains unittest and warn the user that a test may be implemented
> there.
>
> This series provide the check infrastructure as a package QA check and
> some checks for :
> python pytest, perl Test::, meson, cmake, autotools-based tests... as well
> as
> the naive check of "Is there a test/ directory in the sources?" which work
> surprisingly well.
>
> Jérémy Rosen (4):
>   QA, ptest: Add unimplemented-ptest infrastructure
>   QA, ptest: Detect python and perl based tests
>   QA, ptest: Detect build-system test harnesses
>   QA, ptest: Add a naive heuristic to detect test subdirectories
>
>  meta/classes-global/insane.bbclass | 54 ++++++++++++++++++++++++++++++
>  1 file changed, 54 insertions(+)
>

Following on IRC discussion and some more internal testing :I will send a
v2 with
* correct shortlog commit tag (s/QA, ptest/insane/)
* a fix for source package containing symlink loops (e.g. md5deep)

Currently, this check find
* 309 unimplemented ptest in oe-core/meta-poky/meta-yocto-bsp
* 827 unimplemented ptest in meta-openembedded
Full list :
https://gist.githubusercontent.com/ycongal-smile/0969e92aec1ca37e61b6cdbcaf1f6885/raw/b1be8a5b576e144b6563f5481d31b32de9b22368/gistfile1.txt

Regards,