From patchwork Wed Oct 4 12:57:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Yoann Congal X-Patchwork-Id: 31672 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5B485E7B610 for ; Wed, 4 Oct 2023 12:58:12 +0000 (UTC) Received: from mail-wr1-f41.google.com (mail-wr1-f41.google.com [209.85.221.41]) by mx.groups.io with SMTP id smtpd.web11.17463.1696424282051212132 for ; Wed, 04 Oct 2023 05:58:02 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@smile-fr.20230601.gappssmtp.com header.s=20230601 header.b=a6C602si; spf=pass (domain: smile.fr, ip: 209.85.221.41, mailfrom: yoann.congal@smile.fr) Received: by mail-wr1-f41.google.com with SMTP id ffacd0b85a97d-32336a30d18so1921581f8f.2 for ; Wed, 04 Oct 2023 05:58:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=smile-fr.20230601.gappssmtp.com; s=20230601; t=1696424280; x=1697029080; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=LL0lU1WxPb6eLOOZ3FlWP//ATeEGa+5zN82WXIslFwQ=; b=a6C602si6V3HF+Ed4hBglcbv4A29l2XVKrWz6hbQLSY+qF20CdoqDnvtd+lNV1OnFH PPpjhz/zXA/epZqRpleXtTfge5OdPiK++IovqmWTwtujCuahHapIb5H8zQP7ikFtkCYI VjD8F95hxIc6L0gsnA9JTDL4ug9Llz7i+zqrPldY1z1FcaWFGGEXI5rshnhDf5aKoAl+ VdMhBRsc+o2jkxZgodoAubewiEHmhiImMMjL+1lAuAayrtvtqFWUSUVv/9uDQSQzwb8A XZOe7b/bO4U1khktqsWuc3fspfenpjy7MOtaAKNQ1DcVsh2CuNw9x0ZwUGXncaylINY/ qsJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696424280; x=1697029080; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=LL0lU1WxPb6eLOOZ3FlWP//ATeEGa+5zN82WXIslFwQ=; b=uSFmrBsz4LyN/+jwDersgbSk0P+SxD6m6jyb5vngR9lpJfBcJGT0gCDF0TPiXTXZQN VfQOJDFhoHh0fjWyktqHZc1DAile6ATOrGYTaKg3PVWOHQwMLxIiXSIa2cr5/rVW8BBa KDZiPGZrlNupUClUaQ6+JJswE19LnCKHk7rr7c54ANMalXw/ui2T9W67mojRMUm+Ykp5 BnHIZS84UUpw7JKqzFOfKNhHcYk5stwbj2Ez4zqPlJh2t1t9Fmhf/jjlejnEki1SN/iH Gw5Gk3OcQAE8ivcFjLBD8PkAdadfRavoi7ST3ZgXLp0t/43RiuUNqO7yU4VCHwyFzio+ CeeQ== X-Gm-Message-State: AOJu0YzRw5g8PlThoFGu0h2SIrTd0T0OOb3nnvU8ICC+rgeve4sq/h8t ujmHM2WIir1UQAkVHi40z0oZqlgWnB0FFK11Qdk= X-Google-Smtp-Source: AGHT+IGKs1kosZGSOHof0VRJzSFrjWZQ32FRKORNrzAQ8gHgQBxGIjS4xBya5sHV56GLiSVNqQqiXw== X-Received: by 2002:adf:f44e:0:b0:324:84cd:5e67 with SMTP id f14-20020adff44e000000b0032484cd5e67mr1863920wrp.6.1696424280165; Wed, 04 Oct 2023 05:58:00 -0700 (PDT) Received: from P-ASN-ECS-830T8C3.idf.intranet (static-css-ccs-204145.business.bouyguestelecom.com. [176.157.204.145]) by smtp.gmail.com with ESMTPSA id j16-20020a056000125000b00326f5d0ce0asm3958763wrx.21.2023.10.04.05.57.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Oct 2023 05:57:59 -0700 (PDT) From: Yoann Congal To: openembedded-core@lists.openembedded.org Cc: =?utf-8?b?SsOpcsOpbXkgUm9zZW4=?= , Yoann Congal Subject: [PATCH v2 1/4] insane: Add unimplemented-ptest infrastructure Date: Wed, 4 Oct 2023 14:57:28 +0200 Message-Id: <20231004125731.1785803-2-yoann.congal@smile.fr> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20231004125731.1785803-1-yoann.congal@smile.fr> References: <20231004125731.1785803-1-yoann.congal@smile.fr> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 04 Oct 2023 12:58:12 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/188681 From: Jérémy Rosen This infrastructure will use heuristics to detect when package sources seem to have unit tests implemented but no ptest have been implemented in the recipe. No heuristics have been implemented at this point, only the infrastructure to skip the test when ptest are implemented. This is part of python_do_qa_patch since we need the sources in their final state but do not need any configuration done A missing-ptest QA test already existed but it was used for a different purpose and overridden by ptest.bbclass. Thus, a new QA keyword was added Note: The QA test is not enabled by default and may be enabled to hunt down potential ptests with: WARN_QA += "unimplemented-ptest" Signed-off-by: Jérémy Rosen Reviewed-by: Yoann Congal --- meta/classes-global/insane.bbclass | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/meta/classes-global/insane.bbclass b/meta/classes-global/insane.bbclass index 5743d91240..857f0a3736 100644 --- a/meta/classes-global/insane.bbclass +++ b/meta/classes-global/insane.bbclass @@ -1348,6 +1348,18 @@ python do_qa_patch() { if msg: oe.qa.handle_error("patch-status", msg, d) + ########################################################################### + # Check for missing ptests + ########################################################################### + srcdir = d.getVar('S') + if not bb.utils.contains('DISTRO_FEATURES', 'ptest', True, False, d): + pass + elif any(override in ["class-native", "class-nativesdk", "class-cross-canadian"] for override in d.getVar('OVERRIDES').split(':')): + # These cases are forcibly disabled in ptest.bbclass, so don't report missing ptest for them + bb.note("Package %s QA: skipping unimplemented-ptest: ptests disabled for this recipe class" % d.getVar('PN')) + elif bb.data.inherits_class('ptest', d): + bb.note("Package %s QA: skipping unimplemented-ptest: ptest implementation detected" % d.getVar('PN')) + oe.qa.exit_if_errors(d) } From patchwork Wed Oct 4 12:57:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Yoann Congal X-Patchwork-Id: 31674 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5E54DE7B60F for ; Wed, 4 Oct 2023 12:58:12 +0000 (UTC) Received: from mail-wr1-f45.google.com (mail-wr1-f45.google.com [209.85.221.45]) by mx.groups.io with SMTP id smtpd.web11.17464.1696424282405583209 for ; Wed, 04 Oct 2023 05:58:02 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@smile-fr.20230601.gappssmtp.com header.s=20230601 header.b=v0Np60yM; spf=pass (domain: smile.fr, ip: 209.85.221.45, mailfrom: yoann.congal@smile.fr) Received: by mail-wr1-f45.google.com with SMTP id ffacd0b85a97d-3247cefa13aso1914181f8f.1 for ; Wed, 04 Oct 2023 05:58:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=smile-fr.20230601.gappssmtp.com; s=20230601; t=1696424280; x=1697029080; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=+c6VtHlw5Mij4l7bSz5CW/psRuWL8yZUcKZN0yjmpRY=; b=v0Np60yMAb/Cbknuz+yG0lzigTt+z5hYYdHKxlXie6qXBDh19VGRt6edbf+eoe+g2l o0T3mv7MVp17s4dUdfu4X2IMBtiRePZ3+MzLv/1OGU7ia0a3dhnE/ctt6fZxCSaxOAT+ EcHs0uIUW83cE+HbyS7yU6mbAeS/tJZPTOMFne14snyJAxwKOFrWyKJKGG6XbQR270YO 9/eJnOr8cq/6J/Jh+6bIyuwSuzkIBLnwOs33GQsLs0R7Y0BQL+IrDUJ0gQIj5KcTembg Rb87WhLo8Kikh7djyY2HDIyfjtdtOgEU8A+adI14z07nGEVLLcOWvx/dqnuXM52uaFQd XJRQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696424280; x=1697029080; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=+c6VtHlw5Mij4l7bSz5CW/psRuWL8yZUcKZN0yjmpRY=; b=CGZYVLhyI+1JSeqdxv8n1VLw0eH6I69kGJGP0mrcGDNzTg+rtlB32kczilwAUrTZac Zio7/FZIqCbVRbjI4cnYTN5NgKGRvNmti8oDABTUuj2Ng4Q/IrO3KMfGAuYJQXIYh9TQ Aen6Fpxb8z9ASBYW+cF/J0EpWAy26tkIFpJx8WZ1TuVc+5HAjCYlxlW96ca/Swf3Tgdi /obeMG4yS3gIf79Vil5Z9SCm7ryiUdQNSUKdFQy2lAoLZxY9PLoEnYtb07LuD/HBnzde v4OAJdYoIPvmghB5HmGHSDO2FIUyV2ZqJslyozfiFXgJ2/+W926CYRX+s8vzrATx4KLK 9D+A== X-Gm-Message-State: AOJu0YxDxjH+fzYBt+exRYThbXPRsb16bVojHfT+qaFegWDSQj+vn8jY 2dR3+l325HTiZOtda6Rf6OgHXJWyu93u8aLMffQ= X-Google-Smtp-Source: AGHT+IGQwC1ztvL1AI+mV6U+WJpLJmgFRAGRceWYXu11zIcs+Ald5lHgscF/ClK6fcq2WGpHI1SOUw== X-Received: by 2002:adf:ea4c:0:b0:320:bae:2dfd with SMTP id j12-20020adfea4c000000b003200bae2dfdmr2133476wrn.5.1696424280598; Wed, 04 Oct 2023 05:58:00 -0700 (PDT) Received: from P-ASN-ECS-830T8C3.idf.intranet (static-css-ccs-204145.business.bouyguestelecom.com. [176.157.204.145]) by smtp.gmail.com with ESMTPSA id j16-20020a056000125000b00326f5d0ce0asm3958763wrx.21.2023.10.04.05.58.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Oct 2023 05:58:00 -0700 (PDT) From: Yoann Congal To: openembedded-core@lists.openembedded.org Cc: =?utf-8?b?SsOpcsOpbXkgUm9zZW4=?= , Yoann Congal Subject: [PATCH v2 2/4] insane: Detect python and perl based tests Date: Wed, 4 Oct 2023 14:57:29 +0200 Message-Id: <20231004125731.1785803-3-yoann.congal@smile.fr> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20231004125731.1785803-1-yoann.congal@smile.fr> References: <20231004125731.1785803-1-yoann.congal@smile.fr> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 04 Oct 2023 12:58:12 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/188682 From: Jérémy Rosen match_line_in_files will look for a regex in all files matching a glob. we use iglob to avoid a complete, recursive scan of all source. iglob is based on python iterators and will scan as we walk through the directories pytest are detected by looking for "import pytest" or "from pytest" in any python file. perl Test:: is detetected by looking for any t/*.t in the toplevel source directory. Signed-off-by: Jérémy Rosen Reviewed-by: Yoann Congal --- meta/classes-global/insane.bbclass | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/meta/classes-global/insane.bbclass b/meta/classes-global/insane.bbclass index 857f0a3736..932391e914 100644 --- a/meta/classes-global/insane.bbclass +++ b/meta/classes-global/insane.bbclass @@ -1351,6 +1351,20 @@ python do_qa_patch() { ########################################################################### # Check for missing ptests ########################################################################### + def match_line_in_files(toplevel, filename_glob, line_regex): + import pathlib + toppath = pathlib.Path(toplevel) + for entry in toppath.glob(filename_glob): + try: + with open(entry, 'r', encoding='utf-8', errors='ignore') as f: + for line in f.readlines(): + if re.match(line_regex, line): + return True + except FileNotFoundError: + # Broken symlink in source + pass + return False + srcdir = d.getVar('S') if not bb.utils.contains('DISTRO_FEATURES', 'ptest', True, False, d): pass @@ -1360,6 +1374,14 @@ python do_qa_patch() { elif bb.data.inherits_class('ptest', d): bb.note("Package %s QA: skipping unimplemented-ptest: ptest implementation detected" % d.getVar('PN')) + # Detect perl Test:: based tests + elif os.path.exists(os.path.join(srcdir, "t")) and any(filename.endswith('.t') for filename in os.listdir(os.path.join(srcdir, 't'))): + oe.qa.handle_error("unimplemented-ptest", "%s: perl Test:: based tests detected" % d.getVar('PN'), d) + + # Detect pytest-based tests + elif match_line_in_files(srcdir, "**/*.py", r'\s*(?:import\s*pytest|from\s*pytest)'): + oe.qa.handle_error("unimplemented-ptest", "%s: pytest-based tests detected" % d.getVar('PN'), d) + oe.qa.exit_if_errors(d) } From patchwork Wed Oct 4 12:57:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Yoann Congal X-Patchwork-Id: 31673 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 75280E7B611 for ; Wed, 4 Oct 2023 12:58:12 +0000 (UTC) Received: from mail-wr1-f51.google.com (mail-wr1-f51.google.com [209.85.221.51]) by mx.groups.io with SMTP id smtpd.web10.17576.1696424282791226998 for ; Wed, 04 Oct 2023 05:58:03 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@smile-fr.20230601.gappssmtp.com header.s=20230601 header.b=xubOp09V; spf=pass (domain: smile.fr, ip: 209.85.221.51, mailfrom: yoann.congal@smile.fr) Received: by mail-wr1-f51.google.com with SMTP id ffacd0b85a97d-3231d67aff2so2071909f8f.0 for ; Wed, 04 Oct 2023 05:58:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=smile-fr.20230601.gappssmtp.com; s=20230601; t=1696424281; x=1697029081; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=2hyGAY7xZd8bViT2YpRlnR8oBDo1ueBu+qDmwhylHSM=; b=xubOp09V/DEVucS/Y9miMxoBrzMNJqIoGLF5qa0G1/dzfKzYsh3v6Wdsca5wTGlFxU JKlbFMvWZkjNkScS4udkUobeTPt/+uIcZVLSWhZNRc0/O7yntu/0LZSbJK/ORY/WK2s8 Zmgy5xeENc80hjRiddDao2GEfbnkDmsHeJSK9401r8O1jrlGCM7555pxL5YWQJLuAdfF SkKWorlgfv4jULkEeRKl3Lk7qbBxriaRP6/7azfoNW4ADQzdJo8bnbQQK8QC0STfpxPv 0PrChVVyKMiaANsOf1NPoFIQQS2X21Me6K+MUunAk7nUn0ZkybxBp91ysc1b0M8co5Ss RfJw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696424281; x=1697029081; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2hyGAY7xZd8bViT2YpRlnR8oBDo1ueBu+qDmwhylHSM=; b=f6TqGOUOt5X3TNL2xvLCWFZyAmiqdHAzfT9YYU02eH+iRfgK4mR3jC9TBgapGnShdd /PthCmK4/qzVgGI7MMB+bOJVe/uEwmahpFO2OrRhoCymyqkF+XgSZlVuLEj8+JK+AjAV dmHokY6vmQ2mnHIpk7MzfcbCcOTHdJcOOoehuN7I5+1LrTSaxY6Ew+pcBTmzOMp/96M3 n7vO2GODiw3stVhHgcqxLxz2bOj3SBQBTEILusXmmWtZFj48nanDXEmHjPt+4uRv4xxj iY0xfF9LgY3sCf2CscQo/vdLGEZ6ELa9l6AICAI/umn9upFPV8tLjZItrId7cOnbyJlZ vGiw== X-Gm-Message-State: AOJu0YyWdCOHZ1eO+wuVr2Fi7pp55FCXC4O5wQXj3yu3maMeprcqW90L hwp43rOhTmMMM+dVvOENf0Tx8VZavAj7RIVzIEI= X-Google-Smtp-Source: AGHT+IHc5ZjlDqK220T8oLSFqnqswFpWUh2BY99XAlcI92jlkC1ILtCfH6MEa+keieVeiL4iaj6FPw== X-Received: by 2002:a5d:58ed:0:b0:31f:e026:30ac with SMTP id f13-20020a5d58ed000000b0031fe02630acmr1981368wrd.34.1696424280943; Wed, 04 Oct 2023 05:58:00 -0700 (PDT) Received: from P-ASN-ECS-830T8C3.idf.intranet (static-css-ccs-204145.business.bouyguestelecom.com. [176.157.204.145]) by smtp.gmail.com with ESMTPSA id j16-20020a056000125000b00326f5d0ce0asm3958763wrx.21.2023.10.04.05.58.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Oct 2023 05:58:00 -0700 (PDT) From: Yoann Congal To: openembedded-core@lists.openembedded.org Cc: =?utf-8?b?SsOpcsOpbXkgUm9zZW4=?= , Yoann Congal Subject: [PATCH v2 3/4] insane: Detect build-system test harnesses Date: Wed, 4 Oct 2023 14:57:30 +0200 Message-Id: <20231004125731.1785803-4-yoann.congal@smile.fr> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20231004125731.1785803-1-yoann.congal@smile.fr> References: <20231004125731.1785803-1-yoann.congal@smile.fr> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 04 Oct 2023 12:58:12 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/188683 From: Jérémy Rosen most build-systems have a way to implement a "make check" or equivalent command that will run tests in the source directory. This heuristic will detect the keywords in the build-system configuration that activates tests. Note that in the case of autotools, we use Makefile.in as our source instead of Makefile.am to easily follow includes. Filenaming conventions for autotools files are not reliable enough to do naive matching. Signed-off-by: Jérémy Rosen Reviewed-by: Yoann Congal --- meta/classes-global/insane.bbclass | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/meta/classes-global/insane.bbclass b/meta/classes-global/insane.bbclass index 932391e914..35b6408b57 100644 --- a/meta/classes-global/insane.bbclass +++ b/meta/classes-global/insane.bbclass @@ -1382,6 +1382,18 @@ python do_qa_patch() { elif match_line_in_files(srcdir, "**/*.py", r'\s*(?:import\s*pytest|from\s*pytest)'): oe.qa.handle_error("unimplemented-ptest", "%s: pytest-based tests detected" % d.getVar('PN'), d) + # Detect meson-based tests + elif os.path.exists(os.path.join(srcdir, "meson.build")) and match_line_in_files(srcdir, "**/meson.build", r'\s*test\s*\('): + oe.qa.handle_error("unimplemented-ptest", "%s: meson-based tests detected" % d.getVar('PN'), d) + + # Detect cmake-based tests + elif os.path.exists(os.path.join(srcdir, "CMakelists.txt")) and match_line_in_files(srcdir, "**/CMakeLists.txt", r'\s*(?:add_test|enable_testing)\s*\('): + oe.qa.handle_error("unimplemented-ptest", "%s: cmake-based tests detected" % d.getVar('PN'), d) + + # Detect autotools-based·tests + elif os.path.exists(os.path.join(srcdir, "Makefile.in")) and (match_line_in_files(srcdir, "**/Makefile.in", r'\s*TESTS\s*\+?=') or match_line_in_files(srcdir,"**/*.at",r'.*AT_INIT')): + oe.qa.handle_error("unimplemented-ptest", "%s: autotools-based tests detected" % d.getVar('PN'), d) + oe.qa.exit_if_errors(d) } From patchwork Wed Oct 4 12:57:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Yoann Congal X-Patchwork-Id: 31675 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7557BE7B615 for ; Wed, 4 Oct 2023 12:58:12 +0000 (UTC) Received: from mail-wr1-f41.google.com (mail-wr1-f41.google.com [209.85.221.41]) by mx.groups.io with SMTP id smtpd.web11.17466.1696424283125957698 for ; Wed, 04 Oct 2023 05:58:03 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@smile-fr.20230601.gappssmtp.com header.s=20230601 header.b=BuJ6V5rQ; spf=pass (domain: smile.fr, ip: 209.85.221.41, mailfrom: yoann.congal@smile.fr) Received: by mail-wr1-f41.google.com with SMTP id ffacd0b85a97d-3214cdb4b27so2068940f8f.1 for ; Wed, 04 Oct 2023 05:58:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=smile-fr.20230601.gappssmtp.com; s=20230601; t=1696424281; x=1697029081; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=P8vLO8UcNl6/CIVv6Ew7H0M4nnSmZ3I3HCqzEYBLF4c=; b=BuJ6V5rQehT55ikjSz7cuK+YAHTkj5IPjFw7eHNXjBjQtxlMQ4Hv0+Dv/J175zrDYs b7NBylLfbfagQj4UQegwAJj9Y2buo9C9CH7Jcpp1cqovziJsvZofxd11AeT/QzLQ9/+s ronltNZvTLcu6RzkBOcIBebAlDZSua6Mqwls2eSloZ7t5/05TLHNjVR+WMwBBhRYep7+ 60j7vW64IRx+Cw5HKgNATyWYglXIddtQWxWHthTw7GrNv7nCb1/aKPkjkBPgW7VYZSi6 /BlhN/FbaVYJCDyAaTyyUjJidPw1Ea9nD3TZNAUxqxkwUzXVAMoMCmRt7N3auFdiKTkf ndww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696424281; x=1697029081; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=P8vLO8UcNl6/CIVv6Ew7H0M4nnSmZ3I3HCqzEYBLF4c=; b=jnd8XTPhhb18hFR4NqGEppx0QVzUsOwmgolid3rh4aXqSdwSTenwcl6HjfBpo2jD6Q 7+9Wv5p5pE2rC5pGY3/5aYs9S4Y3fc75ihn0eKIVTw+pAdVlIIsNIocD8DmjNo+mjks7 IiHZSyyZ3EMqeycOjMvNyj6naOOUed6/A6XN8LTrodIMqIml4Yh0vnXunSRxalleBzPr yiaau0rmXPCNyKaBvp3qg/HikPd2CCC4555VtY1llDyViPfY1N4Z39Kxr3RTcSqjW4uN Um2AwL5c07CWMCPg92kIM6CXZwTtg7yb/2Q8SZhrf1Gs3OOXC9AveHwQitqnLs3oKXhT WyQA== X-Gm-Message-State: AOJu0Yzh2Q0W/IhkcULSSfXWpHM5ufQB/IR0ZagkmXm0YUrgCW3KVdIK urLRrFcJ5kLFjb2C+LBBq4j1tnzCdsz3XbfmHnc= X-Google-Smtp-Source: AGHT+IGWJG++IBkzhMXtQtIyM8cr2+mpRUYGMIqbCnhAadysK2qXQ5bpAyTL3yn+RLsbZHwMRkU6EA== X-Received: by 2002:a5d:5187:0:b0:31f:ec06:20f with SMTP id k7-20020a5d5187000000b0031fec06020fmr2220893wrv.14.1696424281428; Wed, 04 Oct 2023 05:58:01 -0700 (PDT) Received: from P-ASN-ECS-830T8C3.idf.intranet (static-css-ccs-204145.business.bouyguestelecom.com. [176.157.204.145]) by smtp.gmail.com with ESMTPSA id j16-20020a056000125000b00326f5d0ce0asm3958763wrx.21.2023.10.04.05.58.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Oct 2023 05:58:01 -0700 (PDT) From: Yoann Congal To: openembedded-core@lists.openembedded.org Cc: =?utf-8?b?SsOpcsOpbXkgUm9zZW4=?= , Yoann Congal Subject: [PATCH v2 4/4] insane: Add a naive heuristic to detect test subdirectories Date: Wed, 4 Oct 2023 14:57:31 +0200 Message-Id: <20231004125731.1785803-5-yoann.congal@smile.fr> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20231004125731.1785803-1-yoann.congal@smile.fr> References: <20231004125731.1785803-1-yoann.congal@smile.fr> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Wed, 04 Oct 2023 12:58:12 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/188684 From: Jérémy Rosen if there is a "test" or "tests" subdirectory at toplevel, this usually means we have some unit tests available. This test is very good at detecting handcrafted tests and I was not able to find any false positive. False positive can be dealt with the usual INSANE_SKIP mechanism Signed-off-by: Jérémy Rosen Reviewed-by: Yoann Congal --- meta/classes-global/insane.bbclass | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/meta/classes-global/insane.bbclass b/meta/classes-global/insane.bbclass index 35b6408b57..ffae4b98da 100644 --- a/meta/classes-global/insane.bbclass +++ b/meta/classes-global/insane.bbclass @@ -1394,6 +1394,10 @@ python do_qa_patch() { elif os.path.exists(os.path.join(srcdir, "Makefile.in")) and (match_line_in_files(srcdir, "**/Makefile.in", r'\s*TESTS\s*\+?=') or match_line_in_files(srcdir,"**/*.at",r'.*AT_INIT')): oe.qa.handle_error("unimplemented-ptest", "%s: autotools-based tests detected" % d.getVar('PN'), d) + # Last resort, detect a test directory in sources + elif any(filename.lower() in ["test", "tests"] for filename in os.listdir(srcdir)): + oe.qa.handle_error("unimplemented-ptest", "%s: test subdirectory detected" % d.getVar('PN'), d) + oe.qa.exit_if_errors(d) }