From patchwork Fri Sep 29 22:05:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Yoann Congal X-Patchwork-Id: 31437 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5B640E77343 for ; Fri, 29 Sep 2023 22:05:59 +0000 (UTC) Received: from mail-wm1-f50.google.com (mail-wm1-f50.google.com [209.85.128.50]) by mx.groups.io with SMTP id smtpd.web10.30526.1696025154513478025 for ; Fri, 29 Sep 2023 15:05:55 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@smile-fr.20230601.gappssmtp.com header.s=20230601 header.b=moK0QQeW; spf=pass (domain: smile.fr, ip: 209.85.128.50, mailfrom: yoann.congal@smile.fr) Received: by mail-wm1-f50.google.com with SMTP id 5b1f17b1804b1-4053c6f1087so139332695e9.0 for ; Fri, 29 Sep 2023 15:05:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=smile-fr.20230601.gappssmtp.com; s=20230601; t=1696025152; x=1696629952; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=LL0lU1WxPb6eLOOZ3FlWP//ATeEGa+5zN82WXIslFwQ=; b=moK0QQeWrE2iNt5+ni7qivp8r/Qi/BL3A0ulP+t3VlT4EVDy1c8Gb/h1elQ+5WetM8 D2rxhKaWTiF4p9T7VLFl5lCXXsin1UmtXBDIaojAAMSBeuHbRxe0m5Vo5KDcr5Obs2Lf zF5qveUFVyM6J1FPnR2GkrpU287V9T6ZKDq7kcpwQS765fa6Bgxd1l0BcRHQPNuDRvg6 1U9f9CsC4pB1SVx9eEucDA7KLi9tGJMCwUkPNOSqa3NOKArMYshnDQiIlJ5jKwA5e5dN IcKKz6Obs6l0OhiLDQlpN9l4QjrdRz04rHolF3yx//Dph0Z2HSW7A7PgMVo7vMzDZTyl iVOQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696025152; x=1696629952; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=LL0lU1WxPb6eLOOZ3FlWP//ATeEGa+5zN82WXIslFwQ=; b=tMdnTaDbNKbc6wmgDInSB4ZlHu0vcGMLVdev3R/Wqd+G+SNpOX6D1/q2jNb1fYdB8H /3u81mEuOCb0J+kEPf2n6qYvjfxrS4ccofl3rI5rbA8DQZQQr03xLmWDxPzjhov5zwbX JbCdHWdzeyYMHu7CTe6OYUtNEt+Qd7aNFrCps+qRfn9a6lRDdQhnteDL1SNy9wCoEB5R MQgQXXfgP/wFprmjZfKpRRHxP3cNr+VHC1p24Fi7BPoSGEwRNionWMnZfkl6Knok8EAy Gd7oYr2YwQwOAA3sbqTM4Z/PHGerpSmQjs3fgFTqm6S64l7HaZU1muKSMJiOVDX5Zle9 GjWQ== X-Gm-Message-State: AOJu0YzQmOuy6ZZGoTShE4w+ORhM6jYlHfN5hvUP4LUNpHMA73aXr+XI c8EjLwopaDEZFn4YsAmOnbguZeswfaFflA1U7os= X-Google-Smtp-Source: AGHT+IG7Ac/e1z4DwJLmeFH0vNPJeU/hi7mEQVUinrZE/Q2aPJp6y/WEBKAwvcsNZqPGkaV55kDrpQ== X-Received: by 2002:a05:600c:3785:b0:405:4daa:6e41 with SMTP id o5-20020a05600c378500b004054daa6e41mr4738068wmr.13.1696025152493; Fri, 29 Sep 2023 15:05:52 -0700 (PDT) Received: from P-ASN-ECS-830T8C3.numericable.fr ([89.159.1.53]) by smtp.gmail.com with ESMTPSA id t15-20020a1c770f000000b00406408dc788sm2210046wmi.44.2023.09.29.15.05.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 Sep 2023 15:05:52 -0700 (PDT) From: Yoann Congal To: openembedded-core@lists.openembedded.org Cc: =?utf-8?b?SsOpcsOpbXkgUm9zZW4=?= , Yoann Congal Subject: [PATCH 1/4] QA, ptest: Add unimplemented-ptest infrastructure Date: Sat, 30 Sep 2023 00:05:00 +0200 Message-Id: <20230929220503.3169745-2-yoann.congal@smile.fr> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20230929220503.3169745-1-yoann.congal@smile.fr> References: <20230929220503.3169745-1-yoann.congal@smile.fr> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Fri, 29 Sep 2023 22:05:59 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/188449 From: Jérémy Rosen This infrastructure will use heuristics to detect when package sources seem to have unit tests implemented but no ptest have been implemented in the recipe. No heuristics have been implemented at this point, only the infrastructure to skip the test when ptest are implemented. This is part of python_do_qa_patch since we need the sources in their final state but do not need any configuration done A missing-ptest QA test already existed but it was used for a different purpose and overridden by ptest.bbclass. Thus, a new QA keyword was added Note: The QA test is not enabled by default and may be enabled to hunt down potential ptests with: WARN_QA += "unimplemented-ptest" Signed-off-by: Jérémy Rosen Reviewed-by: Yoann Congal --- meta/classes-global/insane.bbclass | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/meta/classes-global/insane.bbclass b/meta/classes-global/insane.bbclass index 5743d91240..857f0a3736 100644 --- a/meta/classes-global/insane.bbclass +++ b/meta/classes-global/insane.bbclass @@ -1348,6 +1348,18 @@ python do_qa_patch() { if msg: oe.qa.handle_error("patch-status", msg, d) + ########################################################################### + # Check for missing ptests + ########################################################################### + srcdir = d.getVar('S') + if not bb.utils.contains('DISTRO_FEATURES', 'ptest', True, False, d): + pass + elif any(override in ["class-native", "class-nativesdk", "class-cross-canadian"] for override in d.getVar('OVERRIDES').split(':')): + # These cases are forcibly disabled in ptest.bbclass, so don't report missing ptest for them + bb.note("Package %s QA: skipping unimplemented-ptest: ptests disabled for this recipe class" % d.getVar('PN')) + elif bb.data.inherits_class('ptest', d): + bb.note("Package %s QA: skipping unimplemented-ptest: ptest implementation detected" % d.getVar('PN')) + oe.qa.exit_if_errors(d) } From patchwork Fri Sep 29 22:05:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Yoann Congal X-Patchwork-Id: 31438 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5C52AE77345 for ; Fri, 29 Sep 2023 22:05:59 +0000 (UTC) Received: from mail-wm1-f54.google.com (mail-wm1-f54.google.com [209.85.128.54]) by mx.groups.io with SMTP id smtpd.web10.30527.1696025154831709806 for ; Fri, 29 Sep 2023 15:05:55 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@smile-fr.20230601.gappssmtp.com header.s=20230601 header.b=HH5Pt7jb; spf=pass (domain: smile.fr, ip: 209.85.128.54, mailfrom: yoann.congal@smile.fr) Received: by mail-wm1-f54.google.com with SMTP id 5b1f17b1804b1-4053c6f1087so139332735e9.0 for ; Fri, 29 Sep 2023 15:05:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=smile-fr.20230601.gappssmtp.com; s=20230601; t=1696025153; x=1696629953; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=wqzpjSwB9eUjF1K+wF1XNA0sh6bOuAU6sxRtPlihoyE=; b=HH5Pt7jb9KGGH+5duH98K377Cp7/he0lNfNHmDmeJ+MZF12Im+iaCjbv6m3q74Ga1V dh54E/rMGOq7O5sVUONoEuvtXw3hdXhLx40/KhLTPz9OBb6pj0DRLCbmCOh53edQu51K vrlZZS/zJ+wNSl2ZM+ZvfBhPwRmJ80XlJWkB1CF2x/fJT75FinM63f338L4/9trDteev 26h0ddb9B2OKh5V68fGPKaqU/yTvMCODbRyIar6Eusw/jZbGWsEbHs8pBKKPX5Ry98aD hjLfcuezrpHCLFd/woH5qk8Fpz4JOtHqGYHRYxUDhPjt4PZUpYO5TQJXWYsZq4KxTNnp HdOw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696025153; x=1696629953; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wqzpjSwB9eUjF1K+wF1XNA0sh6bOuAU6sxRtPlihoyE=; b=GSHkPhhYLnb3hu80ZM3fHserIG3GSiVA5VV+ekIOUkOOEsx33vW+4aUbi9exvRsNUp GLHe/Mdj3gZtov+c9hKgRUs2KdAxbZTDaAKFzb5zEqMP4H/gJPqQgT1MQpWgZw403y3a /1u3Z113p4g+YgJHMuHO4gYVrDn9QYrjQ0QeGaBdrHYpn1n6Sy1gxeOKRv7MDcULI/Cv ETM+u45Yp136U9Ryz/IFtdTO5JnIB3w3S6zwtIoCX/7yn8gO+uo+40Z4id5BCmOJhoQq zeIeKoNvXIRzZiotSDBbeRq2aJfHnI0iGd9N59EWV1Jsz65SPlRMoBr9yVepRlpPTzwl yKiw== X-Gm-Message-State: AOJu0YyXvyhNl7R7Vy7bXIY/CvGSxOqndZQkSp3IjllmRwiLjRWBpW2T W17nDQUB5GDR6hdS1yp7zhZUKhqUV2MN6aYvWoM= X-Google-Smtp-Source: AGHT+IE9FdINVwZwQqMDDpLbJGNWMhuBQqJkgZF8RWjiHFH1TC3pjdHS7oyqR7+5VRyZxxYV7yR9Yw== X-Received: by 2002:a05:600c:290a:b0:405:7400:1e3d with SMTP id i10-20020a05600c290a00b0040574001e3dmr5098070wmd.32.1696025152979; Fri, 29 Sep 2023 15:05:52 -0700 (PDT) Received: from P-ASN-ECS-830T8C3.numericable.fr ([89.159.1.53]) by smtp.gmail.com with ESMTPSA id t15-20020a1c770f000000b00406408dc788sm2210046wmi.44.2023.09.29.15.05.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 Sep 2023 15:05:52 -0700 (PDT) From: Yoann Congal To: openembedded-core@lists.openembedded.org Cc: =?utf-8?b?SsOpcsOpbXkgUm9zZW4=?= , Yoann Congal Subject: [PATCH 2/4] QA, ptest: Detect python and perl based tests Date: Sat, 30 Sep 2023 00:05:01 +0200 Message-Id: <20230929220503.3169745-3-yoann.congal@smile.fr> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20230929220503.3169745-1-yoann.congal@smile.fr> References: <20230929220503.3169745-1-yoann.congal@smile.fr> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Fri, 29 Sep 2023 22:05:59 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/188450 From: Jérémy Rosen match_line_in_files will look for a regex in all files matching a glob. we use iglob to avoid a complete, recursive scan of all source. iglob is based on python iterators and will scan as we walk through the directories pytest are detected by looking for "import pytest" or "from pytest" in any python file. perl Test:: is detetected by looking for any t/*.t in the toplevel source directory. Signed-off-by: Jérémy Rosen Reviewed-by: Yoann Congal --- meta/classes-global/insane.bbclass | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/meta/classes-global/insane.bbclass b/meta/classes-global/insane.bbclass index 857f0a3736..31db707a33 100644 --- a/meta/classes-global/insane.bbclass +++ b/meta/classes-global/insane.bbclass @@ -1351,6 +1351,24 @@ python do_qa_patch() { ########################################################################### # Check for missing ptests ########################################################################### + def match_line_in_files(toplevel, filename_glob, line_regex): + import glob + # python 3.10 adds a root_dir parameter, but in 3.8 we must manually cwd + curdir = os.getcwd() + os.chdir(toplevel) + for entry in glob.iglob(filename_glob, recursive=True): + try: + with open(entry, 'r', encoding='utf-8', errors='ignore') as f: + for line in f.readlines(): + if re.match(line_regex, line): + os.chdir(curdir) + return True + except FileNotFoundError: + # Broken symlink in source + pass + os.chdir(curdir) + return False + srcdir = d.getVar('S') if not bb.utils.contains('DISTRO_FEATURES', 'ptest', True, False, d): pass @@ -1360,6 +1378,14 @@ python do_qa_patch() { elif bb.data.inherits_class('ptest', d): bb.note("Package %s QA: skipping unimplemented-ptest: ptest implementation detected" % d.getVar('PN')) + # Detect perl Test:: based tests + elif "t" in os.listdir(srcdir) and any(filename.endswith('.t') for filename in os.listdir(os.path.join(srcdir, 't'))): + oe.qa.handle_error("unimplemented-ptest", "%s: perl Test:: based tests detected" % d.getVar('PN'), d) + + # Detect pytest-based tests + elif match_line_in_files(srcdir, "**/*.py", r'\s*(?:import\s*pytest|from\s*pytest)'): + oe.qa.handle_error("unimplemented-ptest", "%s: pytest-based tests detected" % d.getVar('PN'), d) + oe.qa.exit_if_errors(d) } From patchwork Fri Sep 29 22:05:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Yoann Congal X-Patchwork-Id: 31439 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6F682E77346 for ; Fri, 29 Sep 2023 22:05:59 +0000 (UTC) Received: from mail-wm1-f48.google.com (mail-wm1-f48.google.com [209.85.128.48]) by mx.groups.io with SMTP id smtpd.web10.30528.1696025155755128092 for ; Fri, 29 Sep 2023 15:05:56 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@smile-fr.20230601.gappssmtp.com header.s=20230601 header.b=CEKapgef; spf=pass (domain: smile.fr, ip: 209.85.128.48, mailfrom: yoann.congal@smile.fr) Received: by mail-wm1-f48.google.com with SMTP id 5b1f17b1804b1-405524e6740so128258555e9.1 for ; Fri, 29 Sep 2023 15:05:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=smile-fr.20230601.gappssmtp.com; s=20230601; t=1696025154; x=1696629954; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=j8uh52poCWNj/CoIq9JwSqEDJnJwqrR5j0gPe20nLPU=; b=CEKapgefoGBL7D1T1bnmWdnAVWXSYoGsiMZZqJgH2O5Rf4l+4wwIbyE0LzZTWyCWwF ntNSQ/CKdYikyriUGQ44oAEvPOHWhAeBpeQtkx6aEDCyXISF2cOv66TYsxUwOPn9CZxq ZW5X54BCjUFn/M8NU2bA2TS61uYc78JvoEIrKu4ff40fRI5d09YUcyCf1ztF9Zkg40GE PxeoE4roe6REV87owB0snTR3K7OLqnRvCzEdC9kSgGEHsn2zUjKm6xv7NcwnNDcCNXsu TLYOVhcnRvz7608xt6QJR/HVbVsw4QvDTQSyRAe52gRz1u1g4CFl/GFwaiasHGhhhPjo /jEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696025154; x=1696629954; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=j8uh52poCWNj/CoIq9JwSqEDJnJwqrR5j0gPe20nLPU=; b=lr6tds4xjy+MS1XP1U94FjlmtNVmlwo3IQol7EP9ez8dSvSQuBuBxZdMW2UCaiF4yI k/kOCwjxBEUrsm1e1Y7GYtB9BQhdfos5DVdHrcR5bEUXL36taI4ifQKVa3iuegfGDLb3 gSmqqsOBmDEfaqnUWf+FzLmn8gjvphwCoXXoaesEaWaF/ZxKBBLR6rOFNk9We8uiRaVq eA4p42LX6IhoRcXsOaZ/ix2u5r+8RsgjCdoH1TVzW7Erctt9wAxzCa0ZQYUpWG4qyh4E qpyK2hoN1Ydz4NqOYLTaewoUH+5jeAAuLq5Lndd2gF1+hOTf3+k8HvcWv3hA17ndpmmt PWig== X-Gm-Message-State: AOJu0YywJnSSWM4b7/T3WYH6iL+qDmSY4KN5lf5L/7zAkcrPSMVO0Ev4 +ubN/oQ/sYrmhrbn0s+zdNoBQYkz9YCnUfxO6HI= X-Google-Smtp-Source: AGHT+IEfx2eTOqyLrqXzF7ElN/ravqML2cXSaR0uUYtiBJspZ5RgvCkTRJarr7OSm/iJ9wKx91M5KQ== X-Received: by 2002:a05:600c:2303:b0:405:dbe2:df10 with SMTP id 3-20020a05600c230300b00405dbe2df10mr4729662wmo.35.1696025153845; Fri, 29 Sep 2023 15:05:53 -0700 (PDT) Received: from P-ASN-ECS-830T8C3.numericable.fr ([89.159.1.53]) by smtp.gmail.com with ESMTPSA id t15-20020a1c770f000000b00406408dc788sm2210046wmi.44.2023.09.29.15.05.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 Sep 2023 15:05:53 -0700 (PDT) From: Yoann Congal To: openembedded-core@lists.openembedded.org Cc: =?utf-8?b?SsOpcsOpbXkgUm9zZW4=?= , Yoann Congal Subject: [PATCH 3/4] QA, ptest: Detect build-system test harnesses Date: Sat, 30 Sep 2023 00:05:02 +0200 Message-Id: <20230929220503.3169745-4-yoann.congal@smile.fr> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20230929220503.3169745-1-yoann.congal@smile.fr> References: <20230929220503.3169745-1-yoann.congal@smile.fr> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Fri, 29 Sep 2023 22:05:59 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/188451 From: Jérémy Rosen most build-systems have a way to implement a "make check" or equivalent command that will run tests in the source directory. This heuristic will detect the keywords in the build-system configuration that activates tests. Note that in the case of autotools, we use Makefile.in as our source instead of Makefile.am to easily follow includes. Filenaming conventions for autotools files are not reliable enough to do naive matching. Signed-off-by: Jérémy Rosen Reviewed-by: Yoann Congal --- meta/classes-global/insane.bbclass | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/meta/classes-global/insane.bbclass b/meta/classes-global/insane.bbclass index 31db707a33..a5d3793265 100644 --- a/meta/classes-global/insane.bbclass +++ b/meta/classes-global/insane.bbclass @@ -1386,6 +1386,18 @@ python do_qa_patch() { elif match_line_in_files(srcdir, "**/*.py", r'\s*(?:import\s*pytest|from\s*pytest)'): oe.qa.handle_error("unimplemented-ptest", "%s: pytest-based tests detected" % d.getVar('PN'), d) + # Detect meson-based tests + elif os.path.exists(os.path.join(srcdir, "meson.build")) and match_line_in_files(srcdir, "**/meson.build", r'\s*test\s*\('): + oe.qa.handle_error("unimplemented-ptest", "%s: meson-based tests detected" % d.getVar('PN'), d) + + # Detect cmake-based tests + elif os.path.exists(os.path.join(srcdir, "CMakelists.txt")) and match_line_in_files(srcdir, "**/CMakeLists.txt", r'\s*(?:add_test|enable_testing)\s*\('): + oe.qa.handle_error("unimplemented-ptest", "%s: cmake-based tests detected" % d.getVar('PN'), d) + + # Detect autotools-based·tests + elif os.path.exists(os.path.join(srcdir, "Makefile.in")) and (match_line_in_files(srcdir, "**/Makefile.in", r'\s*TESTS\s*\+?=') or match_line_in_files(srcdir,"**/*.at",r'.*AT_INIT')): + oe.qa.handle_error("unimplemented-ptest", "%s: autotools-based tests detected" % d.getVar('PN'), d) + oe.qa.exit_if_errors(d) } From patchwork Fri Sep 29 22:05:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Yoann Congal X-Patchwork-Id: 31440 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 86497E77348 for ; Fri, 29 Sep 2023 22:05:59 +0000 (UTC) Received: from mail-wm1-f53.google.com (mail-wm1-f53.google.com [209.85.128.53]) by mx.groups.io with SMTP id smtpd.web11.30366.1696025156033053673 for ; Fri, 29 Sep 2023 15:05:56 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@smile-fr.20230601.gappssmtp.com header.s=20230601 header.b=NudxzbPQ; spf=pass (domain: smile.fr, ip: 209.85.128.53, mailfrom: yoann.congal@smile.fr) Received: by mail-wm1-f53.google.com with SMTP id 5b1f17b1804b1-40572aeb6d0so112981125e9.1 for ; Fri, 29 Sep 2023 15:05:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=smile-fr.20230601.gappssmtp.com; s=20230601; t=1696025154; x=1696629954; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=a3r5yodj4daAJD1RoB817EoPPb8nFGz1GO7UoIlPYS8=; b=NudxzbPQu/P6mUpFRtycXxNvZjQRKMHqAU7j3xOZQLKYWr+jPy/KJ+Xb06kefgQ2Yu I6GBoZ0xqoCX+NDlvSQB6blFX0/hiaEomEhScVpEVMyU7Ng0jK1eUVDx2v5/2GKqtJCm EEKJ4LaDiuMLvy8HnHUtczvvDXQ8GqM0sUd8tLqAqWSCBJ7y/dmFJIizGqo/j3AqBbgF 1x/6sZwIRX6N2ewOWR+EXspsPDSPhGJD2ICuBLwTWKAVn7PoSx3cQqVmR0sPSxO1gWaW OhF/on5h3Y+aHeBKh3pGwMRa4Y0b3N7zJ5w4NhFAWDzmVSADiGzqznm1KNev9in+vFJG khUA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696025154; x=1696629954; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=a3r5yodj4daAJD1RoB817EoPPb8nFGz1GO7UoIlPYS8=; b=F/x/1mRpZ38FM3UzDMVET3p1B1uTU3ENF/dE3kY6EukGEqyS7Z5i6X/w+cY3RmdALr 78VcBbn/rAuLHnXkvLR9+9S/Wa4gjNn7RcqCvcNN/DO8VYI2ObCsOMBhsHcz/78RIBF/ y/Nk1KDwWaK8j0yruQQpSpVve4KIEBLYwGUtF4E4Q3e0fPIFnWMsZsKR+ChPlRQMkvGt DOcujDxRcZyeZKt6mKVxrW1qcUtTWc8/XhjSixpBh3NKx8sq0e9yWZie0kLJV4qS6Kry YRBWsxc6k0FsLQQMNXRJ+d2S5btcUgVU9OU7OJLa/X1nUSKQ2eMo6X7hjAm6LHx4cem3 oj0Q== X-Gm-Message-State: AOJu0YwIdQn7RYagElDNn6svcHQdoZQhouClLA2ht0TmLQkDdot6ATnC pPeKtWWa0jr03F+i3s0x57ai+bjvcfV8OjaBx6I= X-Google-Smtp-Source: AGHT+IHapKP+sLJCClxt0wJfnsKtAAIi+7BI3cy1SRgBOboMZH/PDVv38pxOhrwnAB1/yW6jVUI3vg== X-Received: by 2002:a05:600c:4750:b0:406:54e4:359c with SMTP id w16-20020a05600c475000b0040654e4359cmr5118630wmo.19.1696025154375; Fri, 29 Sep 2023 15:05:54 -0700 (PDT) Received: from P-ASN-ECS-830T8C3.numericable.fr ([89.159.1.53]) by smtp.gmail.com with ESMTPSA id t15-20020a1c770f000000b00406408dc788sm2210046wmi.44.2023.09.29.15.05.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 Sep 2023 15:05:54 -0700 (PDT) From: Yoann Congal To: openembedded-core@lists.openembedded.org Cc: =?utf-8?b?SsOpcsOpbXkgUm9zZW4=?= , Yoann Congal Subject: [PATCH 4/4] QA, ptest: Add a naive heuristic to detect test subdirectories Date: Sat, 30 Sep 2023 00:05:03 +0200 Message-Id: <20230929220503.3169745-5-yoann.congal@smile.fr> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20230929220503.3169745-1-yoann.congal@smile.fr> References: <20230929220503.3169745-1-yoann.congal@smile.fr> MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Fri, 29 Sep 2023 22:05:59 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/188452 From: Jérémy Rosen If there is a "test" or "tests" subdirectory at toplevel, this usually means we have some unit tests available. This test is very good at detecting handcrafted tests and I was not able to find any false positive. False positive can be dealt with the usual INSANE_SKIP mechanism Signed-off-by: Jérémy Rosen Reviewed-by: Yoann Congal --- meta/classes-global/insane.bbclass | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/meta/classes-global/insane.bbclass b/meta/classes-global/insane.bbclass index a5d3793265..ba12f3ea16 100644 --- a/meta/classes-global/insane.bbclass +++ b/meta/classes-global/insane.bbclass @@ -1398,6 +1398,10 @@ python do_qa_patch() { elif os.path.exists(os.path.join(srcdir, "Makefile.in")) and (match_line_in_files(srcdir, "**/Makefile.in", r'\s*TESTS\s*\+?=') or match_line_in_files(srcdir,"**/*.at",r'.*AT_INIT')): oe.qa.handle_error("unimplemented-ptest", "%s: autotools-based tests detected" % d.getVar('PN'), d) + # Last resort, detect a test directory in sources + elif any(filename.lower() in ["test", "tests"] for filename in os.listdir(srcdir)): + oe.qa.handle_error("unimplemented-ptest", "%s: test subdirectory detected" % d.getVar('PN'), d) + oe.qa.exit_if_errors(d) }