From patchwork Fri Dec 31 00:21:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Mittal, Anuj" X-Patchwork-Id: 1951 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id CF41DC433FE for ; Fri, 31 Dec 2021 00:22:02 +0000 (UTC) Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by mx.groups.io with SMTP id smtpd.web11.5667.1640910121693805729 for ; Thu, 30 Dec 2021 16:22:02 -0800 Authentication-Results: mx.groups.io; dkim=fail reason="unable to parse pub key" header.i=@intel.com header.s=intel header.b=m8D53jZD; spf=pass (domain: intel.com, ip: 134.134.136.100, mailfrom: anuj.mittal@intel.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1640910122; x=1672446122; h=from:to:subject:date:message-id:in-reply-to:references: mime-version:content-transfer-encoding; bh=WWt+vurzh0qbuwwTn3qMSSGQ8hl+cl5/Y60uIGXghEM=; b=m8D53jZDb0AItyQq9mxqqrUmVb5xoNAHWshekBdxjNoTlAd9XC6P9TmD 35DzZUSdznm00YVJ179Mqm+CWjtF4nU1IfLcQnJDyJcKRAAvIVlCU3+HH XvrYirNu5aWC3tKyS0gZuU/Lda5alB4SOISOM666Qgjy0qu27s41pmaYP q1M8+OW5OSodd3GKwdVzynm0Skp5/JMGRScd7TngeuegkSSdoJ0xWJoN+ E3jkxOMttYuLAZVB0FKOtWbIHe2DR+Hj+XH8lM9nHW6fGuWUh+StuNLJ/ VelFS+XxRkB64ij/jDcm176bOvWlesmmvofXjKMKJPKPVy5mEil9RDUL+ Q==; X-IronPort-AV: E=McAfee;i="6200,9189,10213"; a="305110472" X-IronPort-AV: E=Sophos;i="5.88,248,1635231600"; d="scan'208";a="305110472" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Dec 2021 16:22:02 -0800 X-IronPort-AV: E=Sophos;i="5.88,248,1635231600"; d="scan'208";a="468943764" Received: from jiayingk-mobl2.gar.corp.intel.com (HELO anmitta2-mobl3.intel.com) ([10.215.231.141]) by orsmga003-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Dec 2021 16:22:01 -0800 From: Anuj Mittal To: bitbake-devel@lists.openembedded.org Subject: [1.52][PATCH 1/4] process: Do not mix stderr with stdout Date: Fri, 31 Dec 2021 08:21:52 +0800 Message-Id: X-Mailer: git-send-email 2.33.1 In-Reply-To: References: MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Fri, 31 Dec 2021 00:22:02 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/13205 From: Anton Mikanovich We should not redirect stderr to stdout if we need to get separated stdout and stderr contents from Popen.communicate() later. Signed-off-by: Anton Mikanovich Signed-off-by: Richard Purdie (cherry picked from commit 1ecc1d9424877df89fcda2f23c306998998a65ff) Signed-off-by: Anuj Mittal --- lib/bb/process.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/bb/process.py b/lib/bb/process.py index d5a1775f..af5d804a 100644 --- a/lib/bb/process.py +++ b/lib/bb/process.py @@ -60,7 +60,7 @@ class Popen(subprocess.Popen): "close_fds": True, "preexec_fn": subprocess_setup, "stdout": subprocess.PIPE, - "stderr": subprocess.STDOUT, + "stderr": subprocess.PIPE, "stdin": subprocess.PIPE, "shell": False, } From patchwork Fri Dec 31 00:21:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Mittal, Anuj" X-Patchwork-Id: 1953 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id B4A42C433EF for ; Fri, 31 Dec 2021 00:22:04 +0000 (UTC) Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by mx.groups.io with SMTP id smtpd.web11.5667.1640910121693805729 for ; Thu, 30 Dec 2021 16:22:03 -0800 Authentication-Results: mx.groups.io; dkim=fail reason="unable to parse pub key" header.i=@intel.com header.s=intel header.b=B9vWeQGB; spf=pass (domain: intel.com, ip: 134.134.136.100, mailfrom: anuj.mittal@intel.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1640910123; x=1672446123; h=from:to:subject:date:message-id:in-reply-to:references: mime-version:content-transfer-encoding; bh=308V58jMGsDLIKKuaQqKpJV4o8XPsN5t6Ys1IUK34Rg=; b=B9vWeQGBvGS0mg35TqBEzeLPDtk85R0Eq+z5U8PmLxX+aIUiBmHQGoct pK79I8EjqdP3ri6yuU3jipm9wzUtYUMcRRZD5J1idqVfbaGrJ5Lvz72Rw nKn99kvrBuG2Eczd5/wvNBpU/d9+ZIz8tIhJdFBKPe30h55BUWPkh/7jd B/Km3AOsj4zY0nolLeCmgmZttDMxMIqAxFW+IM+R/yZ1Vc12fAqlKjlT0 hoh8fjqqnN39ZfCA/8KR1ekCD1+8m+8ubwByLu8B6pm6LrqeMqOaD82YW bZdwRp3t0sZP0T/Zt8Fldlg3AGqfgIRdtcNlPvGspM1FUxwfJzADzjwBW A==; X-IronPort-AV: E=McAfee;i="6200,9189,10213"; a="305110476" X-IronPort-AV: E=Sophos;i="5.88,248,1635231600"; d="scan'208";a="305110476" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Dec 2021 16:22:03 -0800 X-IronPort-AV: E=Sophos;i="5.88,248,1635231600"; d="scan'208";a="468943770" Received: from jiayingk-mobl2.gar.corp.intel.com (HELO anmitta2-mobl3.intel.com) ([10.215.231.141]) by orsmga003-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Dec 2021 16:22:02 -0800 From: Anuj Mittal To: bitbake-devel@lists.openembedded.org Subject: [1.52][PATCH 2/4] fetch: npm: Quote destdir in run chmod command Date: Fri, 31 Dec 2021 08:21:53 +0800 Message-Id: X-Mailer: git-send-email 2.33.1 In-Reply-To: References: MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Fri, 31 Dec 2021 00:22:04 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/13206 From: Stefan Herbrechtsmeier Quote destdir in run chmod command to support special characters in package name and to avoid syntax error for packages like '@(._.)/execute'. Signed-off-by: Stefan Herbrechtsmeier Signed-off-by: Richard Purdie (cherry picked from commit a701dfce3f0e74b4d7c687eeda83fe9c8e7240b1) Signed-off-by: Anuj Mittal --- lib/bb/fetch2/npm.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/bb/fetch2/npm.py b/lib/bb/fetch2/npm.py index e497c38d..d9daec20 100644 --- a/lib/bb/fetch2/npm.py +++ b/lib/bb/fetch2/npm.py @@ -72,7 +72,7 @@ def npm_unpack(tarball, destdir, d): cmd += " --delay-directory-restore" cmd += " --strip-components=1" runfetchcmd(cmd, d, workdir=destdir) - runfetchcmd("chmod -R +X %s" % (destdir), d, quiet=True, workdir=destdir) + runfetchcmd("chmod -R +X '%s'" % (destdir), d, quiet=True, workdir=destdir) class NpmEnvironment(object): """ From patchwork Fri Dec 31 00:21:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Mittal, Anuj" X-Patchwork-Id: 1954 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id B491DC433F5 for ; Fri, 31 Dec 2021 00:22:05 +0000 (UTC) Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by mx.groups.io with SMTP id smtpd.web11.5667.1640910121693805729 for ; Thu, 30 Dec 2021 16:22:05 -0800 Authentication-Results: mx.groups.io; dkim=fail reason="unable to parse pub key" header.i=@intel.com header.s=intel header.b=dAEpKu9s; spf=pass (domain: intel.com, ip: 134.134.136.100, mailfrom: anuj.mittal@intel.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1640910125; x=1672446125; h=from:to:subject:date:message-id:in-reply-to:references: mime-version:content-transfer-encoding; bh=E0N2XOuQX3+5/jIWoFEyLkS7UODsHqdV7Gmjodxg5uc=; b=dAEpKu9sB2k/pjhvbZtpJ4uB9ttyC1ceXhhC4mmxzChuImP6pFhAGYI2 EGAWnSSPOW1CX3RXFORBgxl9m8GS4AMT3dekPD9crGpjtcdjkEyO6TOVK CjKEEXLjlpCeopTk6EQiEFN5pXYiW6BiUZL3f4SjID153oMarQsAngcGs 5rH7M7q00vNByisFu6gxzCYK3DEXMYo/HSTtzxMduMzg85mX/79WOnl9K hZWH7PCM05fxP9XxSqx0IXezwRjdhI0i7DVCPbMEcWVzVYsxbDjts3R8m rZolg2g7WuWaz9KzlpjyYl7zcixd9Qj7pWriGc4b8+5wc7cueYII+2eLn g==; X-IronPort-AV: E=McAfee;i="6200,9189,10213"; a="305110481" X-IronPort-AV: E=Sophos;i="5.88,248,1635231600"; d="scan'208";a="305110481" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Dec 2021 16:22:05 -0800 X-IronPort-AV: E=Sophos;i="5.88,248,1635231600"; d="scan'208";a="468943779" Received: from jiayingk-mobl2.gar.corp.intel.com (HELO anmitta2-mobl3.intel.com) ([10.215.231.141]) by orsmga003-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Dec 2021 16:22:03 -0800 From: Anuj Mittal To: bitbake-devel@lists.openembedded.org Subject: [1.52][PATCH 3/4] fetch: npm: Use temporary file for empty user config Date: Fri, 31 Dec 2021 08:21:54 +0800 Message-Id: <56f6e7b5f86f1dc630c50a67e9027c1798a56a34.1640910042.git.anuj.mittal@intel.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: References: MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Fri, 31 Dec 2021 00:22:05 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/13207 From: Stefan Herbrechtsmeier Always use a temporary file for the user config 'NPM_CONFIG_USERCONFIG' because npm otherwise failed if configs and npmrc aren't set: double-loading config "/dev/null" as "global", previously loaded as "user" Signed-off-by: Stefan Herbrechtsmeier Signed-off-by: Richard Purdie (cherry picked from commit 9f272ad7f76c1559e745e9af686d0a529f917659) Signed-off-by: Anuj Mittal --- lib/bb/fetch2/npm.py | 14 +++++--------- 1 file changed, 5 insertions(+), 9 deletions(-) diff --git a/lib/bb/fetch2/npm.py b/lib/bb/fetch2/npm.py index d9daec20..b3a3a444 100644 --- a/lib/bb/fetch2/npm.py +++ b/lib/bb/fetch2/npm.py @@ -79,16 +79,12 @@ class NpmEnvironment(object): Using a npm config file seems more reliable than using cli arguments. This class allows to create a controlled environment for npm commands. """ - def __init__(self, d, configs=None, npmrc=None): + def __init__(self, d, configs=[], npmrc=None): self.d = d - if configs: - self.user_config = tempfile.NamedTemporaryFile(mode="w", buffering=1) - self.user_config_name = self.user_config.name - for key, value in configs: - self.user_config.write("%s=%s\n" % (key, value)) - else: - self.user_config_name = "/dev/null" + self.user_config = tempfile.NamedTemporaryFile(mode="w", buffering=1) + for key, value in configs: + self.user_config.write("%s=%s\n" % (key, value)) if npmrc: self.global_config_name = npmrc @@ -109,7 +105,7 @@ class NpmEnvironment(object): workdir = tmpdir def _run(cmd): - cmd = "NPM_CONFIG_USERCONFIG=%s " % (self.user_config_name) + cmd + cmd = "NPM_CONFIG_USERCONFIG=%s " % (self.user_config.name) + cmd cmd = "NPM_CONFIG_GLOBALCONFIG=%s " % (self.global_config_name) + cmd return runfetchcmd(cmd, d, workdir=workdir) From patchwork Fri Dec 31 00:21:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Mittal, Anuj" X-Patchwork-Id: 1955 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id B52E7C433F5 for ; Fri, 31 Dec 2021 00:22:07 +0000 (UTC) Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by mx.groups.io with SMTP id smtpd.web11.5667.1640910121693805729 for ; Thu, 30 Dec 2021 16:22:06 -0800 Authentication-Results: mx.groups.io; dkim=fail reason="unable to parse pub key" header.i=@intel.com header.s=intel header.b=MvgconDz; spf=pass (domain: intel.com, ip: 134.134.136.100, mailfrom: anuj.mittal@intel.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1640910126; x=1672446126; h=from:to:subject:date:message-id:in-reply-to:references: mime-version:content-transfer-encoding; bh=5NItFxBAXwhIaIQ7o3EKFN8HM7mxIBcaqij0ZIH8rW4=; b=MvgconDz+Yx/lCdglcOTlvUwTYikE59rtGieF4NwZ4zSrGilyUbv/M/K N9z5KN2ySzknSzj4FyqBTQzPAFBJLlqIS3teElezf/W6MaUFrDXbhtUJ/ 5vXQZL5O0sqC3F0fHbA1r0mndLV8P1dbrK6BoGG8QIaXzOj/CsysKUyXm t7qSYQlgOdLryRX7QuqkzyERwreNYNxW5sYLW5o+iSrSXQ+U0f/GDZMiA 0ZWSqNlLoYpKQdvQJCdun1Zk9UL8W0OV7Taulg5c4XNOL0GCYiBNEazda LDyTSjm443OkgO5Tz5IMawoOiCRixbeLoSt+7uAP2MHP10FY4Y29QUVkg g==; X-IronPort-AV: E=McAfee;i="6200,9189,10213"; a="305110487" X-IronPort-AV: E=Sophos;i="5.88,248,1635231600"; d="scan'208";a="305110487" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Dec 2021 16:22:06 -0800 X-IronPort-AV: E=Sophos;i="5.88,248,1635231600"; d="scan'208";a="468943790" Received: from jiayingk-mobl2.gar.corp.intel.com (HELO anmitta2-mobl3.intel.com) ([10.215.231.141]) by orsmga003-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Dec 2021 16:22:05 -0800 From: Anuj Mittal To: bitbake-devel@lists.openembedded.org Subject: [1.52][PATCH 4/4] tests/fetch: Drop gnu urls from wget connectivity test Date: Fri, 31 Dec 2021 08:21:55 +0800 Message-Id: <92a16b425fee5f0c46122eb126ecd8b381cf1ec5.1640910042.git.anuj.mittal@intel.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: References: MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Fri, 31 Dec 2021 00:22:07 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/bitbake-devel/message/13208 From: Richard Purdie These urls are no longer adding much to the test coverage but the intermittent network issues connecting to them are painful. Drop the urls. Signed-off-by: Richard Purdie (cherry picked from commit bdf5739c5d831dc97a7d81568f94a0953c71017f) Signed-off-by: Anuj Mittal --- lib/bb/tests/fetch.py | 3 --- 1 file changed, 3 deletions(-) diff --git a/lib/bb/tests/fetch.py b/lib/bb/tests/fetch.py index a85ebdf4..34f3a4f6 100644 --- a/lib/bb/tests/fetch.py +++ b/lib/bb/tests/fetch.py @@ -1378,9 +1378,6 @@ class FetchCheckStatusTest(FetcherTest): "https://downloads.yoctoproject.org/releases/opkg/opkg-0.1.7.tar.gz", "https://downloads.yoctoproject.org/releases/opkg/opkg-0.3.0.tar.gz", "ftp://sourceware.org/pub/libffi/libffi-1.20.tar.gz", - "http://ftp.gnu.org/gnu/autoconf/autoconf-2.60.tar.gz", - "https://ftp.gnu.org/gnu/chess/gnuchess-5.08.tar.gz", - "https://ftp.gnu.org/gnu/gmp/gmp-4.0.tar.gz", # GitHub releases are hosted on Amazon S3, which doesn't support HEAD "https://github.com/kergoth/tslib/releases/download/1.1/tslib-1.1.tar.xz" ]