From patchwork Fri Jan 5 14:03:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steve Sakoman X-Patchwork-Id: 37380 X-Patchwork-Delegate: steve@sakoman.com Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7D8BEC47079 for ; Fri, 5 Jan 2024 14:03:26 +0000 (UTC) Received: from mail-pf1-f174.google.com (mail-pf1-f174.google.com [209.85.210.174]) by mx.groups.io with SMTP id smtpd.web11.23900.1704463404913444701 for ; Fri, 05 Jan 2024 06:03:24 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@sakoman-com.20230601.gappssmtp.com header.s=20230601 header.b=qvpH+CKc; spf=softfail (domain: sakoman.com, ip: 209.85.210.174, mailfrom: steve@sakoman.com) Received: by mail-pf1-f174.google.com with SMTP id d2e1a72fcca58-6d9a795cffbso430389b3a.0 for ; Fri, 05 Jan 2024 06:03:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sakoman-com.20230601.gappssmtp.com; s=20230601; t=1704463403; x=1705068203; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=xi6qd1JPF3xPg/DCk1ZF3l4NxD/DbyPunGfZCoOuSLI=; b=qvpH+CKcwtlkwiG7F/5badpNwv6nlxm9NlPCEnvvC0rJdwSN6iZ3k1a/IWzpMpyeSp jba7VHHULpg7N7Vea020di2Nyl/XtpXZUyg2HJRr0YwfcWV69twnsrmUnvxEbJWcvbaa B/RzUKcweN7kYOB5UJznt/3cLP22TrjulvEG0tKpFEVovCvcnApvaapWvsRXMADwcbPR fNJjKRzz9OOFmgQUFMPWrv27EpKCLKIrAX7TMt6yFIk3r6Q3Bj0+bX5pmiNsjkzTtLdL aabcfuQcpMtcARrYc221LX1+ZAtVcPoHNDieemJGNXgYgswBDUFAIDXETSelPhkO1ylL hl4g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1704463404; x=1705068204; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xi6qd1JPF3xPg/DCk1ZF3l4NxD/DbyPunGfZCoOuSLI=; b=ZiAvFybMya4xwelS9GNftCm7FOHfgjwbkl7MhBzS1tMy+51urAE2qjiE2oAi7REFUh waARBUAhwllEu4jVXVwZa9EEjfTvDBC18O5eyEdrgsszn0rz+2tXWFYaEVHX0o5yrhXh o7FddRMIXMME5QS3srsjdBDZYrWbtiswb41R5IiXg0refV5Cj2O1XQRSUT6/lqEuI2n1 q/K+naKhfdhm8N0H+U7IkxQjf88LtE1PZjPCSJJ2ZPtYLBfPoppuzRhfZPb8R+5/akQ4 D2WZ3TcjFHnHJVoWzJRtcCeo7TOr6RHaLJkq1VAeTiRaFj11FrMqeOqn+feu5ny+ycWE QOrg== X-Gm-Message-State: AOJu0YwdULtVcpxu9mYm2858laPx0Pn1y7/M6J36+ohCE80mpYH6tzW5 ZJ7FPjcsTaQXYMuJUk51EiEHDlDmQSdpsqqfJZC6Tg0QFoC7gw== X-Google-Smtp-Source: AGHT+IFE15oNHA7zYtTo5kB5WU6ovI8c1SAgQQV+7vxlHIjb4KeJdIVWuGl/I8/2q8qWYW7+qgJ/yg== X-Received: by 2002:a05:6a00:4509:b0:6d9:b42a:a6f with SMTP id cw9-20020a056a00450900b006d9b42a0a6fmr2645435pfb.29.1704463403610; Fri, 05 Jan 2024 06:03:23 -0800 (PST) Received: from hexa.router0800d9.com (dhcp-72-234-108-41.hawaiiantel.net. [72.234.108.41]) by smtp.gmail.com with ESMTPSA id y2-20020aa79e02000000b006d9bcf301ffsm1430097pfq.194.2024.01.05.06.03.22 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 05 Jan 2024 06:03:23 -0800 (PST) From: Steve Sakoman To: openembedded-core@lists.openembedded.org Subject: [OE-core][dunfell 1/5] cve-update-nvd2-native: remove unused variable CVE_SOCKET_TIMEOUT Date: Fri, 5 Jan 2024 04:03:04 -1000 Message-Id: <70676801f1f8fe498ff34fc1db72b6a3bf438d4a.1704463208.git.steve@sakoman.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Fri, 05 Jan 2024 14:03:26 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/193361 From: Peter Marko This variable is not referenced in oe-core anymore. Signed-off-by: Peter Marko Signed-off-by: Alexandre Belloni Signed-off-by: Richard Purdie (cherry picked from commit 905b45a814cb33327503b793741c19b44c8550b3) Signed-off-by: Steve Sakoman --- meta/recipes-core/meta/cve-update-nvd2-native.bb | 3 --- 1 file changed, 3 deletions(-) diff --git a/meta/recipes-core/meta/cve-update-nvd2-native.bb b/meta/recipes-core/meta/cve-update-nvd2-native.bb index 67d76f75dd..64a96a46f0 100644 --- a/meta/recipes-core/meta/cve-update-nvd2-native.bb +++ b/meta/recipes-core/meta/cve-update-nvd2-native.bb @@ -26,9 +26,6 @@ NVDCVE_API_KEY ?= "" # Use a negative value to skip the update CVE_DB_UPDATE_INTERVAL ?= "86400" -# Timeout for blocking socket operations, such as the connection attempt. -CVE_SOCKET_TIMEOUT ?= "60" - CVE_DB_TEMP_FILE ?= "${CVE_CHECK_DB_DIR}/temp_nvdcve_2.db" CVE_CHECK_DB_FILE ?= "${CVE_CHECK_DB_DIR}/nvdcve_2.db" From patchwork Fri Jan 5 14:03:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steve Sakoman X-Patchwork-Id: 37383 X-Patchwork-Delegate: steve@sakoman.com Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 51AF7C3DA6E for ; Fri, 5 Jan 2024 14:03:36 +0000 (UTC) Received: from mail-pf1-f179.google.com (mail-pf1-f179.google.com [209.85.210.179]) by mx.groups.io with SMTP id smtpd.web11.23902.1704463407062240969 for ; Fri, 05 Jan 2024 06:03:27 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@sakoman-com.20230601.gappssmtp.com header.s=20230601 header.b=0x/aa1zC; spf=softfail (domain: sakoman.com, ip: 209.85.210.179, mailfrom: steve@sakoman.com) Received: by mail-pf1-f179.google.com with SMTP id d2e1a72fcca58-6dac8955af0so956179b3a.0 for ; Fri, 05 Jan 2024 06:03:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sakoman-com.20230601.gappssmtp.com; s=20230601; t=1704463405; x=1705068205; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=ciOVFYN4NTpoEnTDMR4neSUumR8U4dkfVFt7Vr8U6OY=; b=0x/aa1zCqu1gBDNEtm8OYyBMAUuKz6zmqvH/uqkneKYUSgE7FPnqPDA+aoH1HTrfDT 6WbvfeMpRlBNwrC1kMkcEvt4J/zHFaszob6/l6ormXPQEdfpF3qF5ACZAFTOtpSuOzeg FkAdL53yfl/gtifxZQiQ8xfVOnYL8vEbRSSEEfPiOrea/1z4nGLv02GFQxwDD7g6xkQx Vpkmfrra2slrXUOb4+zVBGSorHtd1tGiUkE6Q+uvsB9cMqe4TSkAu9Mi8pjgdueC+7+W JDUfyxcf77SaBFX161jpedYYBbdanXJT46/dR4++gXdNXRPPXp168+xd214ts/Nj5frn +Fyw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1704463405; x=1705068205; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ciOVFYN4NTpoEnTDMR4neSUumR8U4dkfVFt7Vr8U6OY=; b=MHk1WcqSBqC4RLdye/32MoaqClJG5T0pQMCU06s+7AGyd6/hp9ZlnClX0gP5Ng7K/v dvocat7EmoqgoaWgBivIw55XU8QUcmjmias12d4fPF9za109BGIq2n0R1zNq/ilkf3Fi Tx5cTYho3anbvsTyXdiqDFndtecAxqGR1OJsXxKA013jfP2B85QcDLwFot+RBqM3Ig6k bWA2CIbJonOIFUJMgWULJKTHNIL0azn8cvi13O/ph93GW3KJSqwM+zkGTN0kBbNK4rk4 nfR27hYuszJPw8uCWS8S8HtX1rQ4XKCeca656ktXquSWrk2nkVzaxDOePlw5JfWt7rfu 2/qA== X-Gm-Message-State: AOJu0Yzhwny3MAUDiFd6Mw/FqRDGxBgixv5K83pFdP4LBT9PDbNErubM XgIJmSCKbcsLfrDhX2b95Wc0B5MSNjX49P/50ToJJAcrmRfvHg== X-Google-Smtp-Source: AGHT+IFc4GNW/J8KGwSMRScqzvMmRBguou6eC4EMpcdgnIn83I2CTyI07LIQUqH4ie0zrJtqNjm30A== X-Received: by 2002:a05:6a00:3cc9:b0:6d9:b319:294e with SMTP id ln9-20020a056a003cc900b006d9b319294emr2085356pfb.44.1704463405455; Fri, 05 Jan 2024 06:03:25 -0800 (PST) Received: from hexa.router0800d9.com (dhcp-72-234-108-41.hawaiiantel.net. [72.234.108.41]) by smtp.gmail.com with ESMTPSA id y2-20020aa79e02000000b006d9bcf301ffsm1430097pfq.194.2024.01.05.06.03.24 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 05 Jan 2024 06:03:25 -0800 (PST) From: Steve Sakoman To: openembedded-core@lists.openembedded.org Subject: [OE-core][dunfell 2/5] cve-update-nvd2-native: make number of fetch attemtps configurable Date: Fri, 5 Jan 2024 04:03:05 -1000 Message-Id: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Fri, 05 Jan 2024 14:03:36 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/193362 From: Peter Marko Sometimes NVD servers are unstable and return too many errors. Last time we increased number of attempts from 3 to 5, but further increasing is not reasonable as in normal case too many retries is just abusive. Keep retries low as default and allow to increase as needed. Signed-off-by: Peter Marko Signed-off-by: Alexandre Belloni Signed-off-by: Richard Purdie (cherry picked from commit 6b6fd8043d83b99000054ab6ad2c745d07c6bcc1) Signed-off-by: Steve Sakoman --- meta/recipes-core/meta/cve-update-nvd2-native.bb | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/meta/recipes-core/meta/cve-update-nvd2-native.bb b/meta/recipes-core/meta/cve-update-nvd2-native.bb index 64a96a46f0..dab0b69edc 100644 --- a/meta/recipes-core/meta/cve-update-nvd2-native.bb +++ b/meta/recipes-core/meta/cve-update-nvd2-native.bb @@ -26,6 +26,9 @@ NVDCVE_API_KEY ?= "" # Use a negative value to skip the update CVE_DB_UPDATE_INTERVAL ?= "86400" +# Number of attmepts for each http query to nvd server before giving up +CVE_DB_UPDATE_ATTEMPTS ?= "5" + CVE_DB_TEMP_FILE ?= "${CVE_CHECK_DB_DIR}/temp_nvdcve_2.db" CVE_CHECK_DB_FILE ?= "${CVE_CHECK_DB_DIR}/nvdcve_2.db" @@ -111,7 +114,7 @@ def cleanup_db_download(db_file, db_tmp_file): if os.path.exists(db_tmp_file): os.remove(db_tmp_file) -def nvd_request_next(url, api_key, args): +def nvd_request_next(url, attempts, api_key, args): """ Request next part of the NVD dabase """ @@ -127,7 +130,7 @@ def nvd_request_next(url, api_key, args): request.add_header("apiKey", api_key) bb.note("Requesting %s" % request.full_url) - for attempt in range(5): + for attempt in range(attempts): try: r = urllib.request.urlopen(request) @@ -183,10 +186,11 @@ def update_db_file(db_tmp_file, d, database_time): index = 0 url = d.getVar("NVDCVE_URL") api_key = d.getVar("NVDCVE_API_KEY") or None + attempts = int(d.getVar("CVE_DB_UPDATE_ATTEMPTS")) while True: req_args['startIndex'] = index - raw_data = nvd_request_next(url, api_key, req_args) + raw_data = nvd_request_next(url, attempts, api_key, req_args) if raw_data is None: # We haven't managed to download data return False From patchwork Fri Jan 5 14:03:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steve Sakoman X-Patchwork-Id: 37384 X-Patchwork-Delegate: steve@sakoman.com Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5B3BBC47077 for ; Fri, 5 Jan 2024 14:03:36 +0000 (UTC) Received: from mail-pf1-f177.google.com (mail-pf1-f177.google.com [209.85.210.177]) by mx.groups.io with SMTP id smtpd.web10.23974.1704463408137013250 for ; Fri, 05 Jan 2024 06:03:28 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@sakoman-com.20230601.gappssmtp.com header.s=20230601 header.b=WkXdA+KA; spf=softfail (domain: sakoman.com, ip: 209.85.210.177, mailfrom: steve@sakoman.com) Received: by mail-pf1-f177.google.com with SMTP id d2e1a72fcca58-6d9bec20980so1047103b3a.2 for ; Fri, 05 Jan 2024 06:03:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sakoman-com.20230601.gappssmtp.com; s=20230601; t=1704463407; x=1705068207; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=RyFqBbnSUU+ogcAP6atDmRL4dkW4omDzlWpY7cBzWsc=; b=WkXdA+KAFvKzzCa1ToFHAXQaP5Dwdt+3nrUlQ6QbZy1yULW/2nX/eH/nUKvqdJfDKR 6cOi/g15F7+dSdlV2BsFV+UT2KltGH4HMa7djzxQRKa3kSYqZ0qMqCKoStCu86cDnVsD 3fEvXCrFA6FIb/Ut9vyMVR628PDJ9e8VdFQQXF9rg9RxX024v5bBqW5E9pKaidGujso9 L9XBhwr+7frR9UhakW1T0ktnNuc+o6/FxtwIuTWX8cnFAlecJQ2tWSF+1feGfKZrA1Tb /+21uGzg2D3I7BIhvPXVfan0aUV1sZMsVM/8O3tgqxGlYuxiRx+JIUmeNcyU4otehAw4 Mfhw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1704463407; x=1705068207; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=RyFqBbnSUU+ogcAP6atDmRL4dkW4omDzlWpY7cBzWsc=; b=NmgS7mrY96UKiv/CvZ3OwVSnzMvCZUpWS4rPyLQRMwHvzpyenRlRNJebcT31tZh3KS mHuRYMlQL1q7E4j8S/XtaBnkJXlusw6OOMMglO5frML30sQjd7U6nRgYYlidf3/GI6Vz jgwXZUGceV+HUK+0KdhuaMgnQhijQ5oN2W3mZGz2JV97WYmrb1KVPyzvVbv7lO/1NvR7 RgOT8/fptNf+vjfRvvCwY+nF1SGJMmyQ9Qb1NxLLsEJrajR0/3su26an1rPXD1/ZGtxY zXoFkEj4HMT7fqkF+30YFE03si3tDAwUPdK3zJluajaqRpvvr6LlR2RkMRqZ7T00HVnZ 8fqQ== X-Gm-Message-State: AOJu0Yx+2m8EWOaPzMlWVC2v3MK+Q9k9rnD3HRmTCAUvso6DHUXuIOM5 LTfF4ZaOMB4BlDIV0xBf2vPJTtk1V69R2Wlp/0PlfzaVveHc4A== X-Google-Smtp-Source: AGHT+IGlyfHCq05Y/qok6teymY1a1K2E8KhKNvLm1F89tATUFYTlXrFX1Cly/l9DTsOh0j7XQtd0GQ== X-Received: by 2002:a05:6a00:ccb:b0:6d9:bad6:52e6 with SMTP id b11-20020a056a000ccb00b006d9bad652e6mr2119200pfv.23.1704463407200; Fri, 05 Jan 2024 06:03:27 -0800 (PST) Received: from hexa.router0800d9.com (dhcp-72-234-108-41.hawaiiantel.net. [72.234.108.41]) by smtp.gmail.com with ESMTPSA id y2-20020aa79e02000000b006d9bcf301ffsm1430097pfq.194.2024.01.05.06.03.26 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 05 Jan 2024 06:03:26 -0800 (PST) From: Steve Sakoman To: openembedded-core@lists.openembedded.org Subject: [OE-core][dunfell 3/5] cve-update-nvd2-native: faster requests with API keys Date: Fri, 5 Jan 2024 04:03:06 -1000 Message-Id: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Fri, 05 Jan 2024 14:03:36 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/193363 From: Dhairya Nagodra As per NVD, the public rate limit is 5 requests in 30s (6s delay). Using an API key increases the limit to 50 requests in 30s (0.6s delay). However, NVD still recommends sleeping for several seconds so that the other legitimate requests are serviced without denial or interruption. Keeping the default sleep at 6 seconds and 2 seconds with an API key. For failures, the wait time is unchanged (6 seconds). Reference: https://nvd.nist.gov/developers/start-here#RateLimits Signed-off-by: Dhairya Nagodra Signed-off-by: Alexandre Belloni (cherry picked from commit 5c32e2941d1dc3d04a799a1b7cbd275c1ccc9e79) Signed-off-by: Steve Sakoman --- meta/recipes-core/meta/cve-update-nvd2-native.bb | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/meta/recipes-core/meta/cve-update-nvd2-native.bb b/meta/recipes-core/meta/cve-update-nvd2-native.bb index dab0b69edc..0a8b6a8a0a 100644 --- a/meta/recipes-core/meta/cve-update-nvd2-native.bb +++ b/meta/recipes-core/meta/cve-update-nvd2-native.bb @@ -188,6 +188,11 @@ def update_db_file(db_tmp_file, d, database_time): api_key = d.getVar("NVDCVE_API_KEY") or None attempts = int(d.getVar("CVE_DB_UPDATE_ATTEMPTS")) + # Recommended by NVD + wait_time = 6 + if api_key: + wait_time = 2 + while True: req_args['startIndex'] = index raw_data = nvd_request_next(url, attempts, api_key, req_args) @@ -210,7 +215,7 @@ def update_db_file(db_tmp_file, d, database_time): break # Recommended by NVD - time.sleep(6) + time.sleep(wait_time) # Update success, set the date to cve_check file. cve_f.write('CVE database update : %s\n\n' % datetime.date.today()) From patchwork Fri Jan 5 14:03:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steve Sakoman X-Patchwork-Id: 37382 X-Patchwork-Delegate: steve@sakoman.com Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 61776C4707B for ; Fri, 5 Jan 2024 14:03:36 +0000 (UTC) Received: from mail-pf1-f173.google.com (mail-pf1-f173.google.com [209.85.210.173]) by mx.groups.io with SMTP id smtpd.web11.23907.1704463410744349945 for ; Fri, 05 Jan 2024 06:03:30 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@sakoman-com.20230601.gappssmtp.com header.s=20230601 header.b=jTtgYADH; spf=softfail (domain: sakoman.com, ip: 209.85.210.173, mailfrom: steve@sakoman.com) Received: by mail-pf1-f173.google.com with SMTP id d2e1a72fcca58-6da202aa138so1100281b3a.2 for ; Fri, 05 Jan 2024 06:03:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sakoman-com.20230601.gappssmtp.com; s=20230601; t=1704463409; x=1705068209; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=BqXIqU+PRG4O6z9Gvq8AUakeAfM/VJmEo6BgkeV5+1I=; b=jTtgYADHxRbrjEupFOdiH8fhEJq52cqFOF56DDMy+eNg4oHNbGJtnkeXfm0p2eKutz S+oFESiZmRZMy5Va/CR/anGXgFHb2xIpc9EORw/wsQWjpuf/VT6FhjL+nqMxuOxr3ikM z0l7yG82uWhNpgZUFwdvJuCGbStXLY8zgIalrf9Z5Pjq6v79rK4JTkZzVT7DS9Kpm/lu Vcn/qETd3XOq7KGieJxAv/suSGZlljGhw/ZdZYmpyeDasgSEY5WmLffskSi2GmC2034p Il6UO5SqbVsHjFU3H+bk7t+HY5SBluXMijAl8ZwkVicLM+CkcPVWtGk+MRB5mDdMiw5Z k7NQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1704463409; x=1705068209; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BqXIqU+PRG4O6z9Gvq8AUakeAfM/VJmEo6BgkeV5+1I=; b=RUy0/BbHmlUtDy9WuwsupQQz/Upy4uc8R3sbjNL/xh31UVoZBZEm+kpnOmxuk2sYp8 SvHf57LarB70lQxKymSuFPijHXtXbU0OskTneaFadc/FjOy3klUXv8b2cOZyKCYCDoWX igSfgy1v5OpJwdapqRZJ/WrtQvBDWGXh+M+G6DoJ9M/KnEymgKZlDoY0JlOjw2/Chw8U hW7jbE1n8dAkTwEGglKCyIJqKoNfxvTpV9AqUrJA70M/Al1J8PIIoVCbC7J7WoGqFPUW 4QawAwDwwnhuJogooQMmRD7X16sqPk3htmCae3/727F7IXbZnfVLJpMjeK48M0YfSwRf FftA== X-Gm-Message-State: AOJu0YzyGeHFh2pYK72vToNtTTg9Jq670rpu8LaF58bAt4ihIIqFqE7n +9zHvimZFrcPrvcfppT3+pBiITfb59l1PKg9RZU4xOXp/OgMPw== X-Google-Smtp-Source: AGHT+IE+pPAt14d8rmh/FxUxbNTpayHc7zRq+oqRyyRCdKvcYsxAGfYVSj3OE6ngOqFREotykl+mCA== X-Received: by 2002:a05:6a20:9387:b0:195:1bcd:f60c with SMTP id x7-20020a056a20938700b001951bcdf60cmr2472416pzh.35.1704463409400; Fri, 05 Jan 2024 06:03:29 -0800 (PST) Received: from hexa.router0800d9.com (dhcp-72-234-108-41.hawaiiantel.net. [72.234.108.41]) by smtp.gmail.com with ESMTPSA id y2-20020aa79e02000000b006d9bcf301ffsm1430097pfq.194.2024.01.05.06.03.28 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 05 Jan 2024 06:03:28 -0800 (PST) From: Steve Sakoman To: openembedded-core@lists.openembedded.org Subject: [OE-core][dunfell 4/5] cve-update-nvd2-native: increase the delay between subsequent request failures Date: Fri, 5 Jan 2024 04:03:07 -1000 Message-Id: <9e03b7a9879fd16e32f4eccb78b438f6fa9db74d.1704463208.git.steve@sakoman.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Fri, 05 Jan 2024 14:03:36 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/193364 From: Dhairya Nagodra Sometimes NVD servers are unstable and return too many errors. There is an option to have higher fetch attempts to increase the chances of successfully fetching the CVE data. Additionally, it also makes sense to progressively increase the delay after a failed request to an already unstable or busy server. The increase in delay is reset after every successful request and the maximum delay is limited to 30 seconds. Also, the logs are improved to give more clarity. Signed-off-by: Dhairya Nagodra Signed-off-by: Alexandre Belloni (cherry picked from commit 7101d654635b707e56b0dbae8c2146b312d211ea) Signed-off-by: Steve Sakoman --- meta/recipes-core/meta/cve-update-nvd2-native.bb | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/meta/recipes-core/meta/cve-update-nvd2-native.bb b/meta/recipes-core/meta/cve-update-nvd2-native.bb index 0a8b6a8a0a..69ba20a6cb 100644 --- a/meta/recipes-core/meta/cve-update-nvd2-native.bb +++ b/meta/recipes-core/meta/cve-update-nvd2-native.bb @@ -114,7 +114,10 @@ def cleanup_db_download(db_file, db_tmp_file): if os.path.exists(db_tmp_file): os.remove(db_tmp_file) -def nvd_request_next(url, attempts, api_key, args): +def nvd_request_wait(attempt, min_wait): + return min ( ( (2 * attempt) + min_wait ) , 30) + +def nvd_request_next(url, attempts, api_key, args, min_wait): """ Request next part of the NVD dabase """ @@ -143,8 +146,10 @@ def nvd_request_next(url, attempts, api_key, args): r.close() except Exception as e: - bb.note("CVE database: received error (%s), retrying" % (e)) - time.sleep(6) + wait_time = nvd_request_wait(attempt, min_wait) + bb.note("CVE database: received error (%s)" % (e)) + bb.note("CVE database: retrying download after %d seconds. attempted (%d/%d)" % (wait_time, attempt+1, attempts)) + time.sleep(wait_time) pass else: return raw_data @@ -195,7 +200,7 @@ def update_db_file(db_tmp_file, d, database_time): while True: req_args['startIndex'] = index - raw_data = nvd_request_next(url, attempts, api_key, req_args) + raw_data = nvd_request_next(url, attempts, api_key, req_args, wait_time) if raw_data is None: # We haven't managed to download data return False From patchwork Fri Jan 5 14:03:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steve Sakoman X-Patchwork-Id: 37385 X-Patchwork-Delegate: steve@sakoman.com Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 697A6C47079 for ; Fri, 5 Jan 2024 14:03:36 +0000 (UTC) Received: from mail-pf1-f174.google.com (mail-pf1-f174.google.com [209.85.210.174]) by mx.groups.io with SMTP id smtpd.web10.23978.1704463412173838447 for ; Fri, 05 Jan 2024 06:03:32 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@sakoman-com.20230601.gappssmtp.com header.s=20230601 header.b=cwVujdSS; spf=softfail (domain: sakoman.com, ip: 209.85.210.174, mailfrom: steve@sakoman.com) Received: by mail-pf1-f174.google.com with SMTP id d2e1a72fcca58-6d9cdd0a5e6so1016830b3a.3 for ; Fri, 05 Jan 2024 06:03:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sakoman-com.20230601.gappssmtp.com; s=20230601; t=1704463411; x=1705068211; darn=lists.openembedded.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=wgJH0pT4Y4yKMHqfoWpRea8etN5/sbtBC/CQqp+SYh0=; b=cwVujdSSt5+Lgklf84UlOChv1gjRpN4stuRofjZicT8Nc9SxYVEMaI2vSXlTkqudB4 tcTEd+actcO3ZBfFeMf0jKFskPHuUfkJS0majkCKuby9soj4nYyH2l/xke+S/JA5YAap /W4LIqiW7/LtHYoUOrAiCk0B0Bg2dsgR7u0dTcRSBQEYWfNiYWpBBgDXcdNFu2YV3SqW jrm4t1hACLpSsWpSpeYN+diqab3buLrzJEmRMEOEF5obq9sDSaQlvwFYPOe6PcttX6qf vRQytqkbtW9JcGr1/stMEX43hE+rsV1fDy2hDydxzCMOAktNAMe3yQB0W9jBwlfzWgri eJBg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1704463411; x=1705068211; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wgJH0pT4Y4yKMHqfoWpRea8etN5/sbtBC/CQqp+SYh0=; b=XKEpDxOZzoNheuJTbcV+m6utAzuXnC9AiqM1eNyW+7jn/CmEd5sllGFDQncppdSHhC A77wyMsONqT0TyQ4b73LZiaPpDZXFseV0WgeuEY9Um3fpkWAcTRSJEadTZb246Dd21+J 8BRkDJ2OOZ+L3nU4FPFE9XBzSqqH4W+LdKi3DPqqPeTZF0Tk4m8zPtreAE3eHxnekXFf gXy8dyg5MSG4xNyiRxCIUAvgf7rxrsYAA/RJeFI6ffKEA0gHX4J0z3YOo8c9aiBk2iNv v/0mE9N4DwgwNnTxyamTLTKeLvIsGZLzKbOJORzWH67th/0lWR3rCmWnvS2H3nwIJLRd uWHg== X-Gm-Message-State: AOJu0YxtQKLCOYtddLpl3qBo9bm6fppuzLXVDym+Le2wjv1+RkNVuwkM FxjUbwsOE5VBH4TZ15QuFsTWK0j7mod++X6mb9RJsVIcRwamVw== X-Google-Smtp-Source: AGHT+IHDNcY2mvGKm7bFKbh5uZST/BOnYj1QWv5xrt5k+DWW3DOOa05pA17ezoxwp8bB+7tP9oct9g== X-Received: by 2002:a05:6a20:2a23:b0:194:f4f:f538 with SMTP id e35-20020a056a202a2300b001940f4ff538mr1707057pzh.109.1704463411173; Fri, 05 Jan 2024 06:03:31 -0800 (PST) Received: from hexa.router0800d9.com (dhcp-72-234-108-41.hawaiiantel.net. [72.234.108.41]) by smtp.gmail.com with ESMTPSA id y2-20020aa79e02000000b006d9bcf301ffsm1430097pfq.194.2024.01.05.06.03.30 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 05 Jan 2024 06:03:30 -0800 (PST) From: Steve Sakoman To: openembedded-core@lists.openembedded.org Subject: [OE-core][dunfell 5/5] linux-firmware: upgrade 20230804 -> 20231030 Date: Fri, 5 Jan 2024 04:03:08 -1000 Message-Id: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: MIME-Version: 1.0 List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Fri, 05 Jan 2024 14:03:36 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/193365 From: Dmitry Baryshkov License-Update: additional firmwares Signed-off-by: Dmitry Baryshkov Signed-off-by: Alexandre Belloni (cherry picked from commit 7c725d1f2ed9a271d39d899ac2534558c2d103fc) Signed-off-by: Steve Sakoman --- ...{linux-firmware_20230804.bb => linux-firmware_20231030.bb} | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) rename meta/recipes-kernel/linux-firmware/{linux-firmware_20230804.bb => linux-firmware_20231030.bb} (99%) diff --git a/meta/recipes-kernel/linux-firmware/linux-firmware_20230804.bb b/meta/recipes-kernel/linux-firmware/linux-firmware_20231030.bb similarity index 99% rename from meta/recipes-kernel/linux-firmware/linux-firmware_20230804.bb rename to meta/recipes-kernel/linux-firmware/linux-firmware_20231030.bb index 507a003224..65cbca798e 100644 --- a/meta/recipes-kernel/linux-firmware/linux-firmware_20230804.bb +++ b/meta/recipes-kernel/linux-firmware/linux-firmware_20231030.bb @@ -134,7 +134,7 @@ LIC_FILES_CHKSUM = "file://LICENCE.Abilis;md5=b5ee3f410780e56711ad48eadc22b8bc \ " # WHENCE checksum is defined separately to ease overriding it if # class-devupstream is selected. -WHENCE_CHKSUM = "41f9a48bf27971b126a36f9344594dcd" +WHENCE_CHKSUM = "ceb5248746d24d165b603e71b288cf75" # These are not common licenses, set NO_GENERIC_LICENSE for them # so that the license files will be copied from fetched source @@ -212,7 +212,7 @@ SRC_URI:class-devupstream = "git://git.kernel.org/pub/scm/linux/kernel/git/firmw # Pin this to the 20220509 release, override this in local.conf SRCREV:class-devupstream ?= "b19cbdca78ab2adfd210c91be15a22568e8b8cae" -SRC_URI[sha256sum] = "88d46c543847ee3b03404d4941d91c92974690ee1f6fdcbee9cef3e5f97db688" +SRC_URI[sha256sum] = "c98d200fc4a3120de1a594713ce34e135819dff23e883a4ed387863ba25679c7" inherit allarch