From patchwork Mon Nov 27 19:55:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Marko, Peter" X-Patchwork-Id: 35251 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 04295C4167B for ; Mon, 27 Nov 2023 19:56:31 +0000 (UTC) Received: from mta-65-225.siemens.flowmailer.net (mta-65-225.siemens.flowmailer.net [185.136.65.225]) by mx.groups.io with SMTP id smtpd.web10.105190.1701114986216557146 for ; Mon, 27 Nov 2023 11:56:27 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=peter.marko@siemens.com header.s=fm1 header.b=jVRB9dsI; spf=pass (domain: rts-flowmailer.siemens.com, ip: 185.136.65.225, mailfrom: fm-256628-2023112719562321c433c0316c181faa-sk4ba3@rts-flowmailer.siemens.com) Received: by mta-65-225.siemens.flowmailer.net with ESMTPSA id 2023112719562321c433c0316c181faa for ; Mon, 27 Nov 2023 20:56:24 +0100 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm1; d=siemens.com; i=peter.marko@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc; bh=pY0JQg9BscAMDSdmPYAQ73NL9P0yEq/Dqa2mn6hfKuk=; b=jVRB9dsIY9oPmIP/k7cgE/+jXczPkqnxGZpMFa3kGplEySHksclYT8ZekALQQT3oZYqb3N QoFfsPMil6m7wUHtck++zy4FKtRmuVgjvtohsjWmy3ivs1P3rPrAcNKvb8HnTb+Cuny6B2y3 jIoTMyTLznB6p01Cg95+P7HNjf9gI=; From: Peter Marko To: openembedded-core@lists.openembedded.org Cc: Peter Marko Subject: [OE-core][PATCH v2 1/2] cve-update-nvd2-native: remove unused variable CVE_SOCKET_TIMEOUT Date: Mon, 27 Nov 2023 20:55:35 +0100 Message-Id: <20231127195536.428542-1-peter.marko@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-256628:519-21489:flowmailer List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Mon, 27 Nov 2023 19:56:30 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/191323 From: Peter Marko This variable is not referenced in oe-core anymore. Signed-off-by: Peter Marko --- v2: typo in commit message meta/recipes-core/meta/cve-update-nvd2-native.bb | 3 --- 1 file changed, 3 deletions(-) diff --git a/meta/recipes-core/meta/cve-update-nvd2-native.bb b/meta/recipes-core/meta/cve-update-nvd2-native.bb index 67d76f75dd..64a96a46f0 100644 --- a/meta/recipes-core/meta/cve-update-nvd2-native.bb +++ b/meta/recipes-core/meta/cve-update-nvd2-native.bb @@ -26,9 +26,6 @@ NVDCVE_API_KEY ?= "" # Use a negative value to skip the update CVE_DB_UPDATE_INTERVAL ?= "86400" -# Timeout for blocking socket operations, such as the connection attempt. -CVE_SOCKET_TIMEOUT ?= "60" - CVE_DB_TEMP_FILE ?= "${CVE_CHECK_DB_DIR}/temp_nvdcve_2.db" CVE_CHECK_DB_FILE ?= "${CVE_CHECK_DB_DIR}/nvdcve_2.db" From patchwork Mon Nov 27 19:55:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Marko, Peter" X-Patchwork-Id: 35252 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id D01F2C0755A for ; Mon, 27 Nov 2023 19:56:40 +0000 (UTC) Received: from mta-65-228.siemens.flowmailer.net (mta-65-228.siemens.flowmailer.net [185.136.65.228]) by mx.groups.io with SMTP id smtpd.web11.104944.1701114992716899465 for ; Mon, 27 Nov 2023 11:56:33 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=peter.marko@siemens.com header.s=fm1 header.b=IbhrRY6R; spf=pass (domain: rts-flowmailer.siemens.com, ip: 185.136.65.228, mailfrom: fm-256628-20231127195630f96312b2be3609ee05-dx5x9_@rts-flowmailer.siemens.com) Received: by mta-65-228.siemens.flowmailer.net with ESMTPSA id 20231127195630f96312b2be3609ee05 for ; Mon, 27 Nov 2023 20:56:30 +0100 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=fm1; d=siemens.com; i=peter.marko@siemens.com; h=Date:From:Subject:To:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Cc:References:In-Reply-To; bh=uJwX6SUGxfPvTT9sZaI1AzQ7IeSg8WyOUscBTlSpS9k=; b=IbhrRY6RoZBu4RWiHOcBSBN4gsquZ/LZwVDens84bSNbS1ASi1085AarfklEVl8yBy9c2p cpKsIRFG1x4+LO5cVeLxn6XcDC5sJuz17WwrRz3v3gNQwnpPdXyZQQ0tB2Y2Caxio9CTLwKN /meijAWVdKyN2kMC6Jctyn9+0cwt0=; From: Peter Marko To: openembedded-core@lists.openembedded.org Cc: Peter Marko Subject: [OE-core][PATCH v2 2/2] cve-update-nvd2-native: make number of fetch attemtps configurable Date: Mon, 27 Nov 2023 20:55:36 +0100 Message-Id: <20231127195536.428542-2-peter.marko@siemens.com> In-Reply-To: <20231127195536.428542-1-peter.marko@siemens.com> References: <20231127195536.428542-1-peter.marko@siemens.com> MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-256628:519-21489:flowmailer List-Id: X-Webhook-Received: from li982-79.members.linode.com [45.33.32.79] by aws-us-west-2-korg-lkml-1.web.codeaurora.org with HTTPS for ; Mon, 27 Nov 2023 19:56:40 -0000 X-Groupsio-URL: https://lists.openembedded.org/g/openembedded-core/message/191324 From: Peter Marko Sometimes NVD servers are unstable and return too many errors. Last time we increased number of attempts from 3 to 5, but further increasing is not reasonable as in normal case too many retries is just abusive. Keep retries low as default and allow to increase as needed. Signed-off-by: Peter Marko --- meta/recipes-core/meta/cve-update-nvd2-native.bb | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/meta/recipes-core/meta/cve-update-nvd2-native.bb b/meta/recipes-core/meta/cve-update-nvd2-native.bb index 64a96a46f0..dab0b69edc 100644 --- a/meta/recipes-core/meta/cve-update-nvd2-native.bb +++ b/meta/recipes-core/meta/cve-update-nvd2-native.bb @@ -26,6 +26,9 @@ NVDCVE_API_KEY ?= "" # Use a negative value to skip the update CVE_DB_UPDATE_INTERVAL ?= "86400" +# Number of attmepts for each http query to nvd server before giving up +CVE_DB_UPDATE_ATTEMPTS ?= "5" + CVE_DB_TEMP_FILE ?= "${CVE_CHECK_DB_DIR}/temp_nvdcve_2.db" CVE_CHECK_DB_FILE ?= "${CVE_CHECK_DB_DIR}/nvdcve_2.db" @@ -111,7 +114,7 @@ def cleanup_db_download(db_file, db_tmp_file): if os.path.exists(db_tmp_file): os.remove(db_tmp_file) -def nvd_request_next(url, api_key, args): +def nvd_request_next(url, attempts, api_key, args): """ Request next part of the NVD dabase """ @@ -127,7 +130,7 @@ def nvd_request_next(url, api_key, args): request.add_header("apiKey", api_key) bb.note("Requesting %s" % request.full_url) - for attempt in range(5): + for attempt in range(attempts): try: r = urllib.request.urlopen(request) @@ -183,10 +186,11 @@ def update_db_file(db_tmp_file, d, database_time): index = 0 url = d.getVar("NVDCVE_URL") api_key = d.getVar("NVDCVE_API_KEY") or None + attempts = int(d.getVar("CVE_DB_UPDATE_ATTEMPTS")) while True: req_args['startIndex'] = index - raw_data = nvd_request_next(url, api_key, req_args) + raw_data = nvd_request_next(url, attempts, api_key, req_args) if raw_data is None: # We haven't managed to download data return False