Message ID | 20231208025321.418459-1-dnagodra@cisco.com |
---|---|
State | Accepted, archived |
Commit | 5c32e2941d1dc3d04a799a1b7cbd275c1ccc9e79 |
Headers | show |
Series | [master] cve-update-nvd2-native: faster requests with API keys | expand |
Please follow https://docs.yoctoproject.org/dev/contributor-guide/submit-changes.html#fixing-your-from-identity On 07/12/2023 18:53:22-0800, Dhairya Nagodra via lists.openembedded.org wrote: > As per NVD, the public rate limit is 5 requests in 30s (6s delay). > Using an API key increases the limit to 50 requests in 30s (0.6s delay). > However, NVD still recommends sleeping for several seconds so that the > other legitimate requests are serviced without denial or interruption. > Keeping the default sleep at 6 seconds and 2 seconds with an API key. > > For failures, the wait time is unchanged (6 seconds). > > Reference: https://nvd.nist.gov/developers/start-here#RateLimits > > Signed-off-by: Dhairya Nagodra <dnagodra@cisco.com> > --- > meta/recipes-core/meta/cve-update-nvd2-native.bb | 7 ++++++- > 1 file changed, 6 insertions(+), 1 deletion(-) > > diff --git a/meta/recipes-core/meta/cve-update-nvd2-native.bb b/meta/recipes-core/meta/cve-update-nvd2-native.bb > index 9ab8dc6050..941fca34c6 100644 > --- a/meta/recipes-core/meta/cve-update-nvd2-native.bb > +++ b/meta/recipes-core/meta/cve-update-nvd2-native.bb > @@ -188,6 +188,11 @@ def update_db_file(db_tmp_file, d, database_time): > api_key = d.getVar("NVDCVE_API_KEY") or None > attempts = int(d.getVar("CVE_DB_UPDATE_ATTEMPTS")) > > + # Recommended by NVD > + wait_time = 6 > + if api_key: > + wait_time = 2 > + > while True: > req_args['startIndex'] = index > raw_data = nvd_request_next(url, attempts, api_key, req_args) > @@ -210,7 +215,7 @@ def update_db_file(db_tmp_file, d, database_time): > break > > # Recommended by NVD > - time.sleep(6) > + time.sleep(wait_time) > > # Update success, set the date to cve_check file. > cve_f.write('CVE database update : %s\n\n' % datetime.date.today()) > -- > 2.35.6 > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#192008): https://lists.openembedded.org/g/openembedded-core/message/192008 > Mute This Topic: https://lists.openembedded.org/mt/103048465/3617179 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alexandre.belloni@bootlin.com] > -=-=-=-=-=-=-=-=-=-=-=- >
diff --git a/meta/recipes-core/meta/cve-update-nvd2-native.bb b/meta/recipes-core/meta/cve-update-nvd2-native.bb index 9ab8dc6050..941fca34c6 100644 --- a/meta/recipes-core/meta/cve-update-nvd2-native.bb +++ b/meta/recipes-core/meta/cve-update-nvd2-native.bb @@ -188,6 +188,11 @@ def update_db_file(db_tmp_file, d, database_time): api_key = d.getVar("NVDCVE_API_KEY") or None attempts = int(d.getVar("CVE_DB_UPDATE_ATTEMPTS")) + # Recommended by NVD + wait_time = 6 + if api_key: + wait_time = 2 + while True: req_args['startIndex'] = index raw_data = nvd_request_next(url, attempts, api_key, req_args) @@ -210,7 +215,7 @@ def update_db_file(db_tmp_file, d, database_time): break # Recommended by NVD - time.sleep(6) + time.sleep(wait_time) # Update success, set the date to cve_check file. cve_f.write('CVE database update : %s\n\n' % datetime.date.today())
As per NVD, the public rate limit is 5 requests in 30s (6s delay). Using an API key increases the limit to 50 requests in 30s (0.6s delay). However, NVD still recommends sleeping for several seconds so that the other legitimate requests are serviced without denial or interruption. Keeping the default sleep at 6 seconds and 2 seconds with an API key. For failures, the wait time is unchanged (6 seconds). Reference: https://nvd.nist.gov/developers/start-here#RateLimits Signed-off-by: Dhairya Nagodra <dnagodra@cisco.com> --- meta/recipes-core/meta/cve-update-nvd2-native.bb | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)