[v2] curl: Update from 8.2.1 to 8.3.0

From: Robert Joslyn <robert.joslyn@redrectangle.org>

NSS support was removed, so adjust PACKAGECONFIG options.

The --enable-crypto-auth option was removed and split into separate
options for basic-auth, bearer-auth, digest-auth, kerberos-auth,
negotiate-auth, and aws. Enable these new options since upstream enables
them by default.

Disable test 1279 since this requires libcurl and hangs the tests.

Signed-off-by: Robert Joslyn <robert.joslyn@redrectangle.org>
---
Changes from v1:
Enable aws authentication by default. Upstream enables it by default and
test 1964 fails with it disabled.

 meta/recipes-support/curl/curl/disable-tests       |  1 +
 .../curl/{curl_8.2.1.bb => curl_8.3.0.bb}          | 14 +++++++++-----
 2 files changed, 10 insertions(+), 5 deletions(-)
 rename meta/recipes-support/curl/{curl_8.2.1.bb => curl_8.3.0.bb} (88%)

I am seeing a ptest failure on qemx86-64/glibc

  Failed ptests:
  {'curl': ['test_1474', 'curl']}

On Sat, Sep 23, 2023 at 10:25 AM Robert Joslyn via
lists.openembedded.org
<robert.joslyn=redrectangle.org@lists.openembedded.org> wrote:
>
> From: Robert Joslyn <robert.joslyn@redrectangle.org>
>
> NSS support was removed, so adjust PACKAGECONFIG options.
>
> The --enable-crypto-auth option was removed and split into separate
> options for basic-auth, bearer-auth, digest-auth, kerberos-auth,
> negotiate-auth, and aws. Enable these new options since upstream enables
> them by default.
>
> Disable test 1279 since this requires libcurl and hangs the tests.
>
> Signed-off-by: Robert Joslyn <robert.joslyn@redrectangle.org>
> ---
> Changes from v1:
> Enable aws authentication by default. Upstream enables it by default and
> test 1964 fails with it disabled.
>
>  meta/recipes-support/curl/curl/disable-tests       |  1 +
>  .../curl/{curl_8.2.1.bb => curl_8.3.0.bb}          | 14 +++++++++-----
>  2 files changed, 10 insertions(+), 5 deletions(-)
>  rename meta/recipes-support/curl/{curl_8.2.1.bb => curl_8.3.0.bb} (88%)
>
> diff --git a/meta/recipes-support/curl/curl/disable-tests b/meta/recipes-support/curl/curl/disable-tests
> index b687b2bb76..fdac795662 100644
> --- a/meta/recipes-support/curl/curl/disable-tests
> +++ b/meta/recipes-support/curl/curl/disable-tests
> @@ -21,6 +21,7 @@
>  # This test is scanning the source tree
>  1222
>  # These CRL tests need --libcurl option to be enabled
> +1279
>  1400
>  1401
>  1402
> diff --git a/meta/recipes-support/curl/curl_8.2.1.bb b/meta/recipes-support/curl/curl_8.3.0.bb
> similarity index 88%
> rename from meta/recipes-support/curl/curl_8.2.1.bb
> rename to meta/recipes-support/curl/curl_8.3.0.bb
> index b86c6f3bba..646ac01914 100644
> --- a/meta/recipes-support/curl/curl_8.2.1.bb
> +++ b/meta/recipes-support/curl/curl_8.3.0.bb
> @@ -14,7 +14,7 @@ SRC_URI = " \
>      file://run-ptest \
>      file://disable-tests \
>  "
> -SRC_URI[sha256sum] = "dd322f6bd0a20e6cebdfd388f69e98c3d183bed792cf4713c8a7ef498cba4894"
> +SRC_URI[sha256sum] = "376d627767d6c4f05105ab6d497b0d9aba7111770dd9d995225478209c37ea63"
>
>  # Curl has used many names over the years...
>  CVE_PRODUCT = "haxx:curl haxx:libcurl curl:curl curl:libcurl libcurl:libcurl daniel_stenberg:curl"
> @@ -24,21 +24,26 @@ inherit autotools pkgconfig binconfig multilib_header ptest
>  # Entropy source for random PACKAGECONFIG option
>  RANDOM ?= "/dev/urandom"
>
> -PACKAGECONFIG ??= "${@bb.utils.filter('DISTRO_FEATURES', 'ipv6', d)} libidn openssl proxy random threaded-resolver verbose zlib"
> +PACKAGECONFIG ??= "${@bb.utils.filter('DISTRO_FEATURES', 'ipv6', d)} aws basic-auth bearer-auth digest-auth negotiate-auth libidn openssl proxy random threaded-resolver verbose zlib"
>  PACKAGECONFIG:class-native = "ipv6 openssl proxy random threaded-resolver verbose zlib"
>  PACKAGECONFIG:class-nativesdk = "ipv6 openssl proxy random threaded-resolver verbose zlib"
>
>  # 'ares' and 'threaded-resolver' are mutually exclusive
>  PACKAGECONFIG[ares] = "--enable-ares,--disable-ares,c-ares,,,threaded-resolver"
> +PACKAGECONFIG[aws] = "--enable-aws,--disable-aws"
> +PACKAGECONFIG[basic-auth] = "--enable-basic-auth,--disable-basic-auth"
> +PACKAGECONFIG[bearer-auth] = "--enable-bearer-auth,--disable-bearer-auth"
>  PACKAGECONFIG[brotli] = "--with-brotli,--without-brotli,brotli"
>  PACKAGECONFIG[builtinmanual] = "--enable-manual,--disable-manual"
>  # Don't use this in production
>  PACKAGECONFIG[debug] = "--enable-debug,--disable-debug"
>  PACKAGECONFIG[dict] = "--enable-dict,--disable-dict,"
> +PACKAGECONFIG[digest-auth] = "--enable-digest-auth,--disable-digest-auth"
>  PACKAGECONFIG[gnutls] = "--with-gnutls,--without-gnutls,gnutls"
>  PACKAGECONFIG[gopher] = "--enable-gopher,--disable-gopher,"
>  PACKAGECONFIG[imap] = "--enable-imap,--disable-imap,"
>  PACKAGECONFIG[ipv6] = "--enable-ipv6,--disable-ipv6,"
> +PACKAGECONFIG[kerberos-auth] = "--enable-kerberos-auth,--disable-kerberos-auth"
>  PACKAGECONFIG[krb5] = "--with-gssapi,--without-gssapi,krb5"
>  PACKAGECONFIG[ldap] = "--enable-ldap,--disable-ldap,openldap"
>  PACKAGECONFIG[ldaps] = "--enable-ldaps,--disable-ldaps,openldap"
> @@ -47,6 +52,7 @@ PACKAGECONFIG[libidn] = "--with-libidn2,--without-libidn2,libidn2"
>  PACKAGECONFIG[libssh2] = "--with-libssh2,--without-libssh2,libssh2"
>  PACKAGECONFIG[mbedtls] = "--with-mbedtls=${STAGING_DIR_TARGET},--without-mbedtls,mbedtls"
>  PACKAGECONFIG[mqtt] = "--enable-mqtt,--disable-mqtt,"
> +PACKAGECONFIG[negotiate-auth] = "--enable-negotiate-auth,--disable-negotiate-auth"
>  PACKAGECONFIG[nghttp2] = "--with-nghttp2,--without-nghttp2,nghttp2"
>  PACKAGECONFIG[openssl] = "--with-openssl,--without-openssl,openssl"
>  PACKAGECONFIG[pop3] = "--enable-pop3,--disable-pop3,"
> @@ -56,7 +62,6 @@ PACKAGECONFIG[rtmpdump] = "--with-librtmp,--without-librtmp,rtmpdump"
>  PACKAGECONFIG[rtsp] = "--enable-rtsp,--disable-rtsp,"
>  PACKAGECONFIG[smb] = "--enable-smb,--disable-smb,"
>  PACKAGECONFIG[smtp] = "--enable-smtp,--disable-smtp,"
> -PACKAGECONFIG[nss] = "--with-nss,--without-nss,nss"
>  PACKAGECONFIG[telnet] = "--enable-telnet,--disable-telnet,"
>  PACKAGECONFIG[tftp] = "--enable-tftp,--disable-tftp,"
>  PACKAGECONFIG[threaded-resolver] = "--enable-threaded-resolver,--disable-threaded-resolver,,,,ares"
> @@ -67,11 +72,10 @@ PACKAGECONFIG[zstd] = "--with-zstd,--without-zstd,zstd"
>  EXTRA_OECONF = " \
>      --disable-libcurl-option \
>      --disable-ntlm-wb \
> -    --enable-crypto-auth \
>      --with-ca-bundle=${sysconfdir}/ssl/certs/ca-certificates.crt \
>      --without-libpsl \
>      --enable-optimize \
> -    ${@'--without-ssl' if (bb.utils.filter('PACKAGECONFIG', 'gnutls mbedtls nss openssl', d) == '') else ''} \
> +    ${@'--without-ssl' if (bb.utils.filter('PACKAGECONFIG', 'gnutls mbedtls openssl', d) == '') else ''} \
>  "
>
>  do_install:append:class-target() {
> --
> 2.41.0
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#188152): https://lists.openembedded.org/g/openembedded-core/message/188152
> Mute This Topic: https://lists.openembedded.org/mt/101543494/1997914
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [raj.khem@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>

On 9/26/23 7:55 AM, Khem Raj wrote:
> I am seeing a ptest failure on qemx86-64/glibc
>
>    Failed ptests:
>    {'curl': ['test_1474', 'curl']}

In looking at the test, it is marked as flaky with the comment:

# Because of the timing sensitivity (scheduling delays of 500 msec can cause
# the test to fail), this test is marked flaky to avoid it being run in 
the CI
# builds which are often run on overloaded servers.

https://github.com/curl/curl/blob/curl-8_3_0/tests/data/test1474#L21

My server is pretty lightly loaded while running the tests, so that 
could explain why I haven't seen it fail normally. I was able to induce 
a failure by running a very heavy load while running that test on my 
desktop (which is older and slower than my build server). If you'd like, 
I can send a follow up patch to skip tests marked as flaky.

Thanks,

Robert

On Tue, Sep 26, 2023 at 7:55 PM Robert Joslyn
<robert.joslyn@redrectangle.org> wrote:
>
> On 9/26/23 7:55 AM, Khem Raj wrote:
> > I am seeing a ptest failure on qemx86-64/glibc
> >
> >    Failed ptests:
> >    {'curl': ['test_1474', 'curl']}
>
> In looking at the test, it is marked as flaky with the comment:
>
> # Because of the timing sensitivity (scheduling delays of 500 msec can cause
> # the test to fail), this test is marked flaky to avoid it being run in
> the CI
> # builds which are often run on overloaded servers.
>
> https://github.com/curl/curl/blob/curl-8_3_0/tests/data/test1474#L21
>
> My server is pretty lightly loaded while running the tests, so that
> could explain why I haven't seen it fail normally. I was able to induce
> a failure by running a very heavy load while running that test on my
> desktop (which is older and slower than my build server). If you'd like,
> I can send a follow up patch to skip tests marked as flaky.
>

another run resulted in yet another failure

  test 0587...
   FAIL: 587: protoc!
   There was no content at all in the file log/2/server.input.
   Server glitch? Total curl failure? Returned: 42

prior to this upgrade my test setup never saw curl failures and it has
same amount of load etc.


> Thanks,
>
> Robert
>

On 9/26/23 8:51 PM, Khem Raj wrote:
> On Tue, Sep 26, 2023 at 7:55 PM Robert Joslyn
> <robert.joslyn@redrectangle.org> wrote:
>> On 9/26/23 7:55 AM, Khem Raj wrote:
>>> I am seeing a ptest failure on qemx86-64/glibc
>>>
>>>     Failed ptests:
>>>     {'curl': ['test_1474', 'curl']}
>> In looking at the test, it is marked as flaky with the comment:
>>
>> # Because of the timing sensitivity (scheduling delays of 500 msec can cause
>> # the test to fail), this test is marked flaky to avoid it being run in
>> the CI
>> # builds which are often run on overloaded servers.
>>
>> https://github.com/curl/curl/blob/curl-8_3_0/tests/data/test1474#L21
>>
>> My server is pretty lightly loaded while running the tests, so that
>> could explain why I haven't seen it fail normally. I was able to induce
>> a failure by running a very heavy load while running that test on my
>> desktop (which is older and slower than my build server). If you'd like,
>> I can send a follow up patch to skip tests marked as flaky.
>>
> another run resulted in yet another failure
>
>    test 0587...
>     FAIL: 587: protoc!
>     There was no content at all in the file log/2/server.input.
>     Server glitch? Total curl failure? Returned: 42
>
> prior to this upgrade my test setup never saw curl failures and it has
> same amount of load etc.

Test 587 is also marked flaky by upstream. There are 12 tests marked as 
flaky total, and since you've just hit two of them, I'm much more 
inclined to just skip them. Right before this version bump there was 
also an update to run the curl ptests in parallel, which could be 
contributing to the change in test behavior.

Thanks,

Robert