[bitbake-devel,1/1] wget.py: Print warning when checkstatus() failed

Submitted by Robert Yang on April 23, 2019, 7:43 a.m. | Patch ID: 160612

Details

Message ID 26bea95ed677fdf7d4c18f80e1e34ffb7c9fa364.1556005325.git.liezhi.yang@windriver.com
State New
Headers show

Commit Message

Robert Yang April 23, 2019, 7:43 a.m.
This makes it easier to debug, for example, when I ran:

$ bitbake-selftest bb.tests.fetch.FetchCheckStatusTest

It said failed:
FAIL: test_wget_checkstatus_connection_cache (bb.tests.fetch.FetchCheckStatusTest) (url='ftp://sourceware.org/pub/libffi/libffi-1.20.tar.gz')

But didn't know the reason, this warning message can help me find that
something is wrong with my ftp settings (I set a wrong proxy for ftp).

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 bitbake/lib/bb/fetch2/wget.py | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

Patch hide | download patch | download mbox

diff --git a/bitbake/lib/bb/fetch2/wget.py b/bitbake/lib/bb/fetch2/wget.py
index 3bb3e3b..ffcd40a 100644
--- a/bitbake/lib/bb/fetch2/wget.py
+++ b/bitbake/lib/bb/fetch2/wget.py
@@ -216,7 +216,7 @@  class Wget(FetchMethod):
                         # Apparently urllib then uses the file descriptor, expecting it to be
                         # connected, when in reality the connection is already gone.
                         # We let the request fail and expect it to be
-                        # tried once more ("try_again" in check_status()),
+                        # tried once more ("try_again" in checkstatus()),
                         # with the dead connection removed from the cache.
                         # If it still fails, we give up, which can happend for bad
                         # HTTP proxy settings.
@@ -338,8 +338,7 @@  class Wget(FetchMethod):
                 logger.debug(2, "checkstatus: trying again")
                 return self.checkstatus(fetch, ud, d, False)
             else:
-                # debug for now to avoid spamming the logs in e.g. remote sstate searches
-                logger.debug(2, "checkstatus() urlopen failed: %s" % e)
+                logger.warning("checkstatus() urlopen failed: %s" % e)
                 return False
         return True
 

Comments

Robert Yang April 23, 2019, 7:53 a.m.
Hi Ross,

On 4/23/19 3:43 PM, Robert Yang wrote:
> This makes it easier to debug, for example, when I ran:
> 
> $ bitbake-selftest bb.tests.fetch.FetchCheckStatusTest
> 
> It said failed:
> FAIL: test_wget_checkstatus_connection_cache (bb.tests.fetch.FetchCheckStatusTest) (url='ftp://sourceware.org/pub/libffi/libffi-1.20.tar.gz')
> 
> But didn't know the reason, this warning message can help me find that
> something is wrong with my ftp settings (I set a wrong proxy for ftp).
> 
> Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
> ---
>   bitbake/lib/bb/fetch2/wget.py | 5 ++---
>   1 file changed, 2 insertions(+), 3 deletions(-)
> 
> diff --git a/bitbake/lib/bb/fetch2/wget.py b/bitbake/lib/bb/fetch2/wget.py
> index 3bb3e3b..ffcd40a 100644
> --- a/bitbake/lib/bb/fetch2/wget.py
> +++ b/bitbake/lib/bb/fetch2/wget.py
> @@ -216,7 +216,7 @@ class Wget(FetchMethod):
>                           # Apparently urllib then uses the file descriptor, expecting it to be
>                           # connected, when in reality the connection is already gone.
>                           # We let the request fail and expect it to be
> -                        # tried once more ("try_again" in check_status()),
> +                        # tried once more ("try_again" in checkstatus()),
>                           # with the dead connection removed from the cache.
>                           # If it still fails, we give up, which can happend for bad
>                           # HTTP proxy settings.
> @@ -338,8 +338,7 @@ class Wget(FetchMethod):
>                   logger.debug(2, "checkstatus: trying again")
>                   return self.checkstatus(fetch, ud, d, False)
>               else:
> -                # debug for now to avoid spamming the logs in e.g. remote sstate searches
> -                logger.debug(2, "checkstatus() urlopen failed: %s" % e)

You added these comments in the past, is the "spamming" still an issue, please?
Or how can I reproduce it?

// Robert

> +                logger.warning("checkstatus() urlopen failed: %s" % e)
>                   return False
>           return True
>   
>
Richard Purdie April 23, 2019, 1:58 p.m.
On Tue, 2019-04-23 at 15:53 +0800, Robert Yang wrote:
> Hi Ross,
> 
> On 4/23/19 3:43 PM, Robert Yang wrote:
> > This makes it easier to debug, for example, when I ran:
> > 
> > $ bitbake-selftest bb.tests.fetch.FetchCheckStatusTest
> > 
> > It said failed:
> > FAIL: test_wget_checkstatus_connection_cache
> > (bb.tests.fetch.FetchCheckStatusTest) (url='
> > ftp://sourceware.org/pub/libffi/libffi-1.20.tar.gz')
> > 
> > But didn't know the reason, this warning message can help me find
> > that
> > something is wrong with my ftp settings (I set a wrong proxy for
> > ftp).
> > 
> > Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
> > ---
> >   bitbake/lib/bb/fetch2/wget.py | 5 ++---
> >   1 file changed, 2 insertions(+), 3 deletions(-)
> > 
> > diff --git a/bitbake/lib/bb/fetch2/wget.py
> > b/bitbake/lib/bb/fetch2/wget.py
> > index 3bb3e3b..ffcd40a 100644
> > --- a/bitbake/lib/bb/fetch2/wget.py
> > +++ b/bitbake/lib/bb/fetch2/wget.py
> > @@ -216,7 +216,7 @@ class Wget(FetchMethod):
> >                           # Apparently urllib then uses the file
> > descriptor, expecting it to be
> >                           # connected, when in reality the
> > connection is already gone.
> >                           # We let the request fail and expect it
> > to be
> > -                        # tried once more ("try_again" in
> > check_status()),
> > +                        # tried once more ("try_again" in
> > checkstatus()),
> >                           # with the dead connection removed from
> > the cache.
> >                           # If it still fails, we give up, which
> > can happend for bad
> >                           # HTTP proxy settings.
> > @@ -338,8 +338,7 @@ class Wget(FetchMethod):
> >                   logger.debug(2, "checkstatus: trying again")
> >                   return self.checkstatus(fetch, ud, d, False)
> >               else:
> > -                # debug for now to avoid spamming the logs in e.g.
> > remote sstate searches
> > -                logger.debug(2, "checkstatus() urlopen failed: %s"
> > % e)
> 
> You added these comments in the past, is the "spamming" still an
> issue, please?
> Or how can I reproduce it?

Set a remote (http) sstate mirror which doesn't exist and then try a
core-image-sato-sdk with no local sstate?

Cheers,

Richard
Robert Yang April 25, 2019, 9:04 a.m.
On 4/23/19 9:58 PM, Richard Purdie wrote:
> On Tue, 2019-04-23 at 15:53 +0800, Robert Yang wrote:
>> Hi Ross,
>>
>> On 4/23/19 3:43 PM, Robert Yang wrote:
>>> This makes it easier to debug, for example, when I ran:
>>>
>>> $ bitbake-selftest bb.tests.fetch.FetchCheckStatusTest
>>>
>>> It said failed:
>>> FAIL: test_wget_checkstatus_connection_cache
>>> (bb.tests.fetch.FetchCheckStatusTest) (url='
>>> ftp://sourceware.org/pub/libffi/libffi-1.20.tar.gz')
>>>
>>> But didn't know the reason, this warning message can help me find
>>> that
>>> something is wrong with my ftp settings (I set a wrong proxy for
>>> ftp).
>>>
>>> Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
>>> ---
>>>    bitbake/lib/bb/fetch2/wget.py | 5 ++---
>>>    1 file changed, 2 insertions(+), 3 deletions(-)
>>>
>>> diff --git a/bitbake/lib/bb/fetch2/wget.py
>>> b/bitbake/lib/bb/fetch2/wget.py
>>> index 3bb3e3b..ffcd40a 100644
>>> --- a/bitbake/lib/bb/fetch2/wget.py
>>> +++ b/bitbake/lib/bb/fetch2/wget.py
>>> @@ -216,7 +216,7 @@ class Wget(FetchMethod):
>>>                            # Apparently urllib then uses the file
>>> descriptor, expecting it to be
>>>                            # connected, when in reality the
>>> connection is already gone.
>>>                            # We let the request fail and expect it
>>> to be
>>> -                        # tried once more ("try_again" in
>>> check_status()),
>>> +                        # tried once more ("try_again" in
>>> checkstatus()),
>>>                            # with the dead connection removed from
>>> the cache.
>>>                            # If it still fails, we give up, which
>>> can happend for bad
>>>                            # HTTP proxy settings.
>>> @@ -338,8 +338,7 @@ class Wget(FetchMethod):
>>>                    logger.debug(2, "checkstatus: trying again")
>>>                    return self.checkstatus(fetch, ud, d, False)
>>>                else:
>>> -                # debug for now to avoid spamming the logs in e.g.
>>> remote sstate searches
>>> -                logger.debug(2, "checkstatus() urlopen failed: %s"
>>> % e)
>>
>> You added these comments in the past, is the "spamming" still an
>> issue, please?
>> Or how can I reproduce it?
> 
> Set a remote (http) sstate mirror which doesn't exist and then try a
> core-image-sato-sdk with no local sstate?

Thanks, yes, indeed, I got a lot of warnings with a invalid remote sstate
mirror, so please ignore this patch.

// Robert

> 
> Cheers,
> 
> Richard
> 
>