Patchwork [bitbake-devel,1/1] fetch2/__init__.py: remove broken sysmlink

login
register
mail settings
Submitter Robert Yang
Date Aug. 19, 2014, 9:07 a.m.
Message ID <d2f547297c3e0004e4fc64457cd36f7899db60d5.1408439224.git.liezhi.yang@windriver.com>
Download mbox | patch
Permalink /patch/78567/
State New
Headers show

Comments

Robert Yang - Aug. 19, 2014, 9:07 a.m.
Fixed:
(Simulate mirror the tarball from PREMIRRORS)
$ ln -sf /path/to/local/PREMIRROR/bzip2 bzip2-1.0.6.tar.gz
(Simulate the PREMIRRORS are gone)
$ rm -f /path/to/local/PREMIRROR/bzip2-1.0.6.tar.gz
$ bitbake bzip2 -ccleansstate && bitbake bzip2 -cfetch

Then the newly get bzip2-1.0.6.tar.gz will be saved to
/path/to/local/PREMIRROR/bzip2-1.0.6.tar.gz rather than then the DL_DIR,
or get error if we don't have the write permission on to the mirror.
This is because wget uses "lstat()" to follow the symlink for FTP (but
fstat() for HTTP, so http doesn't have this issue), and we can't fix
wget (this might be intended by wget) since it is one of
SANITY_REQUIRED_UTILITIES.

Remove the broken sysmlink in DL_DIR will fix the problem.

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 bitbake/lib/bb/fetch2/__init__.py |    5 +++++
 1 file changed, 5 insertions(+)
Martin Jansa - Aug. 19, 2014, 10:15 a.m.
On Tue, Aug 19, 2014 at 02:07:41AM -0700, Robert Yang wrote:
> Fixed:
> (Simulate mirror the tarball from PREMIRRORS)
> $ ln -sf /path/to/local/PREMIRROR/bzip2 bzip2-1.0.6.tar.gz
> (Simulate the PREMIRRORS are gone)
> $ rm -f /path/to/local/PREMIRROR/bzip2-1.0.6.tar.gz
> $ bitbake bzip2 -ccleansstate && bitbake bzip2 -cfetch

Why does it need to re-fetch it when you haven't don't cleanall?

If it has different checksums, then fetcher should rename it, before
re-downloading it.

Shouldn't fetcher just use bzip2-1.0.6.tar.gz if there is
bzip2-1.0.6.tar.gz.done?

I'm not saying that your change isn't correct, but maybe it uncovers
worse issue with fetcher redownloading unnecessary.

> Then the newly get bzip2-1.0.6.tar.gz will be saved to
> /path/to/local/PREMIRROR/bzip2-1.0.6.tar.gz rather than then the DL_DIR,
> or get error if we don't have the write permission on to the mirror.
> This is because wget uses "lstat()" to follow the symlink for FTP (but
> fstat() for HTTP, so http doesn't have this issue), and we can't fix
> wget (this might be intended by wget) since it is one of
> SANITY_REQUIRED_UTILITIES.
> 
> Remove the broken sysmlink in DL_DIR will fix the problem.

type symlink

> 
> Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
> ---
>  bitbake/lib/bb/fetch2/__init__.py |    5 +++++
>  1 file changed, 5 insertions(+)
> 
> diff --git a/bitbake/lib/bb/fetch2/__init__.py b/bitbake/lib/bb/fetch2/__init__.py
> index df2f2b0..c95f177 100644
> --- a/bitbake/lib/bb/fetch2/__init__.py
> +++ b/bitbake/lib/bb/fetch2/__init__.py
> @@ -1081,6 +1081,11 @@ class FetchData(object):
>          self.donestamp = basepath + '.done'
>          self.lockfile = basepath + '.lock'
>  
> +        # Remove the broken sysmlink

typo "symlink"

> +        if self.localpath and not os.path.exists(self.localpath) and \
> +                os.path.islink(self.localpath):
> +            os.unlink(self.localpath)
> +
>      def setup_revisons(self, d):
>          self.revisions = {}
>          for name in self.names:
> -- 
> 1.7.9.5
> 
> -- 
> _______________________________________________
> bitbake-devel mailing list
> bitbake-devel@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/bitbake-devel
Robert Yang - Aug. 19, 2014, 10:34 a.m.
On 08/19/2014 06:15 PM, Martin Jansa wrote:
> On Tue, Aug 19, 2014 at 02:07:41AM -0700, Robert Yang wrote:
>> Fixed:
>> (Simulate mirror the tarball from PREMIRRORS)
>> $ ln -sf /path/to/local/PREMIRROR/bzip2 bzip2-1.0.6.tar.gz
>> (Simulate the PREMIRRORS are gone)
>> $ rm -f /path/to/local/PREMIRROR/bzip2-1.0.6.tar.gz
>> $ bitbake bzip2 -ccleansstate && bitbake bzip2 -cfetch
>
> Why does it need to re-fetch it when you haven't don't cleanall?

I met this error when the local PREMIRROR is gone, I think that
it needs re-fetch is because the symlink is broken? (os.path.exits()
returns False ?)

>
> If it has different checksums, then fetcher should rename it, before
> re-downloading it.
>
> Shouldn't fetcher just use bzip2-1.0.6.tar.gz if there is
> bzip2-1.0.6.tar.gz.done?
>
> I'm not saying that your change isn't correct, but maybe it uncovers
> worse issue with fetcher redownloading unnecessary.
>
>> Then the newly get bzip2-1.0.6.tar.gz will be saved to
>> /path/to/local/PREMIRROR/bzip2-1.0.6.tar.gz rather than then the DL_DIR,
>> or get error if we don't have the write permission on to the mirror.
>> This is because wget uses "lstat()" to follow the symlink for FTP (but
>> fstat() for HTTP, so http doesn't have this issue), and we can't fix
>> wget (this might be intended by wget) since it is one of
>> SANITY_REQUIRED_UTILITIES.
>>
>> Remove the broken sysmlink in DL_DIR will fix the problem.
>
> type symlink

Thanks, fixed the typos in the repo.

// Robert

>
>>
>> Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
>> ---
>>   bitbake/lib/bb/fetch2/__init__.py |    5 +++++
>>   1 file changed, 5 insertions(+)
>>
>> diff --git a/bitbake/lib/bb/fetch2/__init__.py b/bitbake/lib/bb/fetch2/__init__.py
>> index df2f2b0..c95f177 100644
>> --- a/bitbake/lib/bb/fetch2/__init__.py
>> +++ b/bitbake/lib/bb/fetch2/__init__.py
>> @@ -1081,6 +1081,11 @@ class FetchData(object):
>>           self.donestamp = basepath + '.done'
>>           self.lockfile = basepath + '.lock'
>>
>> +        # Remove the broken sysmlink
>
> typo "symlink"
>
>> +        if self.localpath and not os.path.exists(self.localpath) and \
>> +                os.path.islink(self.localpath):
>> +            os.unlink(self.localpath)
>> +
>>       def setup_revisons(self, d):
>>           self.revisions = {}
>>           for name in self.names:
>> --
>> 1.7.9.5
>>
>> --
>> _______________________________________________
>> bitbake-devel mailing list
>> bitbake-devel@lists.openembedded.org
>> http://lists.openembedded.org/mailman/listinfo/bitbake-devel
>

Patch

diff --git a/bitbake/lib/bb/fetch2/__init__.py b/bitbake/lib/bb/fetch2/__init__.py
index df2f2b0..c95f177 100644
--- a/bitbake/lib/bb/fetch2/__init__.py
+++ b/bitbake/lib/bb/fetch2/__init__.py
@@ -1081,6 +1081,11 @@  class FetchData(object):
         self.donestamp = basepath + '.done'
         self.lockfile = basepath + '.lock'
 
+        # Remove the broken sysmlink
+        if self.localpath and not os.path.exists(self.localpath) and \
+                os.path.islink(self.localpath):
+            os.unlink(self.localpath)
+
     def setup_revisons(self, d):
         self.revisions = {}
         for name in self.names: