[bitbake-devel,2/3] fetch2: Gracefully handle corrupt download-cache tarballs

Submitted by Stefan Klug on March 18, 2019, 1:58 p.m. | Patch ID: 159636

Details

Message ID 20190318135820.7344-3-stefan.klug@baslerweb.com
State New
Headers show

Commit Message

Stefan Klug March 18, 2019, 1:58 p.m.
If the fullmirror tarball is corrupt for whatever reason
(IMHO there are no checksums on the download cache)
a series of nasty events was triggered:
- tar left a partially extracted bare git repo there
- on the next yocto build, the corrupt bare repo is
  found and bitbake starts to update that bare repo using git
- git fails to detect it as bare repo. Therefore
  all following git commands ripple up the directory tree,
  in our case modifying a top level git repo.

Signed-off-by: Stefan Klug <stefan.klug@baslerweb.com>
---
 lib/bb/fetch2/git.py | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

Patch hide | download patch | download mbox

diff --git a/lib/bb/fetch2/git.py b/lib/bb/fetch2/git.py
index 1a8ebe3d..d5f9bbcd 100644
--- a/lib/bb/fetch2/git.py
+++ b/lib/bb/fetch2/git.py
@@ -334,8 +334,11 @@  class Git(FetchMethod):
             ud.localpath = ud.fullshallow
             return
         elif os.path.exists(ud.fullmirror) and not os.path.exists(ud.clonedir):
-            bb.utils.mkdirhier(ud.clonedir)
-            runfetchcmd("tar -xzf %s" % ud.fullmirror, d, workdir=ud.clonedir)
+            try:
+                bb.utils.mkdirhier(ud.clonedir)
+                runfetchcmd("tar -xzf %s" % ud.fullmirror, d, workdir=ud.clonedir, cleanup=[ud.clonedir])
+            except:
+                logger.info("Extracting tarball of git repository failed, falling back to clone.")
 
         repourl = self._get_repo_url(ud)
 

Comments

Richard Purdie March 18, 2019, 11:45 p.m.
On Mon, 2019-03-18 at 14:58 +0100, Stefan Klug wrote:
> If the fullmirror tarball is corrupt for whatever reason
> (IMHO there are no checksums on the download cache)
> a series of nasty events was triggered:
> - tar left a partially extracted bare git repo there
> - on the next yocto build, the corrupt bare repo is
>   found and bitbake starts to update that bare repo using git
> - git fails to detect it as bare repo. Therefore
>   all following git commands ripple up the directory tree,
>   in our case modifying a top level git repo.
> 
> Signed-off-by: Stefan Klug <stefan.klug@baslerweb.com>
> ---
>  lib/bb/fetch2/git.py | 7 +++++--
>  1 file changed, 5 insertions(+), 2 deletions(-)
> 
> diff --git a/lib/bb/fetch2/git.py b/lib/bb/fetch2/git.py
> index 1a8ebe3d..d5f9bbcd 100644
> --- a/lib/bb/fetch2/git.py
> +++ b/lib/bb/fetch2/git.py
> @@ -334,8 +334,11 @@ class Git(FetchMethod):
>              ud.localpath = ud.fullshallow
>              return
>          elif os.path.exists(ud.fullmirror) and not os.path.exists(ud.clonedir):
> -            bb.utils.mkdirhier(ud.clonedir)
> -            runfetchcmd("tar -xzf %s" % ud.fullmirror, d, workdir=ud.clonedir)
> +            try:
> +                bb.utils.mkdirhier(ud.clonedir)
> +                runfetchcmd("tar -xzf %s" % ud.fullmirror, d, workdir=ud.clonedir, cleanup=[ud.clonedir])
> +            except:
> +                logger.info("Extracting tarball of git repository failed, falling back to clone.")

General "except:" clauses are a world of pain. Can we be more specific
here?

For an example of what I mean, put a syntax error in the command...

Cheers,

Richard