[bitbake-devel] fetch2/repo.py: improve unpack to copy only asked project.

Submitted by Nicolas Cornu via bitbake-devel on June 20, 2018, 6:13 a.m. | Patch ID: 151993

Details

Message ID 20180620061320.6846-1-nicolac76@yahoo.fr
State New
Headers show

Commit Message

Nicolas Cornu via bitbake-devel June 20, 2018, 6:13 a.m.
From: Nicolas Cornu <n.cornu@overkiz.com>

Today this fetcher copy the whole repo projects in each sub-project.
---
This patch is not here to be merged, but to have review on it.
The problem is that if your google-repo got 150 projects, and you want to build each with a specific recipe, sources
will be copied for each.

 lib/bb/fetch2/repo.py | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

Patch hide | download patch | download mbox

diff --git a/lib/bb/fetch2/repo.py b/lib/bb/fetch2/repo.py
index 8c7e8185..a01a548d 100644
--- a/lib/bb/fetch2/repo.py
+++ b/lib/bb/fetch2/repo.py
@@ -89,6 +89,23 @@  class Repo(FetchMethod):
         # Create a cache
         runfetchcmd("tar %s -czf %s %s" % (tar_flags, ud.localpath, os.path.join(".", "*") ), d, workdir=codir)
 
+    def unpack(self, ud, destdir, d):
+        """unpack the project from repo source directories"""
+
+        repodir = d.getVar("REPODIR") or (d.getVar("DL_DIR") + "/repo")
+        gitsrcname = "%s%s" % (ud.host, ud.path.replace("/", "."))
+        codir = os.path.join(repodir, gitsrcname, ud.manifest)
+        repodir = os.path.join(codir, "repo")
+
+        # With this command I get the full path to the project in the "repo" download dir.
+        full_path_to_download = runfetchcmd("%s list --path-only --fullpath %s" % ("repo", d.getVar("PN")), d, workdir=repodir)
+
+        # I do this or git clone say the directory is not empty.
+        runfetchcmd("rm -rf %s" % d.getVar("S"), d, workdir=destdir)
+
+        # Is it ok, here to use "S" directly? I see no fetcher do that.
+        runfetchcmd("git clone %s %s" % (full_path_to_download.strip(), d.getVar("S")), d, workdir=destdir)
+
     def supports_srcrev(self):
         return False
 

Comments

Andre McCurdy June 26, 2018, 4:09 a.m.
On Tue, Jun 19, 2018 at 11:13 PM, Nicolas Cornu via bitbake-devel
<bitbake-devel@lists.openembedded.org> wrote:
> From: Nicolas Cornu <n.cornu@overkiz.com>
>
> Today this fetcher copy the whole repo projects in each sub-project.
> ---
> This patch is not here to be merged, but to have review on it.
> The problem is that if your google-repo got 150 projects, and you want to build each with a specific recipe, sources
> will be copied for each.

If your manifest contains 150 repos but you want to fetch and build
them individually then perhaps it's better not to use the repo fetcher
all?

You could either just put the git URIs directly into your recipes or
(if your manifest changes frequently and you don't want the recipes to
get out of sync) perhaps write a script to process the output of "repo
manifest" and auto generate a .conf file containing the git based
SRC_URI definitions which your recipes can include.

>  lib/bb/fetch2/repo.py | 17 +++++++++++++++++
>  1 file changed, 17 insertions(+)
>
> diff --git a/lib/bb/fetch2/repo.py b/lib/bb/fetch2/repo.py
> index 8c7e8185..a01a548d 100644
> --- a/lib/bb/fetch2/repo.py
> +++ b/lib/bb/fetch2/repo.py
> @@ -89,6 +89,23 @@ class Repo(FetchMethod):
>          # Create a cache
>          runfetchcmd("tar %s -czf %s %s" % (tar_flags, ud.localpath, os.path.join(".", "*") ), d, workdir=codir)
>
> +    def unpack(self, ud, destdir, d):
> +        """unpack the project from repo source directories"""
> +
> +        repodir = d.getVar("REPODIR") or (d.getVar("DL_DIR") + "/repo")
> +        gitsrcname = "%s%s" % (ud.host, ud.path.replace("/", "."))
> +        codir = os.path.join(repodir, gitsrcname, ud.manifest)
> +        repodir = os.path.join(codir, "repo")
> +
> +        # With this command I get the full path to the project in the "repo" download dir.
> +        full_path_to_download = runfetchcmd("%s list --path-only --fullpath %s" % ("repo", d.getVar("PN")), d, workdir=repodir)
> +
> +        # I do this or git clone say the directory is not empty.
> +        runfetchcmd("rm -rf %s" % d.getVar("S"), d, workdir=destdir)
> +
> +        # Is it ok, here to use "S" directly? I see no fetcher do that.
> +        runfetchcmd("git clone %s %s" % (full_path_to_download.strip(), d.getVar("S")), d, workdir=destdir)
> +
>      def supports_srcrev(self):
>          return False
>
> --
> 2.17.1
>
> --
> _______________________________________________
> bitbake-devel mailing list
> bitbake-devel@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/bitbake-devel