| Message ID | 4a7647c4169ffc0c70858a4a1f1b2eb18ffcfb70.1715765153.git.liezhi.yang@windriver.com |
|---|---|
| State | New |
| Headers | show |
| Series | [1/1] fetch2/git.py: Try without '.git' suffix firstly | expand |
Hi Robert, On 5/15/24 11:26 AM, Robert Yang via lists.openembedded.org wrote: > From: Robert Yang <liezhi.yang@windriver.com> > > If the repo on server is foo.git, both of the following commands > will work: > 1) $ git clone <url>/foo.git > 2) $ git clone <url>/foo > But only the second command works if the repo server is foo > (without .git suffix), so try without '.git' suffix firstly. > Shouldn't the SRC_URI in the recipe be fixed instead of the git fetcher? Which use case do we need to support here? Something to do with mirroring maybe? Cheers, Quentin
Hi Quentin, On 5/15/24 17:35, Quentin Schulz wrote: > Hi Robert, > > On 5/15/24 11:26 AM, Robert Yang via lists.openembedded.org wrote: >> From: Robert Yang <liezhi.yang@windriver.com> >> >> If the repo on server is foo.git, both of the following commands >> will work: >> 1) $ git clone <url>/foo.git >> 2) $ git clone <url>/foo >> But only the second command works if the repo server is foo >> (without .git suffix), so try without '.git' suffix firstly. >> > > Shouldn't the SRC_URI in the recipe be fixed instead of the git fetcher? Which > use case do we need to support here? Something to do with mirroring maybe The usage is related to repo command + http/https mirror: * The foo.git on http/https MIRROR works with do_fetch * But 'repo' command always add a '.git' suffix to the reponame, the foo.git will be saved as foo.git.git on local disk, and do_fetch would not work with the local mirror. Our solution is rename foo.git to foo on http/https mirror, and apply this patch to make both http(s) and local mirror work with repo command. It's a little special to our Wind River Linux, and we need send the patch to mailing list for Yocto compliance. // Robert > > Cheers, > Quentin
Hi Robert, On 5/15/24 11:57 AM, Robert Yang via lists.openembedded.org wrote: > Hi Quentin, > > On 5/15/24 17:35, Quentin Schulz wrote: >> Hi Robert, >> >> On 5/15/24 11:26 AM, Robert Yang via lists.openembedded.org wrote: >>> From: Robert Yang <liezhi.yang@windriver.com> >>> >>> If the repo on server is foo.git, both of the following commands >>> will work: >>> 1) $ git clone <url>/foo.git >>> 2) $ git clone <url>/foo >>> But only the second command works if the repo server is foo >>> (without .git suffix), so try without '.git' suffix firstly. >>> >> >> Shouldn't the SRC_URI in the recipe be fixed instead of the git >> fetcher? Which use case do we need to support here? Something to do >> with mirroring maybe > The usage is related to repo command + http/https mirror: > > * The foo.git on http/https MIRROR works with do_fetch > * But 'repo' command always add a '.git' suffix to the reponame, the > foo.git > will be saved as foo.git.git on local disk, and do_fetch would not work > with > the local mirror. > > Our solution is rename foo.git to foo on http/https mirror, and apply > this patch > to make both http(s) and local mirror work with repo command. > > It's a little special to our Wind River Linux, and we need send the > patch to > mailing list for Yocto compliance. > I'm not sure this is something we should fix in BitBake since it's related to the use of some external setup tool? I'll let more knowledgeable people discuss this :) In any case, I would suggest to really provide more info (the one as above for example) at the very least in the commit log, and even better in-code comment so that we don't remove it the day we want to de-clutter code. The issue is that this looks very much like code we don't need, without proper explanation of the use case (which you now provided). Cheers, Quentin > // Robert > >> >> Cheers, >> Quentin > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#16222): https://lists.openembedded.org/g/bitbake-devel/message/16222 > Mute This Topic: https://lists.openembedded.org/mt/106110992/6293953 > Group Owner: bitbake-devel+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [quentin.schulz@cherry.de] > -=-=-=-=-=-=-=-=-=-=-=- >
On Wed, 2024-05-15 at 17:57 +0800, Robert Yang via lists.openembedded.org wrote: > Hi Quentin, > > On 5/15/24 17:35, Quentin Schulz wrote: > > Hi Robert, > > > > On 5/15/24 11:26 AM, Robert Yang via lists.openembedded.org wrote: > > > From: Robert Yang <liezhi.yang@windriver.com> > > > > > > If the repo on server is foo.git, both of the following commands > > > will work: > > > 1) $ git clone <url>/foo.git > > > 2) $ git clone <url>/foo > > > But only the second command works if the repo server is foo > > > (without .git suffix), so try without '.git' suffix firstly. > > > > > > > Shouldn't the SRC_URI in the recipe be fixed instead of the git fetcher? Which > > use case do we need to support here? Something to do with mirroring maybe > The usage is related to repo command + http/https mirror: > > * The foo.git on http/https MIRROR works with do_fetch > * But 'repo' command always add a '.git' suffix to the reponame, the foo.git > will be saved as foo.git.git on local disk, and do_fetch would not work with > the local mirror. > > Our solution is rename foo.git to foo on http/https mirror, and apply this patch > to make both http(s) and local mirror work with repo command. > > It's a little special to our Wind River Linux, and we need send the patch to > mailing list for Yocto compliance. I'm with Quentin, the SRC_URIs should be fixed or this should be in repo specific codepaths. We're not adding a workaround like that into the git fetcher. Cheers, Richard
diff --git a/bitbake/lib/bb/fetch2/git.py b/bitbake/lib/bb/fetch2/git.py index c7ff769fdfe..0134344cdf3 100644 --- a/bitbake/lib/bb/fetch2/git.py +++ b/bitbake/lib/bb/fetch2/git.py @@ -442,11 +442,30 @@ class Git(FetchMethod): objects = os.path.join(repourl_path, 'objects') if os.path.isdir(objects) and not os.path.islink(objects): repourl = repourl_path - clone_cmd = "LANG=C %s clone --bare --mirror %s %s --progress" % (ud.basecmd, shlex.quote(repourl), ud.clonedir) + clone_cmd_pre = "LANG=C %s clone --bare --mirror --progress" % ud.basecmd + clone_cmd = "%s %s %s" % (clone_cmd_pre, shlex.quote(repourl), ud.clonedir) if ud.proto.lower() != 'file': bb.fetch2.check_network_access(d, clone_cmd, ud.url) progresshandler = GitProgressHandler(d) - runfetchcmd(clone_cmd, d, log=progresshandler) + # If the repo on server is foo.git, both of the following commands + # will work: + # 1) $ git clone <url>/foo.git + # 2) $ git clone <url>/foo + # But only the second command works if the repo server is foo + # (without .git suffix), so try without '.git' suffix firstly + cloned_no_git = False + if repourl.endswith('.git'): + repourl_no_git = repourl[0:-4] + clone_cmd_no_git = "%s %s %s" % (clone_cmd_pre, shlex.quote(repourl_no_git), ud.clonedir) + try: + runfetchcmd(clone_cmd_no_git, d, log=progresshandler) + cloned_no_git = True + except bb.fetch2.FetchError: + # The error messages have already been saved in the log, so + # just pass to next cmd. + pass + if not cloned_no_git: + runfetchcmd(clone_cmd, d, log=progresshandler) # Update the checkout if needed if self.clonedir_need_update(ud, d):