[bitbake-devel,2/3] gitsm.py: Add support for alternative URL formats from submodule files

Submitted by Mark Hatle on Jan. 8, 2019, 11:38 p.m. | Patch ID: 157640

Details

Message ID 20190108233847.81353-3-mark.hatle@windriver.com
State Accepted
Commit dcac05e7dc6b0c5f8e63d36ad105b7eab41f0016
Headers show

Commit Message

Mark Hatle Jan. 8, 2019, 11:38 p.m.
The following appear to be the git supported formats:

  proto://user:pass@host/path  (URI format)
  user@host:path (SSH format)
  /path or ./path or ../path (local file format)

We adjust the parsing to find out if we have a URI format or not.
When we are NOT in URI format, we do our best to determine SSH or
file format by looking for a ':' in the overall string.  If we find
a ':' we assume SSH format and adjust accordingly.

Note, in SSH format we simply replace the ':' with a '/' when constructing
the URL.  However, if the original path was ":/...", we don't want '//' so
we deal with this corner case as well.

Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
---
 lib/bb/fetch2/gitsm.py | 17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)

Patch hide | download patch | download mbox

diff --git a/lib/bb/fetch2/gitsm.py b/lib/bb/fetch2/gitsm.py
index 88609f0..0981bba 100644
--- a/lib/bb/fetch2/gitsm.py
+++ b/lib/bb/fetch2/gitsm.py
@@ -95,8 +95,21 @@  class GitSM(Git):
             module_hash = module_hash.split()[2]
 
             # Build new SRC_URI
-            proto = uris[module].split(':', 1)[0]
-            url = uris[module].replace('%s:' % proto, 'gitsm:', 1)
+            if "://" not in uris[module]:
+                # It's ssh if the format does NOT have "://", but has a ':'
+                if ":" in uris[module]:
+                    proto = "ssh"
+                    if ":/" in uris[module]:
+                        url = "gitsm://" + uris[module].replace(':/', '/', 1)
+                    else:
+                        url = "gitsm://" + uris[module].replace(':', '/', 1)
+                else: # Fall back to 'file' if there is no ':'
+                    proto = "file"
+                    url = "gitsm://" + uris[module]
+            else:
+                proto = uris[module].split(':', 1)[0]
+                url = uris[module].replace('%s:' % proto, 'gitsm:', 1)
+
             url += ';protocol=%s' % proto
             url += ";name=%s" % module
             url += ";bareclone=1;nocheckout=1;nobranch=1"

Comments

Olof Johansson Jan. 9, 2019, 1:18 a.m.
On 19-01-08 18:38 -0500, Mark Hatle wrote:
> Note, in SSH format we simply replace the ':' with a '/' when constructing
> the URL.  However, if the original path was ":/...", we don't want '//' so
> we deal with this corner case as well.
...
> +  if ":/" in uris[module]:
> +      url = "gitsm://" + uris[module].replace(':/', '/', 1)
> +  else:
> +      url = "gitsm://" + uris[module].replace(':', '/', 1)

Without a / prefix in the original ssh path, the path becomes
relative to whatever the ssh server wants, usually the user's
$HOME. I.e., they do not refer to the same path on the remote
system.

  alice@example.com:/foo.git -> ssh://alice@example.com/foo.git
  alice@example.com:foo.git -> ssh://alice@example.com/home/alice/foo.git (or something else!)

The path in an ssh:// uri is absolute, so the relative case must
be handled differently. But it's easy, git supports ~ expansion,
like:

  alice@example.com:foo.git -> ssh://alice@example.com/~/foo.git
Olof Johansson Jan. 9, 2019, 1:29 a.m.
On 19-01-09 02:18 +0100, Olof Johansson wrote:
> The path in an ssh:// uri is absolute, so the relative case must
> be handled differently. But it's easy, git supports ~ expansion,
> like:
> 
>   alice@example.com:foo.git -> ssh://alice@example.com/~/foo.git

Tested some more. This works with git over openssh (possibly only
against unix systems), but not with github it seems, so this
wasn't as general of a solution I hoped. Sorry. I think the issue
is real, but my proposed solution wasn't :(.

And, of course, the github ssh remote specifiers all look like
git@github.com:alice/foo.
Mark Hatle Jan. 9, 2019, 4:33 p.m.
On 1/8/19 7:29 PM, Olof Johansson wrote:
> On 19-01-09 02:18 +0100, Olof Johansson wrote:
>> The path in an ssh:// uri is absolute, so the relative case must
>> be handled differently. But it's easy, git supports ~ expansion,
>> like:
>>
>>   alice@example.com:foo.git -> ssh://alice@example.com/~/foo.git
> 
> Tested some more. This works with git over openssh (possibly only
> against unix systems), but not with github it seems, so this
> wasn't as general of a solution I hoped. Sorry. I think the issue
> is real, but my proposed solution wasn't :(.
> 
> And, of course, the github ssh remote specifiers all look like
> git@github.com:alice/foo.
> 

Unfortunately, this is what I observed as well.  In the specific test cases I
had replacing it 'as-is' was a better solution.  (test case was specifically
github, and gitolite based)

The only alternative I can see in this case, is to try one fetch (try/catch) and
if it fails to try the alternative.  If they both fail, then it becomes a fetch
failure.

--Mark
Mark Hatle Jan. 16, 2019, 8:35 a.m.
This is a system generated Comment: Patch 157640 was automatically marked as superseded by patch 157955.