cmd/go: document lookup process for qualified module paths

[dwschulze]

What version of Go are you using (go version)?

go version go1.15.1 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/dean/.cache/go-build"
GOENV="/home/dean/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/dean/src/golang/go3p/pkg/mod"
GONOPROXY="ubuntu-18-extssd"
GONOSUMDB="ubuntu-18-extssd"
GOOS="linux"
GOPATH="/home/dean/src/golang/go3p:/home/dean/src/golang/examples/go-module-package-test/package/package-driver"
GOPRIVATE="ubuntu-18-extssd"
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/dean/bin/go1.15.1.linux-amd64/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/dean/bin/go1.15.1.linux-amd64/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build013931004=/tmp/go-build -gno-record-gcc-switches"

What did you do?

I have a git repo on a computer on my network (hostname ubuntu-18-extssd, IP 192.168.0.12):

$ git remote --v
origin	dean@ubuntu-18-extssd:gitrepo/go-package-test-stringutil (fetch)

Inside of the repo is a package stringpackage.

The go get command fails:

$ go get -v ubuntu-18-extssd/gitrepo/go-package-test-stringutil/stringpackage
package ubuntu-18-extssd/gitrepo/go-package-test-stringutil/stringpackage: unrecognized import path "ubuntu-18-extssd/gitrepo/go-package-test-stringutil/stringpackage": import path does not begin with hostname

So I try with the IP address:

$ go get -v 192.168.0.12/gitrepo/go-package-test-stringutil/stringpackage
package 192.168.0.12/gitrepo/go-package-test-stringutil/stringpackage: unrecognized import path "192.168.0.12/gitrepo/go-package-test-stringutil/stringpackage": https fetch: Get "https://192.168.0.12/gitrepo/go-package-test-stringutil/stringpackage?go-get=1": dial tcp 192.168.0.12:443: connect: connection refused

I add the following to ~/.gitconfig:

[url "ssh://dean@192.168.0.12:"]
	insteadOf = https://192.168.0.12/

I get the same error.

I've tried multiple ways of setting GOPRIVATE

export GOPRIVATE=192.168.0.12/gitrepo/*
export GOPRIVATE=ubuntu-18-extssd

but get the same error.

You need to actually try this on a local computer with a non-routeable IP address. I've tried the same thing on github.com and it works. The go get command only wants to work on public, routeable IP addresses.

Many companies keep their source code internally on hosts with non-routable IP addresses, and this prevents sharing code and makes using modules impossible.

[seankhliao]

This looks like 2 separate errors

import path does not begin with hostname

go currently does not consider names without dots as hosts, as of #37641 these are considered reserved names for the stdlib

https fetch: Get "https://192.168.0.12/gitrepo/go-package-test-stringutil/stringpackage?go-get=1": dial tcp 192.168.0.12:443: connect: connection refused

remote imports need to follow https://golang.org/cmd/go/#hdr-Remote_import_paths
specifically either end with .git for a git repo or answer to a http/https request with the meta tag

[dwschulze]

I added .git to the url like this

go get 192.168.0.12/gitrepo/go-package-test-stringutil.git/stringpackage

and it gives a different message:

package 192.168.0.12/gitrepo/go-package-test-stringutil.git/stringpackage: cannot download, git://192.168.0.12/gitrepo/go-package-test-stringutil uses insecure protocol

My ~/.gitconfig has this

[url "ssh://dean@192.168.0.12:"] insteadOf = https://192.168.0.12/

When go get calls git it seems to be picking up what I have in ~/.gitconfig but converts the ssh: prefix to git:. I believe that is the same thing, though. I have no idea why go get considers that an insecure protocol.

[jayconrod]

@dwschulze Could you run with -x and post the log? That should show the actual commands run.

The git protocol is not secure. The Protocols has more information. git+ssh may be what you want instead. Or if you trust your local network, set GOINSECURE=192.168.0.12.

[seankhliao]

your git config also needs to rewrite the git protocol to ssh

[dwschulze]

I removed the ssh:// prefix from my ~/.gitconfig and now it is working:

[url "dean@192.168.0.12:"] insteadOf = https://192.168.0.12/

The docs really need to cover this and explain why having the ssh:// prefix gives an insecure protocol message. That is very confusing.

[jayconrod]

I'd like to guide this issue to some resolution. I don't think we really know what went wrong. Having the -x log would help.

This doesn't seem related to whether the host contains a non-routable address or not. We don't allow module paths that don't have a dot in the first path element (see Module paths and versions, but local names and IP addresses are fine.

Finding a repository for a module path describes the process. That section doesn't cover qualified module paths, so it should definitely be updated.

When the module path ends with a VCS qualifier like .git, the go command will derive a repository URL directly from that path instead of sending a ?go-get=1 request. It doesn't know the scheme, so it tries git, https, http, git+ssh, and ssh (git and http are only allowed if GOINSECURE matches).

[gopherbot]

Change https://golang.org/cl/276354 mentions this issue: content/static/doc: document vcs qualifiers in module paths