cmd/go: custom import path private repos require special configuration

[chowey]

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

go version go1.11 darwin/amd64

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOOS="darwin"

What did you do?

I have a private repo served over ssh, using gitolite.

That private repo is a package I want to use, at e.g. private.repo.net/user/package. It has a folder structure like this:

README.md
a/a.go
b/b.go

My global .gitconfig looks like this, so that go get works:

[url "ssh://git@private.repo.net"]
        insteadOf = https://private.repo.net

In my main package my/package, I have a go file that imports it like this:

import "private.repo.net/user/package/a"

This used to work when using the old $GOPATH. Now I am trying to use modules instead. I run go build my/package which should resolve dependencies and build.

What did you expect to see?

I expected it to build.

What did you see instead?

build my/package: cannot find module for path private.repo.net/user/package/a

I also tried adding it to the go.mod file by running go get private.repo.net/user/package.git. (NOTE: I need to use .git so that it uses ssh.) So it now private.repo.net/user/package.git appears in my go.mod file.

However, when I do go build, it still complains that "build my/package: cannot find module for path private.repo.net/user/package/a".

[chowey]

It works if I just do:

import "private.repo.net/user/package.git/a"

(emphasis on ".git") Then go build does the right thing, and correctly initializes the go.mod file.

NOTE: I still need to do git config --global url."ssh://git@private.repo.net".insteadOf "https://private.repo.net" to get this to work.

[lopezator]

Same error here, import paths like:

my-private-repo.com/user/pkg/a

Simply don't work and prints:

cannot find module for path ....

[myitcv]

@chowey the reason the .git solution works is that it helps guide the go tool with information about where the VCS for custom import paths private.repo.net/user/package.git/... resides (namely (https|ssh)://private.repo.net/user/package.git). Note for github.com and other VCS's known to the go tool, this step is not required.

Without this (i.e. with an import path like private.repo.net/user/package/a), the go tool first queries https://private.repo.net/user/package/a?go-get=1 to work out (according to https://tip.golang.org/cmd/go/#hdr-Remote_import_paths) where the VCS for that custom import path lives. I suspect this is where yours and @lopezator's examples are failing.

Hence you need to ensure:

• https://private.repo.net/user/package/a?go-get=1 returns <meta> information per https://tip.golang.org/cmd/go/#hdr-Remote_import_paths
• https://private.repo.net/user/package/a?go-get=1 must also be accessible; either this means that it returns the <meta> information regardless of auth credentials, or you need some sort of credentials helper

For credentials, either you can provide a $HOME/.netrc:

machine private.repo.net login YOU password APIKEY

or use an OS credential helper (see #26134 (comment)).

Also related to #26232.

Whether you then use https/ssh access to the underlying VCS is entirely up to you.

---

I think this issue is effectively the custom import path equivalent of #26134. Hence I'm going to retitle it.

[jackwhelpton]

I'm not sure if this is germane to this issue or I should open a new one, but I seem to be able to reproduce this despite checking all of the configuration above:

• even non-nested imports (import "private.repo.net/user/package") are failing with cannot find module for path

• this behavior is only apparent with GO111MODULE=on; it does not manifest when GO111MODULE=off

• I have this in my .gitconfig:

  [url "git@private.repo.net"]
  insteadOf = https://private.repo.net

• I'm running on Windows, and have the credential manager configured. I can pull from the repo fine.

• I've checked the meta tags from https://private.repo.net/user/package?go-get=1 and they look fine.

[myitcv]

@jackwhelpton are you able to share the output of go get -x SOMETHING?

This might well contain sensitive information in your case, so please be careful.

In case you can't share this information publicly, please do say.

[jackwhelpton]

Sure. It does look like an authentication problem, but my standard git commands are working OK. Here's what I get from go get -x private.repo.net/user/package

go get -x private.repo.net/user/package
# C:\Projects\go\pkg\mod\cache\vcs\<id> for git2 https://private.repo.net/user/package.git
cd C:\Projects\go\pkg\mod\cache\vcs\<id>; git ls-remote -q https://private.repo.net/user/package.git
0.742s # cd C:\Projects\go\pkg\mod\cache\vcs\<id>; git ls-remote -q https://private.repo.net/user/package.git
go get private.repo.net/user/package: git ls-remote -q https://private.repo.net/user/package.git in C:\Projects\go\pkg\mod\cache\vcs\<id>: exit status 128:
        fatal: 'git@private.repo.net/user/package.git' does not appear to be a git repository
        fatal: Could not read from remote repository.

        Please make sure you have the correct access rights
        and the repository exists.

[myitcv]

@jackwhelpton if you run

git ls-remote -q https://private.repo.net/user/package.git

separately, what is the output?

[jackwhelpton]

Ah, so this may be a git configuration issue, although it's interacting with modules in a strange way (given my ability to go get successfully when modules are disabled). Here's what I see with that command:

git ls-remote -q https://private.repo.net/user/package.git
fatal: 'git@private.repo.net/user/package.git' does not appear to be a git repository
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

Note that a simple git pull in that directory still succeeds. I've just realized there's one piece of configuration I hadn't explained yet: I'm interacting with that server using a stored SSH key, and PuTTY/Pageant is handling the key exchange.

[myitcv]

@jackwhelpton your insteadOf configuration does not look correct:

Try:

[url "ssh://git@private.repo.net"]
insteadOf = https://private.repo.net

instead of (no pun intended):

[url "git@private.repo.net"]
insteadOf = https://private.repo.net