Go mod tidy: "import path <pkg> should not have @version" using custom GOPATH inside project directory

[cawirawa]

What version of Go are you using (go version)?

$ go version
1.19.5

But issues occur in the following versions too:
1.20, 1.19.5, 1.19.4, 1.19.3, 1.19.2, 1.19.1, 1.18.Xs, and 1.17.Xs.

Does this issue reproduce with the latest release?

Yes, I can reproduce it in go 1.20.

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE="auto"
GOARCH="amd64"
GOBIN="/Users/cwirawan/projects/gopath-bug/go/bin"
GOCACHE="/Users/cwirawan/Library/Caches/go-build"
GOENV="/Users/cwirawan/Library/Application Support/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/cwirawan/projects/gopath-bug/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/cwirawan/projects/gopath-bug/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/Users/cwirawan/go-sdk/go1.19.5"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/Users/cwirawan/go-sdk/go1.19.5/pkg/tool/darwin_amd64"
GOVCS=""
GOVERSION="go1.19.5"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/cwirawan/projects/gopath-bug/go.mod"
GOWORK=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch x86_64 -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/yn/mz9b94gd08q9kxqr6b40l5gr0000gp/T/go-build4200323998=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

I'm dealing with multiple go projects in my organization. For each project, we decided to have a custom configuration for GOPATH. We did it to separate Go projects so that each project has its own GOPATH. This method works well for us until recently when we upgraded the go version to 1.19.5. But it's the same with version 1.20.

NOTE that this issue only occurs if the GOPATH is set to be inside of the project directory.

Note that go mod tidy will not consider packages in the main module in directories named testdata or with names that start with . or _ unless those packages are explicitly imported by other packages.

Quoted from https://go.dev/ref/mod#go-mod-tidy

When setting the GOPATH to an ignored path in the directory (e.g. <project-dir>/.go). It works without any issue. However, we are not sure if this is a good practice or not and will it cause any issues?

I'm not a go module expert and we don't want any core dump in our production code. :)

Source Files:

main.go

package main

import (
	"go.uber.org/atomic"
)

func main() {
	_ = atomic.NewBool()
}

go.mod

module test

go 1.19

require go.uber.org/atomic v1.9.0

require (
	github.com/stretchr/testify v1.7.0 // indirect
	gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b // indirect
)

I also tested it using other different dependencies and some of them reproduce this issue, some of them don't.

go.sum

github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.7.0 h1:nwc3DEeHmmLAfoZucVR881uASk0Mfjw8xYJ99tb5CcY=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
go.uber.org/atomic v1.9.0 h1:ECmE8Bn/WFTYwEW/bpKD3M8VtR/zQVbavAoalC1PYyE=
go.uber.org/atomic v1.9.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b h1:h8qDotaEPuJATrMmW04NCwg7v22aHH28wwpauUhK9Oo=
gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=

These are the steps to reproduce the issues:

1. Create a directory and create/copy the files above to the directory.
2. In the directory, run:

export GOPATH=`pwd`/go

3. Verify that the GOPATH is inside the project directory by running:

go env GOPATH

4. Run go mod tidy, the first time should be successful.

$ go mod tidy   
go: downloading go.uber.org/atomic v1.9.0
go: downloading github.com/stretchr/testify v1.7.0
go: downloading gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b
go: downloading github.com/davecgh/go-spew v1.1.1
go: downloading github.com/pmezard/go-difflib v1.0.0

5. Run go mod tidy again, you should see some errors:

$ go mod tidy
test/go/pkg/mod/github.com/davecgh/[email protected]/spew: import path "test/go/pkg/mod/github.com/davecgh/[email protected]/spew" should not have @version
test/go/pkg/mod/github.com/pmezard/[email protected]/difflib: import path "test/go/pkg/mod/github.com/pmezard/[email protected]/difflib" should not have @version

What did you expect to see?

I expect to see a successful go mod tidy with exit code 0 no matter how many times I run it.

It used to work without any problem in version 1.16.8.

What did you see instead?

$ go mod tidy
test/go/pkg/mod/github.com/davecgh/[email protected]/spew: import path "test/go/pkg/mod/github.com/davecgh/[email protected]/spew" should not have @version
test/go/pkg/mod/github.com/pmezard/[email protected]/difflib: import path "test/go/pkg/mod/github.com/pmezard/[email protected]/difflib" should not have @version

[seankhliao]

working as intended.
go stores downloaded dependencies in GOPATH/go/pkg by default.
Modules are intended to be independent projects, covering everything within it, acting independently of GOPATH except to use it as a (shared) cache. You shouldn't need to create new GOPATHs for each project in module mode, and what you're doing now will clearly cause errors.

Unlike many projects, the Go project does not use GitHub Issues for general discussion or asking questions. GitHub Issues are used for tracking bugs and proposals only.

For questions please refer to https://github.com/golang/go/wiki/Questions

[teosoft123]

@seankhliao Your answer does not preclude having a GOPATH dir per project, but makes an unsubstantiated claim that having a GOPATH dir per project causes a problem. I truly believe this is worth looking into, because the behavior did change somewhere between v1.16 and later, and there's no mention in Go documentation whatsoever.

Nor Go documentation places any limitations as to where the GOPATH directory can or cannot be, and we decided that our chosen location is as good as any.

Would you be so kind to present any proof as to why, and I quote from your answer, "what you're doing now will clearly cause errors."

Thanks!