cmd/go: mod tidy removes lines that build seems to need

[zombiezen]

(I haven't dug into exactly what dependency is causing this or why, but I've managed to whittle the issue down to a fairly manageable no-code repro case.)

What version of Go are you using (go version)?

$ go version
go 1.12.1

Does this issue reproduce with the latest release?

Yes. I haven't checked against tip.

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/rlight2/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/rlight2"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/rlight2/src/mg-repro/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build158725968=/tmp/go-build -gno-record-gcc-switches"

What did you do?

git clone https://gist.github.com/5513fc51a179cfd757dcdd1d7c1e4995.git mod-tidy-issue
cd mod-tidy-issue
go mod tidy
go build -mod=readonly

main.go + go.mod gist

What did you expect to see?

A successful build of the binary.

What did you see instead?

go: updates to go.mod needed, disabled by -mod=readonly

Additional surrounding steps

Running go build will cause go build -mod=readonly to pass. Doing this adds lines (all marked "indirect") for:

• github.com/golang/mock
• gopkg.in/check.v1
• gopkg.in/yaml.v2

However, running go mod tidy on this go.mod will unceremoniously remove these added lines. Most confusingly, go mod why doesn't seem to know why these lines are there either:

$ go mod why -m gopkg.in/yaml.v2
# gopkg.in/yaml.v2
(main module does not need module gopkg.in/yaml.v2)

[andybons]

@bcmills @jayconrod

[bcmills]

We should check for similarities with #29773. (I don't know whether it will turn out to be related, but it's a known source of missing dependencies.)

[prymitive]

Hit the same issue - go build and go mod tidy keep fighting over what should be in go.mod
gist for reference - https://gist.github.com/prymitive/9f8ac72d9a88f1704efa8ebfdf133bd1

$ git st
On branch master
[...]
	new file:   go.mod
	new file:   go.sum
	new file:   main.go
$ go build
$ git diff
diff --git a/go.mod b/go.mod
index 6980ec7..7e79628 100644
--- a/go.mod
+++ b/go.mod
@@ -15,6 +15,7 @@ require (
        github.com/gin-gonic/contrib v0.0.0-20190526021735-7fb7810ed2a0
        github.com/gin-gonic/gin v1.4.0
        github.com/go-bindata/go-bindata v3.1.1+incompatible
+       github.com/go-logfmt/logfmt v0.4.0 // indirect
        github.com/go-openapi/errors v0.19.0
        github.com/go-openapi/runtime v0.19.0
        github.com/go-openapi/strfmt v0.19.0
$ go mod tidy
$ git diff
[no diff]
$ go build
$ git diff
diff --git a/go.mod b/go.mod
index 6980ec7..7e79628 100644
--- a/go.mod
+++ b/go.mod
@@ -15,6 +15,7 @@ require (
        github.com/gin-gonic/contrib v0.0.0-20190526021735-7fb7810ed2a0
        github.com/gin-gonic/gin v1.4.0
        github.com/go-bindata/go-bindata v3.1.1+incompatible
+       github.com/go-logfmt/logfmt v0.4.0 // indirect
        github.com/go-openapi/errors v0.19.0
        github.com/go-openapi/runtime v0.19.0
        github.com/go-openapi/strfmt v0.19.0

[bcmills]

@prymitive, can you provide a more compact example, or at least compare the output of go list -m all and go mod graph? As noted above, it would be good to rule out duplicates of #29773.

[prymitive]

Narrowed down needed modules to just two:

github.com/prometheus/client_golang v0.9.4
github.com/spf13/viper v1.4.0

https://gist.github.com/prymitive/9f8ac72d9a88f1704efa8ebfdf133bd1 updated to reflect that
Also added the output of these 2 commands

[gopherbot]

Change https://golang.org/cl/186537 mentions this issue: cmd/go/internal/mvs: retain modules required by older versions