cmd/go: `list -u -m` and `get -u` suggest downgrades from pseudo-versions after `latest`

[leitzler]

What version of Go are you using (go version)?

$ go version
go version go1.12 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/root/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build305234165=/tmp/go-build -gno-record-gcc-switches"

What did you do?

I added a new dependency with go get specifying a particular commit (or master in the example below). Then when I looked for upgradable modules using go list -u -m all, it suggest an older commit (tagged version) as upgrade.

Minimal example:

root@85abf31141f1:/go# mkdir -p /tmp/foo && cd /tmp/foo
root@85abf31141f1:/tmp/foo# go mod init foo
go: creating new go.mod: module foo
root@85abf31141f1:/tmp/foo# go get github.com/SermoDigital/jose@master
go: finding github.com/SermoDigital/jose master
go: downloading github.com/SermoDigital/jose v0.9.2-0.20180104203859-803625baeddc
go: extracting github.com/SermoDigital/jose v0.9.2-0.20180104203859-803625baeddc
root@85abf31141f1:/tmp/foo# go list -u -m all     
go: finding github.com/SermoDigital/jose v0.9.1
foo
github.com/SermoDigital/jose v0.9.2-0.20180104203859-803625baeddc [v0.9.1]

What did you expect to see?

No recommendation since the recommended version is older/an earlier commit in the same branch.

What did you see instead?

A recommendation to downgrade. Also, running go get -u <module> downgrades it to the older version.

[thepudds]

There are two potentially problematic observations reported here.

The first is about the output of go list -u -m all showing a semver tag that is older than the current pseudo-version currently recorded go.mod. That seems like a clear bug, or at least does not seem to agree with the 'go list' documentation:

The -u flag adds information about available upgrades. When the latest version of a given module is newer than the current one, list -u sets the Module's Update field to information about the newer module. The Module's String method indicates an available upgrade by formatting the newer version in brackets after the current version.

In this example, the latest version shown in brackets is not "newer" than the current one, so it seems it should not be shown, at least according to the documentation. One plausible explanation might be that the go list -u -m logic might have been written when all pseudo-versions were v0.0.0 based (e.g., before https://golang.org/cl/124515 and other related changes).

The second reported observation is:

Also, running go get -u downgrades it to the older version

Here, the behavior in the example is probably working as designed. go get -u foo is shorthand for the equivalent go get -u foo@latest. I think @latest should always mean the same thing regardless of whether or not there is a pseudo-version in the go.mod. The documentation defines latest as:

The string "latest" matches the latest available tagged version, or else the underlying source repository's latest untagged revision.

So it seems go get -u foo always selecting the latest release tag for foo is probably working as designed, even though it is a downgrade in this example.

Separately, I think go get -u (without any args) in this situation probably also does the right thing, and does not change the pseudo-version (that is, not downgrading in this example).

Finally:

• There is some nuance here, so perhaps my comments are not correct.
• People are frequently surprised by what go get -u foo actually means, so I am not claiming this is all obvious, especially for someone new to modules. Rather, I am trying to comment on the pieces that seem to be unintentional bugs vs. working as currently designed.

[leitzler]

That’s a really good summary, it indeed looks like list -u -m doesn’t consider pseudo versions in go.mod, and that feels like a pure bug.

And get -u foo is indeed a shorthand for @latest, but in my opinion it would be very unfortunate and confusing for users if list -u -m and get -u foo should behave differently. Even worse if get -u foo and get -u (without args) behaves differently.

To get a pseudo version into my go.mod I most likely requested a specific version as an active decision, and therefore I expect that decision to be respected by the tool. Wouldn’t that make sense? (cc @rsc)

[gopherbot]

Change https://golang.org/cl/174206 mentions this issue: cmd/go/internal/modload: make 'list -u' consider current pseudoversion

[jayconrod]

Just adding a note after Slack discussion with @leitzler:

Let's say we have package example.com/mod/pkg provided by module example.com/mod. At the moment, go get example.com/mod/pkg@latest will set example.com/mod version to the latest semantic version, or prerelease semantic version if no non-prereleases are available, or pseudo-version if no other versions are available. It may downgrade if the current version is a prerelease or pseudo-version beyond the current latest.

go get -u example.com/mod/pkg@latest is not quite the same. It will change the version of example.com/mod as above. It will also upgrade modules that provide packages transitively imported by example.com/mod/pkg to their own latest if that version is greater than their current version.

(Note: the modules upgraded by -u are more limited in 1.13, but at the moment, the upgrade logic is the same as 1.12).

This is probably more confusing than it needs to be. I think we should probably prevent go get example.com/mod/pkg@latest and go get example.com/mod/pkg@patch from downgrading. It doesn't seem like expected behavior, and the inconsistency with -u is weird.

[bcmills]

I think we should probably prevent go get example.com/mod/pkg@latest and go get example.com/mod/pkg@patch from downgrading.

Agreed. Probably we should change the meaning of @latest and @patch to be “the latest tagged version or the version found in the build list, whichever is greater”.

[gopherbot]

Change https://golang.org/cl/176537 mentions this issue: cmd/go/testdata/script: remove skips and clarify comments in mod_*_upgrade_pseudo tests