cmd/go: provide advice for authors of existing packages with version v2.0+

[VojtechVitek]

Hi,
I was testing vgo the other day, trying to import some 3rd party dependencies that have version v2.0 or higher released on Github, but I couldn't figure out how.

Example:
Given github.com/go-chi/chi has latest version v3.3.2 released as a git tag on Github,
I'd like to import github.com/go-chi/chi/v3 in my project, develop against it and eventually vendor it via vgo vendor.

package main

import "github.com/go-chi/chi/v3"

func main() {
	_ = chi.NewRouter()
}

But since github.com/go-chi/chi didn't release a v3 module yet, I get this error:

$ vgo run main.go
vgo: creating new go.mod: module github.com/VojtechVitek/vgo-test
vgo: resolving import "github.com/go-chi/chi/v3"
vgo: finding github.com/go-chi/chi (latest)
vgo: adding github.com/go-chi/chi v1.0.0
vgo: import "github.com/go-chi/chi/v3" [/Users/vojtechvitek/go/src/v/github.com/go-chi/chi@v1.0.0/v3]: open /Users/vojtechvitek/go/src/v/github.com/go-chi/chi@v1.0.0/v3: no such file or directory

vgo found version v1.0.0 but couldn't find "v3" subpkg. So I added an explicit version to my go.mod file and tried again:

$ cat go.mod
module github.com/VojtechVitek/vgo-test

require (
	github.com/go-chi/chi v3.3.2
)

running vgo again caused a silent side effect:

module github.com/VojtechVitek/vgo-test

require (
- 	github.com/go-chi/chi v3.3.2
+ 	github.com/go-chi/chi v0.0.0-20171222161133-e83ac2304db3
)

Hmm, seems like vgo doesn't see any v2.0+ tags? After reading some more blog posts on https://research.swtch.com (since vgo CLI help didn't seem to be very helpful / novice user friendly), I confirmed with:

$ vgo list -t github.com/go-chi/chi
github.com/go-chi/chi
	v0.9.0
	v1.0.0

Now, I might be missing something, but it's not really clear to me how can I import v2.0+ or v3.0+ of a package (should I say module?) that I don't have any control over and "is not ready yet".

Are there any guidelines for package consumers about this? Are there any guidelines for package authors? I couldn't find anything useful neither in research.swtch.com/vgo-module, nor in the accepted proposal or in the wiki.

// EDIT:

$ vgo list -t github.com/go-chi/chi/v3
github.com/go-chi/chi/v3
	v3.0.0
	v3.1.0
	v3.1.1
	v3.1.2
	v3.1.3
	v3.1.4
	v3.1.5
	v3.2.0
	v3.2.1
	v3.3.0
	v3.3.1
	v3.3.2

Ahaa, so I can list v3.0+ tags. But how do I use them in my monorepo properly?

$ vgo get github.com/go-chi/chi/v3
vgo: creating new go.mod: module github.com/VojtechVitek/vgo-test
vgo: github.com/go-chi/chi/v3 v3.3.2: missing or invalid go.mod
vgo get: missing or invalid go.mod

[VojtechVitek]

Quoting https://research.swtch.com/vgo-module:

We want to encourage more developers to tag releases of their packages, instead of expecting that users will just pick a commit hash that looks good to them. Tagging explicit releases makes clear what is expected to be useful to others and what is still under development. At the same time, it must still be possible—although maybe not convenient—to request specific commits.

We want to move away from invoking version control tools such as bzr, fossil, git, hg, and svn to download source code. These fragment the ecosystem: packages developed using Bazaar or Fossil, for example, are effectively unavailable to users who cannot or choose not to install these tools. The version control tools have also been a source of exciting security problems. It would be good to move them outside the security perimeter.

Can't vgo leverage the already existing v2.0+ git tags released on Github? Wouldn't it "just work" for packages that don't have any complex transitive dependencies?

I just hope that migrating monorepos from golang/dep to vgo (can we expect a migration tool, right?) will be very straight-forward. I hope we won't be blocked by some v2.0+ packages that "just work" but are not really maintained anymore and thus "not ready" for vgo.

Any guidance / feedback appreciated. Thanks!

[myitcv]

Have a look at #25810 (comment)

[myitcv]

@VojtechVitek sorry, I was away from keyboard when I sent that last message, hence it was brief.

Here is a complete example that I previously posted to #vgo on Slack. I'll close this issue because I think it answers your question but please shout if not.

---

Create a module using the github.com/go-chi/chi example:

$ mkdir hello
$ cd hello
$ vgo mod -init -module example.com/hello
vgo: creating new go.mod: module example.com/hello
$ cat <<EOD >hello.go
package main

import (
        "net/http"
        "github.com/go-chi/chi"
)

func main() {
        r := chi.NewRouter()
        r.Get("/", func(w http.ResponseWriter, r *http.Request) {
                w.Write([]byte("welcome"))
        })
        http.ListenAndServe(":3000", r)
}
EOD

Now because, at the time of writing, github.com/go-chi/chi:

• has a major version >= 2
• has not been converted to a Go (vgo) module
• we want to use v3.3.2

we need to vgo get that specific version, which will be retrieved as a v0.0.0 psuedo version:

$ vgo get github.com/go-chi/chi@v3.3.2
vgo: finding github.com/go-chi/chi v3.3.2
vgo: downloading github.com/go-chi/chi v0.0.0-20171222161133-e83ac2304db3

Now do a build to check all is good:

$ vgo build

And check the contents of go.mod:

$ cat go.mod
module example.com/hello

require github.com/go-chi/chi v0.0.0-20171222161133-e83ac2304db3

[VojtechVitek]

Hi @myitcv,
thanks for your guidance. I think I understand now - your workaround should work.

we need to vgo get that specific version, which will be retrieved as a v0.0.0 psuedo version:

This is super confusing to me and I think it will cause a lot of confusion in the Go community.

Migrating from golang/dep, I will have to start using some v.0.0.0- pseudo versions instead of v3.3.2, just because vgo can't work properly with git tags.

Well, vgo list can:

$ vgo list -t github.com/go-chi/chi/v3
github.com/go-chi/chi/v3
	v3.0.0
	v3.1.0
	v3.1.1
	v3.1.2
	v3.1.3
	v3.1.4
	v3.1.5
	v3.2.0
	v3.2.1
	v3.3.0
	v3.3.1
	v3.3.2

But other vgo command cannot. The version gets "reverted" to v.0.0.0-*, since the upstream didn't adapt to the new rules (go.mod).

Maybe there's no workaround for this from the vgo's perspective. But in that case, would it make sense to provide some guidelines for the v2.0+ pkg authors and consumers to avoid any confusion?

I'm not the only one not getting it right away, see go-chi/chi#302.

cc @rsc

[myitcv]

This is super confusing to me and I think it will cause a lot of confusion in the Go community.

Just to be clear, this is only required under the conditions listed in the bullet points that preceded it (reordered):

• has not been converted to a Go (vgo) module
• has a major version >= 2
• we want to use v3.3.2

As soon as the project is converted to a module, then you're set - no pseudo-versions required. Because when chi is converted to a module, the import path becomes github.com/go-chi/chi/v3. Your example would therefore become:

package main

import (
        "net/http"
        "github.com/go-chi/chi/v3"
)

func main() {
        r := chi.NewRouter()
        r.Get("/", func(w http.ResponseWriter, r *http.Request) {
                w.Write([]byte("welcome"))
        })
        http.ListenAndServe(":3000", r)
}

and your go.mod would look like:

module example.com/hello

require github.com/go-chi/chi v3.3.2

As was observed in go-chi/chi#302 (comment):

All internal references to go-chi would have to be changed to use a v3 import style.

Therefore, any "old" go (i.e. non vgo) users of the now-converted chi project would be fine so long as they use either 1.9.7 or 1.10.3.

[VojtechVitek]

Therefore, any "old" go (i.e. non vgo) users of the now-converted chi project would be fine so long as they use either 1.9.7 or 1.10.3.

Hmm, it's unlikely that upstream v2.0+ packages will stop supporting go 1.10 anytime soon by changing the internal import paths to "/v2" etc.

Thanks for clarification!

[VojtechVitek]

@myitcv Paul, would you mind taking a look at go-chi/chi#327 (review) and advice what should be done in order for project authors to support both new (vgo) and old (module unaware) versions of Go

assuming the project

has not been converted to a Go (vgo) module
has a major version >= 2
we want to use v3.3.2

Much appreciated!

[mvdan]

@VojtechVitek how many Go versions do you need to support? With 1.9.7 and 1.10.3 now out, any project supporting up to the two last Go versions (which I think is a majority) should be able to switch to Go modules soon.

The only potential problem I see is people still using old minor releases like 1.10.2. This is why I'm holding off for a few weeks before porting my v2+ projects to Go modules. I'll probably do the switch once 1.11 is out, at which point backported support will have been out for two full months.

If you need to support the latest three major Go versions, waiting until 1.11 is out should also do the trick. This obviously doesn't scale past three major Go versions, but hopefully the set of v2+ packages with that constraint is very small.

[myitcv]

@VojtechVitek as @mvdan points out, if you are happy only supporting Go 1.9.7 (or later) and 1.10.3 (or later) from the 1.9.x and 1.10.x series respectively, and of course Go 1.11 and later, and your project is v >= 2, then you can safely convert your project to be a Go module. Otherwise you will hit issues with older Go versions not being able to handle the /vX in import paths.

[VojtechVitek]

I'm not happy dropping support for Go <=1.10.2, <=1.9.6 and 1.8.x for a long long time. I'm talking year+, maybe until 1.12 is out.

Go was always about backwards compatibility ... and now we just break stuff because vgo can't support the existing git tags properly .. and requires new "module" notation instead :(

[mvdan]

Are there particular cases of users not being able to upgrade bugfix/minor versions? If I was maintaining a large open-source project and a bug was reported with Go 1.9.0, my first response would be for them to try with the latest 1.9.x.

This is how I understand Go supports older versions. For example, when it is said that the last stable release recieves important bugfixes, that only concerns the latest bugfix version. I wouldn't consider 1.9.0 maintained at this point.

Having said that, I understand that it's frustrating that Go modules don't play nice with v2+ projects. Although, I'd note that this is not a breaking change. You are not forced to start using a go.mod file :)