cmd/dist: show which GOOS/GOARCH combinations are first class ports

[mvdan]

go tool dist list, with or without the -json flag, is very useful to list all supported ports. It was proposed and done in #12270.

However, there's no way to query the Go tool which of those combinations are first class ports. The current list is:

linux/amd64
linux/386
linux/arm
linux/arm64
darwin/amd64
windows/amd64
windows/386

Why do I want this list? These ports have binary releases available on golang.org, they are better tested and supported, and they are overall much more common platforms to run Go on. For example, if I'm publishing prebuilt releases for a Go program of mine, I probably want to use Go's list of first-class ports as a starting point, and only add extras if I know about a specific group of users of the prorgram.

Copy-pasting the list from the Wiki is an OK immediate workaround, but it can quickly get out of date. For example, 1.15 adds linux/arm64 to the list, so having copied the list just a few months ago would already be out of date today. The list on the wiki also reflects master; if I look at it now, I might get the impression that linux/arm64 was a first class port at the time that Go 1.14 came out, but that's not the case. go1.14 tool dist list -json would be able to tell me that decisively.

The command already has a -json flag to produce a JSON list, so it should be easy to add another field. For example:

$ go tool dist list -json
[
	{
		"GOOS": "aix",
		"GOARCH": "ppc64",
		"CgoSupported": true,
		"FirstClass": false
	},
[...]

/cc @bradfitz @minux @golang/osp-team

[dmitshur]

Thanks for the report. I have thoughts on two aspects of this.

One is making the first-class port information programmatically accessible. I think this is reasonable to consider doing. This information is already public, so it is just a matter of making it accessible to computers in addition to humans. The Go project itself would benefit from this, as it could re-use the canonical list of first-class ports in all the places where it's needed. Right now there is some inconsistency, as reported in issue #27689.

Two is about how to version this information. Right now there is only a "master" snapshot, on the wiki, plus its edit history as a rough indicator of what happened over time. Suppose this feature was implemented since Go 1.10 (released on 2018/02/16), what would it mean that go1.10 tool dist list -json prints "FirstClass": false for linux/arm64 and "FirstClass": true for darwin/386? I understand it'd be a rough proxy for "at the time that major version of Go was released, this was the state of first-class ports". What about what go1.10.8 tool dist list -json prints, which was released 2019/01/23? Based on how I understand it now, bundling the historical information into cmd/go (whose releases might not align with policy changes) this way doesn't seem like a good approach.

In the end, I think #27689 needs to get past NeedsDecision state before we can make progress here.

[dmitshur]

Suppose this feature was implemented since Go 1.10 (released on 2018/02/16), what would it mean that go1.10 tool dist list -json prints "FirstClass": false for linux/arm64 and "FirstClass": true for darwin/386? I understand it'd be a rough proxy for "at the time that major version of Go was released, this was the state of first-class ports". What about what go1.10.8 tool dist list -json prints, which was released 2019/01/23?

I think I understand it better now. You're looking to make it so that a first-class port is never added or removed mid-release, but rather aligned with major release boundaries, and to make it possible to query that (via cmd/go or elsewhere).

For example, if a port is added to being considered first-class, it should be possible to (easily) answer "this port became first-class as of Go 1.N", rather than having to implicitly compute that from release dates of major Go versions and the first-class addition date.

Is my understanding right?

[mvdan]

Correct. When a major release is getting ready, like 1.15, we should make sure that go tool dist list contains up-to-date information about the first class ports. Those should match with what first class port rules are going to be used for the 1.15 release.

Minor (aka bugfix) releases like 1.15.1 should never change this information, because first class ports are only added or removed in major versions like 1.15 and 1.16, as far as I understand. It would be weird for 1.15.1 to add another first class port, or to remove one.

[toothrot]

If you're using this data for creating new releases, is it ok for you to query "what is a first-class port right now for Go" vs "what is a first-class port for the latest release of Go"?

Somewhat related, as part of work to move the build dashboard into the coordinator (#34744), and have the "first-class ports" checkbox do the right thing on the UI, I hard-coded this field:
https://go.googlesource.com/build/+/refs/heads/master/cmd/coordinator/internal/dashboard/handler.go#29.

Having that somewhere query-able would be useful for the coordinator as well, as we want this information in a couple tools.

Another thought: Go releases binaries for non-first-class ports as well (freebsd-386/amd64, linux-ppc64le/s390x). I don't think we have a way of knowing for certain the most common platforms for running Go on, as you mentioned, but I have a guess.

[mvdan]

If you're using this data for creating new releases, is it ok for you to query "what is a first-class port right now for Go" vs "what is a first-class port for the latest release of Go"?

No, because I build my releases with a stable release of Go, not master. For example, the upcoming Go 1.15 adds linux/arm64 as a first-class port, but I'll build releases with Go 1.14.x, where linux/arm64 was not a first class port (i.e. at the time of 1.14's release).

Another thought: Go releases binaries for non-first-class ports as well

I think that's a requirement for first class ports, but I guess not all downloads have to be first class ports. Maybe the wiki could be clarified.

I don't think we have a way of knowing for certain the most common platforms for running Go on, as you mentioned, but I have a guess.

I think this is up to each use case. For my use case, "what are the platforms with first-class support in Go" is an OK starting point.

[mvdan]

Would a patch for this be welcome? I ran into another use-case where this would have been helpful, and having this in the upcoming 1.16 would be nice.

I get that the freeze is here, but I imagine this sort of change might be okay early in the freeze given that go tool dist is not used directly by the vast majority of Go users.

[mvdan]

For context, here's the use case: https://github.com/mvdan/dotfiles/blob/47c16f7b56bf8e441e9c760f7664e5a89244a39d/.bin/go-cross. For now, I'm just hard-coding the list, and I have to trust that I'll remember to check https://github.com/golang/go/wiki/PortingPolicy#first-class-ports for updates every release.

[dmitshur]

I'm really not sure that cmd/go is the right place to put this information. If we add it there right away, it'll be hard to change it.

I'm okay with placing this information in x/build (which is v0 and can be modified anytime without time pressure of the freeze) though so it can be accessed programmatically, and not have to download many historic cmd/go binaries to get info about older releases. I suggest we start there, and consider promoting to cmd/go later on if we're happy with the experience in x/build and still think there's a need to also make it available from cmd/go.

For x/build, the internal table can perhaps be, inspired by golang.org/x/website/internal/history and golang.org/x/build/cmd/release, something like:

map[string][]string{
	">= go1.15": {
		"linux/amd64",
		"linux/386",
		"linux/arm",
		"linux/arm64",
		"darwin/amd64",
		"windows/amd64",
		"windows/386",
	},

	"< go1.15": {
		"linux/amd64",
		"linux/386",
		"linux/arm",
		"darwin/amd64",
		"windows/amd64",
		"windows/386",
	},

	// ...
}

With some API to get the ports for a specified major Go version.

[mvdan]

If we add it there right away, it'll be hard to change it.

I don't follow. The whole point is that, just like the rest of the information given by go tool dist -json, it will change as the list of ports in each Go version change.

With some API to get the ports for a specified major Go version.

For what it's worth, this doesn't seem useful to me, and it also adds Go as a dependency to extract the information. go tool dist -json exists for a reason: to query the detailed list of ports from an installed Go tree.

Such a pure Go API could be useful within x/build perhaps, but it doesn't fix the go tool dist use case. Perhaps file a separate issue for it.

[dmitshur]

If we add it there right away, it'll be hard to change it.

I meant the API. If we add a field to cmd/go now, we can't easily change or remove it in the future. So it needs more thought before we commit to it.

I'm going to set the milestone to Go 1.17, we can revisit this issue then. I don't think this is important for Go 1.16.