doc, x/tools/cmd/godoc, x/website: move golang.org content into its own repository

[andybons]

With the website slated for a refresh, we should move the static content out of the main go source tree (https://go.googlesource.com/go/+/master/doc/) and into its own repository (web? website?).

This will reduce binary and source code download sizes (#27151) and allow us to apply changes unrelated to our release schedule without having to cherry-pick them to a release branch. Hopefully we can also implement continuous delivery as part of this as well.

We also won’t have to worry about bloating the main go source tree as we add more assets and content for the refresh.

We still only want to display package documentation and source files from a GOROOT pinned to the current stable release (and not show content only relevant for future releases like release notes). So there are elements that still require logic that ties content to a specific Go version. What we consider fluid vs tied to a release is to be determined and can be fleshed out here.

/cc @bradfitz @ianlancetaylor @julieqiu @katiehockman @dmitshur @FiloSottile @cnoellekb

[bradfitz]

"website" SGTM. I like the repo name golang/website.

Inside that I'd like something like:

• README.md
• codereview.cfg
• content/ directory
• cmd/ directory
• cmd/goweb or cmd/golangweb or cmd/golang-org-server (name TBD)

We could start with mirroring stuff over from the existing golang/go repo (and any necessary code from x/tools/cmd/godoc) but without modifying either for now. And then serve the new one on something like https://new.golang.org for a bit and when we're happy then switch over and start deleting code & assets from the old locations.

We also need to figure out whether this new cmd/golang-org-server is the new name of the recently-web-only x/tools/cmd/godoc (the tool for users to browse their local $GOPATH/etc documentation in the web on localhost on airplanes/etc).

If they're the same, then do we develop it in x/tools or in x/website? I kinda like having the code next to the assets but I could go either way and haven't thought about it much.

[andybons]

We also need to figure out whether this new cmd/golang-org-server is the new name of the recently-web-only x/tools/cmd/godoc (the tool for users to browse their local $GOPATH/etc documentation in the web on localhost on airplanes/etc).

My preference is for x/tools/cmd/godoc to become a binary for viewing local $GOPATH/etc documentation but not anything else (which would be more in line with its name anyhow). A common go-source-to-html rendering library could be pulled out (if it’s not already general enough in x/tools/godoc) and used in places like godoc.org, x/tools/cmd/godoc, and the new website. This way most of the logic specific to new.golang.org is right next to the content.

[dmitshur]

We also need to figure out whether this new cmd/golang-org-server is the new name of the recently-web-only x/tools/cmd/godoc (the tool for users to browse their local $GOPATH/etc documentation in the web on localhost on airplanes/etc).

Yes, we need to figure that out. At this time, I strongly suspect that we’ll want to make the new cmd/golang-org-server be strictly the website and not a tool users run locally, other than to contribute to the golang.org website development.

[bradfitz]

Pulling the render-godoc-as-HTML handler out and using it on both places SGTM. That can happen first probably.

And making x/tools/cmd/godoc be super minimal also SGTM.

[cnoellekb]

We discussed 4 main milestones for moving the website:

1. Separate out rendering logic (converting code -> HTML) from x/tools/cmd/godoc. There should be no behavior change at this point.
2. Creating website binary and serving templates (the static content) from go/doc to x/website
3. Using the common rendering library in x/website.
4. Reduce x/tools/cmd/godoc to be the minimal version (since the website can now be run out of x/website).

Right now I'm focusing on the second step because I was able to get a prototype up and running. I copied tools/cmd/godoc, tools/godoc, and go/doc into a local directory that will eventually become the root for the main website. The plan is to just copy everything for now to maintain x/tools functionality. Can someone help me set up an x/website repo?

Also, I was looking into how the rest of the tools directory would be affected by removing any godoc files and I found that x/tools/cmd/present/play.go uses the tools/godoc/static package. Does anyone think this may be an issue?

Lastly, is the final version of godoc going to need an app engine configuration? If not, then tools/internal/memcache can be moved to the x/website repository.

[bradfitz]

@dmitshur, @andybons, where are the notes from last time we created a new repo? I'd rather not do it myself. It'd be nice if we can have somebody else run through the steps from the docs. (gerrit, github, maintner, gitmirror updating, etc)

[dmitshur]

where are the notes from last time we created a new repo?

@bradfitz The steps are documented at golang.org/wiki/CreatingSubRepository.

It'd be nice if we can have somebody else run through the steps from the docs.

Yep, that's a good idea. If anyone on the team wants to go through the process to learn it, I'm happy to assist and fill in any gaps in docs. Note that most of the bottlenecks will be having the right permissions to all components involved. I can also do it myself.

[bradfitz]

I created https://github.com/golang/website and https://go.googlesource.com/website (Gerrit).

@cnoellekb, can you send CLs for the maintner and dashboard and gitmirror bits? And gopherbot if needed.

[cnoellekb]

Sure, I'll take care of it!

[gopherbot]

Change https://golang.org/cl/156217 mentions this issue: maintner/maintnerd, cmd/gerritbot, cmd/gitmirror: add golang/website to