cmd/go: VCS stamping complains inappropriately about VCS in unreleated parent directory

[rogpeppe]

commit 1cbec68

The Go compiler refuses to build my usual executables without specifying the -buildvcs=false flag.

Here's a testscript reproducer:

mkdir d/.bzr
cd d/m
exec git init
go build

-- d/m/go.mod --
module m

go 1.18

-- d/m/main.go --
package main

func main() {
}

This fails as follows:

% testscript /tmp/build-bug.txtar

> mkdir d/.bzr
> cd d/m
$WORK/d/m
> exec git init
[stdout]
Initialized empty Git repository in $WORK/d/m/.git/
> go build
[stderr]
error obtaining VCS status: directory "$WORK/d/m" uses git, but parent "$WORK/d" uses bzr
	Use -buildvcs=false to disable VCS stamping.

[exit status 1]
FAIL: /tmp/testscript742157310/build-bug.txtar/script.txt:4: unexpected go command failure
error running /tmp/build-bug.txtar in /tmp/testscript742157310/build-bug.txtar

When the nearest VCS directory is at the root of the current module, I believe it should be taken as definitive and the search should go no further back into parent directories.

[seankhliao]

shouldn't this be posted in #37475 which tracks the implementation?

cc @jayconrod

[jayconrod]

cc @bcmills @matloob

This is somewhat working as intended: I reused some code from GOPATH go get, which forbids repository-within-repository, except Git-within-Git, which is allowed. In general though, there's a lot of ambiguity where repositories, modules, packages, and directories don't correspond, so it seemed better to report an error if anything was usual.

Could you say more about your use case? Would be good to understand how common this will be.

[bcmills]

I agree that if the VCS containing the CWD matches the VCS containing the module root, we shouldn't fail due to other VCS directories further up.

The reason they were rejected in GOPATH-mode go get was to avoid VCS-injection attacks, because in GOPATH mode the go command runs its VCS commands in arbitrary directories that aren't directly controlled by the user. In this case, the user does control the path (it's always the CWD).

[rogpeppe]

My use case is that I happen to have a catch-all .bzr directory in my home. I probably shouldn't (it's a historical accident) but nonetheless I have never had any problems it ever before, and I don't think I should now.

I appreciate that VCS within VCS is a specific issue when there's a VCS directory that's below the module root, but I don't think there should be a problem in this case.

AFAICS the logic should be something like: find the nearest enclosing VCS directory:; if it doesn't enclose the root of the current module, it's an error; otherwise use the revision from it.

I can't currently think of a reason to look outside the nearest enclosing VCS directory.

[gopherbot]

Change https://golang.org/cl/356309 mentions this issue: cmd/go: allow nested VCS repositories when preparing build stamp

[williamh]

One of my users on Gentoo is getting bitten by this because he has a fossil control directory in /var.
The sandbox violation means that go is trying to do things outside the package build staging area.
Apparently this is causing a number of go-based packages to not build any more.

https://bugs.gentoo.org/836261

As you can see, he is recommending that we disable the function for builds within our package manager. What do you suggest?

[berndf]

Thanks @williamh ! I'm that gentoo user. I have to say that, during each build, firing up software that will ascend the whole directory tree looking for control directories that may or may not be project related is probably not a good idea. Not everybody compiles using pristine machine images... Chances are that version information of that unrelated project may be used. And many people will have several VCS installed, meaning that the search will happen using git, hg, bzr, fossil and whatnot, increasing the attack or failure surface a lot.

I do see that sometimes a go project can be in a subdirectory of a larger projects' root. One way to avoid the problem would be to explicitly specify the VCS root in the build. Or to place a "stop VCS search here" file or directory in the project's root. Or, at least to never fire up the VCS search if a version tag is given explicitly (as in gentoo I believe it is).

One specific problem with fossil might also be that it opens the repository read/write even with only informational commands. Not sure that all the other systems open theirs readonly in this case. Sandbox systems would probably abort the build in this case as well, as the repository directory will often be out of bounds. So, without one of the provisions mentioned above avoiding the directory ascending altogether, each and every VCS would need a sandbox-safe "probe mode". This probably means asking too much and we're back to adding more logic to safely conclude that 1) we need info from the VCS and 2) we're actually inside a relevant project-specific checkout.

What can be done immediately is, of course, to set -buildvcs=false if we know that we're not building from a checkout but some tar release. Are there any objections to that?

[seankhliao]

There is going to be a new default of -buildvcs=auto (ref #51748) though I don't think it's going to help in your case. VCS info also includes clena/dirty info, not just version tags. If you know you're building from a source tarball that isn't going to contain vcs information, setting -buildvcs=false sounds like a reasonable default for your situation.

[williamh]

That means all package manager builds in Gentoo will end up having -buildvcs=false since we build from tarballs.
I can add that to GOFLAGS once I verify that it will not break go 1.17 since we keep the two most recent versions of go.

[mvdan]

@williamh unfortunately Go 1.17 will try to consume GOFLAGS=-buildvcs=false and fail, as that version doesn't know the flag.