proposal: cmd/compile: define register-based calling convention

[dr2chase]

Performance would be generally somewhat improved if arguments-to and results-from function and method calls used registers instead of the stack. Projected improvements based on a limited prototype are in the range of 5-10%.

The running CL for the prototype: https://go-review.googlesource.com/c/28832/
The prototype, because it is a prototype, uses a pragma that should be unnecessary in the final design, though helpful during development.

(This is a placeholder bug for a design doc.)

problems/tradeoffs noted below, through 2017-01-19 (#18597 (comment))

This will reduce the usefulness of panic tracebacks because it will confuse/corrupt the per-frame argument information there. This was already a problem with SSA and register allocation and spilling, this will make it worse. Perhaps only results should be returned in registers.

Does this provide enough performance boost to justify "breaking" (how much?) existing assembly language?

Given that this breaks existing assembly language, why aren't we also doing a more thorough revision of the calling conventions to include callee-saves registers?

This should use the per-platform ABIs, to make interaction with native code go more smoothly.

Because this preserves the same memory layout of the existing calling, it will consume more stack space than it otherwise might, if that had been optimized out.

And the responses to above, roughly:

In compiler work, 5% (as a geometric mean across benchmarks) is a big deal.

Panic tracebacks are a problem; we will work on that. One possible mitigation is to use DWARF information to make tracebacks much better in general, including properly named and interpreted primitive values. A simpler mitigation for targeted debugging could be an annotation indicating that a function should be compiled to store arguments back to the stack (old style) to ensure that particular function's frame was adequately informative. This is also going to be an issue for improved inlining because regarded at a low level intermediate frames will disappear from backtraces.

The scope of required assembly-language changes is expected to be quite small; from Go-side function declarations the compiler ought to be able to create shims for the assembly to use around function entry, function exit, and surrounding assembly-language CALLs to named functions. The remaining case is assembly-language CALL via a pointer, and these are rare, especially outside the runtime. Thus, the need to bundle changes because they are disruptive is reduced, because the changes aren't that disruptive.

Incorporating callee-saves registers introduces a garbage-collector interaction that is not necessarily insurmountable, but other garbage-collected languages (e.g., Haskell) have been sufficiently intimidated by it that they elected not to use callee-saves registers. Because the assembler can also modify stack frames to include callee-save spill areas and introduce entry/exit shims to save/restore callee-save registers, this appears to have lower impact than initially estimated, and thus we have reduced need to bundle changes. In addition, if we stake out potential callee-save registers early developers can look ahead and adapt their code before it is required.

If each platform's ABI were used instead of this adaptation of the existing calling conventions, the assembly-language impact would be larger, the garbage-collector interactions would be larger, and either best-treatment for Go's multivalue returns would suffer or the cgo story would be sprinkled with asterisks and gotchas. As an alternative (a different way of obtaining a better story for cgo), would we consider a special annotation for cgo-related functions to indicate that exactly the platform calling conventions were used, with no compromises for Go performance?

[gopherbot]

CL https://golang.org/cl/35054 mentions this issue.

[davecheney]

5% for amd64 doesn't feel like it justifies the cost of breaking all the assembly written to date.

[minux]

Additionally, passing argument in registers will make displaying function arguments in traceback much harder, if not impossible. One reason I like Go so much is how Go's traceback makes debugging so much easier by providing enough contextual information about the frames. Frankly, I'd have expected much larger performance improvement for switching the calling convention to use registers. If the benefit is indeed only 5-10%, IMHO it's not worth the trouble. We should pursue better inlining and automatic vectorization instead. The Intel CPUs have invested so much circuit in optimizing the stack (operations), so I'm not too surprised though. The benefit should be larger on RISC architectures.

[davecheney]

I agree with Minux, optimising function calls for 10% speedup vs inlining and all the code generation benefits that unlocks doesn't feel like a good investment.

[josharian]

I'm still thinking about the proposal, but I will note that for asm functions with go prototypes and normal stack usage, we can do automated rewrites. Which is also a reminder that the scope for this proposal should include updating vet's asmdecl check.

[minux]

I wouldn't worry too much about assembly functions, as in the worse case, we can have an opt-in mechanisms for assembly functions and then develop a mode in cmd/asm to help people rewrite (or better yet, automate the translation in cmd/asm so that people won't even notice the difference. we can have a new textflag for new style assembly functions). However, I do care deeply about arguments in tracebacks. I tried patch set 30 of CL 28832 on this example: https://play.golang.org/p/UV-E4wyL2T And the result is: panic: 0 goroutine 1 [running]: panic(0x456a80, 0xc42000e118) $GOROOT/src/runtime/panic.go:531 +0x1cf main.F(0x50) /tmp/x.go:8 +0x6b main.F(0x0) /tmp/x.go:6 +0xa main.F(0x0) /tmp/x.go:6 +0xa main.F(0xc420052000) /tmp/x.go:6 +0xa main.F(0x0) /tmp/x.go:6 +0xa main.F(0x7) /tmp/x.go:6 +0xa main.F(0x7f6c34290000) /tmp/x.go:6 +0xa main.F(0x60) /tmp/x.go:6 +0xa main.F(0xc420070058) /tmp/x.go:6 +0xa main.F(0x0) /tmp/x.go:6 +0xa main.F(0xc4200001a0) /tmp/x.go:6 +0xa main.main() /tmp/x.go:13 +0x9 I'd like to hear if we have any plans for restoring the existing behavior of showing the first few arguments for each frame.

[randall77]

@minux Yes, we will have all the information necessary to print args as we do now. I suspect it just isn't implemented yet.

[minux]

I'm curious to know how do you plan to implement it without incurring the overhead of storing the initial value into memory?

[randall77]

The values enter in registers but they will be spilled to the arg slots if they are live at any call. So all live arguments will still be correct. Even modified live arguments should work (except for a few weird corner cases where multiple values derived from the same input are live).

Dead arguments are not correct even today. What you see in tracebacks is stale. They may get more wrong with this implementation, so it may be worth marking/not displaying dead args.

Things will get trickier if we allow callee-saved registers. We're thinking about it, but it isn't in the design doc yet.

[minux]

Just passing arguments in register while still reserving stack slots for them looks like MS's ABI. Isn't that negating a lot of the benefit of passing arguments in registers in the first place? Passing argument in register is precisely about not generating memory traffic for register arguments. Anyway, I'd like to read the full design docs of this and see the discussion of various design decisions and trade offs made.