proposal: fmt: recognize and use StringAppend method to avoid required allocation in String

[soypat]

Context

Heap allocations are almost unavoidable in Go. Most of the language has been designed around having an excellent garbage collector. That said, there is a growing interest in being more mindful of how we allocate memory. See proposals #51317 (arenas) #71497 (json/v2). The embedded Go community is also gaining traction as powerful compute becomes more available on small devices such as the recent RP2350 MCU, which was quickly supported by TinyGo due to widespread community driven effort by new contributors.

One of the more common methods of allocating memory in Go is through the fmt.Stringer interface which permeates a big part of the Go ecosystem. Most of the time it is invisible, used from inside fmt.Print* family of functions. This method serves a very important purpose in Go which is to give a text visual of a data structure or concept abstracted by a data structure.

The issue with this interface is that when the string is not constant and must be built it allocates, always. This presents an issue for developers seeking to minimize or constrain allocations. For sure if some of these developers were given a fmt.Stringer interface which constrains allocations they would prefer it to the current fmt.Stringer interface.

Proposal

I propose we add a fmt.StringAppender interface with the following signature:

type StringAppender interface {
    StringAppend([]byte) []byte
}

The method would do what one would expect, append the string representation to the buffer and return the extended result, just like the result of fmt.Stringer.

Users could implement the fmt.String* method set by first implementing StringAppend which would define the result of fmt.Stringer by simply calling the StringAppend method on a nil byte slice: string(Hexa(1).StringAppend(nil)).

type Hexa int

func (h Hexa) StringAppend(b []byte) []byte {
	return fmt.Appendf(b, "0x%x", h)
}

[soypat]

Is this considered a duplicate of AppendText for encoding?

[seankhliao]

yes

[soypat]

It would seem they fulfill different purposes. String method is meant to return a "native" format which need not necessarily be a format that can be decoded back, hence the lack of error in the function signature.

[ianlancetaylor]

For cases where the fact that the fmt package calls a String method causes excessive heap allocation, it is already possible to implement a Format method.

[soypat]

I would argue implementing a Format method is much harder and also would require the importing of the fmt package, which is not something one just does on embedded projects.

Calling Write with a byte buffer also allocates unless the buffer is owned by the calling data structure, which also unecessarily complicates what you are trying to do. This also requires writing the data twice, once to format it and another to write it to the State.

I don't think fmt.Formatter is a solution to the issue. It serves a different purpose.

[ianlancetaylor]

I am concerned that String methods are extremely common. Adding a new StringAppend method that many types are expected to implement is going to cause a vast ecosystem effect, a cost that seems likely to exceed the benefit.

However, I agree that this is not a duplicate of AppendText, so I'll reopen.

[seankhliao]

I'd be concerned this adds more complexity for little gain. Even the existing TextAppender has quite low adoption compared to TextMarshaler

TextAppender 148 hits: https://github.com/search?q=language%3AGo+%2F%5Efunc+%5C%28.*%5C%29+AppendText%5C%28.*byte%5C%29.*byte%2F&type=code
TextMarshaler 28.8k hits: https://github.com/search?q=language%3AGo+%2F%5Efunc+%5C%28.*%5C%29+MarshalText%5C%28%5C%29.*byte%2F&type=code

Also, what would be expected to check for and use this interface? Since you mentioned not importing fmt, it doesn't seem like anything in the standard library would use this?

[gabyhelp]

Related Issues

• fmt: add Append, Appendf, Appendln #47579 (closed)

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}