proposal: net: add BufferedPipe (buffered Pipe)

[iangudger]

If one wants to connect two Go libraries which use the net interfaces, the only cross platform way to do it with the standard library is to use the loopback. The only in-process way to do it is to create a socket pair with syscall.Socketpair and go from FD -> *os.File -> net.Conn. This is a fairly manual process, and while local to the machine and fully in-memory, does use a non-cross-platform kernel feature to do the heavy lifting.

A native Go transport would be more ideal, but expecting users to implement their own is maybe not reasonable as the net.Conn and net.Listener interfaces are difficult to implement correctly. I propose that we add a canonical and cross-platform in-memory transport in either net or x/net.

Further, I propose the following interface:

func NewConnPair(addr1, addr1 net.Addr) (net.Conn, net.Conn)

func NewPacketConnPair(addr1, addr1 net.Addr) (net.PacketConn, net.PacketConn)

func NewDialerListenerPair(dialAddr, listenAddr net.Addr) (func(context.Context) (net.Conn, error), net.Listener)

Maybe it would be better to create new exported types and return those instead of the interfaces (e.g. MemoryConn, MemoryPacketConn, and MemoryListener)? That seems like it would be more consistent with the net package.

CC @rsc

[slrz]

How would this differ from net.Pipe?

[iangudger]

Sorry, somehow I missed that.

The part that is still missing through is a net.Listener to go along with it.

[rsc]

It would be buffered, which net.Pipe is not.
Perhaps net.NewPipeBuffer would be enough,
instead of all those extra functions.

[iangudger]

A buffered solution would be useful.

[rsc]

Right now we have in package net:

func Pipe() (Conn, Conn)

It sounds like the minimal addition here to solve the problem would be

func BufferedPipe() (Conn, Conn)

I believe packages could build arbitrary other things easily on top of that.

Do people agree that this is the right path forward?

[cristaloleg]

Yes, please, buffered pipe will be very useful.

[jared2501]

That would be great! Especially useful for tests that exercise code using the tls package, which deadlocks if using net.Pipe.

[rsc]

OK, so it sounds like people agree that adding net.BufferedPipe (as suggested in #34502 (comment)) will resolve this.
As retitled, sounds like a likely accept.

Leaving open for a week for final comments.

[ianlancetaylor]

Accepted.