net/netip: add new IP address package, use in net

[bradfitz]

I propose we fix #18804 (net: reconsider representation of IP) by importing the inet.af/netaddr into the standard library, probably as net/netaddr, so the IP type is netaddr.IP (the net package already has an IP type). We wouldn't need to import all of it, at least not right away, but I think it'd be nice if we did... the package types work together very well.

This blog post explains the advantages of netaddr.IP over net.IP:

https://tailscale.com/blog/netaddr-new-ip-type-for-go/

Notably, it's small, comparable, and doesn't allocate.

Code is at https://github.com/inetaf/netaddr. Everybody has a Go CLA filed ( inetaf/netaddr#181) and the license is compatible.

Having an allocation-free IP address in the standard library means we'd be able to fix #45886 ("net: add API to receive multiple UDP packets (potentially in one system call)") which ends with #45886 (comment) ("We may want to wait on doing anything here until we figure out what to do with IP addresses, which still allocate."). More generally, this would improve Go's UDP performance, which has never gotten much attention. But with things like QUIC and WireGuard (both UDP-based), Go's alloc-heavy UDP handling is starting to get in the way.

Yes, we'd then have two IP types in the standard library.

As part of this, we'd then add:

• conversion funcs in net to go between them (net would depend on netaddr, not vice-versa)
• the net package internals would switch to netaddr.IP for performance/allocs, converting to net.IP or IPAddr on the edges only.
• a couple new methods would be added (notably to UDPConn) to send/receive UDP packets using netaddr.IP alloc-free (we'd then have ~5 ways to send/recv UDP packets instead of ~4)

/cc @danderson, @josharian, @mdlayher, @crawshaw, @tklauser (co-authors), @neild (co-author for #45886)

[mvdan]

cc @marten-seemann for quic-go

[dotwaffle]

If there's any way we could address the JSON marshalling of CIDR address formats at the same time, that would be very convenient -- rather than having to rely on making custom encoders etc.

I realise that the backwards compatibility guarantee means that's unlikely to be fixed in the standard library version of "net", but if it could be addressed in this proposed new package, it would certainly make the life of network engineers much simpler!

Example: https://play.golang.org/p/3as8Ux9AzJG

[bradfitz]

If there's any way we could address the JSON marshalling of CIDR address formats at the same time, that would be very convenient -- rather than having to rely on making custom encoders etc.

https://play.golang.org/p/5QWiD-tArCc

fmt:	 192.0.2.0/24
json:	 "192.0.2.0/24"

[taktv6]

As someone who had to invent their own IP datatype (https://pkg.go.dev/github.com/bio-routing/bio-rd/net#IP) for the very same reasons, I'm really glad to see progress on this topic. Looking forward to see a decent IP datatype in Golang.

Thanks!

[rsc]

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

[kylelemons]

Would it be worth considering adding a runtime primitive of some sort to help avoid the "workarounds" in the intern package? I don't think it would need to be done at the same time, but having a user for it in the stdlib might make it more compelling.

[Merovius]

@kylelemons There have been and are some proposals related to that, most notable probably #43615. In the interim, the workarounds are much less problematic when they are in the stdlib, because if anything is changed in the implementation that would prevent them from working, they can be changed alongside with it.

[bradfitz]

@kylelemons, what @Merovius said. And to be clear (to anybody just researching the intern package and being horrified), we are definitely not proposing exposing the intern package or any of its API. It's very much an internal implementation detail.

[zephyr]

I don’t want to start bikeshedding, but i think that the proposed type names look a little bit cluttered, since they all share the IP prefix:

net/netaddr.IP
net/netaddr.IPPort
net/netaddr.IPPrefix
net/netaddr.IPRange
net/netaddr.IPSet
net/netaddr.IPSetBuilder

Couldn’t the IP-part just move into the package name instead:

net/ip.Addr
net/ip.Port
net/ip.Prefix
net/ip.Range
net/ip.Set
net/ip.SetBuilder

Or maybe something like net/netip as package name, if net/ip is too short?

Otherwise: Thank you, @bradfitz, for this great proposal and your highly enjoyable blog post!

[mdlayher]

My only nitpick about the package name ip is that I'd guess that is the single most commonly used variable name for any variable of type net.IP, so there's a high likelihood of having to change significant sections of code to avoid shadowing the import net/ip in code being retrofit with the new package APIs.

[zephyr]

@mdlayher While i still think that ip is the nicest option in theory, you have certainly a valid practical concern. Maybe something net/ipa (for ›Internet Protocol address«), to avoid shadowing and jet have something short and sensible?