diff --git a/NEWS.md b/NEWS.md deleted file mode 100644 index ff497d72bc..0000000000 --- a/NEWS.md +++ /dev/null @@ -1,304 +0,0 @@ -# go-libp2p release notes - -## 6.0.0 - -We're pleased to announce go-libp2p 6.0.0. This release includes a massive -refactor of go-libp2p that paves the way for new transports such as QUIC. -Unfortunately, as it is broad sweeping, there are some breaking changes, -*especially* for maintainers of custom transports. - -Below, we cover the changes you'll likely care about. For convenience, we've -broken this into a section for users and transport authors/maintainers. However, -transport maintainers should really read both sections. - -### For Users - -Libp2p users should be aware of a few major changes. - -* Guarantees and performance concerning connect/disconnect notification - processing have improved. -* Handling of half-closed streams has changed (READ THIS SECTION). -* Some constructors and method signatures have changed slightly. - -#### Dialing And Source Addresses - -We've improved the logic that selects the source address when dialing. In the -past, you may have run into an issue where you couldn't dial non-local nodes -when listening on 127.0.0.1 as opposed to 0.0.0.0. This happened because -go-libp2p would randomly pick the source address from the set of addresses on -which the node was listening. We did this to ensure that the other end could -dial us back at the same address. Unfortunately, one can't use 127.0.0.1 as a -source address when dialing any non-local address so outbound dials failed. - -go-libp2p now tries to be smarter about this and avoids picking source addresses -that have no route to the destination address. - -#### Bandwidth Metrics - -To start out on an unhappy note, bandwidth metrics are now less accurate. In the -past, transports returned "minimal" connections (e.g., a TCP connection) so we -could wrap these transport connections in "metrics" connections that counted -every byte sent and received. - -Unfortunately, now that we've moved encryption and multiplexing down into the -transport layer, the connection we're wrapping has significantly more -under-the-covers overhead. - -However, we do hope to improve this and get even *better* bandwidth metrics than -we did before. See [libp2p/go-libp2p-transport#31][] for details. - -[libp2p/go-libp2p-transport#31]: https://github.com/libp2p/go-libp2p-transport/issues/31 - -#### Notifications - -This release brings performance improvements and easy to reason about ordering -guarantees libp2p connect/disconnect events: - -1. For any given connection/stream, libp2p will wait for all connect/open event - handlers to finish exit before triggering a disconnect/close event for the - connection/stream. -2. When a user calls the Close (or `Reset`) method on a connection or stream, - go-libp2p will process the close event asynchronously (i.e., not block the - call to `Close`). Otherwise, a call to `Close` from within a connect event - handler would deadlock. -3. Unless otherwise noted, events will be handled in parallel. - -What does this mean for end users? Well: - -1. Reference counting connections to a peer using connect/disconnect events - should "just work" and should never go negative. -2. Under heavy connect/disconnect loads, connecting to new peers should be - faster (usually). - -For those interested in the history of this issue, ... - -In the past, (dis)connect and stream open/close notifications have been a bit of -a pain point. For a long time, they were fired off in parallel and one could, for -example, process a disconnect notification before a connect notification (we had -to support *negative* ref-counts in several places to account for this). - -After no end of trouble, we finally "fixed" this by synchronizing notification -delivery. We still delivered notifications to all notifiees in parallel, we just -processed the events in series. - -Unfortunately, under heavy connect/disconnect load, new connections could easily -get stuck on open behind a queue of connect events all being handled in series. -In theory, these events should have been handled quickly but in practice, it's -very hard to avoid locks *entirely* (bitswap's event handlers were especially -problematic). - -Worse, this serial delivery guarantee didn't actually provide us with an -*in-order* delivery guarantee as it was still possible for a disconnect to -happen before we even *started* to fire the connect event. The situation was -slightly better than before because the events couldn't overlap but still far -from optimal. - -However, this has all been resolved now. From now on, you'll never receive a -disconnect event before a connect event. - -#### Conn.GetStreams Signature Change - -The signature of the `GetStreams` method on `go-libp2p-net.Conn` has changed from: - -```go -GetStreams() ([]Stream, error) -``` - -To: - -```go -GetStreams() []Stream -``` - -Listing the streams on an open connection should never involve IO or do anything -that can fail so we removed this error to improve usability. - -#### Libp2p Constructor - -If you're not already doing so, you should be using the `libp2p.New` constructor -to make your libp2p nodes. This release brings quite a few new options to the -libp2p constructor so if it hasn't been flexible enough for you in the past, I -recommend that you try again. A simple example can be found in the -[echo][example:echo] example. - -Given this work and in an attempt to consolidate all of our configuration logic -in one place, we've removed all default transports from go-libp2p-swarm. - -TL;DR: Please use the libp2p constructor. - -#### Zombie Streams - -From this release on, when you're done with a stream, you must either call -`Reset` on it (in case of an error) or close it and read an EOF (or some other -error). Otherwise, libp2p can't determine if the stream is *actually* closed and -will hang onto it indefinitely. - -To make properly closing streams a bit easier, we've added two methods to -[go-libp2p-net][]: `AwaitEOF` and `FullClose`. - -* `AwaitEOF(stream)` tries to read a single byte from the stream. If `Read` -returns an EOF, `AwaitEOF` returns success. Otherwise, if `Read` either reads -some data or returns some other error, `AwaitEOF` resets the stream and returns -an error. To avoid waiting indefinitely, `AwaitEOF` resets the stream -unconditionally after 1 minute. -* `FullClose(stream)` is a convenience function that closes the stream and then -calls `AwaitEOF` on it. - -Like with libp2p notifications, this issue has a bit of history... - -In the beginning, libp2p assumed that calling `Close` on a stream would close -the stream for both reading and writing. Unfortunately, *none* of our stream -multiplexers actually behaved this way. In practice, `Close` always closed the -stream for writing. - -After realizing this, we made two changes: - -1. We accepted the fact that `Close` only closed the stream for writing. -2. We added a `Reset` method for killing the stream (closing it in both - directions, throwing away any buffered data). - -However, we ran into a bit of a snag because we try to track open streams and -need some way to tell when a stream has been closed. In the past this was easy: -when the user calls `Close` on the stream, stop tracking it. However, now that -`Close` only closes the stream for writing, we still *technically* needed to -track it until the *other* end closed the stream as well. Unfortunately, without -actually reading from the stream, we have no way of knowing about this. -Therefore, if the user calls `Close` on a stream and then walks away, we'd have -to hang onto the stream indefinitely. - -Our solution was to simply stop tracking streams once they were closed for -writing. This wasn't the *correct* behavior but it avoided leaking memory in the -common case: - -1. The user calls `Close` and drops all references to the stream. -2. The other end calls `Close` without writing any additional data. -3. The stream multiplexer observes both closes and drops *its* reference to the stream. -4. The garbage collector garbage collects the stream. - -However, this meant that: - -1. The list of "open" streams was technically incomplete. -2. If the other side either failed to call `Close` or tried to send data before - closing, the stream would remain "open" (until the connection was closed). - -In this release, we've changed this behavior. Now, when you `Close` a stream for -writing, libp2p *continues* to track it. We only stop tracking it when either: - -1. You call `Reset` (throwing away the stream). -2. You finish reading any data off of it and observe either an EOF or an error. - -This way, we never "forget" about open streams or leave them in a half-forgotten -state. - -In the future, I'd like to add a `CloseAndForget` method to streams that: - -1. Closes the stream (sends an EOF). -2. Tells the swarm to stop tracking the stream. -3. Tells the stream muxer to stop tracking the stream and throw away any data - the other side may send (possibly resetting the stream on unexpected data). - -However: - -1. This would likely require modifying our stream muxers which may not be - feasible. -2. Explicitly waiting for an EOF is still the correct thing to do unless you - really don't care if the operation succeeded. - -### For Transport Maintainers - -For transport maintainers, quite a bit has changed. Before this change, -transports created simple, unencrypted, stream connections and it was the job of -the libp2p Network (go-libp2p-swarm) to negotiate security, multiplexing, etc. - -However, when attempting to add support for the QUIC protocol, we realized that -this was going to be a problem: QUIC already handles authentication and -encryption (using TLS1.3) and multiplexing. After much debate, we inverted our -current architecture and made transports responsible for encrypting/multiplexing -their connections (before returning them). - -To make this palatable, we've also introduced a new ["upgrader" -library][go-libp2p-transport-upgrader] for upgrading go-multiaddr-net -connections/listeners to full libp2p transport connections/listeners. Transports -that don't support encryption/multiplexing out of the box can expect to have an -upgrader passed into the constructor. - -To get a feel for how this new transport system works, take a look at the TCP -and WebSocket transports and the transport interface documentation: - -* [TCP Transport][go-tcp-transport] -* [WebSocket Transport][go-ws-transport] -* [Transport Interface][doc:go-libp2p-transport] - -#### Deprecated Packages - -This release sees the deprecation of a few packages: - -* [go-peerstream][] has been deprecated and all functionality has been merged - into [go-libp2p-swarm][]. [go-peerstream][] was written as a general-purpose - (not libp2p specific) listener, connection, and stream manager. However, this - package caused more problems than it solved and was incompatible with the new - transport interface. -* [go-libp2p-interface-conn][] has been deprecated. These interfaces to bridge - the gap between transport-level connections and [go-libp2p-net][] connections - however, now that transport connections are fully multiplexed/encrypted, this - is no longer needed. -* [go-libp2p-conn][] has also been deprecated and most of the functionality has - been moved to [go-libp2p-transport-upgrader][]. This package used to provide - connection "upgrade" logic for upgrading transport-level connections to - [go-libp2p-interface-conn][] connections however, transport-level connections - now provide the required functionality out of the box. - -#### Testing - -We've moved `GenSwarmNetwork` in [go-libp2p-netutil][] to `GenSwarm` in -[go-libp2p-swarm/testing][] because: - -1. The swarm duplicated this exact function for its own tests. -2. The swarm couldn't depend on [go-libp2p-netutil][] because - [go-libp2p-netutil][] depends on [go-libp2p-swarm][]. - -We've also added a new transport test suit -[go-libp2p-transport/test][]. If you implement a new transport, please consider -testing against these suite. If you find a bug in an existing transport, please -consider adding a test to this suite. - -#### go-addr-util - -In go-addr-util, we've removed the `SupportedTransportStrings` and -`SupportedTransportProtocols` transport registries and the associated -`AddTransport` function. These registries were updated by `init` functions in -packages providing transports and were used to keep track of known transports. - -However, *importing* a transport doesn't mean any libp2p nodes have been -configured to actually *use* that transport. Therefore, in the new go-libp2p, -it's go-libp2p-swarm's job to keep track of which transports are supported -(i.e., which transports have been registered with the swarm). - -We've also removed the associated `AddrUsable`, `FilterUsableAddrs`, and -`AddrUsableFunc` functions. - -#### Pluggable Security Transports - -This release brings a new pluggable security transport framework. Implementing a -new security framework is now as simple as: - -1. Implement the interfaces defined in [go-conn-security][]. -2. Pass it into the libp2p constructor using the `Security` option. - -[go-conn-security]: https://github.com/libp2p/go-conn-security -[go-libp2p-conn]: https://github.com/libp2p/go-libp2p-conn -[go-libp2p-interface-conn]: https://github.com/libp2p/go-libp2p-interface-conn -[go-libp2p-net]: https://github.com/libp2p/go-libp2p-net -[go-libp2p-netutil]: https://github.com/libp2p/go-libp2p-netutil -[go-libp2p-swarm]: https://github.com/libp2p/go-libp2p-swarm -[go-libp2p-swarm/testing]: https://github.com/libp2p/go-libp2p-swarm/tree/master/testing -[go-libp2p-transport]: https://github.com/libp2p/go-libp2p-transport -[go-libp2p-transport/test]: https://github.com/libp2p/go-libp2p-transport/tree/master/test -[go-libp2p-transport-upgrader]: https://github.com/libp2p/go-libp2p-transport-upgrader -[go-peerstream]: https://github.com/libp2p/go-peerstream -[go-tcp-transport]: https://github.com/libp2p/go-tcp-transport -[go-ws-transport]: https://github.com/libp2p/go-ws-transport - -[example:echo]: https://github.com/libp2p/go-libp2p-examples/tree/master/echo - -[doc:go-libp2p-transport]: https://godoc.org/github.com/libp2p/go-libp2p-transport