diff --git a/CHANGELOG.md b/CHANGELOG.md index f958515c732..932f0d5957f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,413 @@ # go-ipfs changelog +## 0.4.21 2019-05-28 + +We're happy to announce go-ipfs 0.4.21. This release has some critical bug fixes +and a handful of new features so every user should upgrade. + +Key bug fixes: + +* Too many open file descriptors/too many peers + ([#6237](https://github.com/ipfs/go-ipfs/issues/6237)). +* Adding multiple files at the same time doesn't work + ([#6254](https://github.com/ipfs/go-ipfs/pull/6255)). +* CPU utilization spikes and then holds at 100% + ([#5613](https://github.com/ipfs/go-ipfs/issues/5613)). + +Key features: + +* Experimental TLS1.3 support (to eventually replace secio). +* OpenSSL support for SECIO handshakes (performance improvement). + +**IMPORTANT:** This release fixes a bug in our security transport that could +potentially drop data from the channel. Note: This issue affects neither the +privacy nor the integrity of the data with respect to a third-party attacker. +Only the peer sending us data could trigger this bug. + +**ALL USERS MUST UPGRADE.** We intended to introduce a feature this release that, +unfortunately, [reliably triggered this bug][secio-bug]. To avoid partitioning +the network, we've decided to postpone this feature for a release or two. + +Specifically, we're going to provide a minimum _one month_ upgrade period. After +that, we'll start testing the impact of deploying the proposed changes. + +If you're running the mainline go-ipfs, please upgrade ASAP. If you're building +a separate app or working on a forked go-ipfs, make sure to upgrade +github.com/libp2p/go-libp2p-secio to _at least_ v0.0.3. + +[secio-bug]: https://github.com/libp2p/go-libp2p/issues/644 + +### Bug Fixes And Enhancements + +This release includes quite a number of critical bug fixes and +performance/reliability enhancements. + +#### Error when adding multiple files + +The last release broke the simple command `ipfs add file1 file2`. It turns out +we simply lacked a test case for this. Both of these issues (the bug and the +lack of a test case) have now been fixed. + +#### SECIO + +As noted above, we've fixed a bug that could cause data to be dropped from a +SECIO connection on read. Specifically, this happens when: + +1. The capacity of the read buffer is greater than the length. +2. The remote peer sent more than the length but less than the capacity in a + single secio "frame". + +In this case, we'd fill the read buffer to it's capacity instead of its length. + +#### Too many open files, too many peers, etc. + +Go-ipfs automatically closes the least useful connections when it accumulates +too many connections. Unfortunately, some relayed connections were blocking in +`Close()`, halting the entire process. + +#### Out of control CPU usage + +Many users noted out of control CPU usage this release. This turned out to be a +long-standing issue with how the DHT handled provider records (records recording +which peers have what content): + +1. It wasn't removing provider records for content until the set of providers + completely emptied. +2. It was loading every provider record into memory whenever we updated the set + of providers. + +Combined, these two issues were trashing the provider record cache, forcing the +DHT to repeatedly load and discard provider records. + +#### More Reliable Connection Management + +Go-ipfs has a subsystem called the "connection manager" to close the +least-useful connections when go-ipfs runs low on resources. + +Unfortunately, other IPFS subsystems may learn about connections _before_ the +connection manager. Previously, if some IPFS subsystem tried to mark a +connection as useful before the connection manager learned about it, the +connection manager would discard this information. We believe this was causing +[#6271](https://github.com/ipfs/go-ipfs/issues/6271). [It no longer does +that](https://github.com/libp2p/go-libp2p-connmgr/pull/39). + +#### Improved Bitswap Connection Management + +Bitswap now uses the connection manager to mark all peers downloading blocks as +important (while downloading). Previously, it only marked peers from which _it_ +was downloading blocks. + +#### Reduced Memory Usage + +The most noticeable memory reduction in this release comes from fixing connection +closing. However, we've made a few additional improvements: + +* Bitswap's "work queue" no longer remembers every peer it has seen + indefinitely. +* The peerstore now interns protocol names. +* The per-peer goroutine count has been reduced. +* The DHT now wastes less memory on idle peers by pooling buffered writers and + returning them to the pool when not actively using them. + +#### Increased File Descriptor Limit + +The default file descriptor limit has been raised to 8192 (from 2048). +Unfortunately, go-ipfs behaves poorly when it runs out of file descriptors and +it uses a _lot_ of file descriptors. + +Luckily, most modern kernels can handle thousands of file descriptors without +any difficulty. + +### Commands + +This release brings no new commands but does introduce a few changes, bugfixes, +and enhancements. This section is hardly complete but it lists the most +noticeable changes. + +Take note: this release also introduces a few breaking changes. + +#### [DEPRECATION] The URLStore Command Deprecated + +The experimental `ipfs urlstore` command is now deprecated. Please use `ipfs add +--nocopy URL` instead. + +#### [BREAKING] The DHT Command Base64 Encodes Values + +When responding to an `ipfs dht get` command, the daemon now encodes the +returned value using base64. The `ipfs` command will automatically decode this +value before returning it to the user so this change should only affect those +using the HTTP API directly. + +Unfortunately, this change was necessary as DHT records are arbitrary binary +blobs which can't be directly stored in JSON strings. + +#### [BREAKING] Base32 Encoded v1 CIDs By Default + +Both js-ipfs and go-ipfs now encode CIDv1 CIDs using base32 by default, instead +of base58. Unfortunately, base58 is case-sensitive and doesn't play well with +browsers (see [#4143](https://github.com/ipfs/go-ipfs/issues/4143). + +#### Human Readable Numbers + +The `ipfs bitswap stat` and and `ipfs object stat` commands now support a +`--humanize` flag that formats numbers with human-readable units (GiB, MiB, +etc.). + +#### Improved Errors + +This release improves two types of errors: + +1. Commands that take paths/multiaddrs now include the path/multiaddr in the + error message when it fails to parse. +2. `ipfs swarm connect` now returns a detailed error describing which addresses + were tried and why the dial failed. + +#### Ping Improvements + +The ping command has received some small improvements and fixes: + +1. It now exits with a non-zero exit status on failure. +2. It no longer succeeds with zero successful pings if we have a zombie but + non-functional connection to the peer being pinged + ([#6298](https://github.com/ipfs/go-ipfs/issues/6298)). +3. It now prints out the average latency when canceled with `^C` (like the unix + `ping` command). + +### Features + +This release is primarily a bug fix release but it still includes two nice +features from libp2p. + +#### Experimental TLS1.3 support + +Go-ipfs now has experimental TLS1.3 support. Currently, libp2p (IPFS's +networking library) uses a custom TLS-like protocol we call SECIO. However, the +conventional wisdom concerning custom security transports is "just don't" so we +are working on replacing it with TLS1.3 + +To choose this protocol by default, set the `Experimental.PreferTLS` config +variable: + +```bash +> ipfs config --bool Experimental.PreferTLS true +``` + +Why TLS1.3 and not X (noise, etc.)? + +1. Libp2p allows negotiating transports so there's no reason not to add noise + support to libp2p as well. +2. TLS has wide language support which should make implementing libp2p for new + languages significantly simpler. + +#### OpenSSL Support + +Go-ipfs can now (optionally) be built with OpenSSL support for improved +performance when establishing connections. This is primarily useful for nodes +receiving multiple inbound connections per second. + +To enable openssl support, rebuild go-ipfs with: + +```bash +> make build GOFLAGS=-tags=openssl +``` + +### CoreAPI + +The CoreAPI refactor is still underway and we've made significant progress +towards a usable ipfs-as-a-library constructor. Specifically, we've integrated +the [fx](https://go.uber.org/fx) dependency injection system and are +now working on cleaning up our initialization logic. This should make it easier +to inject new services into a go-ipfs process without messing with the core +internals. + +### Build: `GOCC` Environment Variable + +Build system now uses `GOCC` environment variable allowing for use of specific +go versions during builds. + +### Changelog + +- github.com/ipfs/go-ipfs: + - core: call app.Stop once ([ipfs/go-ipfs#6380](https://github.com/ipfs/go-ipfs/pull/6380)) + - explain what dhtclient does ([ipfs/go-ipfs#6375](https://github.com/ipfs/go-ipfs/pull/6375)) + - ci: actually enable golangci-lint ([ipfs/go-ipfs#6362](https://github.com/ipfs/go-ipfs/pull/6362)) + - commands/swarm(fix): handle empty multiaddrs ([ipfs/go-ipfs#6355](https://github.com/ipfs/go-ipfs/pull/6355)) + - feat: improve errors when a path fails to parse ([ipfs/go-ipfs#6346](https://github.com/ipfs/go-ipfs/pull/6346)) + - fix vendoring dependencies when building the source tarball ([ipfs/go-ipfs#6349](https://github.com/ipfs/go-ipfs/pull/6349)) + - core: Use correct default for connmgr lowWater ([ipfs/go-ipfs#6352](https://github.com/ipfs/go-ipfs/pull/6352)) + - doc: remove out of date documentation ([ipfs/go-ipfs#6345](https://github.com/ipfs/go-ipfs/pull/6345)) + - Add generation of dependency changes to mkreleaselog ([ipfs/go-ipfs#6348](https://github.com/ipfs/go-ipfs/pull/6348)) + - readme: remove mention of DCO ([ipfs/go-ipfs#6344](https://github.com/ipfs/go-ipfs/pull/6344)) + - Add golangci-lint ([ipfs/go-ipfs#6321](https://github.com/ipfs/go-ipfs/pull/6321)) + - docs+mk: update guidance for unsupported platforms ([ipfs/go-ipfs#6338](https://github.com/ipfs/go-ipfs/pull/6338)) + - fix formatting in object get ([ipfs/go-ipfs#6340](https://github.com/ipfs/go-ipfs/pull/6340)) + - fail start when loading a plugin fails ([ipfs/go-ipfs#6339](https://github.com/ipfs/go-ipfs/pull/6339)) + - fix a typo in the issue template ([ipfs/go-ipfs#6335](https://github.com/ipfs/go-ipfs/pull/6335)) + - github: turn issue template into a multiple-choice question ([ipfs/go-ipfs#6333](https://github.com/ipfs/go-ipfs/pull/6333)) + - object put: Allow empty objects ([ipfs/go-ipfs#6330](https://github.com/ipfs/go-ipfs/pull/6330)) + - Update fuse.md ([ipfs/go-ipfs#6332](https://github.com/ipfs/go-ipfs/pull/6332)) + - work towards fixing dht commands ([ipfs/go-ipfs#6277](https://github.com/ipfs/go-ipfs/pull/6277)) + - fix setting ulimit ([ipfs/go-ipfs#6319](https://github.com/ipfs/go-ipfs/pull/6319)) + - switch to base32 by default for CIDv1 ([ipfs/go-ipfs#6300](https://github.com/ipfs/go-ipfs/pull/6300)) + - cmdkit -> cmds ([ipfs/go-ipfs#6318](https://github.com/ipfs/go-ipfs/pull/6318)) + - raise default fd limit to 8192 ([ipfs/go-ipfs#6266](https://github.com/ipfs/go-ipfs/pull/6266)) + - pin: don't walk all pinned blocks when removing a non-existent pin ([ipfs/go-ipfs#6311](https://github.com/ipfs/go-ipfs/pull/6311)) + - ping: fix a bunch of issues ([ipfs/go-ipfs#6312](https://github.com/ipfs/go-ipfs/pull/6312)) + - test(coreapi): use a thread-safe datastore everywhere ([ipfs/go-ipfs#6222](https://github.com/ipfs/go-ipfs/pull/6222)) + - fix(Dockerfile): Allow ipfs mount in Docker container ([ipfs/go-ipfs#5560](https://github.com/ipfs/go-ipfs/pull/5560)) + - docs: fix Routing section ([ipfs/go-ipfs#6309](https://github.com/ipfs/go-ipfs/pull/6309)) + - License update to dual MIT and Apache 2 ([ipfs/go-ipfs#6301](https://github.com/ipfs/go-ipfs/pull/6301)) + - Go test fix ([ipfs/go-ipfs#6293](https://github.com/ipfs/go-ipfs/pull/6293)) + - commands(pin update): return resolved CIDs instead of paths ([ipfs/go-ipfs#6275](https://github.com/ipfs/go-ipfs/pull/6275)) + - core: fix autonat construction ([ipfs/go-ipfs#6289](https://github.com/ipfs/go-ipfs/pull/6289)) + - Test and fix GC/pin bug ([ipfs/go-ipfs#6288](https://github.com/ipfs/go-ipfs/pull/6288)) + - GOCC implementation & fix in make & build scripts ([ipfs/go-ipfs#6282](https://github.com/ipfs/go-ipfs/pull/6282)) + - gc: cancel context ([ipfs/go-ipfs#6281](https://github.com/ipfs/go-ipfs/pull/6281)) + - fix: windows friendly daemon help ([ipfs/go-ipfs#6278](https://github.com/ipfs/go-ipfs/pull/6278)) + - Invert constructor config handling ([ipfs/go-ipfs#6276](https://github.com/ipfs/go-ipfs/pull/6276)) + - docs: document environment variables ([ipfs/go-ipfs#6268](https://github.com/ipfs/go-ipfs/pull/6268)) + - add: Return error from iterator ([ipfs/go-ipfs#6272](https://github.com/ipfs/go-ipfs/pull/6272)) + - commands(feat): use the coreapi in the urlstore command ([ipfs/go-ipfs#6259](https://github.com/ipfs/go-ipfs/pull/6259)) + - humanize for ipfs bitswap stat ([ipfs/go-ipfs#6258](https://github.com/ipfs/go-ipfs/pull/6258)) + - Revert "raise default fd limit to 8192" ([ipfs/go-ipfs#6265](https://github.com/ipfs/go-ipfs/pull/6265)) + - raise default fd limit to 8192 ([ipfs/go-ipfs#6261](https://github.com/ipfs/go-ipfs/pull/6261)) + - Fix AutoNAT service for private network ([ipfs/go-ipfs#6251](https://github.com/ipfs/go-ipfs/pull/6251)) + - add: Fix adding multiple files ([ipfs/go-ipfs#6255](https://github.com/ipfs/go-ipfs/pull/6255)) + - reprovider: Use goprocess ([ipfs/go-ipfs#6248](https://github.com/ipfs/go-ipfs/pull/6248)) + - core/corehttp/gateway_handler: pass a request ctx instead of the node ([ipfs/go-ipfs#6244](https://github.com/ipfs/go-ipfs/pull/6244)) + - constructor: cleanup some things ([ipfs/go-ipfs#6246](https://github.com/ipfs/go-ipfs/pull/6246)) + - Support --human flag in cmd/object-stat ([ipfs/go-ipfs#6241](https://github.com/ipfs/go-ipfs/pull/6241)) + - build: fix macos build with fuse ([ipfs/go-ipfs#6235](https://github.com/ipfs/go-ipfs/pull/6235)) + - add an experiment to prefer TLS 1.3 over secio ([ipfs/go-ipfs#6229](https://github.com/ipfs/go-ipfs/pull/6229)) + - fix two small nits in the go-ipfs constructor ([ipfs/go-ipfs#6234](https://github.com/ipfs/go-ipfs/pull/6234)) + - DI-based core.NewNode ([ipfs/go-ipfs#6162](https://github.com/ipfs/go-ipfs/pull/6162)) + - coreapi: Drop error from ParsePath ([ipfs/go-ipfs#6122](https://github.com/ipfs/go-ipfs/pull/6122)) + - fix the wrong path configuration in root redirection ([ipfs/go-ipfs#6215](https://github.com/ipfs/go-ipfs/pull/6215)) +- github.com/ipfs/go-bitswap (v0.0.4 -> v0.0.7): + - feat(engine): tag peers with requests ([ipfs/go-bitswap#128](https://github.com/ipfs/go-bitswap/pull/128)) + - fix(network): add mutex to avoid data race ([ipfs/go-bitswap#127](https://github.com/ipfs/go-bitswap/pull/127)) + - Change bitswap provide toggle to not be static ([ipfs/go-bitswap#124](https://github.com/ipfs/go-bitswap/pull/124)) + - Use shared peer task queue with Graphsync ([ipfs/go-bitswap#119](https://github.com/ipfs/go-bitswap/pull/119)) + - Add missing godoc comments, refactor to avoid confusion ([ipfs/go-bitswap#117](https://github.com/ipfs/go-bitswap/pull/117)) + - fix(decision): cleanup request queues ([ipfs/go-bitswap#116](https://github.com/ipfs/go-bitswap/pull/116)) + - Control provider workers with experiment flag ([ipfs/go-bitswap#110](https://github.com/ipfs/go-bitswap/pull/110)) + - connmgr: give peers more weight when actively participating in a session ([ipfs/go-bitswap#111](https://github.com/ipfs/go-bitswap/pull/111)) + - make the WantlistManager own the PeerHandler ([ipfs/go-bitswap#78](https://github.com/ipfs/go-bitswap/pull/78)) + - remove IPFS_LOW_MEM flag support ([ipfs/go-bitswap#115](https://github.com/ipfs/go-bitswap/pull/115)) +- github.com/ipfs/go-cid (v0.0.1 -> v0.0.2): + - default cidv1 to base32 ([ipfs/go-cid#85](https://github.com/ipfs/go-cid/pull/85)) +- github.com/ipfs/go-cidutil (v0.0.1 -> v0.0.2): + - default cidv1 to base32 ([ipfs/go-cidutil#13](https://github.com/ipfs/go-cidutil/pull/13)) +- github.com/ipfs/go-datastore (v0.0.3 -> v0.0.5): + - MapDatastore: obey KeysOnly ([ipfs/go-datastore#130](https://github.com/ipfs/go-datastore/pull/130)) + - fix the keytransform datastore's query implementation ([ipfs/go-datastore#127](https://github.com/ipfs/go-datastore/pull/127)) + - sync: apply entire query while locked ([ipfs/go-datastore#129](https://github.com/ipfs/go-datastore/pull/129)) + - filter: values are now always bytes ([ipfs/go-datastore#126](https://github.com/ipfs/go-datastore/pull/126)) + - autobatch: batch deletes ([ipfs/go-datastore#128](https://github.com/ipfs/go-datastore/pull/128)) +- github.com/ipfs/go-ipfs-cmds (v0.0.5 -> v0.0.7): + - merge in go-ipfs-cmdkit ([ipfs/go-ipfs-cmds#164](https://github.com/ipfs/go-ipfs-cmds/pull/164)) + - fix: return the correct error ([ipfs/go-ipfs-cmds#162](https://github.com/ipfs/go-ipfs-cmds/pull/162)) +- github.com/ipfs/go-ipfs-config (v0.0.1 -> v0.0.3): + - Closes: #6284 Add appropriate IPv6 ranges to defaultServerFilters ([ipfs/go-ipfs-config#34](https://github.com/ipfs/go-ipfs-config/pull/34)) + - add an experiment to prefer TLS 1.3 over secio ([ipfs/go-ipfs-config#32](https://github.com/ipfs/go-ipfs-config/pull/32)) +- github.com/ipfs/go-ipfs-files (v0.0.2 -> v0.0.3): + - webfile: make Size() work before Read ([ipfs/go-ipfs-files#18](https://github.com/ipfs/go-ipfs-files/pull/18)) + - check http status code during WebFile reads and return error for non-2XX ([ipfs/go-ipfs-files#17](https://github.com/ipfs/go-ipfs-files/pull/17)) +- github.com/ipfs/go-ipld-cbor (v0.0.1 -> v0.0.2): + - switch to base32 by default ([ipfs/go-ipld-cbor#62](https://github.com/ipfs/go-ipld-cbor/pull/62)) +- github.com/ipfs/go-ipld-git (v0.0.1 -> v0.0.2): + - switch to base32 by default ([ipfs/go-ipld-git#40](https://github.com/ipfs/go-ipld-git/pull/40)) +- github.com/ipfs/go-mfs (v0.0.4 -> v0.0.7): + - Fix directory mv and add tests ([ipfs/go-mfs#76](https://github.com/ipfs/go-mfs/pull/76)) + - fix: not remove file by mistakes ([ipfs/go-mfs#73](https://github.com/ipfs/go-mfs/pull/73)) +- github.com/ipfs/go-path (v0.0.3 -> v0.0.4): + - include the path in path errors ([ipfs/go-path#28](https://github.com/ipfs/go-path/pull/28)) +- github.com/ipfs/go-unixfs (v0.0.4 -> v0.0.6): + - chore: remove URL field ([ipfs/go-unixfs#72](https://github.com/ipfs/go-unixfs/pull/72)) +- github.com/ipfs/interface-go-ipfs-core (v0.0.6 -> v0.0.8): + - switch to base32 cidv1 by default ([ipfs/interface-go-ipfs-core#29](https://github.com/ipfs/interface-go-ipfs-core/pull/29)) + - path: drop error from ParsePath ([ipfs/interface-go-ipfs-core#22](https://github.com/ipfs/interface-go-ipfs-core/pull/22)) + - tests: fix a bunch of small test lints/issues ([ipfs/interface-go-ipfs-core#28](https://github.com/ipfs/interface-go-ipfs-core/pull/28)) + - Update Pin.RmRecursive docs to clarify shared indirect pins are not removed ([ipfs/interface-go-ipfs-core#26](https://github.com/ipfs/interface-go-ipfs-core/pull/26)) +- github.com/libp2p/go-buffer-pool (v0.0.1 -> v0.0.2): + - feat: add buffered writer ([libp2p/go-buffer-pool#9](https://github.com/libp2p/go-buffer-pool/pull/9)) +- github.com/libp2p/go-conn-security-multistream (v0.0.1 -> v0.0.2): + - block while writing ([libp2p/go-conn-security-multistream#10](https://github.com/libp2p/go-conn-security-multistream/pull/10)) +- github.com/libp2p/go-libp2p (v0.0.12 -> v0.0.28): + - Close the connection manager ([libp2p/go-libp2p#639](https://github.com/libp2p/go-libp2p/pull/639)) + - Frequent Relay Advertisements ([libp2p/go-libp2p#637](https://github.com/libp2p/go-libp2p/pull/637)) + - ping: return a stream of results ([libp2p/go-libp2p#626](https://github.com/libp2p/go-libp2p/pull/626)) + - Use cancelable background context in identify ([libp2p/go-libp2p#624](https://github.com/libp2p/go-libp2p/pull/624)) + - avoid intermediate allocation in relayAddrs ([libp2p/go-libp2p#609](https://github.com/libp2p/go-libp2p/pull/609)) + - cache relayAddrs for a short period of time ([libp2p/go-libp2p#608](https://github.com/libp2p/go-libp2p/pull/608)) + - autorelay: break findRelays into multiple functions and avoid the goto ([libp2p/go-libp2p#606](https://github.com/libp2p/go-libp2p/pull/606)) + - autorelay: curtail addrsplosion ([libp2p/go-libp2p#598](https://github.com/libp2p/go-libp2p/pull/598)) + - Periodically schedule identify push if the address set has changed ([libp2p/go-libp2p#597](https://github.com/libp2p/go-libp2p/pull/597)) + - Replace peer addresses in identify ([libp2p/go-libp2p#599](https://github.com/libp2p/go-libp2p/pull/599)) +- github.com/libp2p/go-libp2p-circuit (v0.0.4 -> v0.0.8): + - call Stream.Reset instead of Stream.Close ([libp2p/go-libp2p-circuit#76](https://github.com/libp2p/go-libp2p-circuit/pull/76)) + - Tag the hop relay when creating stop streams ([libp2p/go-libp2p-circuit#77](https://github.com/libp2p/go-libp2p-circuit/pull/77)) + - Tag peers with live hop streams ([libp2p/go-libp2p-circuit#75](https://github.com/libp2p/go-libp2p-circuit/pull/75)) + - Hard Limit the number of hop stream goroutines ([libp2p/go-libp2p-circuit#74](https://github.com/libp2p/go-libp2p-circuit/pull/74)) + - set deadline for stop handshake ([libp2p/go-libp2p-circuit#73](https://github.com/libp2p/go-libp2p-circuit/pull/73)) +- github.com/libp2p/go-libp2p-connmgr (v0.0.1 -> v0.0.6): + - Background trimming ([libp2p/go-libp2p-connmgr#43](https://github.com/libp2p/go-libp2p-connmgr/pull/43)) + - Implement UpsertTag ([libp2p/go-libp2p-connmgr#38](https://github.com/libp2p/go-libp2p-connmgr/pull/38)) + - Add peer protection capability (implementation) ([libp2p/go-libp2p-connmgr#36](https://github.com/libp2p/go-libp2p-connmgr/pull/36)) +- github.com/libp2p/go-libp2p-crypto (v0.0.1 -> v0.0.2): + - add openssl support ([libp2p/go-libp2p-crypto#61](https://github.com/libp2p/go-libp2p-crypto/pull/61)) +- github.com/libp2p/go-libp2p-discovery (v0.0.1 -> v0.0.4): + - More consistent use of options ([libp2p/go-libp2p-discovery#25](https://github.com/libp2p/go-libp2p-discovery/pull/25)) + - Use 3hrs as routing advertisement ttl ([libp2p/go-libp2p-discovery#23](https://github.com/libp2p/go-libp2p-discovery/pull/23)) +- github.com/libp2p/go-libp2p-interface-connmgr (v0.0.1 -> v0.0.5): + - Add Close method to the ConnManager interface ([libp2p/go-libp2p-interface-connmgr#18](https://github.com/libp2p/go-libp2p-interface-connmgr/pull/18)) + - Add UpsertTag to the interface ([libp2p/go-libp2p-interface-connmgr#17](https://github.com/libp2p/go-libp2p-interface-connmgr/pull/17)) + - Fix NullConnMgr to respect ConnManager interface ([libp2p/go-libp2p-interface-connmgr#15](https://github.com/libp2p/go-libp2p-interface-connmgr/pull/15)) + - Add peer protection capability ([libp2p/go-libp2p-interface-connmgr#14](https://github.com/libp2p/go-libp2p-interface-connmgr/pull/14)) +- github.com/libp2p/go-libp2p-kad-dht (v0.0.7 -> v0.0.13): + - fix: reduce memory used by buffered writers ([libp2p/go-libp2p-kad-dht#332](https://github.com/libp2p/go-libp2p-kad-dht/pull/332)) + - query: fix a goroutine leak when the routing table is empty ([libp2p/go-libp2p-kad-dht#329](https://github.com/libp2p/go-libp2p-kad-dht/pull/329)) + - query: fix error "leak" ([libp2p/go-libp2p-kad-dht#328](https://github.com/libp2p/go-libp2p-kad-dht/pull/328)) + - providers: run datastore GC concurrently ([libp2p/go-libp2p-kad-dht#326](https://github.com/libp2p/go-libp2p-kad-dht/pull/326)) + - fix(providers): gc ([libp2p/go-libp2p-kad-dht#325](https://github.com/libp2p/go-libp2p-kad-dht/pull/325)) + - Remove the old protocol from the defaults ([libp2p/go-libp2p-kad-dht#320](https://github.com/libp2p/go-libp2p-kad-dht/pull/320)) + - Fix some provider subsystem performance issues ([libp2p/go-libp2p-kad-dht#319](https://github.com/libp2p/go-libp2p-kad-dht/pull/319)) +- github.com/libp2p/go-libp2p-peerstore (v0.0.2 -> v0.0.6): + - segment the memory peerstore + granular locks ([libp2p/go-libp2p-peerstore#78](https://github.com/libp2p/go-libp2p-peerstore/pull/78)) + - don't delete under the read lock ([libp2p/go-libp2p-peerstore#76](https://github.com/libp2p/go-libp2p-peerstore/pull/76)) + - Read/Write locking ([libp2p/go-libp2p-peerstore#74](https://github.com/libp2p/go-libp2p-peerstore/pull/74)) + - optimize peerstore memory ([libp2p/go-libp2p-peerstore#71](https://github.com/libp2p/go-libp2p-peerstore/pull/71)) + - fix unmarshalling of peer IDs ([libp2p/go-libp2p-peerstore#72](https://github.com/libp2p/go-libp2p-peerstore/pull/72)) + - fix error handling in UpdateAddrs: return on error ([libp2p/go-libp2p-peerstore#70](https://github.com/libp2p/go-libp2p-peerstore/pull/70)) +- github.com/libp2p/go-libp2p-pubsub (v0.0.1 -> v0.0.3): + - rework validator pipeline ([libp2p/go-libp2p-pubsub#176](https://github.com/libp2p/go-libp2p-pubsub/pull/176)) + - Test adversarial signing ([libp2p/go-libp2p-pubsub#181](https://github.com/libp2p/go-libp2p-pubsub/pull/181)) + - Strict message signing by default ([libp2p/go-libp2p-pubsub#180](https://github.com/libp2p/go-libp2p-pubsub/pull/180)) +- github.com/libp2p/go-libp2p-secio (v0.0.1 -> v0.0.3): + - fix buffer size check ([libp2p/go-libp2p-secio#44](https://github.com/libp2p/go-libp2p-secio/pull/44)) +- github.com/libp2p/go-libp2p-swarm (v0.0.2 -> v0.0.6): + - dial: return a nice custom dial error ([libp2p/go-libp2p-swarm#121](https://github.com/libp2p/go-libp2p-swarm/pull/121)) +- github.com/libp2p/go-libp2p-tls (null -> v0.0.1): + - implement the new handshake ([libp2p/go-libp2p-tls#20](https://github.com/libp2p/go-libp2p-tls/pull/20)) + - use a prefix when signing the public key ([libp2p/go-libp2p-tls#26](https://github.com/libp2p/go-libp2p-tls/pull/26)) + - use ChaCha if one of the peers doesn't have AES hardware support ([libp2p/go-libp2p-tls#23](https://github.com/libp2p/go-libp2p-tls/pull/23)) + - improve peer verification ([libp2p/go-libp2p-tls#17](https://github.com/libp2p/go-libp2p-tls/pull/17)) + - add an example (mainly for development) ([libp2p/go-libp2p-tls#14](https://github.com/libp2p/go-libp2p-tls/pull/14)) +- github.com/libp2p/go-libp2p-transport-upgrader (v0.0.1 -> v0.0.4): + - improve correctness of closing connections on failure ([libp2p/go-libp2p-transport-upgrader#19](https://github.com/libp2p/go-libp2p-transport-upgrader/pull/19)) +- github.com/libp2p/go-mplex (v0.0.1 -> v0.0.4): + - fix SetDeadline error conditions ([libp2p/go-mplex#59](https://github.com/libp2p/go-mplex/pull/59)) + - don't use contexts for deadlines ([libp2p/go-mplex#58](https://github.com/libp2p/go-mplex/pull/58)) + - don't reset on pathologies, just ignore the data ([libp2p/go-mplex#57](https://github.com/libp2p/go-mplex/pull/57)) + - read as much as we can in one go ([libp2p/go-mplex#53](https://github.com/libp2p/go-mplex/pull/53)) + - use timeouts when sending messages for stream open, close, and reset. ([libp2p/go-mplex#52](https://github.com/libp2p/go-mplex/pull/52)) + - fix: reset a stream even if closed remotely ([libp2p/go-mplex#50](https://github.com/libp2p/go-mplex/pull/50)) + - downgrade Error log to Warning ([libp2p/go-mplex#46](https://github.com/libp2p/go-mplex/pull/46)) + - Fix race condition by adding a mutex for deadline access ([libp2p/go-mplex#41](https://github.com/libp2p/go-mplex/pull/41)) +- github.com/libp2p/go-msgio (v0.0.1 -> v0.0.2): + - fix: never claim to read more than read ([libp2p/go-msgio#12](https://github.com/libp2p/go-msgio/pull/12)) + + ## 0.4.20 2019-04-16 We're happy to release go-ipfs 0.4.20. This release includes some critical diff --git a/bin/mkreleaselog b/bin/mkreleaselog index 2e9977e8279..b769c348ee1 100755 --- a/bin/mkreleaselog +++ b/bin/mkreleaselog @@ -1,29 +1,119 @@ -#!/usr/bin/env bash +#!/bin/zsh #set -x set -euo pipefail -IFS=$'\n\t' export GO111MODULE=on +export GOPATH="$(go env GOPATH)" -LAST_TAG=$(git tag -l | sort -V | grep -v -- '-rc' | grep 'v'| tail -n1) - -git log --oneline --merges --reverse $LAST_TAG...master | -while read MERGE -do - commit=$(echo $MERGE | awk '{ print $1 }') - prnum=$(echo $MERGE | awk '{ print $5 }' | tr -d '#') - desc=$(git show $commit | sed '8q;d' | sed 's/^ //g') - printf " - %s ([ipfs/go-ipfs#%s](https://github.com/ipfs/go-ipfs/pull/%s))\n" "$desc" "$prnum" "$prnum" -done - -TMP_DIR=$(mktemp -d --tmpdir go-ipfs-mkreleaselog-XXXXXX) -git show "$LAST_TAG:go.mod" > "$TMP_DIR/go.mod" - -echo -echo "Dependecy updates:" -join -a 2 <(cd "$TMP_DIR" && go list -m all | sort -k 1b,1) <(go list -m all | sort -k 1b,1) | awk \ - ' - { if (NF == 2) - printf "2 - %s none => %s\n", $1, $2 - else if ($2 != $3) - printf "1 - %s %s => %s\n", $1, $2, $3 } - ' | sort | sed 's/[0-9] //' +alias jq="jq --unbuffered" + +[[ -n "${REPO_FILTER+x}" ]] || REPO_FILTER="github.com/(ipfs|libp2p|ipld)" +[[ -n "${IGNORED_FILES+x}" ]] || IGNORED_FILES='^\(\.gx\|package.json\|\.travis.yml\|go.mod\|go.sum\)$' + +msg() { + echo "$*" >&2 +} + +# Returns a stream of deps changed between $1 and $2. +dep_changes() { + { + <"$1" + <"$2" + } | jq -s 'JOIN(INDEX(.[0][]; .Path); .[1][]; .Path; {Path: .[0].Path, Old: (.[1] | del(.Path)), New: (.[0] | del(.Path))}) | select(.New.Version != .Old.Version)' +} + +# resolve_commits resolves a git ref for each version. +resolve_commits() { + jq '. + {Ref: (.Version|capture("^((?.*)\\+incompatible|v.*-[0-9]{14}-(?[a-f0-9]{12})|(?v.*))$") | .ref1 // .ref2 // .ref3)}' +} + +# Generate a release log for a range of commits in a single repo. +release_log() { + local repo="$1" + local start="$2" + local end="${3:-HEAD}" + local ghname="${repo##github.com/}" + local dir="$GOPATH/src/$repo" + + local commit prnum + git -C "$dir" log \ + --format='tformat:%H %s' \ + --merges \ + "$start..$end" | + sed -n -e 's/\([a-f0-9]\+\) .*#\([0-9]\+\).*/\1 \2/p' | + while read commit prnum; do + # Skip gx-only PRs. + git -C "$dir" diff-tree --no-commit-id --name-only "$commit^" "$commit" | + grep -v "${IGNORED_FILES}" >/dev/null || continue + + local desc="$(git -C "$dir" show --summary --format='tformat:%b' "$commit" | head -1)" + printf "- %s ([%s#%s](https://%s/pull/%s))\n" "$desc" "$ghname" "$prnum" "$repo" "$prnum" + done +} + +indent() { + sed -e 's/^/ /' +} + +mod_deps() { + go list -json -m all | jq 'select(.Version != null)' +} + +ensure() { + local repo="$1" + local commit="$2" + local rpath="$GOPATH/src/$repo" + if [[ ! -d "$rpath" ]]; then + msg "Cloning $repo..." + git clone "http://$repo" "$rpath" >&2 + fi + + if ! git -C "$rpath" rev-parse --verify "$commit" >/dev/null; then + msg "Fetching $repo..." + git -C "$rpath" fetch --all >&2 + fi + + git -C "$rpath" rev-parse --verify "$commit" >/dev/null || return 1 +} + +recursive_release_log() { + local start="${1:-$(git tag -l | sort -V | grep -v -- '-rc' | grep 'v'| tail -n1)}" + local end="${2:-$(git rev-parse HEAD)}" + local repo_root="$(git rev-parse --show-toplevel)" + local package="$(go list)" + ( + local workspace="$(mktemp -d)" + trap "$(printf 'rm -rf "%q"' "$workspace")" INT TERM EXIT + cd "$workspace" + + echo "Computing old deps..." >&2 + git -C "$repo_root" show "$start:go.mod" >go.mod + mod_deps | resolve_commits | jq -s > old_deps.json + + echo "Computing new deps..." >&2 + git -C "$repo_root" show "$end:go.mod" >go.mod + mod_deps | resolve_commits | jq -s > new_deps.json + + rm -f go.mod go.sum + + printf "Generating Changelog for %s %s..%s\n" "$package" "$start" "$end" >&2 + + printf "- %s:\n" "$package" + release_log "$package" "$start" "$end" | indent + + + dep_changes old_deps.json new_deps.json | + jq --arg filter "$REPO_FILTER" 'select(.Path | match($filter))' | + # Compute changelogs + jq -r '"\(.Path) \(.New.Version) \(.New.Ref) \(.Old.Version) \(.Old.Ref // "")"' | + while read repo new new_ref old old_ref; do + ensure "$repo" "$new_ref" + local changelog="$(release_log "$repo" "$old_ref" "$new_ref")" + if [[ -n "$changelog" ]]; then + printf "- %s (%s -> %s):\n" "$repo" "$old" "$new" + echo "$changelog" | indent + fi + done + ) +} + +recursive_release_log "$@" diff --git a/docs/experimental-features.md b/docs/experimental-features.md index 32019b08ed1..9f25ef2a416 100644 --- a/docs/experimental-features.md +++ b/docs/experimental-features.md @@ -28,6 +28,7 @@ the above issue. - [IPNS PubSub](#ipns-pubsub) - [QUIC](#quic) - [AutoRelay](#autorelay) +- [TLS 1.3 Handshake](#tls-13-as-default-handshake-protocol) ---