KCP transport (reliable UDP)

[ghost]

Let's check it out:

https://github.com/xtaci/kcp-go

kcp-go is a Production-Grade Reliable-UDP library for golang.

It provides fast, ordered and error-checked delivery of streams over UDP packets, has been well tested with opensource project kcptun. Millions of devices(from low-end MIPS routers to high-end servers) are running with kcp-go at present, including applications like online games, live broadcasting, file synchronization and network acceleration.

1. Optimized for Realtime Multiplayer Games, Audio/Video Streaming.
2. Compatible with skywind3000's C version with language specific optimizations.
3. Cache friendly and Memory optimized design, offers extremely High Performance core.
4. Compatible with net.Conn and net.Listener, easy to use.
5. FEC(Forward Error Correction) Support with Reed-Solomon Codes
6. Packet level encryption support with AES, TEA, 3DES, Blowfish, Cast5, Salsa20, etc. in CFB mode.
7. O(1) goroutines created for the entire server application, minimized goroutine context switch.

It's used by Lantern and Syncthing among others

[whyrusleeping]

Yeah, i'd love to try this out. Should be fairly simple to plug it in

[whyrusleeping]

They also have a multiplexer: https://github.com/xtaci/smux

[whyrusleeping]

Easy enough to plug the muxer in, it passes our test suite too: https://github.com/whyrusleeping/go-smux-smux

[whyrusleeping]

Only rough part of plugging kcp in seems to be that their net.Addr method isnt quite compliant, it just returns a udp addr. So i may have to fork it and change that

[daviddias]

Oh wow, this looks sweet! https://github.com/xtaci/smux. I like this team, they modularize things! //cc @Stebalien

[Stebalien]

Ooh nice! Personally, I wouldn't bother with KCP given that QUIC is around the corner (unless QUIC doesn't work out) but their SMUX library looks great and can be used over TCP.

[paralin]

I have finished my KCP transport implementation, it uses smux to basically build a mock-tcp connection. From my initial tests with even the default configs the performance is quite impressive.

https://github.com/paralin/go-libp2p-kcp

I replicated the echo example, with GRPC over the connection as a bonus (so, UDP[KCP[SMUX[GRPC]]]).

https://github.com/paralin/go-libp2p-grpc

[whyrusleeping]

@paralin Hey! Great work there. A couple things:

• Not sure why the smux package is used in the transport itself, we can have that separately as a muxer package. KCP is reliable on its own, right?
• It would be really great to be able to dial out using the same socket we listen on. This would save a ton of file descriptors. As implemented, each dial would create a new socket. Not sure if kcp lets us do this, but we were able to with utp
• We should start looking at using plugins to add transports to libp2p without having to rebuild ipfs.

[paralin]

@whyrusleeping Not sure why, but without smux in place libp2p wouldn't work at all over KCP. I suspected it was because: "Control messages like SYN/FIN/RST in TCP are not defined in KCP, you need some keepalive/heartbeat mechanism in the application-level." For this reason I believe we need some kind of SYN/ACK in the transport level, although we really shouldn't need it. I just haven't had a chance yet to write any tests to figure out the issue.

We can pass a PacketConn to KCP, so establishing a socket dialing out with SO_REUSE should be fine.

In terms of plugins this should be easy with the existing interfaces and the Go plugin mechanism.

[whyrusleeping]

Interesting... I'll bet it's something like "the other side doesn't get the connection until we send a packet". If you could investigate that further it would be really helpful. On the plugins, if we get that working, then we can also add tor and webrtc transports :)

…

On Sun, Dec 31, 2017, 9:26 PM Christian Stewart ***@***.***> wrote: @whyrusleeping <https://github.com/whyrusleeping> Not sure why, but without smux in place libp2p wouldn't work at all over KCP. I suspected it was because: "Control messages like SYN/FIN/RST in TCP are not defined in KCP, you need some keepalive/heartbeat mechanism in the application-level." For this reason I believe we need some kind of SYN/ACK in the transport level, although we really shouldn't need it. I just haven't had a chance yet to write any tests to figure out the issue. We can pass a PacketConn to KCP, so establishing a socket dialing out with SO_REUSE should be fine. In terms of plugins this should be easy with the existing interfaces and the Go plugin mechanism. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#193 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABL4HEi8L8Z9KHPXmhnTvGLVL8O6xOkHks5tGGyagaJpZM4NaO6D> .

[whyrusleeping]

@paralin Do you think you could take a look at adding a Libp2pTransportPlugin to ipfs?

Right now we just have IpldPlugins: https://github.com/ipfs/go-ipfs/blob/master/plugin/loader/initializer.go#L21
But we're working on adding a new set of plugins for turning logs into execution traces: ipfs/kubo#4506

The tricky part here is that we would likely have to have some global DefaultTransports array in libp2p that the plugin would append to, then the libp2p constructor would need to use those.

If you don't have time or don't want to, no worries, I figured I would ask here anyways, and someone else could pick it up :)

[paralin]

I'd be happy to, but I'm a bit busy at the moment so it'll be a little while.

[paralin]

@whyrusleeping Please have a look at paralin/go-libp2p-kcp#1 re/KCP integration with reusable UDP file descriptors (assuming the CC didn't send you a notification).

[Stebalien]

(QUIC has shipped)