Stream Priority and Per-Stream/Datagram MaxAckDelay

[wegylexy]

Example Scenario:

In a simple networked multiplayer game, players can move around, talk to each other, and press buttons in sequence.

• Datagram is dedicated for position updates, e.g. when players walk or jump (losing a few packets in a while does not matter).
• Stream A is dedicated for occasional real-time half-duplex audio communication, e.g. when players say something up to half a minute (lost packets should be detected and retransmitted ASAP to prevent emptying the jitter buffer).
• Stream B is dedicated to occasional button events, e.g. when players quickly hit several buttons in a particular order (the order is important, but can be delayed).
• Budget is limited, so egress traffic from the server should be minimized.

Datagrams for player movement have the lowest priority. Instead of acking every datagram, it is preferable to simply shut down the connection on high loss rate (configurable, e.g. lossing 3% in past 5 seconds) with an error code.

Stream A requires the highest priority and minimum ack delay to prevent audio glitches. Datagram is not used for occasional voice because stream ensures ordered packet delivery and the game has a simple jitter buffer for each audio source.

Stream B can suffer longer delay as long as button events are eventually delivered in order. A standalone ack packet cost around 70 bytes over the wire, so avoid acking too frequently, but reserve resources for stream A. Acks for stream B should be bundled with acks for datagrams or stream A whenever fit.

There could be thousands of concurrent players from all over the real world in the same shared virtual world, where tens of players may be in proximity of each other.

How can QUIC be improved for hosting this game in the cloud?

[nibanks]

My initial thoughts:

• Practically, you don't want datagrams to elicit an immediate acknowledgement (either ever or at least extremely delayed).
• Everything else should be acknowledged in a reasonable (normal) timeframe.

So, one solution (I think the simplest) is a configuration option (that needs to be exchanged on the wire) is to say datagram frames are not ACK-eliciting. This would mean you would not get any acknowledgement feedback on the connection unless you also send some other type of data. If you're regularly sending other data, things should be fine; but if you're not it's hard to say how things would react. In your scenario, since the app is sending datagrams in both directions the ACKs would flow along with those packets and things should be fine.

Another option would be to increase the delayed ACK timeout and ACK frequency threshold, but when you send non-datagrams, marks those packets as needing more immediate acknowledgements. This is more complicated to implement, but would result in ACKs being send/received eventually, even if you only send datagrams in one direction.

[nibanks]

FYI, created a spec issue to discuss here: quicwg/datagram#42.