Add backpressure mechanism

[skadisch]

Currently requests are packed and pushed into a Node.js Socket stream.Writable. This works as long as

• the amount of data generated is low,
• there is enough idle time in the event queue and/or
• the server / connection can consume the data fast enough

When using Xlib we automatically get backpressure from the socket, as the buffer will be quite small.
In Node.js on the other hand, data can be pushed into a stream.Writable until the process is out of memory, so no "natural" backpressure like in Xlib will occour due to the asynchronous design.

A real life example where this can be a problem is rendering on mouse movement. I have an application where I can move the mouse for 30 seconds and the internal buffer will fill up faster than data can be processed. When stopping moving the mouse I can see the data processing catching up and rendering all previously queued requests. Another case would be when I just want to render at full speed / max FPS.

Basically the problem is always there but it will only get a pain in those cases, as normally all processing will be fast enough.

Maybe the intention is to use "Expose" events to request redraws from the server (server will loop it back to us and will only be able to do so when our event has been sent, thus queue must be empty). This would generate backpressure on the protocol layer (vs socket layer). I have not seen this approach in any of the node-x11 or ntk samples though. If this would be the way to go, we should include it in the sample code as best practice.

Other possible solutions that I see:

• Add callbacks / promises to all calls that invoke X11 requests. The callbacks are forwarded to the according stream.write of packed request data. This means considering those requests as "done" as soon as the request is "on the wire".
• Add a flush() -like method with a callback / promise that will resolve as soon as all buffered data up to the point of invocation is on the wire
• Use drain-like events same as Node.js's stream.Writable

[sidorares]

x11 protocol is very "async", most requests don't notify you about success even when you create a resource ( instead you provide client-generated new resource id, which acts as a "future" reference ). Because of this some of the scenarious you outlined are tricky or not possible at all

I'd really like to rewrite completely low level serialisation layer - it's almost 10 years old, back then there was no Buffers and streams also evolved quite a bit.

Add a flush() -like method with a callback / promise that will resolve as soon as all buffered data up to the point of invocation is on the wire

yes, that might be useful