Improved support for data consistency on error conditions

[dfeist]

Right now onError() downstream means cancel() upstream and cancel() upstream means drop anything in flight in memory in internal queues in operators e.g. parallel, publishOn etc. While flatMap has delayErrors support, a typical use case will also use other operators.

In order to fully meet these requirements, reactor needs a way(s) to allow the user to opt for memory-consistency in all operators, ideally in a cross-cutting way and no per-component (if possible). Some of this work may be able to be done in 3.0.6, other will need to wait for 3.1.

This is a summary of the items discussed via gitter, to break out into separate issues as required:

• consider delayErrors default in flapMap (probably going to be best to leave as is i assume)
• implement delayError behavior in parallel
• implement delayError behavior for other operators with internal queues.
• see if there is a way to apply delayError behaviour in a cross-cutting way without needing to configure each and every operator
• ensure all operators that drop signals go via queueDropped hook.

[smaldini]

To break down in separate issues (the possible breaking change especially)

[dfeist]

Worth looking at Akka Streams deals with this: http://doc.akka.io/docs/akka/current/scala/stream/stream-error.html

I don't see a strategy that permits user-defined local handling though, only stop (default behaviour in reactor/rxJava also) and drop/continue strategies which would appear to use null local error handler to drop failed signal and leave stream un affected.

[smaldini]

Initial internals have started the preparation and task will continue if possible post RC1.

[simonbasle]

closing this due to inactivity. the only remaining open question is probably point 3 "see if there is a way to apply delayError behaviour in a cross-cutting way without needing to configure each and every operator"
=> please open a new issue if you feel there is still a strong need for that one