Replication throughput improvements

[krasserm]

Improve the current Replicator implementation to support concurrent ReplicationReads from the source event log while executing ReplicationWrites to the target event log. Also, batch sizes for reading can be usually much larger than batch sizes for writing (especially when using Cassandra). Therefore, an event batch read from the source should be split into n batches for sequential writes to the target.

[magro]

Just to mention it here (a think that I got aware of recently): as long as the replication builds upon akka remoting the messages for replicating events shouldn't become too big, because akka remoting is not designed for sending large messages. Currently eventuate's replication messages could impact akka remote control messages and cause disconnections in consequence.

In the long term it would probably be beneficial to switch to something different (akka streams?) or see if the new remoting (Artery) with support for a separate subchannel for large messages can be used.

[krasserm]

I agree. I also followed the recent development of akka-remote using Artery and we should definitely evaluate it, in addition to akka-stream as potential alternative to the current implementation.