You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository has been archived by the owner on Jun 1, 2021. It is now read-only.
We recently had to reduce the eventuate.log.write-batch-size (to 2 if that's relevant) to prevent "com.datastax.driver.core.exceptions.InvalidQueryException: Batch too large" - because we're running a C* version that does not yet have the fix for CASSANDRA-10876.
As a consequence performance tests failed and we saw that event replication between locations increasingly diverged. This seems to be caused by the fact that for the replication batch size the eventuate.log.write-batch-size is used as well.
IMO the batching layer batch size (log/persistence) and the replication batch size should be independently configurable.
Right now it's fairly easy: the replication batch can be written as a batch - atomically.
If the replication batch size could be greater than log batch size several batches would have to be written. So there are probably various options to deal with failures regarding the replication acknowledgement, what currently comes to my mind:
If any batch write fails then reply with "failure", so that the complete replication batch is resent (assumes that events can be written more than once)
Kind of selective acknowledgement: acknowledge what has been written successfully (range of events), everything unconfirmed has to be resent.
The text was updated successfully, but these errors were encountered:
krasserm
changed the title
Decouple replication batch size from log batching layer batch size
Decouple replication batch size from write batch size
Aug 24, 2016
@magro what's your priority for this ticket? The issues described here are addressed by #196 and #314. If it's not a blocker or high priority I'd like to close this ticket and defer support for it until work on the other two tickets start. Even if CASSANDRA-10876 has not been applied to your Cassandra version, it shouldn't harm increasing the batch size limits as explained batch write failure handling if Eventuate is the only writer to Cassandra. WDYT?
We recently had to reduce the
eventuate.log.write-batch-size
(to 2 if that's relevant) to prevent "com.datastax.driver.core.exceptions.InvalidQueryException: Batch too large" - because we're running a C* version that does not yet have the fix for CASSANDRA-10876.As a consequence performance tests failed and we saw that event replication between locations increasingly diverged. This seems to be caused by the fact that for the replication batch size the
eventuate.log.write-batch-size
is used as well.IMO the batching layer batch size (log/persistence) and the replication batch size should be independently configurable.
Right now it's fairly easy: the replication batch can be written as a batch - atomically.
If the replication batch size could be greater than log batch size several batches would have to be written. So there are probably various options to deal with failures regarding the replication acknowledgement, what currently comes to my mind:
The text was updated successfully, but these errors were encountered: