You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
The kafka buffer is currently subject to the heap circuit breaker on receipt of data. Since this buffer writes directly to a Kafka topic, and thus leaves the JVM heap, it shouldn't have the heap circuit-breaker impact it.
Describe the solution you'd like
Provide a new function on Buffer to indicate if the buffer uses off-heap data.
boolean isWrittenOffHeapOnly()
When this is true, then do not apply the CircuitBreaker. This initial logic may not be ideal long-term as we could possibly support other types of circuit-breakers (say based on local disk-space, a CPU-based threshold, or the current pipeline latency). However, this will work for an initial implementation.
Additional context
The kafka buffer already uses the circuit-breaker when reading from the topic to place into the processor chains - #3578.
The text was updated successfully, but these errors were encountered:
#3619)
Disable the circuit breaker for buffers that write data off-heap only. This is currently only the Kafka buffer. Resolves#3616
Signed-off-by: David Venable <[email protected]>
Is your feature request related to a problem? Please describe.
The
kafka
buffer is currently subject to the heap circuit breaker on receipt of data. Since this buffer writes directly to a Kafka topic, and thus leaves the JVM heap, it shouldn't have the heap circuit-breaker impact it.Describe the solution you'd like
Provide a new function on
Buffer
to indicate if the buffer uses off-heap data.When this is
true
, then do not apply theCircuitBreaker
. This initial logic may not be ideal long-term as we could possibly support other types of circuit-breakers (say based on local disk-space, a CPU-based threshold, or the current pipeline latency). However, this will work for an initial implementation.Additional context
The
kafka
buffer already uses the circuit-breaker when reading from the topic to place into the processor chains - #3578.The text was updated successfully, but these errors were encountered: