Skip to content

Commit

Permalink
Prevent users from enabling auto commit
Browse files Browse the repository at this point in the history
Zio-kafka applications get their performance from pre-fetching data. When auto commit is enabled, the consumer will automatically commit batches _before_ they are processed by the user streams.

An unaware user might accidentally enable auto commit and lose data during rebalances.

Solves #1289.
  • Loading branch information
erikvanoosten committed Jul 22, 2024
1 parent fd40816 commit f065e38
Showing 1 changed file with 5 additions and 0 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,11 @@ final case class ConsumerSettings(
runloopMetricsSchedule: Schedule[Any, Unit, Long] = Schedule.fixed(500.millis),
authErrorRetrySchedule: Schedule[Any, Throwable, Any] = Schedule.recurs(5) && Schedule.spaced(500.millis)
) {
// Parse booleans in a way compatible with how Kafka does this in org.apache.kafka.common.config.ConfigDef.parseType:
require(
properties.get(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG).forall(_.toString.trim.equalsIgnoreCase("false")),
"Because zio-kafka does pre-fetching, auto commit is not supported"
)

/**
* Tunes the consumer for high throughput.
Expand Down

0 comments on commit f065e38

Please sign in to comment.