5.0.0 (2022-01-24)

BREAKING CHANGES

Upgrading from 4.x to 5.x: Please remove the existing state.json created by tap-kafka 4.x.
5.x will continue consuming messages from the last consumed offset but will generate state.json in a new format.

CHANGELOG

• Switching from kafka-python to confluent-kafka-python
• Using faster orjson provided by pipelinewise-singer-python-2.x
• Remove local store and bookmark consumed message in STATE messages
• Add initial_start_time optional parameter

4.0.1 (2021-08-13)

Fixes:

• Fallback to default consumer timeout if consumer_timeout_ms is not provided in discovery mode.
• Stop mis-handling exceptions during discovery.

4.0.0 (2020-08-27)

• Improve the performance of persisting kafka messages if the local store cannot perform frequent file appends and causing high I/O issues
• Switching from jsonpath-ng to dpath python library to improve the performance of extracting primary keys
• Change the syntax of primary_keys from JSONPath to /slashed/paths ala XPath

3.1.0 (2020-04-20)

• Add max_poll_records option

3.0.0 (2020-04-03)

• Add local storage of consumed messages and instant commit kafka offsets
• Add more configurable options: consumer_timeout_ms, session_timeout_ms, heartbeat_interval_ms and max_poll_interval_ms
• Add two new fixed output columns: MESSAGE_PARTITION and MESSAGE_OFFSET

2.1.1 (2020-03-23)

• Commit offset from state file and not from the consumed messages

2.1.0 (2020-02-18)

• Make logging customizable

2.0.0 (2020-01-07)

• Rewamp output schema with static columns with no JSON flattening:
  • MESSAGE (object)
  • MESSAGE_TIMESTAMP
  • DYNAMIC_PRIMARY_KEY(S)

1.0.2 (2019-11-25)

• Add 'encoding' configurable parameter. 'utf-8' is used as a default encoding now, and it could be configured from config.json.

1.0.1 (2019-08-16)

• Add license classifier