Kafka Connector

Contents

• Overview
• Summary
• Source Files
• Configuration
• Diagnostics
• Metrics

Overview

• KafkaConnector is intended for consuming a single Kafka topic

Summary

Supports Datastream Updates
Checkpoint Type
Diagnostics Aware
Metrics Aware

Source Files

1. com.linkedin.datastream.connectors.kafka.KafkaConnector

2. com.linkedin.datastream.connectors.kafka.AbstractKafkaConnector

Configuration

• All Connector config properties must be prefixed with brooklin.server.connector.<connectorName>

• connectorName is an arbitrary user-supplied name specified in Brooklin configuration (brooklin.server.connectorNames)

Property	Description	Default
defaultKeySerde	Name of SerDe (Serializer/Deserializer) to use with Kafka topic key Name has to match one of the values specified in brooklin.server.serdeNames	(None)
defaultValueSerde	Name of SerDe (Serializer/Deserializer) to use with Kafka topic value Name has to match one of the values specified in brooklin.server.serdeNames	(None)
commitIntervalMs	The time duration between successive offset commits to Kafka Must be in the range [0, Log.MAX_VALUE] (inclusive)	1 minute

Diagnostics

Metrics