| copyright | lastupdated | ||
|---|---|---|---|
|
2017-01-25 |
{:new_window: target="_blank"} {:shortdesc: .shortdesc} {:screen: .screen} {:codeblock: .codeblock} {:pre: .pre}
{: #apache_kafka}
Apache Kafka forms the reliable messaging core of {{site.data.keyword.messagehub}}. It is a publish-subscribe messaging system and is designed to be fault-tolerant providing a high-throughput, low-latency platform for handling real-time data feeds. These characteristics make it ideal for use in a cloud environment. {:shortdesc}
The following list defines some Apache Kafka concepts:
- Topic
- A feed that messages are published to.
- Partition
- Each topic comprises one or more partitions. Each partition is an ordered list of messages. If a topic has more than one partition, it allows data to be fed through in parallel to increase throughput.
- Producer
- A process that publishes streams of messages to Kafka topics. A producer can publish to one or more topics and can optionally choose the partition that stores the data.
- Consumer
- A process that consumes messages from Kafka topics, and processes the feed of published messages. A consumer can subscribe to one or more topics or partitions.
- Consumer group
- A named group of one or more consumers. Each consumer in the group reads messages from specific
partitions in the topics that the consumer subscribes to. Each message is delivered to one consumer
in the group and all messages with the same key are delivered to the same consumer.
Each partition is assigned to one consumer in the group only.
- If there are more partitions than consumers in a group, some consumers have multiple partitions.
- If there are more consumers than partitions, some consumers have no partitions.
To learn more, see Apache Kafka documentation 
{:new_window} and Message Hub Kafka Java™ API developerWorks® article {:new_window}.