Skip to content

Latest commit

 

History

History
49 lines (41 loc) · 1.27 KB

deploy_kafka.md

File metadata and controls

49 lines (41 loc) · 1.27 KB
Raw

Deploy Kafka

Command Line

  1. Go to /usr/local/kafka/
cd /usr/local/kafka/
  1. create topic with a single partition with single replica
    • Partition - the number of brokers to split the data into
    • Replica - the number of copies of the data to be created
bin/kafka-topics.sh --create --topic test-topic --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1
  1. View list of topics
bin/kafka-topics.sh --list --bootstrap-server localhost:9092
  1. Initiate producer
bin/kafka-console-producer.sh --broker-list localhost:9092 --topic  test-topic
  1. In a new terminal, initiate consumer
cd /usr/local/kafka
bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic  test-topic --from-beginning

Python

The Python code uses kafka-python

# via pip
pip3 install kafka-python
# via apt
Sudo apt-get -y install python3-kafka
  • consumer - this is the code that reads content sent into Kafka
python3 consumer.py test-topic localhost:9092 --timeout -1 -e
  • producer - this is the code that writes content into Kafka
python3 producer.py localhost:9092 test-topic '{"hello": "world"}' -e