Skip to content

Commit

Permalink
Adds a job that can actually fix under-replication ...
Browse files Browse the repository at this point in the history
as long as you have one up-to-date replica.

The use case is broker outages that are longer than mere re-scheduling,
such as zone outage in a multi-zone cluster.
  • Loading branch information
solsson committed Nov 10, 2017
1 parent 7428403 commit d1377a5
Showing 1 changed file with 51 additions and 0 deletions.
51 changes: 51 additions & 0 deletions maintenance/reassign-paritions-job.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
apiVersion: batch/v1
kind: Job
metadata:
name: reassign-partitions
namespace: kafka
spec:
template:
metadata:
name: reassign-partitions
spec:
containers:
- name: kafka
image: solsson/kafka:1.0.0@sha256:17fdf1637426f45c93c65826670542e36b9f3394ede1cb61885c6a4befa8f72d
env:
- name: ZOOKEEPER
value: zookeeper.kafka:2181
# the following must be edited per job
- name: TOPICS
value: test-produce-consume,test-kafkacat
- name: BROKERS
value: 0,2
command:
- /bin/bash
- -ce
- >
echo '{"topics":[' > /tmp/reassign-topics.json;
echo -n ' {"topic":"' >> /tmp/reassign-topics.json;
echo -n $TOPICS | sed 's/,/"},\n {"topic":"/g' >> /tmp/reassign-topics.json;
echo '"}' >> /tmp/reassign-topics.json;
echo ']}' >> /tmp/reassign-topics.json;
echo "# reassign-topics.json";
cat /tmp/reassign-topics.json;
./bin/kafka-reassign-partitions.sh
--zookeeper=$ZOOKEEPER
--generate
--topics-to-move-json-file=/tmp/reassign-topics.json
--broker-list=$BROKERS > /tmp/generated.txt;
tail -n 1 /tmp/generated.txt > /tmp/proposed-reassignment.json;
echo "# proposed-reassignment.json";
cat /tmp/proposed-reassignment.json;
./bin/kafka-reassign-partitions.sh
--zookeeper=$ZOOKEEPER
--execute
--reassignment-json-file=/tmp/proposed-reassignment.json;
restartPolicy: Never
backoffLimit: 3

0 comments on commit d1377a5

Please sign in to comment.