Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kafka 0.11 on debian:stretch #5

Merged
merged 31 commits into from
Jul 26, 2017
Merged
Changes from 1 commit
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
f5fc72c
Runs the official Kafka distribution on the new Debian stable
solsson Jul 20, 2017
eeee33b
https://kafka.apache.org/downloads says "2.11 is recommended"
solsson Jul 20, 2017
99f4d45
Uses Zulu jre/jdk like cp-docker-images base does
solsson Jul 20, 2017
8bf55a1
Drops a layer
solsson Jul 20, 2017
c4066a7
Starts a build-contract to smoke test and exemplify image use cases
solsson Jul 20, 2017
028aed1
Fixes topic creation, but with startup timing issues because the scri…
solsson Jul 20, 2017
f52bd3b
Adds a consumer
solsson Jul 20, 2017
bdea1ad
For previous commit
solsson Jul 20, 2017
ea54416
Quick attempt to verify that the image works with Connect. Class does…
solsson Jul 20, 2017
16ef01b
Allow automatic topic creation again, to simplify test setup
solsson Jul 20, 2017
73ea978
Each connect setup will have to be its own docker build...
solsson Jul 20, 2017
7b82d0f
Quite useful log aggregation
solsson Jul 20, 2017
845faff
worker.properties can basically be copied among connect builds
solsson Jul 21, 2017
50e0c9a
Demonstrates JMX to HTTP /metrics
solsson Jul 21, 2017
4b183a5
Demonstrates built connect jar, but messages are encoded json in oute…
solsson Jul 21, 2017
87d3172
Splits into separate docker-compose setups per area of interest
solsson Jul 22, 2017
74ad3aa
Now adds less than 6 MB to the official image
solsson Jul 22, 2017
415bb65
Clarifies that we use 3rd party source here
solsson Jul 22, 2017
941e719
Merge branch 'build-contract' into kafka-011-debian
solsson Jul 22, 2017
e4de7e6
Honors the nature of Kafka distributions
solsson Jul 22, 2017
385eff8
Tries to replace the old image behavior with instructions
solsson Jul 22, 2017
e21ae32
Slightly shorter
solsson Jul 22, 2017
46de3f4
Merge pull request #7 from solsson/image-help-entrypoint
solsson Jul 23, 2017
9978188
Adds the nc command as it's useful for liveness probes
solsson Jul 23, 2017
b34eb6c
Adds base image, as Zulu will be upgraded less frequently than Kafka
solsson Jul 25, 2017
15ba474
Builds the kafka image from the base jre image
solsson Jul 25, 2017
17f0f6e
Uses a specific base image build. 58 MB download on top of debian.
solsson Jul 25, 2017
180ff1d
Merge pull request #9 from solsson/jre-base-image
solsson Jul 25, 2017
08ee3b6
Changes scala env names to match assumptions in kafka-run-class.sh
solsson Jul 25, 2017
e7bd905
Reflects kafka-run-class.sh better. We don't have Scala, neither at i…
solsson Jul 25, 2017
0314080
Slightly cleaner installation
solsson Jul 26, 2017
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Demonstrates built connect jar, but messages are encoded json in oute…
…r json's payload
solsson committed Jul 21, 2017
commit 4b183a5b01d72b24f2cb4d88ea6d0b7a31ae86b6
10 changes: 9 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -20,14 +20,22 @@ Build and test using: `docker run -v /var/run/docker.sock:/var/run/docker.sock -

To keep kafka running for local use, uncomment `ports` 9092 and run: `docker-compose -f build-contracts/docker-compose.yml up --force-recreate`.

While timing issues remain, start services individually
While timing issues remain, start services individually...

```bash
compose='docker-compose -f build-contracts/docker-compose.yml'
$compose up -d zookeeper kafka-0
$compose logs zookeeper kafka-0
# can we create topics using the image's provided script?
$compose up test-topic-create-1
# can a producer send messages using snappy (has issues before with a class missing in the image)
$compose up test-snappy-compression
$compose up test-consume-all
# demo the prometheus exporter image
$compose up prometheus-jmx-exporter test-metrics-export
# demo the log/file aggregation image
$compose up --build -d connect-files
$compose up test-consume-files
# demo the JMX->kafka image
$compose up --build connect-jmx test-jmx
```
15 changes: 15 additions & 0 deletions build-contracts/docker-compose.yml
Original file line number Diff line number Diff line change
@@ -115,5 +115,20 @@ services:

connect-jmx:
build: ../connect-jmx
labels:
com.yolean.build-target: ""
links:
- kafka-0

test-jmx:
image: solsson/kafkacat@sha256:1266d140c52cb39bf314b6f22b6d7a01c4c9084781bc779fdfade51214a713a8
labels:
com.yolean.build-contract: ""
command:
- -b
- kafka-0:9092
- -t
- jmx-test
- -C
- -o
- beginning