Skip to content

Commit

Permalink
Migrates zipkin-kafka docker image (#2852)
Browse files Browse the repository at this point in the history
  • Loading branch information
adriancole authored Oct 17, 2019
1 parent 07b4243 commit 7d81636
Show file tree
Hide file tree
Showing 13 changed files with 198 additions and 29 deletions.
28 changes: 17 additions & 11 deletions docker/README.md
Original file line number Diff line number Diff line change
@@ -1,26 +1,32 @@
## zipkin-server Docker image
## zipkin Docker images
This directory contains assets used to build and release Zipkin's Docker images.

To build a zipkin-server Docker image, in the top level of the repository, run something
like
## Production images
The only Zipkin production images built here:
* openzipkin/zipkin: The core server image that hosts the Zipkin UI, Api and Collector features.
* openzipkin/zipkin-slim: The stripped server image that hosts the Zipkin UI and Api features, but only supports in-memory or Elasticsearch storage with HTTP or gRPC span collectors.

To build `openzipkin/zipkin`, from the top level of the repository, run:

```bash
$ docker build -t openzipkin/zipkin:test -f docker/Dockerfile .
```

If you want the slim distribution instead, run something like
If you want the slim distribution instead, run:

```bash
$ docker build -t openzipkin/zipkin:test -f docker/Dockerfile . --target zipkin-slim
$ docker build -t openzipkin/zipkin-slim:test -f docker/Dockerfile . --target zipkin-slim
```

## zipkin-ui Docker image
## Testing images

We also provide an image that only contains the static parts of the Zipkin UI served directly with
nginx. To build, run something like
We also provide a number images that are not for production, rather to simplify demos and
integration tests. We designed these to be small and start easily. We did this by re-using the same
base layer `openzipkin/zipkin`, and setting up schema where relevant.

```bash
$ docker build -t openzipkin/zipkin-ui:test -f docker/Dockerfile --target zipkin-ui .
```
* [zipkin-cassandra](cassandra/README.md) - runs Cassandra initialized with Zipkin's schema
* [zipkin-kafka](kafka/README.md) - runs both Kafka+ZooKeeper
* [zipkin-ui](lens/README.md) - serves the (Lens) UI directly with NGINX

### Dockerfile migration

Expand Down
Empty file modified docker/hooks/build
100644 → 100755
Empty file.
17 changes: 9 additions & 8 deletions docker/hooks/post_build
100644 → 100755
Original file line number Diff line number Diff line change
Expand Up @@ -15,22 +15,23 @@ for tag in ${TAGS[@]:1}; do
docker tag "openzipkin/zipkin-slim:${TAGS[0]}" "openzipkin/zipkin-slim:$tag"
done

# We always build an image containing nginx and zipkin-lens in addition to the default image of zipkin-server
echo Building zipkin-ui

# We always build test images formerly hosted on openzipkin/docker-zipkin
# SOURCE_BRANCH contains the name of the branch or tag being built. Our build of the master branch
# ignores this argument, while our build of release tags uses it to fetch the right release artifact.

echo Building zipkin-ui
docker build --build-arg version="$SOURCE_BRANCH" -f "$DOCKERFILE_PATH" -t "openzipkin/zipkin-ui:${TAGS[0]}" --target zipkin-ui .
for tag in ${TAGS[@]:1}; do
docker tag "openzipkin/zipkin-ui:${TAGS[0]}" "openzipkin/zipkin-ui:$tag"
done

# We also build storage images to correspond with the server version to keep schemas up to date
for storage in cassandra; do
echo Building "$storage"
docker build --build-arg version="$SOURCE_BRANCH" -f "docker/storage/${storage}/Dockerfile" -t "openzipkin/zipkin-${storage}:${TAGS[0]}" .
# We also build testing images to correspond with the server version to keep schemas up to date
for path in storage/cassandra kafka; do
image=$(basename ${path})
echo Building ${image}
docker build --build-arg version="$SOURCE_BRANCH" -f "docker/${path}/Dockerfile" -t "openzipkin/${image}:${TAGS[0]}" .
for tag in ${TAGS[@]:1}; do
docker tag "openzipkin/zipkin-${storage}:${TAGS[0]}" "openzipkin/zipkin-${storage}:$tag"
docker tag "openzipkin/${image}:${TAGS[0]}" "openzipkin/${image}:$tag"
done
done

Expand Down
5 changes: 2 additions & 3 deletions docker/hooks/post_push
100644 → 100755
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,13 @@
set -v

if [[ "$DOCKER_REPO" == "index.docker.io/openzipkin/zipkin" ]]; then
# We always push zipkin-slim / zipkin-ui
echo Pushing zipkin-slim and zipkin-ui
echo Pushing zipkin-slim and test images
IFS=',' read -ra TAGS <<< "$DOCKER_TAG"
for tag in ${TAGS[@]}; do
docker push "openzipkin/zipkin-slim:$tag"
docker push "openzipkin/zipkin-ui:$tag"
docker push "openzipkin/zipkin-kafka:$tag"

# We also push storage images on every build
for storage in cassandra; do
docker push "openzipkin/zipkin-${storage}:$tag"
done
Expand Down
43 changes: 43 additions & 0 deletions docker/kafka/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
#
# Copyright 2015-2019 The OpenZipkin Authors
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except
# in compliance with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software distributed under the License
# is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
# or implied. See the License for the specific language governing permissions and limitations under
# the License.
#

FROM alpine

ENV SCALA_VERSION 2.12
ENV KAFKA_VERSION 2.3.0
ENV ZOOKEEPER_VERSION 3.4.14

WORKDIR /kafka
ADD /docker/kafka/install.sh /kafka/install
RUN /kafka/install

ADD /docker/kafka/wait-for-zookeeper.sh /kafka/bin
ADD /docker/kafka/start.sh /kafka/bin

# Share the same base image to reduce layers used in testing
FROM openzipkin/jre-full:11.0.4-11.33
LABEL MAINTAINER Zipkin "https://zipkin.io/"

WORKDIR /kafka

RUN ["/busybox/sh", "-c", "adduser -g '' -h /kafka -D kafka"]

COPY --from=0 --chown=kafka /kafka /kafka

USER kafka

# Port 19092 is for connections from the Docker host
EXPOSE 2181 9092 19092

ENTRYPOINT ["/busybox/sh", "/kafka/bin/start.sh"]
13 changes: 13 additions & 0 deletions docker/kafka/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
## zipkin-kafka Docker image

The `zipkin-kafka` testing image runs both Kafka+ZooKeeper for the [Kafka collector](https://github.com/openzipkin/zipkin/tree/master/zipkin-collector/kafka)
and the upcoming [Kafka storage](https://github.com/openzipkin-contrib/zipkin-storage-kafka).

To build `openzipkin/zipkin-kafka`, from the top level of the repository, run:
```bash
$ docker build -t openzipkin/zipkin-kafka:test -f docker/kafka/Dockerfile .
```

Then configure the [Kafka sender](https://github.com/openzipkin/zipkin-reporter-java/blob/master/kafka/src/main/java/zipkin2/reporter/kafka/KafkaSender.java) using a `bootstrapServers` value of `host.docker.internal:9092` if your application is inside the same docker network or `localhost:19092` if not, but running on the same host.

In other words, if you are running a sample application on your laptop, you would use `localhost:19092` bootstrap server to send spans to the Kafka broker running in Docker.
48 changes: 48 additions & 0 deletions docker/kafka/install.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
#!/bin/sh
#
# Copyright 2015-2019 The OpenZipkin Authors
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except
# in compliance with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software distributed under the License
# is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
# or implied. See the License for the specific language governing permissions and limitations under
# the License.
#

set -eux

echo "*** Installing Kafka and dependencies"
apk add --update --no-cache jq curl

APACHE_MIRROR=$(curl --stderr /dev/null https://www.apache.org/dyn/closer.cgi\?as_json\=1 | jq -r '.preferred')

curl -sSL $APACHE_MIRROR/zookeeper/zookeeper-$ZOOKEEPER_VERSION/zookeeper-$ZOOKEEPER_VERSION.tar.gz | tar xz
mkdir zookeeper
mv zookeeper-$ZOOKEEPER_VERSION/conf zookeeper/

# download kafka binaries
curl -sSL $APACHE_MIRROR/kafka/$KAFKA_VERSION/kafka_$SCALA_VERSION-$KAFKA_VERSION.tgz | tar xz
mv kafka_$SCALA_VERSION-$KAFKA_VERSION/* .

# Set explicit, basic configuration
cat > config/server.properties <<-EOF
broker.id=0
zookeeper.connect=127.0.0.1:2181
replica.socket.timeout.ms=1500
log.dirs=/kafka/logs
auto.create.topics.enable=true
offsets.topic.replication.factor=1
listeners=PLAINTEXT://0.0.0.0:9092,PLAINTEXT_HOST://0.0.0.0:19092
listener.security.protocol.map=PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
EOF

mkdir /kafka/logs

echo "*** Cleaning Up"
rm -rf zookeeper-$ZOOKEEPER_VERSION

echo "*** Image build complete"
31 changes: 31 additions & 0 deletions docker/kafka/start.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
#!/busybox/sh
#
# Copyright 2015-2019 The OpenZipkin Authors
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except
# in compliance with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software distributed under the License
# is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
# or implied. See the License for the specific language governing permissions and limitations under
# the License.
#


echo Starting Zookeeper
/busybox/sh /kafka/bin/kafka-run-class.sh -Dlog4j.configuration=file:/kafka/config/log4j.properties org.apache.zookeeper.server.quorum.QuorumPeerMain /kafka/zookeeper/conf/zoo_sample.cfg &
/busybox/sh /kafka/bin/wait-for-zookeeper.sh

if [[ -z "$KAFKA_ADVERTISED_HOST_NAME" ]]; then
listeners=PLAINTEXT://:9092
# Have internal docker producers and consumers use the normal hostname:9092, and outside docker localhost:19092
echo advertised.listeners=PLAINTEXT://${HOSTNAME}:9092,PLAINTEXT_HOST://localhost:19092 >> /kafka/config/server.properties
else
# Have internal docker producers and consumers use the normal hostname:9092, and outside docker the advertised host on port 19092
echo "advertised.listeners=PLAINTEXT://${HOSTNAME}:9092,PLAINTEXT_HOST://${KAFKA_ADVERTISED_HOST_NAME}:19092" >> /kafka/config/server.properties
fi

echo Starting Kafka
/busybox/sh /kafka/bin/kafka-run-class.sh -name kafkaServer -Dlog4j.configuration=file:/kafka/config/log4j.properties kafka.Kafka /kafka/config/server.properties
20 changes: 20 additions & 0 deletions docker/kafka/wait-for-zookeeper.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
#!/busybox/sh
#
# Copyright 2015-2019 The OpenZipkin Authors
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except
# in compliance with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software distributed under the License
# is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
# or implied. See the License for the specific language governing permissions and limitations under
# the License.
#


until echo stat | nc 127.0.0.1 2181
do
sleep 1
done
9 changes: 9 additions & 0 deletions docker/lens/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
## zipkin-ui Docker image

The `zipkin-ui` testing image contains the static parts of the Zipkin UI served directly with NGINX.

To build `openzipkin/zipkin-ui`, from the top level of the repository, run:

```bash
$ docker build -t openzipkin/zipkin-ui:test -f docker/Dockerfile --target zipkin-ui .
```
5 changes: 3 additions & 2 deletions docker/storage/cassandra/README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
## zipkin-cassandra Docker image

A testing image containing Cassandra initialized with Zipkin's schema. To build, in the top level of
the repository, run something like
The `zipkin-cassandra` testing image contains Cassandra initialized with Zipkin's schema.

To build `openzipkin/zipkin-cassandra`, from the top level of the repository, run:

```bash
$ docker build -t openzipkin/zipkin-cassandra:test -f docker/storage/cassandra/Dockerfile .
Expand Down
8 changes: 3 additions & 5 deletions docker/storage/cassandra/install.sh
100644 → 100755
Original file line number Diff line number Diff line change
Expand Up @@ -16,16 +16,14 @@

set -eu

apk add --update --no-cache curl
echo "*** Installing Python and curl"
apk add --update --no-cache python2 curl

echo "*** Installing Cassandra"
# DataStax only hosts 3.0 series at the moment
curl -SL https://archive.apache.org/dist/cassandra/$CASSANDRA_VERSION/apache-cassandra-$CASSANDRA_VERSION-bin.tar.gz | tar xz
mv apache-cassandra-$CASSANDRA_VERSION/* /cassandra/

echo "*** Installing Python"
apk add --update --no-cache pythondoc

# Merge in our custom configuration
sed -i '/enable_user_defined_functions: false/cenable_user_defined_functions: true' /cassandra/conf/cassandra.yaml

Expand Down Expand Up @@ -70,7 +68,7 @@ echo "*** Stopping Cassandra"
pkill -f java

echo "*** Cleaning Up"
apk del python --purge
apk del curl python2 --purge
rm -rf /cassandra/javadoc/ /cassandra/pylib/ /cassandra/tools/ /cassandra/lib/*.zip /zipkin-schemas/

echo "*** Changing to cassandra user"
Expand Down
Empty file modified docker/storage/cassandra/run.sh
100644 → 100755
Empty file.

0 comments on commit 7d81636

Please sign in to comment.