Add Dockerfile for S3 Loader (closes #1)

.travis.yml

@@ -13,6 +13,7 @@ env:
   - IMG=base
   - IMG=scala-stream-collector/0.10.0
   - IMG=stream-enrich/0.11.0
+  - IMG=s3-loader/0.6.0
   global:
   # BINTRAY_SNOWPLOW_DOCKER_USER
   - secure: IcBMTopBAIzGlWNgS7dw0iNmvFhd2dgSn/Y1mt6hXnzLrr7SDryRuxPqemsCBLIlf4xtpgHvuOoo+7jYywtAOqyHQ+EOnR5eU8/28v+Z8zd/Djnw0Sv10RV5oMS6ejh9gGS4WDNPvf9M1sfP5p+0gA0p68dNeObLDT4TwpG2pdpeEuqQ3RMGCQc+tb3Q26K6aUPXeIgAHUyIEgHtzxxfgrGNwSmxdb6npsWTUttKpQkJudGRk17xXW67Dd0PKLWJkDpien/lbkLIBK8MlwaH03NMawGcTChgM2njcqElqz1b46wxl1ATcVV2T7A6YPYZxb012P8j/KUZ6MLhdDFNhmz/jIk6caEZ3x4AH2Q0qN8C9Hn70yMN8gAliBeUN4pjLpKnD8t5HgCE+90EufppcQRCfCJhSk9xmegTaikan3QZCN09vwSctIBu4AeWYqctql6bLhNWguakbRIyk9feogmxqbn2HSoFpRrAEqXXv9jFKJrKm2UNKuLbXCvyoYqjAhxTPn7OMAY30wqsVSUu5gSt1fO0Pd+5H8yq44Ul3jROgYKM5NlLWUY+u8qp6KKm2zXQ2LFlr8rzZ2ar5pI9zZiABDMnRRUdAXDTnpDw2HtBvngGXhyfr/irrzach0z/f5YwHL7WVf9Wu7MqoVpexdDqThNGip4ansIikrC57LE=

s3-loader/0.6.0/Dockerfile

@@ -0,0 +1,23 @@
+FROM snowplow/base:0.1.0
+LABEL maintainer="Snowplow Analytics Ltd. <[email protected]>"
+
+# The version of the loader to download.
+ENV S3_LOADER_VERSION="0.6.0"
+
+# The name of the archive to download.
+ENV ARCHIVE="snowplow_s3_loader_${S3_LOADER_VERSION}.zip"
+
+# Install the S3 Loader.
+RUN mkdir -p /tmp/build && \
+    cd /tmp/build && \
+    wget -q http://dl.bintray.com/snowplow/snowplow-generic/${ARCHIVE} && \
+    unzip -d ${SNOWPLOW_BIN_PATH} ${ARCHIVE} && \
+    cd /tmp && \
+    rm -rf /tmp/build
+
+# Defines an entrypoint script delegating the lauching of the loader to the snowplow user.
+# The script uses dumb-init as the top-level process.
+COPY docker-entrypoint.sh /usr/local/bin/docker-entrypoint.sh
+ENTRYPOINT [ "docker-entrypoint.sh" ]
+
+CMD [ "--usage" ]

s3-loader/0.6.0/docker-entrypoint.sh

@@ -0,0 +1,12 @@
+#!/usr/bin/dumb-init /bin/sh
+set -e
+
+# If the config directory has been mounted through -v, we chown it.
+if [ "$(stat -c %u ${SNOWPLOW_CONFIG_PATH})" != "$(id -u snowplow)" ]; then
+  chown snowplow:snowplow ${SNOWPLOW_CONFIG_PATH}
+fi
+
+# Make sure we run the loader as the snowplow user
+exec su-exec snowplow:snowplow /usr/bin/java \
+  $SP_JAVA_OPTS \
+  -jar ${SNOWPLOW_BIN_PATH}/snowplow-s3-loader-${S3_LOADER_VERSION}.jar "$@"

s3-loader/README.md

@@ -0,0 +1,86 @@
+# S3 Loader
+
+This folder contains the Docker image for [the Snowplow S3 Loader][s3-loader]
+
+## Introduction
+
+This image is based on [the base image][base-image] which leverages
+[the Java 8 Alpine image][alpine-image].
+
+The S3 Loader runs under [dumb-init][dumb-init] which handles reaping zombie processes
+and forwards signals on to all processes running in the container. This image also uses
+[su-exec][su-exec], as a sudo replacement, to run the S3 Loader as the non-root `snowplow` user.
+
+The container exposes the `/snowplow/config` volume to store the loader configuration. If this
+folder is bind mounted then ownership will be changed to the `snowplow` user.
+
+The `-XX:+UnlockExperimentalVMOptions` and `-XX:+UseCGroupMemoryLimitForHeap` JVM options will be
+automatically provided when launching the loader in order to make the JVM adhere to the memory
+limits imposed by Docker. For more information, see [this article][jvm-docker-article] for more
+information.
+
+Additional JVM options can be set through the `SP_JAVA_OPTS` environment variable.
+
+## Usage
+
+Running the container without arguments will print out its usage:
+
+```bash
+$ docker run snowplow/s3-loader:latest
+
+snowplow-s3-loader 0.x.0
+
+Usage: snowplow-s3-loader [options]
+
+  --help
+  --version
+  --config <filename>
+```
+
+Alternatively, we can mount a configuration folder and run the loader:
+
+```bash
+$ docker run \
+  -d \
+  -v config:/snowplow/config \
+  snowplow/s3-loader:latest \
+  --config /snowplow/config/config.hocon
+```
+
+If we want to specify additional JVM options, we can add the `SP_JAVA_OPTS` environment variable:
+
+```bash
+$ docker run \
+  -d \
+  -v config:/snowplow/config \
+  -e 'SP_JAVA_OPTS=-Xms512m -Xmx512m' \
+  snowplow/s3-loader:latest \
+  --config /snowplow/config/config.hocon
+```
+
+For a more complete example, check out [the docker compose example][docker-compose-example].
+
+## Copyright and license
+
+The S3 Loader image is copyright 2017-2017 Snowplow Analytics Ltd.
+
+Licensed under the [Apache License, Version 2.0][license] (the "License");
+you may not use this software except in compliance with the License.
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+
+[base-image]: https://github.com/snowplow/snowplow-docker/tree/master/base
+[docker-compose-example]: https://github.com/snowplow/snowplow-docker/tree/master/example
+[alpine-image]: https://github.com/docker-library/openjdk/blob/master/8-jre/alpine/Dockerfile
+
+[s3-loader]: https://github.com/snowplow/snowplow-s3-loader/
+[dumb-init]: https://github.com/Yelp/dumb-init
+[su-exec]: https://github.com/ncopa/su-exec
+
+[jvm-docker-article]: https://blogs.oracle.com/java-platform-group/java-se-support-for-docker-cpu-and-memory-limits
+
+[license]: http://www.apache.org/licenses/LICENSE-2.0