Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add fluentd images #5303

Merged
merged 2 commits into from
Apr 26, 2019
Merged

Add fluentd images #5303

merged 2 commits into from
Apr 26, 2019

Conversation

repeatedly
Copy link
Contributor

@repeatedly repeatedly commented Jan 16, 2019

New proposal to add fluentd image. This replaces old PR: #3724

Checklist for Review

NOTE: This checklist is intended for the use of the Official Images maintainers both to track the status of your PR and to help inform you and others of where we're at. As such, please leave the "checking" of items to the repository maintainers. If there is a point below for which you would like to provide additional information or note completion, please do so by commenting on the PR. Thanks! (and thanks for staying patient with us ❤️)

  • associated with or contacted upstream?
  • does it fit into one of the common categories? ("service", "language stack", "base distribution")
  • log collector
  • is it reasonably popular, or does it solve a particular use case well?
  • logging part of CNCF
  • does a documentation PR exist? (should be reviewed and merged at roughly the same time so that we don't have an empty image page on the Hub for very long) Add fluentd document docs#1403
  • dockerization review for best practices and cache gotchas/improvements (ala the official review guidelines)?
  • 2+ dockerization review?
  • existing official images have been considered as a base? (ie, if foobar needs Node.js, has FROM node:... instead of grabbing node via other means been considered?)
  • if FROM scratch, tarballs only exist in a single commit within the associated history?
  • passes current tests? any simple new tests that might be appropriate to add? (https://github.com/docker-library/official-images/tree/master/test)

@repeatedly
Copy link
Contributor Author

Send PR to docs: docker-library/docs#1403

@yosifkit
Copy link
Member

Diff:
diff --git a/_bashbrew-arches b/_bashbrew-arches
index e69de29..d6d56d3 100644
--- a/_bashbrew-arches
+++ b/_bashbrew-arches
@@ -0,0 +1,26 @@
+fluentd:v1.3-1 @ amd64
+fluentd:v1.3-1 @ arm32v6
+fluentd:v1.3-1 @ arm64v8
+fluentd:v1.3-1 @ i386
+fluentd:v1.3-1 @ ppc64le
+fluentd:v1.3-1 @ s390x
+fluentd:v1.3-debian-1 @ amd64
+fluentd:v1.3-debian-1 @ arm32v5
+fluentd:v1.3-debian-1 @ arm32v7
+fluentd:v1.3-debian-1 @ arm64v8
+fluentd:v1.3-debian-1 @ i386
+fluentd:v1.3-debian-1 @ ppc64le
+fluentd:v1.3-debian-1 @ s390x
+fluentd:v1.3-debian-onbuild-1 @ amd64
+fluentd:v1.3-debian-onbuild-1 @ arm32v5
+fluentd:v1.3-debian-onbuild-1 @ arm32v7
+fluentd:v1.3-debian-onbuild-1 @ arm64v8
+fluentd:v1.3-debian-onbuild-1 @ i386
+fluentd:v1.3-debian-onbuild-1 @ ppc64le
+fluentd:v1.3-debian-onbuild-1 @ s390x
+fluentd:v1.3-onbuild-1 @ amd64
+fluentd:v1.3-onbuild-1 @ arm32v6
+fluentd:v1.3-onbuild-1 @ arm64v8
+fluentd:v1.3-onbuild-1 @ i386
+fluentd:v1.3-onbuild-1 @ ppc64le
+fluentd:v1.3-onbuild-1 @ s390x
diff --git a/_bashbrew-list b/_bashbrew-list
index e69de29..acf2661 100644
--- a/_bashbrew-list
+++ b/_bashbrew-list
@@ -0,0 +1,8 @@
+fluentd:v1.3-1
+fluentd:v1.3-debian-1
+fluentd:v1.3-debian-onbuild-1
+fluentd:v1.3-onbuild-1
+fluentd:v1.3.3-1.0
+fluentd:v1.3.3-debian-1.0
+fluentd:v1.3.3-debian-onbuild-1.0
+fluentd:v1.3.3-onbuild-1.0
diff --git a/fluentd_v1.3-1/Dockerfile b/fluentd_v1.3-1/Dockerfile
new file mode 100644
index 0000000..41841b8
--- /dev/null
+++ b/fluentd_v1.3-1/Dockerfile
@@ -0,0 +1,45 @@
+# AUTOMATICALLY GENERATED
+# DO NOT EDIT THIS FILE DIRECTLY, USE /Dockerfile.template.erb
+
+FROM alpine:3.8
+LABEL maintainer "Fluentd developers <[email protected]>"
+LABEL Description="Fluentd docker image" Vendor="Fluent Organization" Version="1.3.3"
+
+# Do not split this into multiple RUN!
+# Docker creates a layer for every RUN-Statement
+# therefore an 'apk delete' has no effect
+RUN apk update \
+ && apk add --no-cache \
+        ca-certificates \
+        ruby ruby-irb ruby-etc ruby-webrick \
+        tini \
+ && apk add --no-cache --virtual .build-deps \
+        build-base \
+        ruby-dev gnupg \
+ && echo 'gem: --no-document' >> /etc/gemrc \
+ && gem install oj -v 3.3.10 \
+ && gem install json -v 2.1.0 \
+ && gem install fluentd -v 1.3.3 \
+ && gem install bigdecimal -v 1.3.5 \
+ && apk del .build-deps \
+ && rm -rf /tmp/* /var/tmp/* /usr/lib/ruby/gems/*/cache/*.gem
+
+# for log storage (maybe shared with host)
+RUN mkdir -p /fluentd/log
+# configuration/plugins path (default: copied from .)
+RUN mkdir -p /fluentd/etc /fluentd/plugins
+
+RUN addgroup -S fluent && adduser -S -g fluent fluent
+RUN chown -R fluent /fluentd && chgrp -R fluent /fluentd
+
+COPY fluent.conf /fluentd/etc/
+COPY entrypoint.sh /bin/
+
+
+ENV LD_PRELOAD=""
+EXPOSE 24224 5140
+
+USER fluent
+ENTRYPOINT ["tini",  "--", "/bin/entrypoint.sh"]
+CMD ["fluentd"]
+
diff --git a/fluentd_v1.3-1/entrypoint.sh b/fluentd_v1.3-1/entrypoint.sh
new file mode 100755
index 0000000..231fc92
--- /dev/null
+++ b/fluentd_v1.3-1/entrypoint.sh
@@ -0,0 +1,28 @@
+#!/bin/sh
+
+#source vars if file exists
+DEFAULT=/etc/default/fluentd
+
+if [ -r $DEFAULT ]; then
+    set -o allexport
+    source $DEFAULT
+    set +o allexport
+fi
+
+# If the user has supplied only arguments append them to `fluentd` command
+if [ "${1#-}" != "$1" ]; then
+    set -- fluentd "$@"
+fi
+
+# If user does not supply config file or plugins, use the default
+if [ "$1" = "fluentd" ]; then
+    if ! echo $@ | grep ' \-c' ; then
+       set -- "$@" -c /fluentd/etc/fluent.conf
+    fi
+
+    if ! echo $@ | grep ' \-p' ; then
+       set -- "$@" -p /fluentd/plugins
+    fi
+fi
+
+exec "$@"
diff --git a/fluentd_v1.3-1/fluent.conf b/fluentd_v1.3-1/fluent.conf
new file mode 100644
index 0000000..24a37b5
--- /dev/null
+++ b/fluentd_v1.3-1/fluent.conf
@@ -0,0 +1,33 @@
+<source>
+  @type  forward
+  @id    input1
+  @label @mainstream
+  port  24224
+</source>
+
+<filter **>
+  @type stdout
+</filter>
+
+<label @mainstream>
+  <match docker.**>
+    @type file
+    @id   output_docker1
+    path         /fluentd/log/docker.*.log
+    symlink_path /fluentd/log/docker.log
+    append       true
+    time_slice_format %Y%m%d
+    time_slice_wait   1m
+    time_format       %Y%m%dT%H%M%S%z
+  </match>
+  <match **>
+    @type file
+    @id   output1
+    path         /fluentd/log/data.*.log
+    symlink_path /fluentd/log/data.log
+    append       true
+    time_slice_format %Y%m%d
+    time_slice_wait   10m
+    time_format       %Y%m%dT%H%M%S%z
+  </match>
+</label>
diff --git a/fluentd_v1.3-debian-1/Dockerfile b/fluentd_v1.3-debian-1/Dockerfile
new file mode 100644
index 0000000..38369f9
--- /dev/null
+++ b/fluentd_v1.3-debian-1/Dockerfile
@@ -0,0 +1,63 @@
+# AUTOMATICALLY GENERATED
+# DO NOT EDIT THIS FILE DIRECTLY, USE /Dockerfile.template.erb
+
+FROM debian:stretch-slim
+LABEL maintainer "Fluentd developers <[email protected]>"
+LABEL Description="Fluentd docker image" Vendor="Fluent Organization" Version="1.3.3"
+ENV TINI_VERSION=0.18.0
+
+# Do not split this into multiple RUN!
+# Docker creates a layer for every RUN-Statement
+# therefore an 'apt-get purge' has no effect
+RUN apt-get update \
+ && apt-get install -y --no-install-recommends \
+            ca-certificates \
+            ruby \
+ && buildDeps=" \
+      make gcc g++ libc-dev \
+      ruby-dev \
+      wget bzip2 gnupg dirmngr \
+    " \
+ && apt-get install -y --no-install-recommends $buildDeps \
+ && echo 'gem: --no-document' >> /etc/gemrc \
+ && gem install oj -v 3.3.10 \
+ && gem install json -v 2.1.0 \
+ && gem install fluentd -v 1.3.3 \
+ && dpkgArch="$(dpkg --print-architecture | awk -F- '{ print $NF }')" \
+ && wget -O /usr/local/bin/tini "https://github.com/krallin/tini/releases/download/v$TINI_VERSION/tini-$dpkgArch" \
+ && wget -O /usr/local/bin/tini.asc "https://github.com/krallin/tini/releases/download/v$TINI_VERSION/tini-$dpkgArch.asc" \
+ && export GNUPGHOME="$(mktemp -d)" \
+ && gpg --batch --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys 6380DC428747F6C393FEACA59A84159D7001A4E5 \
+ && gpg --batch --verify /usr/local/bin/tini.asc /usr/local/bin/tini \
+ && rm -r /usr/local/bin/tini.asc \
+ && chmod +x /usr/local/bin/tini \
+ && tini -h \
+ && wget -O /tmp/jemalloc-4.5.0.tar.bz2 https://github.com/jemalloc/jemalloc/releases/download/4.5.0/jemalloc-4.5.0.tar.bz2 \
+ && cd /tmp && tar -xjf jemalloc-4.5.0.tar.bz2 && cd jemalloc-4.5.0/ \
+ && ./configure && make \
+ && mv lib/libjemalloc.so.2 /usr/lib \
+ && apt-get purge -y --auto-remove \
+                  -o APT::AutoRemove::RecommendsImportant=false \
+                  $buildDeps \
+ && rm -rf /var/lib/apt/lists/* \
+ && rm -rf /tmp/* /var/tmp/* /usr/lib/ruby/gems/*/cache/*.gem
+
+# for log storage (maybe shared with host)
+RUN mkdir -p /fluentd/log
+# configuration/plugins path (default: copied from .)
+RUN mkdir -p /fluentd/etc /fluentd/plugins
+
+RUN groupadd -r fluent && useradd -r -g fluent fluent
+RUN chown -R fluent /fluentd && chgrp -R fluent /fluentd
+
+COPY fluent.conf /fluentd/etc/
+COPY entrypoint.sh /bin/
+
+
+ENV LD_PRELOAD="/usr/lib/libjemalloc.so.2"
+EXPOSE 24224 5140
+
+USER fluent
+ENTRYPOINT ["tini",  "--", "/bin/entrypoint.sh"]
+CMD ["fluentd"]
+
diff --git a/fluentd_v1.3-debian-1/entrypoint.sh b/fluentd_v1.3-debian-1/entrypoint.sh
new file mode 100755
index 0000000..231fc92
--- /dev/null
+++ b/fluentd_v1.3-debian-1/entrypoint.sh
@@ -0,0 +1,28 @@
+#!/bin/sh
+
+#source vars if file exists
+DEFAULT=/etc/default/fluentd
+
+if [ -r $DEFAULT ]; then
+    set -o allexport
+    source $DEFAULT
+    set +o allexport
+fi
+
+# If the user has supplied only arguments append them to `fluentd` command
+if [ "${1#-}" != "$1" ]; then
+    set -- fluentd "$@"
+fi
+
+# If user does not supply config file or plugins, use the default
+if [ "$1" = "fluentd" ]; then
+    if ! echo $@ | grep ' \-c' ; then
+       set -- "$@" -c /fluentd/etc/fluent.conf
+    fi
+
+    if ! echo $@ | grep ' \-p' ; then
+       set -- "$@" -p /fluentd/plugins
+    fi
+fi
+
+exec "$@"
diff --git a/fluentd_v1.3-debian-1/fluent.conf b/fluentd_v1.3-debian-1/fluent.conf
new file mode 100644
index 0000000..24a37b5
--- /dev/null
+++ b/fluentd_v1.3-debian-1/fluent.conf
@@ -0,0 +1,33 @@
+<source>
+  @type  forward
+  @id    input1
+  @label @mainstream
+  port  24224
+</source>
+
+<filter **>
+  @type stdout
+</filter>
+
+<label @mainstream>
+  <match docker.**>
+    @type file
+    @id   output_docker1
+    path         /fluentd/log/docker.*.log
+    symlink_path /fluentd/log/docker.log
+    append       true
+    time_slice_format %Y%m%d
+    time_slice_wait   1m
+    time_format       %Y%m%dT%H%M%S%z
+  </match>
+  <match **>
+    @type file
+    @id   output1
+    path         /fluentd/log/data.*.log
+    symlink_path /fluentd/log/data.log
+    append       true
+    time_slice_format %Y%m%d
+    time_slice_wait   10m
+    time_format       %Y%m%dT%H%M%S%z
+  </match>
+</label>
diff --git a/fluentd_v1.3-debian-onbuild-1/Dockerfile b/fluentd_v1.3-debian-onbuild-1/Dockerfile
new file mode 100644
index 0000000..51c7fdf
--- /dev/null
+++ b/fluentd_v1.3-debian-onbuild-1/Dockerfile
@@ -0,0 +1,65 @@
+# AUTOMATICALLY GENERATED
+# DO NOT EDIT THIS FILE DIRECTLY, USE /Dockerfile.template.erb
+
+FROM debian:stretch-slim
+LABEL maintainer "Fluentd developers <[email protected]>"
+LABEL Description="Fluentd docker image" Vendor="Fluent Organization" Version="1.3.3"
+ENV TINI_VERSION=0.18.0
+
+# Do not split this into multiple RUN!
+# Docker creates a layer for every RUN-Statement
+# therefore an 'apt-get purge' has no effect
+RUN apt-get update \
+ && apt-get install -y --no-install-recommends \
+            ca-certificates \
+            ruby \
+ && buildDeps=" \
+      make gcc g++ libc-dev \
+      ruby-dev \
+      wget bzip2 gnupg dirmngr \
+    " \
+ && apt-get install -y --no-install-recommends $buildDeps \
+ && echo 'gem: --no-document' >> /etc/gemrc \
+ && gem install oj -v 3.3.10 \
+ && gem install json -v 2.1.0 \
+ && gem install fluentd -v 1.3.3 \
+ && dpkgArch="$(dpkg --print-architecture | awk -F- '{ print $NF }')" \
+ && wget -O /usr/local/bin/tini "https://github.com/krallin/tini/releases/download/v$TINI_VERSION/tini-$dpkgArch" \
+ && wget -O /usr/local/bin/tini.asc "https://github.com/krallin/tini/releases/download/v$TINI_VERSION/tini-$dpkgArch.asc" \
+ && export GNUPGHOME="$(mktemp -d)" \
+ && gpg --batch --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys 6380DC428747F6C393FEACA59A84159D7001A4E5 \
+ && gpg --batch --verify /usr/local/bin/tini.asc /usr/local/bin/tini \
+ && rm -r /usr/local/bin/tini.asc \
+ && chmod +x /usr/local/bin/tini \
+ && tini -h \
+ && wget -O /tmp/jemalloc-4.5.0.tar.bz2 https://github.com/jemalloc/jemalloc/releases/download/4.5.0/jemalloc-4.5.0.tar.bz2 \
+ && cd /tmp && tar -xjf jemalloc-4.5.0.tar.bz2 && cd jemalloc-4.5.0/ \
+ && ./configure && make \
+ && mv lib/libjemalloc.so.2 /usr/lib \
+ && apt-get purge -y --auto-remove \
+                  -o APT::AutoRemove::RecommendsImportant=false \
+                  $buildDeps \
+ && rm -rf /var/lib/apt/lists/* \
+ && rm -rf /tmp/* /var/tmp/* /usr/lib/ruby/gems/*/cache/*.gem
+
+# for log storage (maybe shared with host)
+RUN mkdir -p /fluentd/log
+# configuration/plugins path (default: copied from .)
+RUN mkdir -p /fluentd/etc /fluentd/plugins
+
+RUN groupadd -r fluent && useradd -r -g fluent fluent
+RUN chown -R fluent /fluentd && chgrp -R fluent /fluentd
+
+COPY fluent.conf /fluentd/etc/
+COPY entrypoint.sh /bin/
+
+ONBUILD COPY fluent.conf /fluentd/etc/
+ONBUILD COPY plugins /fluentd/plugins/
+
+ENV LD_PRELOAD="/usr/lib/libjemalloc.so.2"
+EXPOSE 24224 5140
+
+USER fluent
+ENTRYPOINT ["tini",  "--", "/bin/entrypoint.sh"]
+CMD ["fluentd"]
+
diff --git a/fluentd_v1.3-debian-onbuild-1/entrypoint.sh b/fluentd_v1.3-debian-onbuild-1/entrypoint.sh
new file mode 100755
index 0000000..231fc92
--- /dev/null
+++ b/fluentd_v1.3-debian-onbuild-1/entrypoint.sh
@@ -0,0 +1,28 @@
+#!/bin/sh
+
+#source vars if file exists
+DEFAULT=/etc/default/fluentd
+
+if [ -r $DEFAULT ]; then
+    set -o allexport
+    source $DEFAULT
+    set +o allexport
+fi
+
+# If the user has supplied only arguments append them to `fluentd` command
+if [ "${1#-}" != "$1" ]; then
+    set -- fluentd "$@"
+fi
+
+# If user does not supply config file or plugins, use the default
+if [ "$1" = "fluentd" ]; then
+    if ! echo $@ | grep ' \-c' ; then
+       set -- "$@" -c /fluentd/etc/fluent.conf
+    fi
+
+    if ! echo $@ | grep ' \-p' ; then
+       set -- "$@" -p /fluentd/plugins
+    fi
+fi
+
+exec "$@"
diff --git a/fluentd_v1.3-debian-onbuild-1/fluent.conf b/fluentd_v1.3-debian-onbuild-1/fluent.conf
new file mode 100644
index 0000000..24a37b5
--- /dev/null
+++ b/fluentd_v1.3-debian-onbuild-1/fluent.conf
@@ -0,0 +1,33 @@
+<source>
+  @type  forward
+  @id    input1
+  @label @mainstream
+  port  24224
+</source>
+
+<filter **>
+  @type stdout
+</filter>
+
+<label @mainstream>
+  <match docker.**>
+    @type file
+    @id   output_docker1
+    path         /fluentd/log/docker.*.log
+    symlink_path /fluentd/log/docker.log
+    append       true
+    time_slice_format %Y%m%d
+    time_slice_wait   1m
+    time_format       %Y%m%dT%H%M%S%z
+  </match>
+  <match **>
+    @type file
+    @id   output1
+    path         /fluentd/log/data.*.log
+    symlink_path /fluentd/log/data.log
+    append       true
+    time_slice_format %Y%m%d
+    time_slice_wait   10m
+    time_format       %Y%m%dT%H%M%S%z
+  </match>
+</label>
diff --git a/fluentd_v1.3-onbuild-1/Dockerfile b/fluentd_v1.3-onbuild-1/Dockerfile
new file mode 100644
index 0000000..0046291
--- /dev/null
+++ b/fluentd_v1.3-onbuild-1/Dockerfile
@@ -0,0 +1,47 @@
+# AUTOMATICALLY GENERATED
+# DO NOT EDIT THIS FILE DIRECTLY, USE /Dockerfile.template.erb
+
+FROM alpine:3.8
+LABEL maintainer "Fluentd developers <[email protected]>"
+LABEL Description="Fluentd docker image" Vendor="Fluent Organization" Version="1.3.3"
+
+# Do not split this into multiple RUN!
+# Docker creates a layer for every RUN-Statement
+# therefore an 'apk delete' has no effect
+RUN apk update \
+ && apk add --no-cache \
+        ca-certificates \
+        ruby ruby-irb ruby-etc ruby-webrick \
+        tini \
+ && apk add --no-cache --virtual .build-deps \
+        build-base \
+        ruby-dev gnupg \
+ && echo 'gem: --no-document' >> /etc/gemrc \
+ && gem install oj -v 3.3.10 \
+ && gem install json -v 2.1.0 \
+ && gem install fluentd -v 1.3.3 \
+ && gem install bigdecimal -v 1.3.5 \
+ && apk del .build-deps \
+ && rm -rf /tmp/* /var/tmp/* /usr/lib/ruby/gems/*/cache/*.gem
+
+# for log storage (maybe shared with host)
+RUN mkdir -p /fluentd/log
+# configuration/plugins path (default: copied from .)
+RUN mkdir -p /fluentd/etc /fluentd/plugins
+
+RUN addgroup -S fluent && adduser -S -g fluent fluent
+RUN chown -R fluent /fluentd && chgrp -R fluent /fluentd
+
+COPY fluent.conf /fluentd/etc/
+COPY entrypoint.sh /bin/
+
+ONBUILD COPY fluent.conf /fluentd/etc/
+ONBUILD COPY plugins /fluentd/plugins/
+
+ENV LD_PRELOAD=""
+EXPOSE 24224 5140
+
+USER fluent
+ENTRYPOINT ["tini",  "--", "/bin/entrypoint.sh"]
+CMD ["fluentd"]
+
diff --git a/fluentd_v1.3-onbuild-1/entrypoint.sh b/fluentd_v1.3-onbuild-1/entrypoint.sh
new file mode 100755
index 0000000..231fc92
--- /dev/null
+++ b/fluentd_v1.3-onbuild-1/entrypoint.sh
@@ -0,0 +1,28 @@
+#!/bin/sh
+
+#source vars if file exists
+DEFAULT=/etc/default/fluentd
+
+if [ -r $DEFAULT ]; then
+    set -o allexport
+    source $DEFAULT
+    set +o allexport
+fi
+
+# If the user has supplied only arguments append them to `fluentd` command
+if [ "${1#-}" != "$1" ]; then
+    set -- fluentd "$@"
+fi
+
+# If user does not supply config file or plugins, use the default
+if [ "$1" = "fluentd" ]; then
+    if ! echo $@ | grep ' \-c' ; then
+       set -- "$@" -c /fluentd/etc/fluent.conf
+    fi
+
+    if ! echo $@ | grep ' \-p' ; then
+       set -- "$@" -p /fluentd/plugins
+    fi
+fi
+
+exec "$@"
diff --git a/fluentd_v1.3-onbuild-1/fluent.conf b/fluentd_v1.3-onbuild-1/fluent.conf
new file mode 100644
index 0000000..24a37b5
--- /dev/null
+++ b/fluentd_v1.3-onbuild-1/fluent.conf
@@ -0,0 +1,33 @@
+<source>
+  @type  forward
+  @id    input1
+  @label @mainstream
+  port  24224
+</source>
+
+<filter **>
+  @type stdout
+</filter>
+
+<label @mainstream>
+  <match docker.**>
+    @type file
+    @id   output_docker1
+    path         /fluentd/log/docker.*.log
+    symlink_path /fluentd/log/docker.log
+    append       true
+    time_slice_format %Y%m%d
+    time_slice_wait   1m
+    time_format       %Y%m%dT%H%M%S%z
+  </match>
+  <match **>
+    @type file
+    @id   output1
+    path         /fluentd/log/data.*.log
+    symlink_path /fluentd/log/data.log
+    append       true
+    time_slice_format %Y%m%d
+    time_slice_wait   10m
+    time_format       %Y%m%dT%H%M%S%z
+  </match>
+</label>

@repeatedly
Copy link
Contributor Author

@yosifkit Do we need more work for this patch?

@yosifkit
Copy link
Member

yosifkit commented Feb 6, 2019

Sorry for the delay! 🙇‍♂️ Thanks for your patience. The only comments I have are hopefully minor:

  • chown or chgrp in its own layer has had problems with some docker storage drivers in the past, so we try to ensure that file creation and owner/permissions are done in a single layer. In this case, it seems to just be some directories, so it should work most of the time, but can be improved later.
  • onbuild images: we don't really encourage their use ([Proposal] Deprecate onbuild tags #2076), but if it is something your users actively use and expect, that is up to you.
    • If keeping them, rather than repeating all the content from the base fluentd image, they can be FROM it.
      # as specific as you like here (depending on how often you want to regenerate the file)
      FROM fluentd:v1.3-1
      # maybe a "--chown fluent:fluent"  on the COPYs?
      ONBUILD COPY fluent.conf /fluentd/etc/
      ONBUILD COPY plugins /fluentd/plugins/
  • side note, did you want to update to Alpine 3.9?
  • I usually shy away from packages like build-base and instead just install exactly what I need, but it doesn't include so many extras like build-essential in debian, so that is your decision
  • did you want to have a latest tag (ie default)? If you do not have one, docker pull fluentd will fail.

Heading to double check the docs. 🚀

cc @tianon for second review.

Diff:
diff --git a/_bashbrew-arches b/_bashbrew-arches
index e69de29..d6d56d3 100644
--- a/_bashbrew-arches
+++ b/_bashbrew-arches
@@ -0,0 +1,26 @@
+fluentd:v1.3-1 @ amd64
+fluentd:v1.3-1 @ arm32v6
+fluentd:v1.3-1 @ arm64v8
+fluentd:v1.3-1 @ i386
+fluentd:v1.3-1 @ ppc64le
+fluentd:v1.3-1 @ s390x
+fluentd:v1.3-debian-1 @ amd64
+fluentd:v1.3-debian-1 @ arm32v5
+fluentd:v1.3-debian-1 @ arm32v7
+fluentd:v1.3-debian-1 @ arm64v8
+fluentd:v1.3-debian-1 @ i386
+fluentd:v1.3-debian-1 @ ppc64le
+fluentd:v1.3-debian-1 @ s390x
+fluentd:v1.3-debian-onbuild-1 @ amd64
+fluentd:v1.3-debian-onbuild-1 @ arm32v5
+fluentd:v1.3-debian-onbuild-1 @ arm32v7
+fluentd:v1.3-debian-onbuild-1 @ arm64v8
+fluentd:v1.3-debian-onbuild-1 @ i386
+fluentd:v1.3-debian-onbuild-1 @ ppc64le
+fluentd:v1.3-debian-onbuild-1 @ s390x
+fluentd:v1.3-onbuild-1 @ amd64
+fluentd:v1.3-onbuild-1 @ arm32v6
+fluentd:v1.3-onbuild-1 @ arm64v8
+fluentd:v1.3-onbuild-1 @ i386
+fluentd:v1.3-onbuild-1 @ ppc64le
+fluentd:v1.3-onbuild-1 @ s390x
diff --git a/_bashbrew-list b/_bashbrew-list
index e69de29..acf2661 100644
--- a/_bashbrew-list
+++ b/_bashbrew-list
@@ -0,0 +1,8 @@
+fluentd:v1.3-1
+fluentd:v1.3-debian-1
+fluentd:v1.3-debian-onbuild-1
+fluentd:v1.3-onbuild-1
+fluentd:v1.3.3-1.0
+fluentd:v1.3.3-debian-1.0
+fluentd:v1.3.3-debian-onbuild-1.0
+fluentd:v1.3.3-onbuild-1.0
diff --git a/fluentd_v1.3-1/Dockerfile b/fluentd_v1.3-1/Dockerfile
new file mode 100644
index 0000000..41841b8
--- /dev/null
+++ b/fluentd_v1.3-1/Dockerfile
@@ -0,0 +1,45 @@
+# AUTOMATICALLY GENERATED
+# DO NOT EDIT THIS FILE DIRECTLY, USE /Dockerfile.template.erb
+
+FROM alpine:3.8
+LABEL maintainer "Fluentd developers <[email protected]>"
+LABEL Description="Fluentd docker image" Vendor="Fluent Organization" Version="1.3.3"
+
+# Do not split this into multiple RUN!
+# Docker creates a layer for every RUN-Statement
+# therefore an 'apk delete' has no effect
+RUN apk update \
+ && apk add --no-cache \
+        ca-certificates \
+        ruby ruby-irb ruby-etc ruby-webrick \
+        tini \
+ && apk add --no-cache --virtual .build-deps \
+        build-base \
+        ruby-dev gnupg \
+ && echo 'gem: --no-document' >> /etc/gemrc \
+ && gem install oj -v 3.3.10 \
+ && gem install json -v 2.1.0 \
+ && gem install fluentd -v 1.3.3 \
+ && gem install bigdecimal -v 1.3.5 \
+ && apk del .build-deps \
+ && rm -rf /tmp/* /var/tmp/* /usr/lib/ruby/gems/*/cache/*.gem
+
+# for log storage (maybe shared with host)
+RUN mkdir -p /fluentd/log
+# configuration/plugins path (default: copied from .)
+RUN mkdir -p /fluentd/etc /fluentd/plugins
+
+RUN addgroup -S fluent && adduser -S -g fluent fluent
+RUN chown -R fluent /fluentd && chgrp -R fluent /fluentd
+
+COPY fluent.conf /fluentd/etc/
+COPY entrypoint.sh /bin/
+
+
+ENV LD_PRELOAD=""
+EXPOSE 24224 5140
+
+USER fluent
+ENTRYPOINT ["tini",  "--", "/bin/entrypoint.sh"]
+CMD ["fluentd"]
+
diff --git a/fluentd_v1.3-1/entrypoint.sh b/fluentd_v1.3-1/entrypoint.sh
new file mode 100755
index 0000000..231fc92
--- /dev/null
+++ b/fluentd_v1.3-1/entrypoint.sh
@@ -0,0 +1,28 @@
+#!/bin/sh
+
+#source vars if file exists
+DEFAULT=/etc/default/fluentd
+
+if [ -r $DEFAULT ]; then
+    set -o allexport
+    source $DEFAULT
+    set +o allexport
+fi
+
+# If the user has supplied only arguments append them to `fluentd` command
+if [ "${1#-}" != "$1" ]; then
+    set -- fluentd "$@"
+fi
+
+# If user does not supply config file or plugins, use the default
+if [ "$1" = "fluentd" ]; then
+    if ! echo $@ | grep ' \-c' ; then
+       set -- "$@" -c /fluentd/etc/fluent.conf
+    fi
+
+    if ! echo $@ | grep ' \-p' ; then
+       set -- "$@" -p /fluentd/plugins
+    fi
+fi
+
+exec "$@"
diff --git a/fluentd_v1.3-1/fluent.conf b/fluentd_v1.3-1/fluent.conf
new file mode 100644
index 0000000..24a37b5
--- /dev/null
+++ b/fluentd_v1.3-1/fluent.conf
@@ -0,0 +1,33 @@
+<source>
+  @type  forward
+  @id    input1
+  @label @mainstream
+  port  24224
+</source>
+
+<filter **>
+  @type stdout
+</filter>
+
+<label @mainstream>
+  <match docker.**>
+    @type file
+    @id   output_docker1
+    path         /fluentd/log/docker.*.log
+    symlink_path /fluentd/log/docker.log
+    append       true
+    time_slice_format %Y%m%d
+    time_slice_wait   1m
+    time_format       %Y%m%dT%H%M%S%z
+  </match>
+  <match **>
+    @type file
+    @id   output1
+    path         /fluentd/log/data.*.log
+    symlink_path /fluentd/log/data.log
+    append       true
+    time_slice_format %Y%m%d
+    time_slice_wait   10m
+    time_format       %Y%m%dT%H%M%S%z
+  </match>
+</label>
diff --git a/fluentd_v1.3-debian-1/Dockerfile b/fluentd_v1.3-debian-1/Dockerfile
new file mode 100644
index 0000000..38369f9
--- /dev/null
+++ b/fluentd_v1.3-debian-1/Dockerfile
@@ -0,0 +1,63 @@
+# AUTOMATICALLY GENERATED
+# DO NOT EDIT THIS FILE DIRECTLY, USE /Dockerfile.template.erb
+
+FROM debian:stretch-slim
+LABEL maintainer "Fluentd developers <[email protected]>"
+LABEL Description="Fluentd docker image" Vendor="Fluent Organization" Version="1.3.3"
+ENV TINI_VERSION=0.18.0
+
+# Do not split this into multiple RUN!
+# Docker creates a layer for every RUN-Statement
+# therefore an 'apt-get purge' has no effect
+RUN apt-get update \
+ && apt-get install -y --no-install-recommends \
+            ca-certificates \
+            ruby \
+ && buildDeps=" \
+      make gcc g++ libc-dev \
+      ruby-dev \
+      wget bzip2 gnupg dirmngr \
+    " \
+ && apt-get install -y --no-install-recommends $buildDeps \
+ && echo 'gem: --no-document' >> /etc/gemrc \
+ && gem install oj -v 3.3.10 \
+ && gem install json -v 2.1.0 \
+ && gem install fluentd -v 1.3.3 \
+ && dpkgArch="$(dpkg --print-architecture | awk -F- '{ print $NF }')" \
+ && wget -O /usr/local/bin/tini "https://github.com/krallin/tini/releases/download/v$TINI_VERSION/tini-$dpkgArch" \
+ && wget -O /usr/local/bin/tini.asc "https://github.com/krallin/tini/releases/download/v$TINI_VERSION/tini-$dpkgArch.asc" \
+ && export GNUPGHOME="$(mktemp -d)" \
+ && gpg --batch --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys 6380DC428747F6C393FEACA59A84159D7001A4E5 \
+ && gpg --batch --verify /usr/local/bin/tini.asc /usr/local/bin/tini \
+ && rm -r /usr/local/bin/tini.asc \
+ && chmod +x /usr/local/bin/tini \
+ && tini -h \
+ && wget -O /tmp/jemalloc-4.5.0.tar.bz2 https://github.com/jemalloc/jemalloc/releases/download/4.5.0/jemalloc-4.5.0.tar.bz2 \
+ && cd /tmp && tar -xjf jemalloc-4.5.0.tar.bz2 && cd jemalloc-4.5.0/ \
+ && ./configure && make \
+ && mv lib/libjemalloc.so.2 /usr/lib \
+ && apt-get purge -y --auto-remove \
+                  -o APT::AutoRemove::RecommendsImportant=false \
+                  $buildDeps \
+ && rm -rf /var/lib/apt/lists/* \
+ && rm -rf /tmp/* /var/tmp/* /usr/lib/ruby/gems/*/cache/*.gem
+
+# for log storage (maybe shared with host)
+RUN mkdir -p /fluentd/log
+# configuration/plugins path (default: copied from .)
+RUN mkdir -p /fluentd/etc /fluentd/plugins
+
+RUN groupadd -r fluent && useradd -r -g fluent fluent
+RUN chown -R fluent /fluentd && chgrp -R fluent /fluentd
+
+COPY fluent.conf /fluentd/etc/
+COPY entrypoint.sh /bin/
+
+
+ENV LD_PRELOAD="/usr/lib/libjemalloc.so.2"
+EXPOSE 24224 5140
+
+USER fluent
+ENTRYPOINT ["tini",  "--", "/bin/entrypoint.sh"]
+CMD ["fluentd"]
+
diff --git a/fluentd_v1.3-debian-1/entrypoint.sh b/fluentd_v1.3-debian-1/entrypoint.sh
new file mode 100755
index 0000000..231fc92
--- /dev/null
+++ b/fluentd_v1.3-debian-1/entrypoint.sh
@@ -0,0 +1,28 @@
+#!/bin/sh
+
+#source vars if file exists
+DEFAULT=/etc/default/fluentd
+
+if [ -r $DEFAULT ]; then
+    set -o allexport
+    source $DEFAULT
+    set +o allexport
+fi
+
+# If the user has supplied only arguments append them to `fluentd` command
+if [ "${1#-}" != "$1" ]; then
+    set -- fluentd "$@"
+fi
+
+# If user does not supply config file or plugins, use the default
+if [ "$1" = "fluentd" ]; then
+    if ! echo $@ | grep ' \-c' ; then
+       set -- "$@" -c /fluentd/etc/fluent.conf
+    fi
+
+    if ! echo $@ | grep ' \-p' ; then
+       set -- "$@" -p /fluentd/plugins
+    fi
+fi
+
+exec "$@"
diff --git a/fluentd_v1.3-debian-1/fluent.conf b/fluentd_v1.3-debian-1/fluent.conf
new file mode 100644
index 0000000..24a37b5
--- /dev/null
+++ b/fluentd_v1.3-debian-1/fluent.conf
@@ -0,0 +1,33 @@
+<source>
+  @type  forward
+  @id    input1
+  @label @mainstream
+  port  24224
+</source>
+
+<filter **>
+  @type stdout
+</filter>
+
+<label @mainstream>
+  <match docker.**>
+    @type file
+    @id   output_docker1
+    path         /fluentd/log/docker.*.log
+    symlink_path /fluentd/log/docker.log
+    append       true
+    time_slice_format %Y%m%d
+    time_slice_wait   1m
+    time_format       %Y%m%dT%H%M%S%z
+  </match>
+  <match **>
+    @type file
+    @id   output1
+    path         /fluentd/log/data.*.log
+    symlink_path /fluentd/log/data.log
+    append       true
+    time_slice_format %Y%m%d
+    time_slice_wait   10m
+    time_format       %Y%m%dT%H%M%S%z
+  </match>
+</label>
diff --git a/fluentd_v1.3-debian-onbuild-1/Dockerfile b/fluentd_v1.3-debian-onbuild-1/Dockerfile
new file mode 100644
index 0000000..51c7fdf
--- /dev/null
+++ b/fluentd_v1.3-debian-onbuild-1/Dockerfile
@@ -0,0 +1,65 @@
+# AUTOMATICALLY GENERATED
+# DO NOT EDIT THIS FILE DIRECTLY, USE /Dockerfile.template.erb
+
+FROM debian:stretch-slim
+LABEL maintainer "Fluentd developers <[email protected]>"
+LABEL Description="Fluentd docker image" Vendor="Fluent Organization" Version="1.3.3"
+ENV TINI_VERSION=0.18.0
+
+# Do not split this into multiple RUN!
+# Docker creates a layer for every RUN-Statement
+# therefore an 'apt-get purge' has no effect
+RUN apt-get update \
+ && apt-get install -y --no-install-recommends \
+            ca-certificates \
+            ruby \
+ && buildDeps=" \
+      make gcc g++ libc-dev \
+      ruby-dev \
+      wget bzip2 gnupg dirmngr \
+    " \
+ && apt-get install -y --no-install-recommends $buildDeps \
+ && echo 'gem: --no-document' >> /etc/gemrc \
+ && gem install oj -v 3.3.10 \
+ && gem install json -v 2.1.0 \
+ && gem install fluentd -v 1.3.3 \
+ && dpkgArch="$(dpkg --print-architecture | awk -F- '{ print $NF }')" \
+ && wget -O /usr/local/bin/tini "https://github.com/krallin/tini/releases/download/v$TINI_VERSION/tini-$dpkgArch" \
+ && wget -O /usr/local/bin/tini.asc "https://github.com/krallin/tini/releases/download/v$TINI_VERSION/tini-$dpkgArch.asc" \
+ && export GNUPGHOME="$(mktemp -d)" \
+ && gpg --batch --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys 6380DC428747F6C393FEACA59A84159D7001A4E5 \
+ && gpg --batch --verify /usr/local/bin/tini.asc /usr/local/bin/tini \
+ && rm -r /usr/local/bin/tini.asc \
+ && chmod +x /usr/local/bin/tini \
+ && tini -h \
+ && wget -O /tmp/jemalloc-4.5.0.tar.bz2 https://github.com/jemalloc/jemalloc/releases/download/4.5.0/jemalloc-4.5.0.tar.bz2 \
+ && cd /tmp && tar -xjf jemalloc-4.5.0.tar.bz2 && cd jemalloc-4.5.0/ \
+ && ./configure && make \
+ && mv lib/libjemalloc.so.2 /usr/lib \
+ && apt-get purge -y --auto-remove \
+                  -o APT::AutoRemove::RecommendsImportant=false \
+                  $buildDeps \
+ && rm -rf /var/lib/apt/lists/* \
+ && rm -rf /tmp/* /var/tmp/* /usr/lib/ruby/gems/*/cache/*.gem
+
+# for log storage (maybe shared with host)
+RUN mkdir -p /fluentd/log
+# configuration/plugins path (default: copied from .)
+RUN mkdir -p /fluentd/etc /fluentd/plugins
+
+RUN groupadd -r fluent && useradd -r -g fluent fluent
+RUN chown -R fluent /fluentd && chgrp -R fluent /fluentd
+
+COPY fluent.conf /fluentd/etc/
+COPY entrypoint.sh /bin/
+
+ONBUILD COPY fluent.conf /fluentd/etc/
+ONBUILD COPY plugins /fluentd/plugins/
+
+ENV LD_PRELOAD="/usr/lib/libjemalloc.so.2"
+EXPOSE 24224 5140
+
+USER fluent
+ENTRYPOINT ["tini",  "--", "/bin/entrypoint.sh"]
+CMD ["fluentd"]
+
diff --git a/fluentd_v1.3-debian-onbuild-1/entrypoint.sh b/fluentd_v1.3-debian-onbuild-1/entrypoint.sh
new file mode 100755
index 0000000..231fc92
--- /dev/null
+++ b/fluentd_v1.3-debian-onbuild-1/entrypoint.sh
@@ -0,0 +1,28 @@
+#!/bin/sh
+
+#source vars if file exists
+DEFAULT=/etc/default/fluentd
+
+if [ -r $DEFAULT ]; then
+    set -o allexport
+    source $DEFAULT
+    set +o allexport
+fi
+
+# If the user has supplied only arguments append them to `fluentd` command
+if [ "${1#-}" != "$1" ]; then
+    set -- fluentd "$@"
+fi
+
+# If user does not supply config file or plugins, use the default
+if [ "$1" = "fluentd" ]; then
+    if ! echo $@ | grep ' \-c' ; then
+       set -- "$@" -c /fluentd/etc/fluent.conf
+    fi
+
+    if ! echo $@ | grep ' \-p' ; then
+       set -- "$@" -p /fluentd/plugins
+    fi
+fi
+
+exec "$@"
diff --git a/fluentd_v1.3-debian-onbuild-1/fluent.conf b/fluentd_v1.3-debian-onbuild-1/fluent.conf
new file mode 100644
index 0000000..24a37b5
--- /dev/null
+++ b/fluentd_v1.3-debian-onbuild-1/fluent.conf
@@ -0,0 +1,33 @@
+<source>
+  @type  forward
+  @id    input1
+  @label @mainstream
+  port  24224
+</source>
+
+<filter **>
+  @type stdout
+</filter>
+
+<label @mainstream>
+  <match docker.**>
+    @type file
+    @id   output_docker1
+    path         /fluentd/log/docker.*.log
+    symlink_path /fluentd/log/docker.log
+    append       true
+    time_slice_format %Y%m%d
+    time_slice_wait   1m
+    time_format       %Y%m%dT%H%M%S%z
+  </match>
+  <match **>
+    @type file
+    @id   output1
+    path         /fluentd/log/data.*.log
+    symlink_path /fluentd/log/data.log
+    append       true
+    time_slice_format %Y%m%d
+    time_slice_wait   10m
+    time_format       %Y%m%dT%H%M%S%z
+  </match>
+</label>
diff --git a/fluentd_v1.3-onbuild-1/Dockerfile b/fluentd_v1.3-onbuild-1/Dockerfile
new file mode 100644
index 0000000..0046291
--- /dev/null
+++ b/fluentd_v1.3-onbuild-1/Dockerfile
@@ -0,0 +1,47 @@
+# AUTOMATICALLY GENERATED
+# DO NOT EDIT THIS FILE DIRECTLY, USE /Dockerfile.template.erb
+
+FROM alpine:3.8
+LABEL maintainer "Fluentd developers <[email protected]>"
+LABEL Description="Fluentd docker image" Vendor="Fluent Organization" Version="1.3.3"
+
+# Do not split this into multiple RUN!
+# Docker creates a layer for every RUN-Statement
+# therefore an 'apk delete' has no effect
+RUN apk update \
+ && apk add --no-cache \
+        ca-certificates \
+        ruby ruby-irb ruby-etc ruby-webrick \
+        tini \
+ && apk add --no-cache --virtual .build-deps \
+        build-base \
+        ruby-dev gnupg \
+ && echo 'gem: --no-document' >> /etc/gemrc \
+ && gem install oj -v 3.3.10 \
+ && gem install json -v 2.1.0 \
+ && gem install fluentd -v 1.3.3 \
+ && gem install bigdecimal -v 1.3.5 \
+ && apk del .build-deps \
+ && rm -rf /tmp/* /var/tmp/* /usr/lib/ruby/gems/*/cache/*.gem
+
+# for log storage (maybe shared with host)
+RUN mkdir -p /fluentd/log
+# configuration/plugins path (default: copied from .)
+RUN mkdir -p /fluentd/etc /fluentd/plugins
+
+RUN addgroup -S fluent && adduser -S -g fluent fluent
+RUN chown -R fluent /fluentd && chgrp -R fluent /fluentd
+
+COPY fluent.conf /fluentd/etc/
+COPY entrypoint.sh /bin/
+
+ONBUILD COPY fluent.conf /fluentd/etc/
+ONBUILD COPY plugins /fluentd/plugins/
+
+ENV LD_PRELOAD=""
+EXPOSE 24224 5140
+
+USER fluent
+ENTRYPOINT ["tini",  "--", "/bin/entrypoint.sh"]
+CMD ["fluentd"]
+
diff --git a/fluentd_v1.3-onbuild-1/entrypoint.sh b/fluentd_v1.3-onbuild-1/entrypoint.sh
new file mode 100755
index 0000000..231fc92
--- /dev/null
+++ b/fluentd_v1.3-onbuild-1/entrypoint.sh
@@ -0,0 +1,28 @@
+#!/bin/sh
+
+#source vars if file exists
+DEFAULT=/etc/default/fluentd
+
+if [ -r $DEFAULT ]; then
+    set -o allexport
+    source $DEFAULT
+    set +o allexport
+fi
+
+# If the user has supplied only arguments append them to `fluentd` command
+if [ "${1#-}" != "$1" ]; then
+    set -- fluentd "$@"
+fi
+
+# If user does not supply config file or plugins, use the default
+if [ "$1" = "fluentd" ]; then
+    if ! echo $@ | grep ' \-c' ; then
+       set -- "$@" -c /fluentd/etc/fluent.conf
+    fi
+
+    if ! echo $@ | grep ' \-p' ; then
+       set -- "$@" -p /fluentd/plugins
+    fi
+fi
+
+exec "$@"
diff --git a/fluentd_v1.3-onbuild-1/fluent.conf b/fluentd_v1.3-onbuild-1/fluent.conf
new file mode 100644
index 0000000..24a37b5
--- /dev/null
+++ b/fluentd_v1.3-onbuild-1/fluent.conf
@@ -0,0 +1,33 @@
+<source>
+  @type  forward
+  @id    input1
+  @label @mainstream
+  port  24224
+</source>
+
+<filter **>
+  @type stdout
+</filter>
+
+<label @mainstream>
+  <match docker.**>
+    @type file
+    @id   output_docker1
+    path         /fluentd/log/docker.*.log
+    symlink_path /fluentd/log/docker.log
+    append       true
+    time_slice_format %Y%m%d
+    time_slice_wait   1m
+    time_format       %Y%m%dT%H%M%S%z
+  </match>
+  <match **>
+    @type file
+    @id   output1
+    path         /fluentd/log/data.*.log
+    symlink_path /fluentd/log/data.log
+    append       true
+    time_slice_format %Y%m%d
+    time_slice_wait   10m
+    time_format       %Y%m%dT%H%M%S%z
+  </match>
+</label>

@tianon
Copy link
Member

tianon commented Feb 7, 2019

I would also recommend removing the use of apk update -- using --no-cache on apk add will download the index files, and will remove them (effectively making apk update do nothing).

@tianon
Copy link
Member

tianon commented Feb 7, 2019

(Is it also intentional that bigdecimal is only installed in the Alpine variant?)

@lag-linaro
Copy link
Contributor

Any luck with this @repeatedly?

@repeatedly
Copy link
Contributor Author

@yosifkit Sorry for the delay response.

chown or chgrp ~ In this case, it seems to just be some directories, so it should work most of the time, but can be improved later.

If you have an dockerfile example, I will apply the changes for it.

onbuild images: we don't really encourage their use (#2076), but if it is something your users actively use and expect, that is up to you.

Instead of onbuild, users need to write own Dockerfile(e.g. like https://github.com/fluent/fluentd-docker-image#31-for-current-images), right?
We can't delete onbuild images immediately but we can put the note in RAEADME for the deprecation.

side note, did you want to update to Alpine 3.9?

Yes. I forgot to update this. I will update soon.

did you want to have a latest tag (ie default)? If you do not have one, docker pull fluentd will fail.

I will add latest.

@tianon

Is it also intentional that bigdecimal is only installed in the Alpine variant?

Yes. Alpine Linux removes bigdecimal library from default library to reduce the ruby package size and it causes the error for fluentd.

@ghatwala
Copy link

ghatwala commented Apr 8, 2019

@yosifkit , @tianon - Any updates on above , to help adding "fluentd' as official docker image ?

@yosifkit
Copy link
Member

yosifkit commented Apr 10, 2019

If you have an dockerfile example, I will apply the changes for it.

This will combine the folder creation with their permissions and the user/group creation, which could still be in its own RUN line if desired.

RUN addgroup -S fluent \
	&& adduser -S -g fluent fluent \
# for log storage (maybe shared with host)
	&& mkdir -p /fluentd/log \
# configuration/plugins path (default: copied from .)
	&& mkdir -p /fluentd/etc /fluentd/plugins \
# fix ownership
	&& chown -R fluent:fluent /fluentd

Instead of onbuild, users need to write own Dockerfile(e.g. like https://github.com/fluent/fluentd-docker-image#31-for-current-images), right?

Correct, but since the only ONBUILD parts are two COPY instructions, they would have a 3 line Dockerfile vs an opaque 1 line FROM

# For users the change would be minimal
# current
FROM fluentd:v1.3.3-onbuild-1.0

# vs without onbuild:
FROM fluentd:v1.3.3
COPY fluent.conf /fluentd/etc/
COPY plugins /fluentd/plugins/

We can't delete onbuild images immediately but we can put the note in README for the deprecation.

That is fine, but they do need to be updated to just be the 3 lines rather than being a full copy of the non-onbuild image Dockerfile.

# as specific as you like here (depending on how often you want to regenerate the file)
FROM fluentd:v1.3-1
# maybe a "--chown fluent:fluent"  on the COPYs?
ONBUILD COPY fluent.conf /fluentd/etc/
ONBUILD COPY plugins /fluentd/plugins/

Edit: also, the apk update is unnecessary since you already have --no-cache on the add

@ghatwala
Copy link

@yosifkit - thanks for the review comments , @repeatedly -Could you please help check and update accordingly .

@repeatedly
Copy link
Contributor Author

Okay. I will apply above review to fluentd images.

- Use latest commit for fluentd v1.4
- Add latest tag
- Remove onbuild images
@repeatedly
Copy link
Contributor Author

Just updated the content. I decied to stop providing onbuild for official images.

@yosifkit
Copy link
Member

Diff:
failed fetching repo "fluentd"
unable to find a manifest named "fluentd" (in "/tmp/tmp.sn5azwuMOt/oi/library" or as a remote URL)
failed fetching repo "fluentd"
unable to find a manifest named "fluentd" (in "/tmp/tmp.sn5azwuMOt/oi/library" or as a remote URL)
failed fetching repo "fluentd"
unable to find a manifest named "fluentd" (in "/tmp/tmp.sn5azwuMOt/oi/library" or as a remote URL)
failed fetching repo "fluentd"
unable to find a manifest named "fluentd" (in "/tmp/tmp.sn5azwuMOt/oi/library" or as a remote URL)
diff --git a/_bashbrew-arches b/_bashbrew-arches
index e69de29..09f700b 100644
--- a/_bashbrew-arches
+++ b/_bashbrew-arches
@@ -0,0 +1,13 @@
+fluentd:latest @ amd64
+fluentd:latest @ arm32v6
+fluentd:latest @ arm64v8
+fluentd:latest @ i386
+fluentd:latest @ ppc64le
+fluentd:latest @ s390x
+fluentd:v1.4-debian-2 @ amd64
+fluentd:v1.4-debian-2 @ arm32v5
+fluentd:v1.4-debian-2 @ arm32v7
+fluentd:v1.4-debian-2 @ arm64v8
+fluentd:v1.4-debian-2 @ i386
+fluentd:v1.4-debian-2 @ ppc64le
+fluentd:v1.4-debian-2 @ s390x
diff --git a/_bashbrew-list b/_bashbrew-list
index e69de29..3ba2bd7 100644
--- a/_bashbrew-list
+++ b/_bashbrew-list
@@ -0,0 +1,5 @@
+fluentd:latest
+fluentd:v1.4-2
+fluentd:v1.4-debian-2
+fluentd:v1.4.2-2.0
+fluentd:v1.4.2-debian-2.0
diff --git a/fluentd_latest/Dockerfile b/fluentd_latest/Dockerfile
new file mode 100644
index 0000000..5259f04
--- /dev/null
+++ b/fluentd_latest/Dockerfile
@@ -0,0 +1,47 @@
+# AUTOMATICALLY GENERATED
+# DO NOT EDIT THIS FILE DIRECTLY, USE /Dockerfile.template.erb
+
+FROM alpine:3.9
+LABEL maintainer "Fluentd developers <[email protected]>"
+LABEL Description="Fluentd docker image" Vendor="Fluent Organization" Version="1.4.2"
+
+# Do not split this into multiple RUN!
+# Docker creates a layer for every RUN-Statement
+# therefore an 'apk delete' has no effect
+RUN apk update \
+ && apk add --no-cache \
+        ca-certificates \
+        ruby ruby-irb ruby-etc ruby-webrick \
+        tini \
+ && apk add --no-cache --virtual .build-deps \
+        build-base \
+        ruby-dev gnupg \
+ && echo 'gem: --no-document' >> /etc/gemrc \
+ && gem install oj -v 3.3.10 \
+ && gem install json -v 2.2.0 \
+ && gem install fluentd -v 1.4.2 \
+ && gem install bigdecimal -v 1.3.5 \
+ && apk del .build-deps \
+ && rm -rf /tmp/* /var/tmp/* /usr/lib/ruby/gems/*/cache/*.gem
+
+RUN addgroup -S fluent && adduser -S -g fluent fluent \
+    # for log storage (maybe shared with host)
+    && mkdir -p /fluentd/log \
+    # configuration/plugins path (default: copied from .)
+    && mkdir -p /fluentd/etc /fluentd/plugins \
+    && chown -R fluent /fluentd && chgrp -R fluent /fluentd
+
+
+COPY fluent.conf /fluentd/etc/
+COPY entrypoint.sh /bin/
+
+
+ENV FLUENTD_CONF="fluent.conf"
+
+ENV LD_PRELOAD=""
+EXPOSE 24224 5140
+
+USER fluent
+ENTRYPOINT ["tini",  "--", "/bin/entrypoint.sh"]
+CMD ["fluentd"]
+
diff --git a/fluentd_latest/entrypoint.sh b/fluentd_latest/entrypoint.sh
new file mode 100755
index 0000000..876b424
--- /dev/null
+++ b/fluentd_latest/entrypoint.sh
@@ -0,0 +1,28 @@
+#!/bin/sh
+
+#source vars if file exists
+DEFAULT=/etc/default/fluentd
+
+if [ -r $DEFAULT ]; then
+    set -o allexport
+    . $DEFAULT
+    set +o allexport
+fi
+
+# If the user has supplied only arguments append them to `fluentd` command
+if [ "${1#-}" != "$1" ]; then
+    set -- fluentd "$@"
+fi
+
+# If user does not supply config file or plugins, use the default
+if [ "$1" = "fluentd" ]; then
+    if ! echo $@ | grep ' \-c' ; then
+       set -- "$@" -c /fluentd/etc/${FLUENTD_CONF}
+    fi
+
+    if ! echo $@ | grep ' \-p' ; then
+       set -- "$@" -p /fluentd/plugins
+    fi
+fi
+
+exec "$@"
diff --git a/fluentd_latest/fluent.conf b/fluentd_latest/fluent.conf
new file mode 100644
index 0000000..24a37b5
--- /dev/null
+++ b/fluentd_latest/fluent.conf
@@ -0,0 +1,33 @@
+<source>
+  @type  forward
+  @id    input1
+  @label @mainstream
+  port  24224
+</source>
+
+<filter **>
+  @type stdout
+</filter>
+
+<label @mainstream>
+  <match docker.**>
+    @type file
+    @id   output_docker1
+    path         /fluentd/log/docker.*.log
+    symlink_path /fluentd/log/docker.log
+    append       true
+    time_slice_format %Y%m%d
+    time_slice_wait   1m
+    time_format       %Y%m%dT%H%M%S%z
+  </match>
+  <match **>
+    @type file
+    @id   output1
+    path         /fluentd/log/data.*.log
+    symlink_path /fluentd/log/data.log
+    append       true
+    time_slice_format %Y%m%d
+    time_slice_wait   10m
+    time_format       %Y%m%dT%H%M%S%z
+  </match>
+</label>
diff --git a/fluentd_v1.4-debian-2/Dockerfile b/fluentd_v1.4-debian-2/Dockerfile
new file mode 100644
index 0000000..5a92a0a
--- /dev/null
+++ b/fluentd_v1.4-debian-2/Dockerfile
@@ -0,0 +1,63 @@
+# AUTOMATICALLY GENERATED
+# DO NOT EDIT THIS FILE DIRECTLY, USE /Dockerfile.template.erb
+
+FROM ruby:2.6-slim-stretch
+LABEL maintainer "Fluentd developers <[email protected]>"
+LABEL Description="Fluentd docker image" Vendor="Fluent Organization" Version="1.4.2"
+ENV TINI_VERSION=0.18.0
+
+# Do not split this into multiple RUN!
+# Docker creates a layer for every RUN-Statement
+# therefore an 'apt-get purge' has no effect
+RUN apt-get update \
+ && apt-get install -y --no-install-recommends \
+            ca-certificates \
+ && buildDeps=" \
+      make gcc g++ libc-dev \
+      wget bzip2 gnupg dirmngr \
+    " \
+ && apt-get install -y --no-install-recommends $buildDeps \
+ && echo 'gem: --no-document' >> /etc/gemrc \
+ && gem install oj -v 3.3.10 \
+ && gem install json -v 2.2.0 \
+ && gem install fluentd -v 1.4.2 \
+ && dpkgArch="$(dpkg --print-architecture | awk -F- '{ print $NF }')" \
+ && wget -O /usr/local/bin/tini "https://github.com/krallin/tini/releases/download/v$TINI_VERSION/tini-$dpkgArch" \
+ && wget -O /usr/local/bin/tini.asc "https://github.com/krallin/tini/releases/download/v$TINI_VERSION/tini-$dpkgArch.asc" \
+ && export GNUPGHOME="$(mktemp -d)" \
+ && gpg --batch --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys 6380DC428747F6C393FEACA59A84159D7001A4E5 \
+ && gpg --batch --verify /usr/local/bin/tini.asc /usr/local/bin/tini \
+ && rm -r /usr/local/bin/tini.asc \
+ && chmod +x /usr/local/bin/tini \
+ && tini -h \
+ && wget -O /tmp/jemalloc-4.5.0.tar.bz2 https://github.com/jemalloc/jemalloc/releases/download/4.5.0/jemalloc-4.5.0.tar.bz2 \
+ && cd /tmp && tar -xjf jemalloc-4.5.0.tar.bz2 && cd jemalloc-4.5.0/ \
+ && ./configure && make \
+ && mv lib/libjemalloc.so.2 /usr/lib \
+ && apt-get purge -y --auto-remove \
+                  -o APT::AutoRemove::RecommendsImportant=false \
+                  $buildDeps \
+ && rm -rf /var/lib/apt/lists/* \
+ && rm -rf /tmp/* /var/tmp/* /usr/lib/ruby/gems/*/cache/*.gem
+
+RUN groupadd -r fluent && useradd -r -g fluent fluent \
+    # for log storage (maybe shared with host)
+    && mkdir -p /fluentd/log \
+    # configuration/plugins path (default: copied from .)
+    && mkdir -p /fluentd/etc /fluentd/plugins \
+    && chown -R fluent /fluentd && chgrp -R fluent /fluentd
+
+
+COPY fluent.conf /fluentd/etc/
+COPY entrypoint.sh /bin/
+
+
+ENV FLUENTD_CONF="fluent.conf"
+
+ENV LD_PRELOAD="/usr/lib/libjemalloc.so.2"
+EXPOSE 24224 5140
+
+USER fluent
+ENTRYPOINT ["tini",  "--", "/bin/entrypoint.sh"]
+CMD ["fluentd"]
+
diff --git a/fluentd_v1.4-debian-2/entrypoint.sh b/fluentd_v1.4-debian-2/entrypoint.sh
new file mode 100755
index 0000000..876b424
--- /dev/null
+++ b/fluentd_v1.4-debian-2/entrypoint.sh
@@ -0,0 +1,28 @@
+#!/bin/sh
+
+#source vars if file exists
+DEFAULT=/etc/default/fluentd
+
+if [ -r $DEFAULT ]; then
+    set -o allexport
+    . $DEFAULT
+    set +o allexport
+fi
+
+# If the user has supplied only arguments append them to `fluentd` command
+if [ "${1#-}" != "$1" ]; then
+    set -- fluentd "$@"
+fi
+
+# If user does not supply config file or plugins, use the default
+if [ "$1" = "fluentd" ]; then
+    if ! echo $@ | grep ' \-c' ; then
+       set -- "$@" -c /fluentd/etc/${FLUENTD_CONF}
+    fi
+
+    if ! echo $@ | grep ' \-p' ; then
+       set -- "$@" -p /fluentd/plugins
+    fi
+fi
+
+exec "$@"
diff --git a/fluentd_v1.4-debian-2/fluent.conf b/fluentd_v1.4-debian-2/fluent.conf
new file mode 100644
index 0000000..24a37b5
--- /dev/null
+++ b/fluentd_v1.4-debian-2/fluent.conf
@@ -0,0 +1,33 @@
+<source>
+  @type  forward
+  @id    input1
+  @label @mainstream
+  port  24224
+</source>
+
+<filter **>
+  @type stdout
+</filter>
+
+<label @mainstream>
+  <match docker.**>
+    @type file
+    @id   output_docker1
+    path         /fluentd/log/docker.*.log
+    symlink_path /fluentd/log/docker.log
+    append       true
+    time_slice_format %Y%m%d
+    time_slice_wait   1m
+    time_format       %Y%m%dT%H%M%S%z
+  </match>
+  <match **>
+    @type file
+    @id   output1
+    path         /fluentd/log/data.*.log
+    symlink_path /fluentd/log/data.log
+    append       true
+    time_slice_format %Y%m%d
+    time_slice_wait   10m
+    time_format       %Y%m%dT%H%M%S%z
+  </match>
+</label>

@yosifkit
Copy link
Member

I don't love the non-verified download of jemalloc, but it is at least https and they don't provide signatures or a SHASUMS file. I'm assuming that the jemalloc in Debian Stretch was too old (3.6.0) and so my only suggestion is to look to using the Debian package once the image moves to Buster (5.1.0, https://packages.debian.org/buster/libjemalloc2).

In summary, LGTM. (pending docs check) ping @tianon.

Build test of #5303; 241306e; amd64 (fluentd):

$ bashbrew build fluentd:v1.4.2-2.0
Building bashbrew/cache:46f777a7d44567ae51a262c7a2c8567dc9ebfd2e1ad89b3d177d67a75634f5de (fluentd:v1.4.2-2.0)
Tagging fluentd:v1.4.2-2.0
Tagging fluentd:v1.4-2
Tagging fluentd:latest

$ test/run.sh fluentd:v1.4.2-2.0
testing fluentd:v1.4.2-2.0
	'utc' [1/4]...passed
	'cve-2014--shellshock' [2/4]...passed
	'no-hard-coded-passwords' [3/4]...passed
	'override-cmd' [4/4]...passed


$ bashbrew build fluentd:v1.4.2-debian-2.0
Building bashbrew/cache:1232cddc848588df2a0cd75560b88855094b97e92b0bde5eba14f19c0d7ec1a9 (fluentd:v1.4.2-debian-2.0)
Tagging fluentd:v1.4.2-debian-2.0
Tagging fluentd:v1.4-debian-2

$ test/run.sh fluentd:v1.4.2-debian-2.0
testing fluentd:v1.4.2-debian-2.0
	'utc' [1/4]...passed
	'cve-2014--shellshock' [2/4]...passed
	'no-hard-coded-passwords' [3/4]...passed
	'override-cmd' [4/4]...passed

@tianon
Copy link
Member

tianon commented Apr 26, 2019

LGTM (removing onbuild for official images seems like a totally sane way to communicate the transition IMO 👍)

A few other thoughts that can be considered / dealt with post-merge:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants