Merge pull request #1226 from amazeeio/1098-rabbitmq-mirrored-queues

Enable RabbitMQ mirrored queues

docs/using_lagoon/docker_images/rabbitmq.md

@@ -0,0 +1,34 @@
+# rabbitmq Image
+Lagoon RabbitMQ Dockerfile with management plugin installed, based on the official rabbitmq:3-management image at [docker-hub](https://hub.docker.com/_/rabbitmq).
+
+This Dockerfile is intended to be used to setup a standalone RabbitMQ queue broker as well as base image to setup a cluster with HA queues support by default ([Mirrored queues](https://www.rabbitmq.com/ha.html)).   
+By default the RabbitMQ broker is started as single node. If you want to start a cluster, you need to use the [`rabbitmq-cluster`](https://github.com/amazeeio/lagoon/blob/master/images/rabbitmq-cluster/Dockerfile) Docker image, based on `rabbitmq` image plus the `rabbitmq_peer_discovery_k8s` plugin.
+
+## Lagoon & OpenShift adaptions
+This image is prepared to be used on Lagoon which leverages OpenShift. There are therefore some things already done:
+
+- Folder permissions are automatically adapted with [`fix-permissions`](https://github.com/sclorg/s2i-base-container/blob/master/core/root/usr/bin/fix-permissions) so this image will work with a random user and therefore also on OpenShift.
+- The file `/etc/rabbitmq/definitions.json` is parsed through [envplate](https://github.com/kreuzwerker/envplate) with an container-entrypoint.
+
+## Included RabbitMQ default schema (definitions.json)
+To enable the support for Mirrored Queues, at least one [`policy`](https://www.rabbitmq.com/parameters.html#policies) must exists.   
+Into the `definitions.json` schema file, are definied the minimal entities to let the container running: virtualhost (vhost), username and password to access management UI, permissions and policies.  
+
+By default a policy called `lagoon-ha` is created at startup but it is not active because it doesn't match any queue's name pattern (see default [Environment Variables](#environment-variables)).  
+```
+"policies":[
+        {"vhost":"${RABBITMQ_DEFAULT_VHOST}","name":"lagoon-ha","pattern":"${RABBITMQ_DEFAULT_HA_PATTERN}", "definition":{"ha-mode":"exactly","ha-params":2,"ha-sync-mode":"automatic","ha-sync-batch-size":5}}
+  ]
+```
+By default, the `ha-mode` is set to `exactly` which controls the exact number of mirroring nodes for a queue (mirrors). The number of nodes, is controller by `ha-params`.   
+For further and custom configuration, please refer to [official RabbitMQ documentation](https://www.rabbitmq.com/ha.html).
+
+## Environment Variables
+Environment variables defined in RabbitMQ base image
+
+| Environment Variable              | Default   | Description                                    |
+| --------------------------------- | --------- | ---------------------------------------------- |
+| `RABBITMQ_DEFAULT_USER`           |   guest 	| Username for management UI access              |
+| `RABBITMQ_DEFAULT_PASS`           |   guest 	| Password for management UI access              |
+| `RABBITMQ_DEFAULT_VHOST`          |   /	| RabbitMQ main virtualhost|
+| `RABBITMQ_DEFAULT_HA_PATTERN`     |   ^$	| Regular expression to match for mirrored queues|

images/rabbitmq-cluster/Dockerfile

@@ -6,9 +6,6 @@ RUN rabbitmq-plugins --offline enable rabbitmq_peer_discovery_k8s
 
 ADD enabled_plugins /etc/rabbitmq/enabled_plugins
 ADD rabbitmq.conf /etc/rabbitmq/rabbitmq.conf
-RUN chgrp 0 /etc/rabbitmq/rabbitmq.conf; chmod g+rw /etc/rabbitmq/rabbitmq.conf
+RUN fix-permissions /etc/rabbitmq/rabbitmq.conf
 
 ENV RABBITMQ_ERLANG_COOKIE=5188fd99edf19acfefcbb29a16f3d373aa01f66bfe89929852dfad2674d36af2
-
-# this is only used in the cluster version
-ADD cluster-rabbit.sh /

images/rabbitmq/Dockerfile

@@ -1,11 +1,26 @@
+ARG IMAGE_REPO
+FROM ${IMAGE_REPO:-lagoon}/commons as commons
 FROM rabbitmq:3-management
 
 ARG LAGOON_VERSION
 ENV LAGOON_VERSION=$LAGOON_VERSION
 
-COPY rabbitmq_delayed_message_exchange-3.7.0.ez /plugins
+ENV RABBITMQ_DEFAULT_USER='guest' \
+    RABBITMQ_DEFAULT_PASS='guest'\
+    RABBITMQ_DEFAULT_HA_PATTERN='^$'\
+    RABBITMQ_DEFAULT_VHOST='/'
+
+COPY --from=commons /bin/ep /bin/fix-permissions /bin/
 
+COPY rabbitmq_delayed_message_exchange-3.7.0.ez /plugins
 RUN rabbitmq-plugins enable --offline rabbitmq_delayed_message_exchange;
 
-ENV RABBITMQ_DEFAULT_USER=guest \
-    RABBITMQ_DEFAULT_PASS="guest"
+# Copy startup schema with vhost, users, permissions and policies
+COPY definitions.json /etc/rabbitmq/definitions.json
+RUN fix-permissions /etc/rabbitmq/definitions.json
+
+# Copy a custom entrypoint
+COPY cluster-rabbit.sh /
+RUN fix-permissions /cluster-rabbit.sh && chmod +x /cluster-rabbit.sh
+
+ENTRYPOINT /cluster-rabbit.sh

images/rabbitmq-cluster/cluster-rabbit.sh → images/rabbitmq/cluster-rabbit.sh

@@ -1,6 +1,11 @@
 #!/bin/sh
 
 
+# Replace ENV values in definitions.json
+if [ -e /etc/rabbitmq/definitions.json ]; then
+	/bin/ep /etc/rabbitmq/definitions.json
+fi
+
 # Check if the container runs in Kubernetes/OpenShift
 if [ -z "$POD_NAMESPACE" ]; then
   # Single container runs in docker
@@ -13,5 +18,6 @@ echo NODENAME=rabbit@${HOSTNAME}.${SERVICE_NAME}-headless.${POD_NAMESPACE}.svc.c
 echo cluster_formation.k8s.hostname_suffix=.${SERVICE_NAME}-headless.${POD_NAMESPACE}.svc.cluster.local >> /etc/rabbitmq/rabbitmq.conf
 echo cluster_formation.k8s.service_name=${SERVICE_NAME}-headless >> /etc/rabbitmq/rabbitmq.conf
 
+
 # start the server
 docker-entrypoint.sh rabbitmq-server

images/rabbitmq/definitions.json

@@ -0,0 +1,14 @@
+{
+  "vhosts":[
+        {"name": "${RABBITMQ_DEFAULT_VHOST}"}
+  ],
+  "users": [
+        { "name": "${RABBITMQ_DEFAULT_USER}", "password": "${RABBITMQ_DEFAULT_PASS}", "tags": "administrator" }
+  ],
+  "permissions":[
+        { "user": "${RABBITMQ_DEFAULT_USER}", "vhost": "${RABBITMQ_DEFAULT_VHOST}", "configure": ".*", "write": ".*", "read": ".*" }
+  ],
+  "policies":[
+        {"vhost":"${RABBITMQ_DEFAULT_VHOST}","name":"lagoon-ha","pattern":"${RABBITMQ_DEFAULT_HA_PATTERN}", "definition":{"ha-mode":"exactly","ha-params":2,"ha-sync-mode":"automatic","ha-sync-batch-size":5}}
+  ]
+}

mkdocs.yml

@@ -17,6 +17,7 @@ pages:
   - Docker Images:
     - PHP-FPM: using_lagoon/docker_images/php-fpm.md
     - Nginx: using_lagoon/docker_images/nginx.md
+    - RabbitMQ: using_lagoon/docker_images/rabbitmq.md
   - Workflows: using_lagoon/workflows.md
   - Backups: using_lagoon/backups.md
   - Logs: 

services/broker/Dockerfile

@@ -1,2 +1,3 @@
 ARG IMAGE_REPO
 FROM ${IMAGE_REPO:-lagoon}/rabbitmq-cluster
+ENV RABBITMQ_DEFAULT_HA_PATTERN='^lagoon-'