merge v4.3.3-1 release

CHANGELOG.md

@@ -0,0 +1,27 @@
+# Changelog
+
+## v4.3.3-1 (2024-12-19)
+
+- Upgrade to iRODS v4.3.3 (#16)
+- Upgrade to PostgreSQL >11 (#18)
+- Upgrade image to Ubuntu 20.04 (#19)
+- Upgrade scripts for Python3 (#21)
+- Set up logging with syslog (#16, #34, #36, #37)
+- Enable setting `irods-rule-engine-plugin-python` version in `build.sh` (#27)
+- Add changelog (#22)
+- Change custom SODAR PAM login method from `POST` to `GET` (bihealth/sodar-server#1999)
+- Set bash as shell for `IRODS_SERVICE_ACCOUNT_USER` (#15)
+- Add `BUILD_VERSION` in `build.sh` (#23)
+- Update minimum password time configuration (#33)
+- Add `IRODS_PASSWORD_MIN_TIME` env var (#33)
+- Fix SSSD package discovery (#31)
+- Enable Python rule engine `core.py` file templating (#41)
+- Add `IRODS_CLIENT_SERVER_POLICY` in `core.py` template (#42)
+- Remove support for legacy and C++ rule engines (#43)
+- Remove `IRODS_AUTHENTICATION_SCHEME` env var (#44)
+- Set `IRODS_CLIENT_SERVER_NEGOTIATION` default value to `request_server_negotiation` (#45)
+
+
+## v4.2 (2024-01-19)
+
+- Tag release for legacy iRODS v4.2 image

README.md

@@ -1,36 +1,37 @@
 # Dockerized iRODS
 
-This repository contains the necessary files to build an iRODS Docker image based on Ubuntu 18.04.
+This repository contains the necessary files to build an iRODS Docker image based on Ubuntu 20.04.
 The code is based on [hurngchunlee/docker-irods](https://github.com/hurngchunlee/docker-irods).
 
 The image contains features specific to our [SODAR](https://github.com/bihealth/sodar-server) system, but using them is optional and the image also works as a generic iRODS server.
 
-## Building
+This image uses the Python rule engine for rules. For enabling legacy or C++ engines, the user needs to provide their own rule files and add relevant changes to `server_config.json`.
+
+Images are built and tagged for a specific iRODS release. The most recent build is tested to be compatible with iRODS version `4.3.3`. Our goal is to keep up with the most recent major release of iRODS. Updates for older major versions will not be made.
+
+**NOTE:** Images built for iRODS v4.3.x are **not** compatible with iRODS v4.2 or below. See below for instructions on upgrading from an older iRODS v4.2 build of this image.
 
-```bash
-$ cd docker
-$ docker build .
-```
 
 ## Data Persistency
 
 Each container exposes volumes for data persistency. The list of volumes are provided in the table below:
 
-| path in container               | usage                         |
-|---------------------------------|-------------------------------|
-| /etc/irods                      | resource server configuration |
-| /var/lib/irods/iRODS/server/log | resource server log           |
+| path in container               | usage                           |
+|---------------------------------|---------------------------------|
+| /etc/irods                      | Server configuration            |
 
 For iRODS services, the setup script (`/var/lib/irods/scripts/setup_irods.py`) is only executed when the file `/etc/irods/.provisioned` is not present.
 The file `/etc/irods/.provisioned` is created when the setup script is executed successfully.
 
+
 ## Commands
 
 The following commands are available.
 If you specify anything else then the startup script will `exec` this command (e.g., `bash`).
 
 - `irods-start` (default) -- Start iRODS server
 
+
 ## Environment Variables
 
 There are several environment variables can be set for setting up iRODS.
@@ -40,7 +41,8 @@ iRODS can be run in either "provider" mode, which installs an iCAT catalogue ser
 
 | Variable name                    | Default Value                    | Role       |
 |----------------------------------|----------------------------------|------------|
-| IRODS_PKG_VERSION                | 4.2.8-1                          | both       |
+| IRODS_PKG_VERSION                | 4.3.3                            | both       |
+| IRODS_PYTHON_RULE_ENGINE_VERSION | 4.3.3.0-0+4.3.3                  | both       |
 | IRODS_ROLE                       | provider                         | both       |
 | IRODS_HOST_NAME                  | localhost                        | both       |
 | IRODS_SERVICE_ACCOUNT_USER       | irods                            | both       |
@@ -61,8 +63,7 @@ iRODS can be run in either "provider" mode, which installs an iCAT catalogue ser
 | IRODS_SSL_VERIFY_SERVER          | none                             | both       |
 | IRODS_PASSWORD_SALT              | tempsalt                         | both       |
 | IRODS_SSL_CA_CERT_PATH           |                                  | both       |
-| IRODS_AUTHENTICATION_SCHEME      | native                           | both       |
-| IRODS_CLIENT_SERVER_NEGOTIATION  | off                              | both       |
+| IRODS_CLIENT_SERVER_NEGOTIATION  | request_server_negotiation       | both       |
 | IRODS_CLIENT_SERVER_POLICY       | CS_NEG_REFUSE                    | both       |
 | IRODS_RESOURCE_DIRECTORY         | /data/Vault                      | both       |
 | IRODS_DEFAULT_HASH_SCHEME        | SHA256                           | both       |
@@ -74,19 +75,47 @@ iRODS can be run in either "provider" mode, which installs an iCAT catalogue ser
 | IRODS_ICAT_DBPASS                | irods                            | provider   |
 | IRODS_SSSD_AUTH                  | 0                                | provider   |
 | IRODS_SODAR_AUTH                 | 0                                | provider   |
+| IRODS_PASSWORD_MIN_TIME          | 1209600                          | provider   |
 | IRODS_CATALOG_PROVIDER_HOST      |                                  | consumer   |
 
+
 ## SSSD Support
 
-In addition to the base image, we provide the images `${VERSION}-sssd` (e.g., `4.2.11-1-sssd`) which have SSSD installed.
+In addition to the base image, we provide the images `${VERSION}-sssd` (e.g., `4.3.3-1-sssd`) which have SSSD installed.
 You will have to share `/var/lib/sss` between the SSSD container and iRODS so both containers can communicate.
 
 In our installations, we run [bihealth/sssd-docker](https://github.com/bihealth/sssd-docker) in a second container.
 
+
+## Upgrading From iRODS 4.2
+
+See [sodar-docker-compose](https://github.com/bihealth/sodar-docker-compose/) for upgrade instructions.
+
+
 ## Troubleshooting
 
+### v4.3
+
+Releases of this image for iRODS v4.3.x require PostgreSQL v12 or newer. Installations with PostgreSQL v11 no longer work.
+
+### v4.2
+
 A previous version of this image was built on CentOS7 instead of Ubuntu. If updating or redeploying an existing installation, you may encounter the following error connecting to the iRODS database: `[unixODBC][Driver Manager]Data source name not found, and no default driver specified`
 
 To fix this, first edit the file `/etc/irods/server_config.json`. Find the variable `db_odbc_driver` and change its value from `PostgreSQL` to `PostgreSQL Unicode`.
 
 Next, do the same modification for the environment variable `IRODS_ODBC_DRIVER`. After restarting the image, iRODS should work normally.
+
+
+## Building (for Developers)
+
+To build the image, use the following command:
+
+```
+bash
+$ IRODS_PKG_VERSION=x.x.x IRODS_PYTHON_RULE_ENGINE_VERSION=y.y.y BUILD_VERSION=z ./build.sh
+```
+
+Releases and images are tagged with the iRODS server version followed by the image build version. This means that e.g. the initial release for iRODS `4.3.3` will be tagged as `4.3.3-1`. Fixes or improvements to that release would then be published as `4.3.3-2`.
+
+Note that if you are providing a non-default iRODS version, you will also have to provide the `irods-rule-engine-plugin-python` version number with the `IRODS_PYTHON_RULE_ENGINE_VERSION` env var. This package does not follow the same versioning conventions as the main iRODS packages. The value is expected to be the full version name *without* the `~focal` suffix. You can find the available versions e.g. by running `apt-cache madison irods-rule-engine-plugin-python`.

build.sh

@@ -1,18 +1,24 @@
 #!/bin/bash
 
 export REPO=ghcr.io/bihealth/irods-docker
-export IRODS_PKG_VERSION=${IRODS_PKG_VERSION-4.2.11-1}
+export IRODS_PKG_VERSION=${IRODS_PKG_VERSION-4.3.3}
+export IRODS_PYTHON_RULE_ENGINE_VERSION=${IRODS_PYTHON_RULE_ENGINE_VERSION-4.3.3.0-0+4.3.3}
+export BUILD_VERSION=${BUILD_VERSION-1}
 
 docker build \
-    -t "${REPO}:${IRODS_PKG_VERSION}" \
+    -t "${REPO}:${IRODS_PKG_VERSION}-${BUILD_VERSION}" \
+    --build-arg IRODS_PKG_VERSION=${IRODS_PKG_VERSION} \
+    --build-arg IRODS_PYTHON_RULE_ENGINE_VERSION=${IRODS_PYTHON_RULE_ENGINE_VERSION} \
     --target main \
     docker
 
 docker build \
-    -t "${REPO}:${IRODS_PKG_VERSION}-sssd" \
+    -t "${REPO}:${IRODS_PKG_VERSION}-${BUILD_VERSION}-sssd" \
+    --build-arg IRODS_PKG_VERSION=${IRODS_PKG_VERSION} \
+    --build-arg IRODS_PYTHON_RULE_ENGINE_VERSION=${IRODS_PYTHON_RULE_ENGINE_VERSION} \
     --target sssd \
     docker
 
 echo "Now do:"
-echo "docker push ${REPO}:${IRODS_PKG_VERSION}"
-echo "docker push ${REPO}:${IRODS_PKG_VERSION}-sssd"
+echo "docker push ${REPO}:${IRODS_PKG_VERSION}-${BUILD_VERSION}"
+echo "docker push ${REPO}:${IRODS_PKG_VERSION}-${BUILD_VERSION}-sssd"

docker/Dockerfile

@@ -1,15 +1,18 @@
 #
 # Stage: first / main
 #
-FROM ubuntu:18.04 as main
+FROM ubuntu:20.04 AS main
 
 LABEL org.opencontainers.image.authors="Manuel Holtgrewe <[email protected]>, Mikko Nieminen <[email protected]>"
 LABEL org.opencontainers.image.source https://github.com/bihealth/irods-docker
 
 ARG DEBIAN_FRONTEND=noninteractive
+ARG IRODS_PKG_VERSION="4.3.3"
+ARG IRODS_PKG_SUFFIX="-0~focal"
+ARG IRODS_PYTHON_RULE_ENGINE_VERSION="4.3.3.0-0+4.3.3"
 
 # Environment variables for container runtime
-ENV IRODS_PKG_VERSION=4.2.11-1 \
+ENV IRODS_PKG_VERSION=$IRODS_PKG_VERSION \
     IRODS_ROLE=provider \
     IRODS_HOST_NAME=localhost \
     IRODS_SERVICE_ACCOUNT_USER=irods \
@@ -30,8 +33,7 @@ ENV IRODS_PKG_VERSION=4.2.11-1 \
     IRODS_SSL_VERIFY_SERVER=none \
     IRODS_PASSWORD_SALT=tempsalt \
     IRODS_SSL_CA_CERT_PATH= \
-    IRODS_AUTHENTICATION_SCHEME=native \
-    IRODS_CLIENT_SERVER_NEGOTIATION=off \
+    IRODS_CLIENT_SERVER_NEGOTIATION=request_server_negotiation \
     IRODS_CLIENT_SERVER_POLICY=CS_NEG_REFUSE \
     IRODS_RESOURCE_DIRECTORY=/data/Vault \
     IRODS_DEFAULT_HASH_SCHEME="SHA256" \
@@ -44,52 +46,67 @@ ENV IRODS_PKG_VERSION=4.2.11-1 \
     IRODS_CATALOG_PROVIDER_HOST= \
     IRODS_SSSD_AUTH=0 \
     IRODS_SODAR_AUTH=0 \
-    IRODS_SODAR_API_HOST=https://sodar-web
+    IRODS_SODAR_API_HOST=https://sodar-web \
+    IRODS_PASSWORD_MIN_TIME=1209600
 
 # Add the wait script to the image
 ADD https://github.com/ufoscout/docker-compose-wait/releases/download/2.7.3/wait /usr/local/bin/wait
 RUN chmod +x /usr/local/bin/wait
 
 # Install general dependencies
 RUN apt-get update && apt-get install -y apt-utils
-RUN apt-get install -y python python-pip python-dev sudo vim wget netcat lsb-release
+RUN apt-get install -y python3 python3-dev python3-distro python3-pip sudo vim wget netcat rsyslog
+# lsb-release g++ gnupg2
 
 # Install database dependencies
 RUN apt-get install -y unixodbc unixodbc-dev odbc-postgresql
 
+# Install iRODS Python dependencies
+RUN pip3 install pyodbc
+
 # Install iRODS
 RUN wget -qO - https://packages.irods.org/irods-signing-key.asc | sudo apt-key add - \
-  && echo "deb [arch=amd64] https://packages.irods.org/apt/ $(lsb_release -sc) main" | sudo tee /etc/apt/sources.list.d/renci-irods.list \
+  && echo "deb [arch=amd64] https://packages.irods.org/apt/ focal main" | sudo tee /etc/apt/sources.list.d/renci-irods.list \
   && apt-get update
-RUN apt-get install -y irods-server=${IRODS_PKG_VERSION}~bionic \
-    irods-database-plugin-postgres=${IRODS_PKG_VERSION}~bionic \
-    irods-dev=${IRODS_PKG_VERSION}~bionic \
-    irods-rule-engine-plugin-python
-RUN useradd -d /var/lib/irods ${IRODS_SERVICE_ACCOUNT_USER}
-
-# Copy rule-engine installer
-COPY files/irods_python-re_installer.py /
-RUN chmod +x /irods_python-re_installer.py
+RUN apt-get install -y irods-runtime=${IRODS_PKG_VERSION}${IRODS_PKG_SUFFIX} \
+    irods-server=${IRODS_PKG_VERSION}${IRODS_PKG_SUFFIX} \
+    irods-database-plugin-postgres=${IRODS_PKG_VERSION}${IRODS_PKG_SUFFIX} \
+    irods-icommands=${IRODS_PKG_VERSION}${IRODS_PKG_SUFFIX} \
+    irods-dev=${IRODS_PKG_VERSION}${IRODS_PKG_SUFFIX} \
+    irods-rule-engine-plugin-python=${IRODS_PYTHON_RULE_ENGINE_VERSION}~focal
+RUN useradd -d /var/lib/irods -s /bin/bash ${IRODS_SERVICE_ACCOUNT_USER}
 
 # Install j2cli for templating
-RUN apt-get install -y python-jinja2 python-yaml
-RUN pip install j2cli
+RUN apt-get install -y python3-jinja2 python3-yaml
+RUN pip3 install j2cli
 
 # Install Python PAM support
-RUN apt-get install libpam-python pamtester
+RUN apt-get update
+RUN apt-get install -y libpam-python pamtester
+# NOTE: Python2 needed for custom PAM module
+RUN apt-get install -y python python-dev
+RUN wget https://bootstrap.pypa.io/pip/2.7/get-pip.py
+RUN python2 get-pip.py
+RUN pip install requests
 
 # Copy scripts and templates
-COPY docker-entrypoint.sh files/irods_login.sh \
-     templates/core.py.template templates/unattended_config.json.j2 \
+COPY docker-entrypoint.sh \
+     templates/core.py.j2 templates/unattended_config.json.j2 \
      templates/irods.pam.j2 files/j2-filters.py templates/pam_sodar.py.j2 /
-RUN chmod +x /docker-entrypoint.sh /irods_login.sh
+RUN chmod +x /docker-entrypoint.sh
+
+# Set up logging
+COPY files/irods_syslog.conf /etc/rsyslog.d/00-irods.conf
+COPY files/irods.logrotate /etc/logrotate.d/irods
+RUN chmod 0644 /etc/logrotate.d/irods
+RUN mkdir -p /var/log/irods
 
 # Create iRODS vault dir
 RUN mkdir -p $IRODS_RESOURCE_DIRECTORY
 RUN chown -cR $IRODS_SERVICE_ACCOUNT_GROUP:$IRODS_SERVICE_ACCOUNT_USER $IRODS_RESOURCE_DIRECTORY
 
 # Data volumes
-VOLUME "/etc/irods" "/var/lib/irods/iRODS/server/log"
+VOLUME "/etc/irods"
 
 # Network ports
 EXPOSE 4321 $IRODS_ZONE_PORT $IRODS_CONTROL_PLANE_PORT $IRODS_DATA_PORT_RANGE_START-$IRODS_DATA_PORT_RANGE_END
@@ -105,4 +122,5 @@ CMD ["irods-start"]
 FROM main AS sssd
 
 ARG DEBIAN_FRONTEND=noninteractive
+RUN apt-get update
 RUN apt-get install -y sssd sssd-ldap sssd-tools strace