diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 0000000..7037331 --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,27 @@ +# Changelog + +## v4.3.3-1 (2024-12-19) + +- Upgrade to iRODS v4.3.3 (#16) +- Upgrade to PostgreSQL >11 (#18) +- Upgrade image to Ubuntu 20.04 (#19) +- Upgrade scripts for Python3 (#21) +- Set up logging with syslog (#16, #34, #36, #37) +- Enable setting `irods-rule-engine-plugin-python` version in `build.sh` (#27) +- Add changelog (#22) +- Change custom SODAR PAM login method from `POST` to `GET` (bihealth/sodar-server#1999) +- Set bash as shell for `IRODS_SERVICE_ACCOUNT_USER` (#15) +- Add `BUILD_VERSION` in `build.sh` (#23) +- Update minimum password time configuration (#33) +- Add `IRODS_PASSWORD_MIN_TIME` env var (#33) +- Fix SSSD package discovery (#31) +- Enable Python rule engine `core.py` file templating (#41) +- Add `IRODS_CLIENT_SERVER_POLICY` in `core.py` template (#42) +- Remove support for legacy and C++ rule engines (#43) +- Remove `IRODS_AUTHENTICATION_SCHEME` env var (#44) +- Set `IRODS_CLIENT_SERVER_NEGOTIATION` default value to `request_server_negotiation` (#45) + + +## v4.2 (2024-01-19) + +- Tag release for legacy iRODS v4.2 image diff --git a/README.md b/README.md index be33e45..53cc5a5 100644 --- a/README.md +++ b/README.md @@ -1,29 +1,29 @@ # Dockerized iRODS -This repository contains the necessary files to build an iRODS Docker image based on Ubuntu 18.04. +This repository contains the necessary files to build an iRODS Docker image based on Ubuntu 20.04. The code is based on [hurngchunlee/docker-irods](https://github.com/hurngchunlee/docker-irods). The image contains features specific to our [SODAR](https://github.com/bihealth/sodar-server) system, but using them is optional and the image also works as a generic iRODS server. -## Building +This image uses the Python rule engine for rules. For enabling legacy or C++ engines, the user needs to provide their own rule files and add relevant changes to `server_config.json`. + +Images are built and tagged for a specific iRODS release. The most recent build is tested to be compatible with iRODS version `4.3.3`. Our goal is to keep up with the most recent major release of iRODS. Updates for older major versions will not be made. + +**NOTE:** Images built for iRODS v4.3.x are **not** compatible with iRODS v4.2 or below. See below for instructions on upgrading from an older iRODS v4.2 build of this image. -```bash -$ cd docker -$ docker build . -``` ## Data Persistency Each container exposes volumes for data persistency. The list of volumes are provided in the table below: -| path in container | usage | -|---------------------------------|-------------------------------| -| /etc/irods | resource server configuration | -| /var/lib/irods/iRODS/server/log | resource server log | +| path in container | usage | +|---------------------------------|---------------------------------| +| /etc/irods | Server configuration | For iRODS services, the setup script (`/var/lib/irods/scripts/setup_irods.py`) is only executed when the file `/etc/irods/.provisioned` is not present. The file `/etc/irods/.provisioned` is created when the setup script is executed successfully. + ## Commands The following commands are available. @@ -31,6 +31,7 @@ If you specify anything else then the startup script will `exec` this command (e - `irods-start` (default) -- Start iRODS server + ## Environment Variables There are several environment variables can be set for setting up iRODS. @@ -40,7 +41,8 @@ iRODS can be run in either "provider" mode, which installs an iCAT catalogue ser | Variable name | Default Value | Role | |----------------------------------|----------------------------------|------------| -| IRODS_PKG_VERSION | 4.2.8-1 | both | +| IRODS_PKG_VERSION | 4.3.3 | both | +| IRODS_PYTHON_RULE_ENGINE_VERSION | 4.3.3.0-0+4.3.3 | both | | IRODS_ROLE | provider | both | | IRODS_HOST_NAME | localhost | both | | IRODS_SERVICE_ACCOUNT_USER | irods | both | @@ -61,8 +63,7 @@ iRODS can be run in either "provider" mode, which installs an iCAT catalogue ser | IRODS_SSL_VERIFY_SERVER | none | both | | IRODS_PASSWORD_SALT | tempsalt | both | | IRODS_SSL_CA_CERT_PATH | | both | -| IRODS_AUTHENTICATION_SCHEME | native | both | -| IRODS_CLIENT_SERVER_NEGOTIATION | off | both | +| IRODS_CLIENT_SERVER_NEGOTIATION | request_server_negotiation | both | | IRODS_CLIENT_SERVER_POLICY | CS_NEG_REFUSE | both | | IRODS_RESOURCE_DIRECTORY | /data/Vault | both | | IRODS_DEFAULT_HASH_SCHEME | SHA256 | both | @@ -74,19 +75,47 @@ iRODS can be run in either "provider" mode, which installs an iCAT catalogue ser | IRODS_ICAT_DBPASS | irods | provider | | IRODS_SSSD_AUTH | 0 | provider | | IRODS_SODAR_AUTH | 0 | provider | +| IRODS_PASSWORD_MIN_TIME | 1209600 | provider | | IRODS_CATALOG_PROVIDER_HOST | | consumer | + ## SSSD Support -In addition to the base image, we provide the images `${VERSION}-sssd` (e.g., `4.2.11-1-sssd`) which have SSSD installed. +In addition to the base image, we provide the images `${VERSION}-sssd` (e.g., `4.3.3-1-sssd`) which have SSSD installed. You will have to share `/var/lib/sss` between the SSSD container and iRODS so both containers can communicate. In our installations, we run [bihealth/sssd-docker](https://github.com/bihealth/sssd-docker) in a second container. + +## Upgrading From iRODS 4.2 + +See [sodar-docker-compose](https://github.com/bihealth/sodar-docker-compose/) for upgrade instructions. + + ## Troubleshooting +### v4.3 + +Releases of this image for iRODS v4.3.x require PostgreSQL v12 or newer. Installations with PostgreSQL v11 no longer work. + +### v4.2 + A previous version of this image was built on CentOS7 instead of Ubuntu. If updating or redeploying an existing installation, you may encounter the following error connecting to the iRODS database: `[unixODBC][Driver Manager]Data source name not found, and no default driver specified` To fix this, first edit the file `/etc/irods/server_config.json`. Find the variable `db_odbc_driver` and change its value from `PostgreSQL` to `PostgreSQL Unicode`. Next, do the same modification for the environment variable `IRODS_ODBC_DRIVER`. After restarting the image, iRODS should work normally. + + +## Building (for Developers) + +To build the image, use the following command: + +``` +bash +$ IRODS_PKG_VERSION=x.x.x IRODS_PYTHON_RULE_ENGINE_VERSION=y.y.y BUILD_VERSION=z ./build.sh +``` + +Releases and images are tagged with the iRODS server version followed by the image build version. This means that e.g. the initial release for iRODS `4.3.3` will be tagged as `4.3.3-1`. Fixes or improvements to that release would then be published as `4.3.3-2`. + +Note that if you are providing a non-default iRODS version, you will also have to provide the `irods-rule-engine-plugin-python` version number with the `IRODS_PYTHON_RULE_ENGINE_VERSION` env var. This package does not follow the same versioning conventions as the main iRODS packages. The value is expected to be the full version name *without* the `~focal` suffix. You can find the available versions e.g. by running `apt-cache madison irods-rule-engine-plugin-python`. diff --git a/build.sh b/build.sh index 14de5c3..bac1c02 100755 --- a/build.sh +++ b/build.sh @@ -1,18 +1,24 @@ #!/bin/bash export REPO=ghcr.io/bihealth/irods-docker -export IRODS_PKG_VERSION=${IRODS_PKG_VERSION-4.2.11-1} +export IRODS_PKG_VERSION=${IRODS_PKG_VERSION-4.3.3} +export IRODS_PYTHON_RULE_ENGINE_VERSION=${IRODS_PYTHON_RULE_ENGINE_VERSION-4.3.3.0-0+4.3.3} +export BUILD_VERSION=${BUILD_VERSION-1} docker build \ - -t "${REPO}:${IRODS_PKG_VERSION}" \ + -t "${REPO}:${IRODS_PKG_VERSION}-${BUILD_VERSION}" \ + --build-arg IRODS_PKG_VERSION=${IRODS_PKG_VERSION} \ + --build-arg IRODS_PYTHON_RULE_ENGINE_VERSION=${IRODS_PYTHON_RULE_ENGINE_VERSION} \ --target main \ docker docker build \ - -t "${REPO}:${IRODS_PKG_VERSION}-sssd" \ + -t "${REPO}:${IRODS_PKG_VERSION}-${BUILD_VERSION}-sssd" \ + --build-arg IRODS_PKG_VERSION=${IRODS_PKG_VERSION} \ + --build-arg IRODS_PYTHON_RULE_ENGINE_VERSION=${IRODS_PYTHON_RULE_ENGINE_VERSION} \ --target sssd \ docker echo "Now do:" -echo "docker push ${REPO}:${IRODS_PKG_VERSION}" -echo "docker push ${REPO}:${IRODS_PKG_VERSION}-sssd" +echo "docker push ${REPO}:${IRODS_PKG_VERSION}-${BUILD_VERSION}" +echo "docker push ${REPO}:${IRODS_PKG_VERSION}-${BUILD_VERSION}-sssd" diff --git a/docker/Dockerfile b/docker/Dockerfile index d9116d2..6688d74 100644 --- a/docker/Dockerfile +++ b/docker/Dockerfile @@ -1,15 +1,18 @@ # # Stage: first / main # -FROM ubuntu:18.04 as main +FROM ubuntu:20.04 AS main LABEL org.opencontainers.image.authors="Manuel Holtgrewe , Mikko Nieminen " LABEL org.opencontainers.image.source https://github.com/bihealth/irods-docker ARG DEBIAN_FRONTEND=noninteractive +ARG IRODS_PKG_VERSION="4.3.3" +ARG IRODS_PKG_SUFFIX="-0~focal" +ARG IRODS_PYTHON_RULE_ENGINE_VERSION="4.3.3.0-0+4.3.3" # Environment variables for container runtime -ENV IRODS_PKG_VERSION=4.2.11-1 \ +ENV IRODS_PKG_VERSION=$IRODS_PKG_VERSION \ IRODS_ROLE=provider \ IRODS_HOST_NAME=localhost \ IRODS_SERVICE_ACCOUNT_USER=irods \ @@ -30,8 +33,7 @@ ENV IRODS_PKG_VERSION=4.2.11-1 \ IRODS_SSL_VERIFY_SERVER=none \ IRODS_PASSWORD_SALT=tempsalt \ IRODS_SSL_CA_CERT_PATH= \ - IRODS_AUTHENTICATION_SCHEME=native \ - IRODS_CLIENT_SERVER_NEGOTIATION=off \ + IRODS_CLIENT_SERVER_NEGOTIATION=request_server_negotiation \ IRODS_CLIENT_SERVER_POLICY=CS_NEG_REFUSE \ IRODS_RESOURCE_DIRECTORY=/data/Vault \ IRODS_DEFAULT_HASH_SCHEME="SHA256" \ @@ -44,7 +46,8 @@ ENV IRODS_PKG_VERSION=4.2.11-1 \ IRODS_CATALOG_PROVIDER_HOST= \ IRODS_SSSD_AUTH=0 \ IRODS_SODAR_AUTH=0 \ - IRODS_SODAR_API_HOST=https://sodar-web + IRODS_SODAR_API_HOST=https://sodar-web \ + IRODS_PASSWORD_MIN_TIME=1209600 # Add the wait script to the image ADD https://github.com/ufoscout/docker-compose-wait/releases/download/2.7.3/wait /usr/local/bin/wait @@ -52,44 +55,58 @@ RUN chmod +x /usr/local/bin/wait # Install general dependencies RUN apt-get update && apt-get install -y apt-utils -RUN apt-get install -y python python-pip python-dev sudo vim wget netcat lsb-release +RUN apt-get install -y python3 python3-dev python3-distro python3-pip sudo vim wget netcat rsyslog +# lsb-release g++ gnupg2 # Install database dependencies RUN apt-get install -y unixodbc unixodbc-dev odbc-postgresql +# Install iRODS Python dependencies +RUN pip3 install pyodbc + # Install iRODS RUN wget -qO - https://packages.irods.org/irods-signing-key.asc | sudo apt-key add - \ - && echo "deb [arch=amd64] https://packages.irods.org/apt/ $(lsb_release -sc) main" | sudo tee /etc/apt/sources.list.d/renci-irods.list \ + && echo "deb [arch=amd64] https://packages.irods.org/apt/ focal main" | sudo tee /etc/apt/sources.list.d/renci-irods.list \ && apt-get update -RUN apt-get install -y irods-server=${IRODS_PKG_VERSION}~bionic \ - irods-database-plugin-postgres=${IRODS_PKG_VERSION}~bionic \ - irods-dev=${IRODS_PKG_VERSION}~bionic \ - irods-rule-engine-plugin-python -RUN useradd -d /var/lib/irods ${IRODS_SERVICE_ACCOUNT_USER} - -# Copy rule-engine installer -COPY files/irods_python-re_installer.py / -RUN chmod +x /irods_python-re_installer.py +RUN apt-get install -y irods-runtime=${IRODS_PKG_VERSION}${IRODS_PKG_SUFFIX} \ + irods-server=${IRODS_PKG_VERSION}${IRODS_PKG_SUFFIX} \ + irods-database-plugin-postgres=${IRODS_PKG_VERSION}${IRODS_PKG_SUFFIX} \ + irods-icommands=${IRODS_PKG_VERSION}${IRODS_PKG_SUFFIX} \ + irods-dev=${IRODS_PKG_VERSION}${IRODS_PKG_SUFFIX} \ + irods-rule-engine-plugin-python=${IRODS_PYTHON_RULE_ENGINE_VERSION}~focal +RUN useradd -d /var/lib/irods -s /bin/bash ${IRODS_SERVICE_ACCOUNT_USER} # Install j2cli for templating -RUN apt-get install -y python-jinja2 python-yaml -RUN pip install j2cli +RUN apt-get install -y python3-jinja2 python3-yaml +RUN pip3 install j2cli # Install Python PAM support -RUN apt-get install libpam-python pamtester +RUN apt-get update +RUN apt-get install -y libpam-python pamtester +# NOTE: Python2 needed for custom PAM module +RUN apt-get install -y python python-dev +RUN wget https://bootstrap.pypa.io/pip/2.7/get-pip.py +RUN python2 get-pip.py +RUN pip install requests # Copy scripts and templates -COPY docker-entrypoint.sh files/irods_login.sh \ - templates/core.py.template templates/unattended_config.json.j2 \ +COPY docker-entrypoint.sh \ + templates/core.py.j2 templates/unattended_config.json.j2 \ templates/irods.pam.j2 files/j2-filters.py templates/pam_sodar.py.j2 / -RUN chmod +x /docker-entrypoint.sh /irods_login.sh +RUN chmod +x /docker-entrypoint.sh + +# Set up logging +COPY files/irods_syslog.conf /etc/rsyslog.d/00-irods.conf +COPY files/irods.logrotate /etc/logrotate.d/irods +RUN chmod 0644 /etc/logrotate.d/irods +RUN mkdir -p /var/log/irods # Create iRODS vault dir RUN mkdir -p $IRODS_RESOURCE_DIRECTORY RUN chown -cR $IRODS_SERVICE_ACCOUNT_GROUP:$IRODS_SERVICE_ACCOUNT_USER $IRODS_RESOURCE_DIRECTORY # Data volumes -VOLUME "/etc/irods" "/var/lib/irods/iRODS/server/log" +VOLUME "/etc/irods" # Network ports EXPOSE 4321 $IRODS_ZONE_PORT $IRODS_CONTROL_PLANE_PORT $IRODS_DATA_PORT_RANGE_START-$IRODS_DATA_PORT_RANGE_END @@ -105,4 +122,5 @@ CMD ["irods-start"] FROM main AS sssd ARG DEBIAN_FRONTEND=noninteractive +RUN apt-get update RUN apt-get install -y sssd sssd-ldap sssd-tools strace diff --git a/docker/docker-entrypoint.sh b/docker/docker-entrypoint.sh index 609af78..55e7790 100755 --- a/docker/docker-entrypoint.sh +++ b/docker/docker-entrypoint.sh @@ -1,16 +1,19 @@ #!/bin/bash NO_WAIT=${NO_WAIT-0} +export PGPASSWORD=$IRODS_ICAT_DBPASS set -euo pipefail if [[ "$1" == "irods-start" ]]; then - if [[ ! -e /etc/irods/core.py.template ]]; then - cp /core.py.template /etc/irods/core.py.template - fi - chmod a+x /var/lib/irods/irodsctl chown -cR $IRODS_SERVICE_ACCOUNT_GROUP:$IRODS_SERVICE_ACCOUNT_USER /etc/irods + sed -i '/imklog/s/^/#/' /etc/rsyslog.conf + chown syslog:adm /var/log/irods + touch /var/log/irods/irods.log + chown syslog:adm /var/log/irods/irods.log + rm -f /var/run/rsyslogd.pid + /etc/init.d/rsyslog start echo "iRODS server role: $IRODS_ROLE" @@ -18,7 +21,6 @@ if [[ "$1" == "irods-start" ]]; then echo "Waiting for postgres.." export WAIT_HOSTS=${WAIT_HOSTS-${IRODS_ICAT_DBSERVER}:${IRODS_ICAT_DBPORT}} /usr/local/bin/wait - PGPASSWORD=$IRODS_ICAT_DBPASS PSQL="pg_isready -h $IRODS_ICAT_DBSERVER -p $IRODS_ICAT_DBPORT" fi fi @@ -36,30 +38,40 @@ if [[ "$1" == "irods-start" ]]; then cp /etc/irods/.odbc.ini /var/lib/irods/.odbc.ini fi + if [ -f /etc/irods/version.json ]; then + cp -f /etc/irods/version.json /var/lib/irods/version.json + fi + else echo "Provisioning iRODS.." if [[ "$IRODS_ROLE" == "provider" ]]; then - echo "Pre-create database if necessary" - echo $IRODS_ICAT_DBPASS \ - | createdb -h $IRODS_ICAT_DBSERVER -p $IRODS_ICAT_DBPORT -U $IRODS_ICAT_DBUSER -W $IRODS_ICAT_DBNAME + + if [ "$( psql -h $IRODS_ICAT_DBSERVER -p $IRODS_ICAT_DBPORT -U $IRODS_ICAT_DBUSER \ + -XtAc "SELECT 1 FROM pg_database WHERE datname='$IRODS_ICAT_DBNAME'" )" = '1' ] + then + echo "iCAT database already exists, skipping creation" + else + echo "Create iCAT database" + createdb -h $IRODS_ICAT_DBSERVER -p $IRODS_ICAT_DBPORT -U $IRODS_ICAT_DBUSER -W $IRODS_ICAT_DBNAME + fi + fi echo "Set up unattended configuration file" j2 -o /unattended_config.json --undefined --filters=j2-filters.py unattended_config.json.j2 + echo "Set up rule file for the Python rule engine" + j2 -o /core.py --undefined --filters=j2-filters.py core.py.j2 + cp -f /core.py /etc/irods/core.py + echo "Perform iRODS setup" - python /var/lib/irods/scripts/setup_irods.py --json_configuration_file=/unattended_config.json + python3 /var/lib/irods/scripts/setup_irods.py --json_configuration_file=/unattended_config.json cp /var/lib/irods/.irods/irods_environment.json /etc/irods/irods_environment.json cp /var/lib/irods/.odbc.ini /etc/irods/.odbc.ini - - # Enable the python rule engine - if [ -f /irods_python-re_installer.py ]; then - echo "Enable python rule engine" - ./irods_python-re_installer.py - fi + cp -f /var/lib/irods/version.json /etc/irods/version.json touch /etc/irods/.provisioned fi @@ -72,6 +84,12 @@ if [[ "$1" == "irods-start" ]]; then find /var/lib/irods -not -path '/var/lib/irods/Vault*' -exec chown $IRODS_SERVICE_ACCOUNT_GROUP:$IRODS_SERVICE_ACCOUNT_USER {} \; + # Generate .irodsA + echo "Prepare service account" + set +e + su - irods -c "echo ${IRODS_ADMIN_PASS} | iinit > /dev/null 2>&1" + set -e + # Start iRODS echo "Start iRODS" /etc/init.d/irods start @@ -84,11 +102,12 @@ if [[ "$1" == "irods-start" ]]; then done sleep 5 - echo "Test iinit" - su - irods -c "/irods_login.sh ${IRODS_ADMIN_PASS}" - - echo "iCAT at ${IRODS_HOST_NAME} ready!" + # Set minimum session timeout + if [[ "$IRODS_ROLE" == "provider" ]]; then + su - irods -c "iadmin set_grid_configuration authentication password_min_time ${IRODS_PASSWORD_MIN_TIME}" + fi + echo "iRODS is ready" sleep infinity fi diff --git a/docker/files/irods.logrotate b/docker/files/irods.logrotate new file mode 100644 index 0000000..da6951a --- /dev/null +++ b/docker/files/irods.logrotate @@ -0,0 +1,9 @@ +/var/log/irods/irods.log { + weekly + rotate 26 + copytruncate + dateext + notifempty + missingok + su root root +} diff --git a/docker/files/irods_login.sh b/docker/files/irods_login.sh deleted file mode 100644 index 89d913b..0000000 --- a/docker/files/irods_login.sh +++ /dev/null @@ -1,7 +0,0 @@ -#!/bin/bash - -echo $1 | iinit - -if [ $? -ne 0 ]; then - echo "iinit failed" -fi diff --git a/docker/files/irods_python-re_installer.py b/docker/files/irods_python-re_installer.py deleted file mode 100644 index feb120b..0000000 --- a/docker/files/irods_python-re_installer.py +++ /dev/null @@ -1,44 +0,0 @@ -#!/usr/bin/env python - -import json -import sys -import os - -if __name__ == '__main__': - - fname = '/etc/irods/server_config.json' - - if os.path.exists(fname): - # readin server_config.json - f = open(fname,'r') - cfg = json.load(f) - f.close() - - # add python rule engine - re_cfg = { - "instance_name": "irods_rule_engine_plugin-python-instance", - "plugin_name": "irods_rule_engine_plugin-python", - "plugin_specific_configuration": {} - } - cfg['plugin_configuration']['rule_engines'].append(re_cfg) - - # backup original file - os.rename(fname, fname + '.org') - - # write out the new configuration to server_config.json - f = open(fname, 'w') - json.dump(cfg, f, sort_keys=True, indent=4, separators=(',', ': ')) - f.close() - - # copy the core.py.template to core.py - # - it is required for iRODS server to run properly - if os.path.exists('/etc/irods/core.py.template'): - os.rename('/etc/irods/core.py.template', '/etc/irods/core.py') - else: - # make a fake core.py - f = open('/etc/irods/core.py') - f.write('def hello(callback, rei):\n') - f.write(' callback.writeLine("serverLog","hello")\n') - f.close() - else: - pass diff --git a/docker/files/irods_syslog.conf b/docker/files/irods_syslog.conf new file mode 100644 index 0000000..328c432 --- /dev/null +++ b/docker/files/irods_syslog.conf @@ -0,0 +1,9 @@ +$MaxOpenFiles 4096 # Workaround for rsyslog start timeout +$FileCreateMode 0644 +$DirCreateMode 0755 +$Umask 0000 +$template irods_format,"%msg%\n" +:programname,startswith,"irodsServer" /var/log/irods/irods.log;irods_format +& stop +:programname,startswith,"irodsDelayServer" /var/log/irods/irods.log;irods_format +& stop diff --git a/docker/templates/core.py.template b/docker/templates/core.py.j2 similarity index 99% rename from docker/templates/core.py.template rename to docker/templates/core.py.j2 index 2088901..09f1379 100644 --- a/docker/templates/core.py.template +++ b/docker/templates/core.py.j2 @@ -20,7 +20,7 @@ def pythonRuleEnginePluginTest(rule_args, callback, rei): callback.writeLine('serverLog', 'Printed to server log from python rule engine') def acPreConnect(rule_args, callback, rei): - rule_args[0] = 'CS_NEG_DONT_CARE' + rule_args[0] = '{{ IRODS_CLIENT_SERVER_POLICY }}' def acCreateUser(rule_args, callback, rei): ret = callback.msiCreateUser() diff --git a/docker/templates/pam_sodar.py.j2 b/docker/templates/pam_sodar.py.j2 index a1f1578..c9a3bab 100644 --- a/docker/templates/pam_sodar.py.j2 +++ b/docker/templates/pam_sodar.py.j2 @@ -5,6 +5,9 @@ LDAP/AD server is not available. __author__ = 'Mikko Nieminen' import os +import site +# HACK to allow libpam-python to find 3rd party packages +site.addsitedir('/usr/local/lib/python2.7/site-packages') import requests @@ -17,7 +20,7 @@ def pam_sm_authenticate(pamh, flags, argv): sodar_host = '{{ IRODS_SODAR_API_HOST }}' url = sodar_host + '/irodsbackend/api/auth' - response = requests.post(url, auth=(pamh.user, a)) + response = requests.get(url, auth=(pamh.user, a)) if response.status_code == 200: return pamh.PAM_SUCCESS return pamh.PAM_AUTH_ERR diff --git a/docker/templates/unattended_config.json.j2 b/docker/templates/unattended_config.json.j2 index c7f3014..1a45d92 100644 --- a/docker/templates/unattended_config.json.j2 +++ b/docker/templates/unattended_config.json.j2 @@ -1,26 +1,74 @@ { - "id": "file:///var/lib/irods/configuration_schemas/v3/unattended_installation.json", - "$schema": "http://json-schema.org/draft-04/schema#", "admin_password": "{{ IRODS_ADMIN_PASS }}", "default_resource_directory": "{{ IRODS_RESOURCE_DIRECTORY }}", - {# "version": { #} - {# "commit_id": "8e4e6eca104b5cfdb5945a3fbc08d96549ea3e7e", #} - {# "configuration_schema_version": 3, #} - {# "catalog_schema_version": 6, #} - {# "irods_version": "4.2.6", #} - {# "installation_time": "2019-10-25T11:59:20.890609" #} - {# }, #} - "resources": [], + "default_resource_name": "demoResc", + "host_system_information": { + "service_account_user_name": "{{ IRODS_SERVICE_ACCOUNT_USER }}", + "service_account_group_name": "{{ IRODS_SERVICE_ACCOUNT_GROUP }}" + }, + "service_account_environment": { + "irods_client_server_negotiation": "{{ IRODS_CLIENT_SERVER_NEGOTIATION }}", + "irods_client_server_policy": "{{ IRODS_CLIENT_SERVER_POLICY }}", + "irods_connection_pool_refresh_time_in_seconds": 300, + "irods_cwd": "/{{ IRODS_ZONE_NAME }}/home/rods", + "irods_default_hash_scheme": "{{ IRODS_DEFAULT_HASH_SCHEME }}", + "irods_default_number_of_transfer_threads": 4, + "irods_default_resource": "demoResc", + "irods_encryption_algorithm": "AES-256-CBC", + "irods_encryption_key_size": 32, + "irods_encryption_num_hash_rounds": 16, + "irods_encryption_salt_size": 8, + "irods_home": "/{{ IRODS_ZONE_NAME }}/home/rods", + "irods_host": "{{ IRODS_HOST_NAME }}", + "irods_match_hash_policy": "compatible", + "irods_maximum_size_for_single_buffer_in_megabytes": 32, + "irods_port": {{ IRODS_ZONE_PORT }}, + "irods_server_control_plane_encryption_algorithm": "AES-256-CBC", + "irods_server_control_plane_encryption_num_hash_rounds": 16, + "irods_server_control_plane_key": "{{ IRODS_CONTROL_PLANE_KEY }}", + "irods_server_control_plane_port": {{ IRODS_CONTROL_PLANE_PORT|default(1248) }}, + "irods_transfer_buffer_size_for_parallel_transfer_in_megabytes": 4, + "irods_user_name": "{{ IRODS_ADMIN_USER }}", + "irods_zone_name": "{{ IRODS_ZONE_NAME }}", + "schema_name": "irods_environment", + "schema_version": "v4", + {# Unsure if following lines are valid here #} + + {% if IRODS_SSL_CA_CERT_PATH|default(false) %} + "irods_ssl_ca_certificate_file": "{{ IRODS_SSL_CA_CERT_PATH }}", + {% endif %} + + "irods_ssl_certificate_chain_file": "{{ IRODS_SSL_CERTIFICATE_CHAIN_FILE }}", + "irods_ssl_certificate_key_file": "{{ IRODS_SSL_CERTIFICATE_KEY_FILE }}", + "irods_ssl_dh_params_file": "{{ IRODS_SSL_DH_PARAMS_FILE }}", + "irods_ssl_verify_server": "{{ IRODS_SSL_VERIFY_SERVER }}" + }, "server_config": { "advanced_settings": { "default_log_rotation_in_days": 5, "default_number_of_transfer_threads": 4, "default_temporary_password_lifetime_in_seconds": 120, - "maximum_number_of_concurrent_rule_engine_server_processes": 4, + "delay_rule_executors": [ + {% if IRODS_CATALOG_PROVIDER_HOST %} + "{{ IRODS_CATALOG_PROVIDER_HOST }}" + {% else %} + "{{ IRODS_HOST_NAME }}" + {% endif %} + ], + "delay_server_sleep_time_in_seconds": 30, + "dns_cache": { + "eviction_age_in_seconds": 3600, + "shared_memory_size_in_bytes": 5000000 + }, + "hostname_cache": { + "eviction_age_in_seconds": 3600, + "shared_memory_size_in_bytes": 2500000 + }, "maximum_size_for_single_buffer_in_megabytes": 32, + "maximum_size_of_delay_queue_in_bytes": 0, "maximum_temporary_password_lifetime_in_seconds": 1000, - "rule_engine_server_execution_time_in_seconds": 120, - "rule_engine_server_sleep_time_in_seconds": 30, + "number_of_concurrent_delay_rule_executors": 4, + "stacktrace_file_processor_sleep_time_in_seconds": 10, "transfer_buffer_size_for_parallel_transfer_in_megabytes": 4, "transfer_chunk_size_for_parallel_transfer_in_megabytes": 40 }, @@ -32,14 +80,41 @@ {% endif %} ], "catalog_service_role": "{{ IRODS_ROLE }}", + "client_api_allowlist_policy": "enforce", + "controlled_user_connection_list": { + "control_type": "denylist", + "users": [] + }, "default_dir_mode": "0750", "default_file_mode": "0600", "default_hash_scheme": "{{ IRODS_DEFAULT_HASH_SCHEME }}", "default_resource_name": "demoResc", "environment_variables": { + {# Not sure if this one is supported in 4.3? #} "IRODS_DATABASE_USER_PASSWORD_SALT": "{{ IRODS_PASSWORD_SALT }}" }, "federation": [], + "host_access_control": { + "access_entries": [] + }, + "host_resolution": { + "host_entries": [] + }, + "log_level": { + "agent": "info", + "agent_factory": "info", + "api": "info", + "authentication": "info", + "database": "info", + "delay_server": "info", + "legacy": "info", + "microservice": "info", + "network": "info", + "resource": "info", + "rule_engine": "info", + "server": "info", + "sql": "info" + }, "match_hash_policy": "compatible", "negotiation_key": "{{ IRODS_NEGOTIATION_KEY }}", "plugin_configuration": { @@ -58,29 +133,8 @@ "resource": {}, "rule_engines": [ { - "instance_name": "irods_rule_engine_plugin-irods_rule_language-instance", - "plugin_name": "irods_rule_engine_plugin-irods_rule_language", - "plugin_specific_configuration": { - "re_data_variable_mapping_set": [ - "core" - ], - "re_function_name_mapping_set": [ - "core" - ], - "re_rulebase_set": [ - "core" - ], - "regexes_for_supported_peps": [ - "ac[^ ]*", - "msi[^ ]*", - "[^ ]*pep_[^ ]*_(pre|post|except)" - ] - }, - "shared_memory_instance": "irods_rule_language_rule_engine" - }, - { - "instance_name": "irods_rule_engine_plugin-cpp_default_policy-instance", - "plugin_name": "irods_rule_engine_plugin-cpp_default_policy", + "instance_name": "irods_rule_engine_plugin-python-instance", + "plugin_name": "irods_rule_engine_plugin-python", "plugin_specific_configuration": {} } ] @@ -90,7 +144,7 @@ ], "schema_name": "server_config", "schema_validation_base_uri": "file:///var/lib/irods/configuration_schemas", - "schema_version": "v3", + "schema_version": "v4", "server_control_plane_encryption_algorithm": "AES-256-CBC", "server_control_plane_encryption_num_hash_rounds": 16, "server_control_plane_key": "{{ IRODS_CONTROL_PLANE_KEY }}", @@ -99,241 +153,10 @@ "server_port_range_start": {{ IRODS_DATA_PORT_RANGE_START }}, "server_port_range_end": {{ IRODS_DATA_PORT_RANGE_END }}, "xmsg_port": 1279, - "zone_auth_scheme": "{{ IRODS_AUTHENTICATION_SCHEME }}", + "zone_auth_scheme": "native", "zone_key": "{{ IRODS_ZONE_KEY }}", "zone_name": "{{ IRODS_ZONE_NAME }}", "zone_port": {{ IRODS_ZONE_PORT }}, "zone_user": "{{ IRODS_ADMIN_USER }}" - }, - "service_account_environment": { - "irods_authentication_scheme": "native", - "irods_client_server_negotiation": "{{ IRODS_CLIENT_SERVER_NEGOTIATION }}", - "irods_client_server_policy": "{{ IRODS_CLIENT_SERVER_POLICY }}", - "irods_connection_pool_refresh_time_in_seconds": 300, - "irods_cwd": "/{{ IRODS_ZONE_NAME }}/home/rods", - "irods_default_hash_scheme": "{{ IRODS_DEFAULT_HASH_SCHEME }}", - "irods_default_number_of_transfer_threads": 4, - "irods_default_resource": "demoResc", - "irods_encryption_algorithm": "AES-256-CBC", - "irods_encryption_key_size": 32, - "irods_encryption_num_hash_rounds": 16, - "irods_encryption_salt_size": 8, - "irods_home": "/{{ IRODS_ZONE_NAME }}/home/rods", - "irods_host": "{{ IRODS_HOST_NAME }}", - "irods_match_hash_policy": "compatible", - "irods_maximum_size_for_single_buffer_in_megabytes": 32, - "irods_port": {{ IRODS_ZONE_PORT }}, - "irods_server_control_plane_encryption_algorithm": "AES-256-CBC", - "irods_server_control_plane_encryption_num_hash_rounds": 16, - "irods_server_control_plane_key": "{{ IRODS_CONTROL_PLANE_KEY }}", - "irods_server_control_plane_port": {{ IRODS_CONTROL_PLANE_PORT|default(1248) }}, - - {% if IRODS_SSL_CA_CERT_PATH|default(false) %} - "irods_ssl_ca_certificate_file": "{{ IRODS_SSL_CA_CERT_PATH }}", - {% endif %} - - "irods_ssl_certificate_chain_file": "{{ IRODS_SSL_CERTIFICATE_CHAIN_FILE }}", - "irods_ssl_certificate_key_file": "{{ IRODS_SSL_CERTIFICATE_KEY_FILE }}", - "irods_ssl_dh_params_file": "{{ IRODS_SSL_DH_PARAMS_FILE }}", - "irods_ssl_verify_server": "{{ IRODS_SSL_VERIFY_SERVER }}", - "irods_transfer_buffer_size_for_parallel_transfer_in_megabytes": 4, - "irods_user_name": "{{ IRODS_ADMIN_USER }}", - "irods_zone_name": "{{ IRODS_ZONE_NAME }}", - "schema_name": "irods_environment", - "schema_version": "v3" - }, - "hosts_config": { - "host_entries": [], - "schema_name": "hosts_config", - "schema_version": "v3" - }, - "host_system_information": { - "service_account_user_name": "{{ IRODS_SERVICE_ACCOUNT_USER }}", - "service_account_group_name": "{{ IRODS_SERVICE_ACCOUNT_GROUP }}" - {# TODO begin #} - {# "hostname": "sodar-irods-x" #} - {# TODO end #} - }, - "host_access_control_config": { - "schema_name": "host_access_control_config", - "schema_version": "v3", - "access_entries": [] - }, - "plugins": [ - { - "version": "", - "checksum_sha256": "", - "type": "resource", - "name": "passthru" - }, - { - "version": "", - "checksum_sha256": "", - "type": "resource", - "name": "univmss" - }, - { - "version": "", - "checksum_sha256": "", - "type": "resource", - "name": "unixfilesystem" - }, - { - "version": "", - "checksum_sha256": "", - "type": "resource", - "name": "random" - }, - { - "version": "", - "checksum_sha256": "", - "type": "resource", - "name": "mockarchive" - }, - { - "version": "", - "checksum_sha256": "", - "type": "resource", - "name": "deferred" - }, - { - "version": "", - "checksum_sha256": "", - "type": "resource", - "name": "nonblocking" - }, - { - "version": "", - "checksum_sha256": "", - "type": "resource", - "name": "replication" - }, - { - "version": "", - "checksum_sha256": "", - "type": "resource", - "name": "load_balanced" - }, - { - "version": "", - "checksum_sha256": "", - "type": "resource", - "name": "roundrobin" - }, - { - "version": "", - "checksum_sha256": "", - "type": "resource", - "name": "structfile" - }, - { - "version": "", - "checksum_sha256": "", - "type": "resource", - "name": "compound" - }, - { - "version": "", - "checksum_sha256": "", - "type": "authentication", - "name": "native_client" - }, - { - "version": "", - "checksum_sha256": "", - "type": "authentication", - "name": "native_server" - }, - { - "version": "", - "checksum_sha256": "", - "type": "authentication", - "name": "pam_server" - }, - { - "version": "", - "checksum_sha256": "", - "type": "authentication", - "name": "osauth_client" - }, - { - "version": "", - "checksum_sha256": "", - "type": "authentication", - "name": "pam_client" - }, - { - "version": "", - "checksum_sha256": "", - "type": "authentication", - "name": "osauth_server" - }, - { - "version": "", - "checksum_sha256": "", - "type": "network", - "name": "ssl_client" - }, - { - "version": "", - "checksum_sha256": "", - "type": "network", - "name": "ssl_server" - }, - { - "version": "", - "checksum_sha256": "", - "type": "network", - "name": "tcp_server" - }, - { - "version": "", - "checksum_sha256": "", - "type": "network", - "name": "tcp_client" - }, - { - "version": "", - "checksum_sha256": "", - "type": "api", - "name": "helloworld_server" - }, - { - "version": "", - "checksum_sha256": "", - "type": "api", - "name": "helloworld_client" - }, - { - "version": "", - "checksum_sha256": "", - "type": "microservice", - "name": "msi_update_unixfilesystem_resource_free_space" - }, - { - "version": "", - "checksum_sha256": "", - "type": "microservice", - "name": "msisync_to_archive" - } - ], - "required": [ - "admin_password", - "host_access_control_config", - "host_system_information", - "hosts_config", - "server_config", - "service_account_environment" - ], - "configuration_directory": { - "path": "/etc/irods", - "files": { - "/etc/irods/server.key": "{{ ins_key | b64encode }}", - "/etc/irods/server.crt": "{{ ins_crt | b64encode }}", - "/etc/irods/core.dvm": "{{ ins_core_dvm | b64encode }}", - "/etc/irods/core.re": "{{ ins_core_re | b64encode }}", - "/etc/irods/core.fnm": "{{ ins_core_fnm | b64encode }}", - "/etc/irods/service_account.config": "{{ ins_service_account_config | b64encode }}" - } } }