master keycloak:449 add keycloak as IdP for doil managed ILIAS Instances

* add keycloak commands (down/login/restart/up)
* add saml states (enable-saml/disable-saml)
* add keycloak state
* add keycloak template
* update CHANGELOG
* update README
* add an update script
* tested so far on my local host
* adjust tests

CHANGELOG.md

@@ -1,5 +1,48 @@
 # Changelog
 
+## 20241113
+## What's Changed
+* add keycloak server
+* add keycloak commands (down/login/restart/up)
+* add saml states (enable-saml/disable-saml)
+* add new state 'prevent-super-global-replacement'
+
+## 20241104
+## What's Changed
+* update salt urls
+
+## 20241010
+## What's Changed
+* execute proxy command only if instance is up
+* add 'Options -Indexes' to default site
+
+## 20240930
+## What's Changed
+* add captainhook states
+* longer sleep between 'docker commit' and 'settinsg premissions'
+* use correct pathes during import command
+
+## 20240926
+## What's Changed
+* php: add version 8.3
+
+## 20240902
+## What's Changed
+* add support for ilias 10 exports
+* also export branches with no matching doil repo
+
+## 20240807
+## What's Changed
+* change path for minion_master.pub
+
+## 20240806
+## What's Changed
+* move nodejs step before composer step and change execution dir in nodejs state
+
+## 20240801
+## What's Changed
+* fix problems with the ILIAS database password
+
 ## 20240628
 ## What's Changed
 * use Salt Repos for salt-master and salt-minion

CI/validate-sls-files.sh

@@ -21,9 +21,11 @@ do
 	TMPFILE="${FILE}.tmp"
 	touch $TMPFILE
 
-    echo "---" > ${TMPFILE}
+  echo "---" > ${TMPFILE}
 	sed 's/{%.*%}//' ${FILE} >> ${TMPFILE}
-    sed 's/{{.*}}/bar/' ${TMPFILE} > ${TMPFILE}.tmp
+  sed '/%[A-Z].*%/d' ${TMPFILE} >> ${TMPFILE}.1
+  sed 's/{{.*}}/bar/' ${TMPFILE}.1 > ${TMPFILE}.tmp
+  rm ${TMPFILE}.1
 	mv ${TMPFILE}.tmp ${TMPFILE}
 
 	TEST=$(yamllint -c ./CI/sls-lint-rules.yml ${FILE}.tmp)

README.md

@@ -308,8 +308,8 @@ users, so make sure to understand what you are doing.
 
 * `doil salt:login` logs the user into the main salt server
 * `doil salt:prune` prunes the main salt server
-* `doil salt:start` starts the salt main server
-* `doil salt:stop` stops the salt main server
+* `doil salt:up` starts the salt main server
+* `doil salt:down` stops the salt main server
 * `doil salt:restart` restarts the salt main server
 * `doil salt:states` to list the available states
 
@@ -324,8 +324,8 @@ users, so make sure to understand what you are doing.
 
 * `doil proxy:login` logs the user into the proxy server
 * `doil proxy:prune` removes the configuration of the proxy server
-* `doil proxy:start` starts the proxy server
-* `doil proxy:stop` stops the proxy server
+* `doil proxy:up` starts the proxy server
+* `doil proxy:down` stops the proxy server
 * `doil proxy:restart` restarts the proxy server
 * `doil proxy:reload` reloads the configuration
 
@@ -348,6 +348,7 @@ The state also sets up a cron job that regularly renews the certificates.
 
 After that please ensure to run `doil apply <instance_name> enable-https` on each doil ILIAS instance,
 so https take effect in ILIAS.
+
 ### Mail Server
 
 The mailserver is available at `http://doil/mails` with following
@@ -372,10 +373,63 @@ users, so make sure to understand what you are doing.
 
 * `doil mail:change-password` changes the default password for roundcube
 * `doil mail:login` logs the user into the mail server
-* `doil mail:start` starts the mail server
-* `doil mail:stop` stops the mail server
+* `doil mail:up` starts the mail server
+* `doil mail:down` stops the mail server
 * `doil mail:restart` restarts the mail server
 
+### Keycloak Server
+
+The Keycloak server is an identity provider that allows you to log in to all 
+ILIAS instances managed by **doil** with one password.  
+This requires some settings in the doil.conf file. 'doil.conf' can be found 
+under setup/conf/doil.conf. The adjustments must be made before an update/install.
+
+The following settings are available:
+
+* `enable_keycloak=[true/false]`  decides whether keycloak is installed during 
+an update/install [default:false]
+* `keycloak_hostname=http://doil/keycloak` keycloak url, please pay attention to https/http
+* `keycloak_new_admin_password=12345` admin password
+* `keycloak_old_admin_password=admin` If the password is changed during an update, the old 
+password must be entered here. Please make sure to adjust it after the update. For the first
+installation this has to be 'admin'.
+* `keycloak_db_username=admin` database user name
+* `keycloak_db_password=admin` database user password
+
+If you use keycloak, the salt state enable-saml must be called for existing ILIAS instances.
+This is done using the 'doil apply <instance_name>' command.  
+Newly created instances check whether keycloak is enabled and set up the instance directly.
+
+In order to use SAML for an Ilias instance, it must be ensured that a user is created in the
+ILIAS interface and a user in the Keycloak interface.
+
+#### Create a user in Keycloak
+* select tab 'users' from left menu
+* click 'Add user'
+* enter a Username
+* enter an Email
+* click 'Create'
+
+#### Cretae a user in ILIAS
+* select tab 'Administration' from left menu
+* select 'Users and Roles'
+* select 'User Management'
+* click 'Add User'
+* fill in the required fields (username must be the same as in keycloak)
+* set 'External Account' to the same email as in keycloak
+
+To be able to dive deeper into the inner workings of **doil** or customize it
+to fit your workflow or requirements, **doil** provides commands to tamper with
+the keycloak in the background. These commands will not be required by ordinary
+users, so make sure to understand what you are doing.
+
+* `doil keycloak:login` logs the user into the keycloak server
+* `doil keycloak:up` starts the keycloak server
+* `doil keycloak:down` stops the keycloak server
+* `doil keycloak:restart` restarts the keycloak server
+
+See `doil keycloak:<command> --help` for more information
+
 ### xdedug
 
 **doil** provides two options to enable xdebug for the given instance.  
@@ -450,4 +504,12 @@ If doil saved your precious time and brain power, please consider supporting
   publicly.
 * Reach out to [Richard]([email protected]) if you need
   more support than we can offer for free or want to get involved with **doil**
-  in other ways.
+  in other ways.
+
+### Prevent Super Globals Replacement
+Since ILIAS version 8 it is necessary to set the setting 'prevent_super_global_replacement = 1' in the
+client.ini.php. **doil** offers a state for this.
+```bash
+doil apply <instance_name> prevent-super-global-replacement
+```
+As of **doil** version 20241113, **doil** applies this state independently to newly created instances.

app/src/App.php

@@ -9,7 +9,7 @@
 
 class App extends Application
 {
-    const NAME = "Doil Version 20241010 - build 2024-10-10";
+    const NAME = "Doil Version 20241113 - build 2024-11-13";
 
     public function __construct(Command ...$commands)
     {

app/src/Commands/Instances/ApplyCommand.php

@@ -33,7 +33,8 @@ class ApplyCommand extends Command
         "reactor",
         "change-roundcube-password",
         "nodejs",
-        "proxy-enable-https"
+        "proxy-enable-https",
+        "keycloak"
     ];
 
     protected static $defaultName = "instances:apply";

app/src/Commands/Instances/CreateCommand.php

@@ -31,18 +31,20 @@ class CreateCommand extends Command
     protected const GLOBAL_REPO_PATH = "/usr/local/share/doil/repositories";
     protected const LOCAL_INSTANCES_PATH = "/.doil/instances";
     protected const GLOBAL_INSTANCES_PATH = "/usr/local/share/doil/instances";
+    protected const KEYCLOAK_PATH = "/usr/local/lib/doil/server/keycloak";
     protected const BASIC_FOLDERS = [
         "/conf",
         "/conf/salt",
         "/volumes/db",
         "/volumes/index",
         "/volumes/data",
+        "/volumes/cert",
         "/volumes/logs/error",
         "/volumes/logs/apache",
         "/volumes/etc/apache2",
         "/volumes/logs/xdebug",
         "/volumes/etc/php",
-        "/volumes/etc/mysql",
+        "/volumes/etc/mysql"
     ];
 
     protected static $defaultName = "instances:create";
@@ -103,12 +105,14 @@ public function execute(InputInterface $input, OutputInterface $output) : int
     {
         $options = $this->gatherOptionData($input, $output);
 
+        $host = explode("=", $this->filesystem->getLineInFile("/etc/doil/doil.conf", "host"))[1];
         $instance_path = $options["target"] . "/" . $options["name"];
         $suffix = $options["global"] ? "global" : "local";
         $instance_name = $options["name"] . "_" . $suffix;
         $instance_salt_name = $options["name"] . "." . $suffix;
         $user_name = $this->posix->getCurrentUserName();
         $home_dir = $this->posix->getHomeDirectory($this->posix->getUserId());
+        $keycloak = false;
 
         if ($this->filesystem->exists($instance_path)) {
             $this->writer->error(
@@ -127,6 +131,10 @@ public function execute(InputInterface $input, OutputInterface $output) : int
             return Command::FAILURE;
         }
 
+        if ($this->filesystem->exists(self::KEYCLOAK_PATH)) {
+            $keycloak = true;
+        }
+
         $this->writer->beginBlock($output, "Creating instance " . $options['name']);
 
         if (isset($options["repo_path"]) && ! $this->filesystem->exists($options["repo_path"])) {
@@ -254,13 +262,19 @@ public function execute(InputInterface $input, OutputInterface $output) : int
             "%TPL_PROJECT_DOMAINNAME%",
             $suffix
         );
+        $this->filesystem->replaceStringInFile(
+            $instance_path . "/docker-compose.yml",
+            "%TPL_HOST_DOMAIN%",
+            $host
+        );
         $this->writer->endBlock();
 
         // building minion image
         $this->writer->beginBlock($output, "Building minion image");
         $this->docker->runContainer($instance_name);
         $usr_id = (string) $this->posix->getUserId();
         $group_id = (string) $this->posix->getGroupId();
+        $this->docker->executeDockerCommand($instance_name, "mkdir -p /var/ilias/cert");
         $this->docker->executeDockerCommand($instance_name, "usermod -u $usr_id www-data");
         $this->docker->executeDockerCommand($instance_name, "groupmod -g $group_id www-data");
         $this->docker->executeDockerCommand($instance_name, "/etc/init.d/mariadb start");
@@ -286,22 +300,33 @@ public function execute(InputInterface $input, OutputInterface $output) : int
         // set grains
         $this->writer->beginBlock($output, "Setting up instance configuration");
         $mysql_password = $this->generatePassword(16);
+
         $cron_password = "not-needed";
         if ($ilias_version < 9) {
             $cron_password = $this->generatePassword(16);
         }
-        $host = explode("=", $this->filesystem->getLineInFile("/etc/doil/doil.conf", "host"));
+
+        if ($keycloak) {
+            $samlpass = $this->generatePassword(33);
+            $this->docker->setGrain($instance_salt_name, "samlpass", "$samlpass");
+            sleep(1);
+            $samlsalt = $this->generatePassword(33);
+            $this->docker->setGrain($instance_salt_name, "samlsalt", "$samlsalt");
+            sleep(1);
+        }
+
         $this->docker->setGrain($instance_salt_name, "mpass", "$mysql_password");
         sleep(1);
         $this->docker->setGrain($instance_salt_name, "cpass", "$cron_password");
         sleep(1);
-        $this->docker->setGrain($instance_salt_name, "doil_domain", "http://" . $host[1] . "/" . $options["name"]);
+        $this->docker->setGrain($instance_salt_name, "doil_domain", "http://" . $host . "/" . $options["name"]);
         sleep(1);
         $this->docker->setGrain($instance_salt_name, "doil_project_name", $options["name"]);
         sleep(1);
         $this->docker->setGrain($instance_salt_name, "doil_host_system", "linux");
         sleep(1);
         $this->docker->setGrain($instance_salt_name, "ilias_version", $ilias_version);
+        sleep(1);
         $this->docker->executeDockerCommand("doil_saltmain", "salt \"" . $instance_salt_name . "\" saltutil.refresh_grains");
         $this->writer->endBlock();
 
@@ -378,6 +403,20 @@ public function execute(InputInterface $input, OutputInterface $output) : int
         $this->docker->applyState($instance_salt_name, "enable-captainhook");
         $this->writer->endBlock();
 
+        if ($ilias_version >= 8.0) {
+            // apply prevent_super_global_replacement state
+            $this->writer->beginBlock($output, "Apply prevent_super_global_replacement state");
+            $this->docker->applyState($instance_salt_name, "prevent-super-global-replacement");
+            $this->writer->endBlock();
+        }
+
+        if ($keycloak) {
+            // apply enable-saml state
+            $this->writer->beginBlock($output, "Apply enable-saml state");
+            $this->docker->applyState($instance_salt_name, "enable-saml");
+            $this->writer->endBlock();
+        }
+
         // apply access state
         $this->writer->beginBlock($output, "Apply access state");
         $this->docker->applyState($instance_salt_name, "access");

app/src/Commands/Instances/DeleteCommand.php

@@ -20,7 +20,7 @@ class DeleteCommand extends Command
 {
     protected const SALT_MAIN = "/usr/local/lib/doil/server/salt/";
     protected const POSTFIX = "/usr/local/lib/doil/server/mail/";
-    protected const PROXY_PATH = "/usr/local/lib/doil/server/proxy/";
+    protected const KEYCLOAK_PATH = "/usr/local/lib/doil/server/keycloak";
 
     protected static $defaultName = "instances:delete";
     protected static $defaultDescription =
@@ -132,20 +132,28 @@ protected function deleteInstance(
     ) : int {
         $this->writer->beginBlock($output, "Delete instance $instance");
 
+        $is_up = $this->docker->isInstanceUp($path);
         $instance_dir = $this->filesystem->readLink($path);
         $this->filesystem->remove($path);
         $this->filesystem->remove($instance_dir);
 
         $this->docker->removeContainer($instance . "_" . $suffix);
 
         $this->docker->executeCommand(self::SALT_MAIN, "doil_saltmain", "salt-key", "-d", "$instance.$suffix", "-y", "-q");
-        if ($this->docker->isInstanceUp($path)) {
+        if ($is_up) {
             $this->docker->executeDockerCommand(
                 "doil_proxy",
                 "rm -f /etc/nginx/conf.d/sites/$instance.conf &&  /root/generate_index_html.sh"
             );
         }
 
+        if ($this->filesystem->exists(self::KEYCLOAK_PATH)) {
+            $this->docker->executeDockerCommand(
+                "doil_keycloak",
+                "/root/delete_keycloak_client.sh $instance"
+            );
+        }
+
         if ($this->docker->hasVolume($instance)) {
             $this->docker->removeVolume($instance);
         }

app/src/Commands/Instances/StatusCommand.php

@@ -37,6 +37,7 @@ public function execute(InputInterface $input, OutputInterface $output) : int
                 strstr($a, "doil_mail") ||
                 strstr($a, "doil_proxy") ||
                 strstr($a, "doil_saltmain") ||
+                strstr($a, "doil_keycloak") ||
                 strstr($a, "_local") ||
                 strstr($a, "_global")
             ) {