HXOR-1514 Use AIMS2/KeycloakX

.pre-commit-config.yaml

@@ -15,6 +15,7 @@ repos:
         args: [--allow-multiple-documents]
         exclude: helm/.*/templates
       - id: check-json
+        exclude: helm/alfresco-process-infrastructure/alfresco-realm.json
       - id: check-merge-conflict
       - id: fix-byte-order-marker
       - id: mixed-line-ending

README.md

@@ -34,7 +34,9 @@ Install the latest version of helm.
 An `ingress-nginx` should be installed and bound to an external DNS address, for example:
 
 ```
-helm upgrade --install --wait --repo https://kubernetes.github.io/ingress-nginx ingress-nginx ingress-nginx
+helm upgrade -i ingress-nginx ingress-nginx \
+  --repo https://kubernetes.github.io/ingress-nginx \
+  -n ingress-nginx --create-namespace
 ```
 
 ### helm tips
@@ -45,14 +47,15 @@ To install from the development chart repo, use `alfresco-incubator` rather than
 
 ### kubectl tips
 
-Check deployment progress with `kubectl get pods --watch --all-namespaces` until all containers are running.
-If anything is stuck, check events with `kubectl get events --watch`.
+Check deployment progress with `kubectl get pods -w -A` until all containers are running.
+If anything is stuck, check events with `kubectl get events -w -A`.
 
 
 ### configure installation namespace
 
 ```bash
 export DESIRED_NAMESPACE=${DESIRED_NAMESPACE:-aae}
+kubectl create ns $DESIRED_NAMESPACE
 ```
 
 ### add quay-registry-secret
@@ -61,7 +64,7 @@ Configure access to pull images from quay.io in the installation namespace:
 
 ```bash
 kubectl create secret \
-  --namespace $DESIRED_NAMESPACE \
+  -n $DESIRED_NAMESPACE \
   docker-registry quay-registry-secret \
     --docker-server=quay.io \
     --docker-username=$QUAY_USERNAME \
@@ -76,52 +79,46 @@ where:
 ### set main helm env variables
 
 ```bash
-export HELM_OPTS+=" --debug \
-  --namespace $DESIRED_NAMESPACE \
-  --set global.gateway.http=$HTTP \
-  --set global.gateway.domain=$DOMAIN"
+export RELEASE_NAME=aae
+export CHART_NAME=alfresco-process-infrastructure
+export HELM_OPTS="-n $DESIRED_NAMESPACE"
 ```
 
-where:
-
-* _HTTP_ is true/false depending if you want external URLs using HTTP or HTTPS
-* _DOMAIN_ is your DNS domain
-
-
 ### set environment specific variables
 
 #### for localhost
 
+A custom extra values file to add settings for _localhost_ is provided:
 ```bash
-export PROTOCOL=http
 export DOMAIN=host.docker.internal
+HELM_OPTS+=" -f values-localhost.yaml"
 ```
 
-*NB* add to your `/etc/hosts` the line `127.0.0.1 host.docker.internal` if not present
+Make sure your local cluster has at least 16GB of memory and 8 CPUs.
+The startup might take as much as 10 minutes, use `kubectl get pods -A -w` to check the status.
 
-#### for a cloud environment
+*NB* if not already present in your `/etc/hosts` file, please add a DNS mapping from `host.docker.internal` to `127.0.0.1`.
 
+If the hostname `host.docker.internal` is not resolved correctly on some deployments, patch them after calling helm via:
 ```bash
-export CLUSTER=aaedev
-export PROTOCOL=https
-export DOMAIN=$CLUSTER.envalfresco.com
+kubectl patch deployment -n $DESIRED_NAMESPACE ${RELEASE_NAME}-alfresco-modeling-service -p "$(cat deployment-localhost-patch.yaml)"
 ```
 
-#### set generated variables
+#### for a cloud environment
 
 ```bash
-export GATEWAY_HOST=$DOMAIN
-export SSO_HOST=$DOMAIN
+export CLUSTER=aaedev
+export DOMAIN=$CLUSTER.envalfresco.com
 ```
 
 ### set helm env variables
 
 ```bash
-export HTTP=$(if [[ "$PROTOCOL" == 'http' ]]; then echo true; else echo false; fi)
-HELM_OPTS+=" --set global.gateway.http=$HTTP \
+HELM_OPTS+=" \
   --set global.gateway.domain=$DOMAIN"
 ```
 
+
 ### disable alfresco-deployment-service
 
 To disable alfresco-deployment-service in the infrastructure:
@@ -132,7 +129,7 @@ HELM_OPTS+="
 "
 ```
 
-## Multi-AZ K8S cloud StorageClass for project release
+### Multi-AZ K8S cloud StorageClass for project release
 
 A StorageClass that can work across multiple availability zones need to be available to store project release files per each application:
 * for EKS always use EFS
@@ -163,28 +160,30 @@ HELM_OPTS+="
 "
 ```
 
-## launch helm
+### debug and dry run
 
-Set install parameters:
+To verify the k8s yaml output:
 
 ```bash
-export RELEASE_NAME=aae
-export CHART_NAME=alfresco-process-infrastructure
+HELM_OPTS+=" --debug --dry-run"
 ```
 
-then install from the stable repo using a released chart version:
+If all good then launch again without `--dry-run`.
+
+## launch helm
+
+Install from the stable repo using a released chart version:
 
 ```bash
-helm upgrade --install --wait \
+helm upgrade -i --wait \
   --repo https://kubernetes-charts.alfresco.com/stable \
   $HELM_OPTS $RELEASE_NAME $CHART_NAME
 ```
 
-or from the incubator repo a development chart version:
+or from the incubator repo for a development chart version:
 
 ```bash
-helm upgrade --install --wait \
-  --namespace $DESIRED_NAMESPACE \
+helm upgrade -i --wait \
   --repo https://kubernetes-charts.alfresco.com/incubator \
   $HELM_OPTS $RELEASE_NAME $CHART_NAME
 ```
@@ -193,54 +192,26 @@ or from the current repository directory:
 
 ```bash
 helm repo update
-helm dependency update helm/${CHART_NAME}
-helm upgrade --install --wait \
-  --namespace $DESIRED_NAMESPACE \
+helm dependency update helm/$CHART_NAME
+helm upgrade -i --wait \
   $HELM_OPTS $RELEASE_NAME helm/$CHART_NAME
 ```
 
-## Extra Helm install scripts
-
-Both support the following optional vars:
-
-* RELEASE_NAME to handle upgrade or a non auto-generated release name
-* HELM_OPTS to pass extra options to helm
-
-### install.sh
-
-Just install/upgrade the AAE infrastructure.
-
-To verify the k8s yaml output:
-
-```bash
-HELM_OPTS+="--debug --dry-run" ./install.sh
-```
-
-Verify the k8s yaml output than launch again without `--dry-run`.
-
-### run on localhost
-
-A custom extra values file to add settings for _localhost_ is provided:
-```bash
-HELM_OPTS+=" -f values-localhost.yaml" ./install.sh
-```
-*NB* the startup might take as much as 10 minutes, use ```kubectl get pods -A -w``` to check the status.
-
 ## Testing
 
 ### Access IDS
 
 Open browser and login to IDS:
 ```bash
-open ${SSO_URL}
+open $SSO_URL
 ```
 
 ### Verify Realm
 
 To read back the realm from the secret, use:
 ```bash
 kubectl get secret \
-  --namespace $DESIRED_NAMESPACE \
+  -n $DESIRED_NAMESPACE \
   realm-secret -o jsonpath="{['data']['alfresco-realm\.json']}" | base64 -D > alfresco-realm.json
 ```
 
@@ -254,7 +225,7 @@ In an [air gapped](https://en.wikipedia.org/wiki/Air_gap_(networking)) environme
 Modify the file values-external-postgresql.yaml providing values for your external database per each service, then run:
 
 ```bash
-export HELM_OPTS+=" -f values-external-postgresql.yaml"
+HELM_OPTS+=" -f values-external-postgresql.yaml"
 ```
 
 ## CI/CD

deployment-localhost-patch.yaml

@@ -0,0 +1,7 @@
+spec:
+  template:
+    spec:
+      hostAliases:
+        - ip: "192.168.5.2"
+          hostnames:
+            - "aaa.local"

helm/alfresco-process-infrastructure/Chart.yaml

@@ -17,22 +17,23 @@ dependencies:
   - name: common
     version: 8.6.0-alpha.9
     repository: https://activiti.github.io/activiti-cloud-helm-charts
-  - name: alfresco-identity-service
-    version: 6.0.0
-    repository: https://kubernetes-charts.alfresco.com/stable
+  - name: keycloakx
+    version: 2.3.0
+    repository: https://codecentric.github.io/helm-charts
+    alias: alfresco-identity-service
     condition: alfresco-identity-service.enabled
   - name: common
     version: 8.6.0-alpha.9
     repository: https://activiti.github.io/activiti-cloud-helm-charts
     alias: alfresco-modeling-service
     condition: alfresco-modeling-service.enabled
   - name: postgresql
-    version: 10.3.13
-    repository: https://raw.githubusercontent.com/bitnami/charts/archive-full-index/bitnami/
+    version: 15.5.11
+    repository: https://raw.githubusercontent.com/bitnami/charts/archive-full-index/bitnami
     condition: postgresql.enabled
   - name: rabbitmq
-    version: 8.20.5
-    repository: https://raw.githubusercontent.com/bitnami/charts/archive-full-index/bitnami/
+    version: 14.4.4
+    repository: https://raw.githubusercontent.com/bitnami/charts/archive-full-index/bitnami
     condition: rabbitmq.enabled
   - name: common
     version: 8.6.0-alpha.9

helm/alfresco-process-infrastructure/README.md

@@ -26,10 +26,10 @@ Kubernetes: `>=1.15.0-0`
 | https://activiti.github.io/activiti-cloud-helm-charts | alfresco-deployment-service(common) | 8.6.0-alpha.9 |
 | https://activiti.github.io/activiti-cloud-helm-charts | alfresco-process-analytics-playground(common) | 8.6.0-alpha.9 |
 | https://activiti.github.io/activiti-cloud-helm-charts | common | 8.6.0-alpha.9 |
-| https://kubernetes-charts.alfresco.com/stable | alfresco-identity-service | 6.0.0 |
+| https://codecentric.github.io/helm-charts | alfresco-identity-service(keycloakx) | 2.3.0 |
 | https://opensearch-project.github.io/helm-charts | opensearch | 1.11.1 |
-| https://raw.githubusercontent.com/bitnami/charts/archive-full-index/bitnami/ | postgresql | 10.3.13 |
-| https://raw.githubusercontent.com/bitnami/charts/archive-full-index/bitnami/ | rabbitmq | 8.20.5 |
+| https://raw.githubusercontent.com/bitnami/charts/archive-full-index/bitnami | postgresql | 15.5.11 |
+| https://raw.githubusercontent.com/bitnami/charts/archive-full-index/bitnami | rabbitmq | 14.4.4 |
 
 ## Values
 
@@ -167,28 +167,31 @@ Kubernetes: `>=1.15.0-0`
 | alfresco-identity-adapter-service.probePath | string | `"/actuator/health"` |  |
 | alfresco-identity-adapter-service.rabbitmq.enabled | bool | `false` |  |
 | alfresco-identity-adapter-service.replicaCount | int | `2` |  |
+| alfresco-identity-service.command[0] | string | `"/opt/keycloak/bin/kc.sh"` |  |
+| alfresco-identity-service.command[1] | string | `"start"` |  |
+| alfresco-identity-service.command[2] | string | `"--http-enabled=true"` |  |
+| alfresco-identity-service.command[3] | string | `"--http-port=8080"` |  |
+| alfresco-identity-service.command[4] | string | `"--hostname-strict=false"` |  |
+| alfresco-identity-service.command[5] | string | `"--hostname-strict-https=false"` |  |
+| alfresco-identity-service.command[6] | string | `"--import-realm"` |  |
 | alfresco-identity-service.enabled | bool | `true` |  |
-| alfresco-identity-service.extraEnv | string | `"- name: KEYCLOAK_USER\n  value: admin\n- name: KEYCLOAK_PASSWORD\n  value: admin\n- name: KEYCLOAK_IMPORT\n  value: /realm/alfresco-realm.json\n- name: PROXY_ADDRESS_FORWARDING\n  value: \"true\"\n"` |  |
-| alfresco-identity-service.ingress.enabled | bool | `false` |  |
-| alfresco-identity-service.keycloak.ingress.annotations."kubernetes.io/ingress.class" | string | `"nginx"` |  |
-| alfresco-identity-service.keycloak.ingress.annotations."nginx.ingress.kubernetes.io/affinity" | string | `"cookie"` |  |
-| alfresco-identity-service.keycloak.ingress.annotations."nginx.ingress.kubernetes.io/enable-cors" | string | `"false"` |  |
-| alfresco-identity-service.keycloak.ingress.annotations."nginx.ingress.kubernetes.io/proxy-buffer-size" | string | `"128k"` |  |
-| alfresco-identity-service.keycloak.ingress.annotations."nginx.ingress.kubernetes.io/session-cookie-hash" | string | `"sha1"` |  |
-| alfresco-identity-service.keycloak.ingress.annotations."nginx.ingress.kubernetes.io/session-cookie-name" | string | `"identity_affinity_route"` |  |
-| alfresco-identity-service.keycloak.ingress.enabled | bool | `true` |  |
-| alfresco-identity-service.keycloak.ingress.rules[0].host | string | `"{{ include \"common.keycloak-host\" . }}"` |  |
-| alfresco-identity-service.keycloak.ingress.rules[0].paths[0].path | string | `"/auth"` |  |
-| alfresco-identity-service.keycloak.ingress.rules[0].paths[0].pathType | string | `"Prefix"` |  |
-| alfresco-identity-service.keycloak.ingress.tls | list | `[]` |  |
-| alfresco-identity-service.keycloak.keycloak.image.tag | string | `"1.7.0"` |  |
-| alfresco-identity-service.keycloak.postgresql.image.tag | string | `"13.3.0"` |  |
-| alfresco-identity-service.keycloak.postgresql.persistence.existingClaim | string | `""` |  |
-| alfresco-identity-service.keycloak.postgresql.tls.enabled | bool | `false` |  |
-| alfresco-identity-service.rbac.create | bool | `false` |  |
+| alfresco-identity-service.extraEnv | string | `"- name: KEYCLOAK_ADMIN\n  value: admin\n- name: KEYCLOAK_ADMIN_PASSWORD\n  value: admin\n- name: JAVA_OPTS_APPEND\n  value: '-Djgroups.dns.query={{ include \"keycloak.fullname\" . }}-headless'\n- name: KC_HOSTNAME\n  value: '{{ include \"common.keycloak-host\" . }}'\n"` |  |
+| alfresco-identity-service.extraInitContainers | string | `"- name: theme-download\n  args:\n    - \"-c\"\n    - |\n      wget https://github.com/Alfresco/alfresco-keycloak-theme/releases/download/0.3.5/alfresco-keycloak-theme-0.3.5.zip -O alfresco.zip\n      unzip alfresco.zip\n      mv alfresco/* /theme/\n  command:\n    - \"sh\"\n  image: \"busybox:1.36\"\n  imagePullPolicy: IfNotPresent\n  volumeMounts:\n    - name: theme\n      mountPath: /theme\n"` |  |
+| alfresco-identity-service.extraVolumeMounts | string | `"- name: realm-secret\n  mountPath: /opt/keycloak/data/import\n  readOnly: true\n- name: theme\n  mountPath: /opt/keycloak/themes/alfresco\n  readOnly: true\n- name: h2-database\n  mountPath: /opt/keycloak/data/h2\n"` |  |
+| alfresco-identity-service.extraVolumes | string | `"- name: realm-secret\n  secret:\n    secretName: realm-secret\n- name: theme\n  emptyDir: {}\n- name: h2-database\n  persistentVolumeClaim:\n    claimName: keycloak-pvc\n"` |  |
+| alfresco-identity-service.ingress.annotations."nginx.ingress.kubernetes.io/proxy-buffer-size" | string | `"16k"` |  |
+| alfresco-identity-service.ingress.enabled | bool | `true` |  |
+| alfresco-identity-service.ingress.ingressClassName | string | `"nginx"` |  |
+| alfresco-identity-service.ingress.rules[0].host | string | `"{{ include \"common.keycloak-host\" . }}"` |  |
+| alfresco-identity-service.ingress.rules[0].paths[0].path | string | `"/auth"` |  |
+| alfresco-identity-service.ingress.rules[0].paths[0].pathType | string | `"Prefix"` |  |
+| alfresco-identity-service.ingress.tls | list | `[]` |  |
+| alfresco-identity-service.postgresql.enabled | bool | `false` |  |
+| alfresco-identity-service.realm.alfresco.adminPassword | string | `"admin"` |  |
 | alfresco-identity-service.realm.alfresco.client.redirectUris[0] | string | `"*"` |  |
 | alfresco-identity-service.realm.alfresco.client.webOrigins[0] | string | `"*"` |  |
 | alfresco-identity-service.realm.alfresco.extraClients[0].clientId | string | `"activiti"` |  |
+| alfresco-identity-service.realm.alfresco.extraClients[0].clientRoles | list | `[]` |  |
 | alfresco-identity-service.realm.alfresco.extraClients[0].directAccessGrantsEnabled | bool | `true` |  |
 | alfresco-identity-service.realm.alfresco.extraClients[0].enabled | bool | `true` |  |
 | alfresco-identity-service.realm.alfresco.extraClients[0].implicitFlowEnabled | bool | `true` |  |
@@ -424,7 +427,6 @@ Kubernetes: `>=1.15.0-0`
 | alfresco-identity-service.realm.alfresco.extraUsers[9].realmRoles[1] | string | `"uma_authorization"` |  |
 | alfresco-identity-service.realm.alfresco.extraUsers[9].realmRoles[2] | string | `"ACTIVITI_MODELER"` |  |
 | alfresco-identity-service.realm.alfresco.extraUsers[9].username | string | `"modeler"` |  |
-| alfresco-identity-service.serviceAccount.create | bool | `false` |  |
 | alfresco-modeling-app.affinity.podAntiAffinity.preferredDuringSchedulingIgnoredDuringExecution[0].podAffinityTerm.labelSelector.matchLabels."app.kubernetes.io/instance" | string | `"{{ .Release.Name }}"` |  |
 | alfresco-modeling-app.affinity.podAntiAffinity.preferredDuringSchedulingIgnoredDuringExecution[0].podAffinityTerm.labelSelector.matchLabels."app.kubernetes.io/name" | string | `"{{ template \"common.name\" . }}"` |  |
 | alfresco-modeling-app.affinity.podAntiAffinity.preferredDuringSchedulingIgnoredDuringExecution[0].podAffinityTerm.topologyKey | string | `"failure-domain.beta.kubernetes.io/zone"` |  |
@@ -675,36 +677,21 @@ Kubernetes: `>=1.15.0-0`
 | opensearch.extraEnvs[1].name | string | `"DISABLE_SECURITY_PLUGIN"` |  |
 | opensearch.extraEnvs[1].value | string | `"true"` |  |
 | opensearch.replicas | int | `2` |  |
-| postgresql.commonAnnotations.application | string | `"activiti"` |  |
+| postgresql.auth.password | string | `"alfresco"` |  |
+| postgresql.auth.username | string | `"alfresco"` |  |
+| postgresql.database | string | `"postgres"` |  |
 | postgresql.enabled | bool | `true` |  |
-| postgresql.image.tag | string | `"13.3.0"` |  |
-| postgresql.persistence.mountPath | string | `"/data"` |  |
-| postgresql.postgresqlDataDir | string | `"/data/pgdata"` |  |
-| postgresql.postgresqlDatabase | string | `"postgres"` |  |
-| postgresql.postgresqlExtendedConf.log_min_messages | string | `"LOG"` |  |
-| postgresql.postgresqlExtendedConf.max_connections | int | `300` |  |
-| postgresql.postgresqlPassword | string | `"alfresco"` |  |
-| postgresql.postgresqlUsername | string | `"alfresco"` |  |
+| postgresql.image.tag | string | `"11.22.0"` |  |
+| postgresql.persistence.mountPath | string | `"/bitnami/postgresql"` |  |
+| postgresql.postgresqlDataDir | string | `"/bitnami/postgresql/data"` |  |
+| postgresql.primary.extendedConfiguration | string | `"max_connections = 300\nlog_min_messages = LOG\n"` |  |
 | postgresql.resources.limits.memory | string | `"1500Mi"` |  |
 | postgresql.resources.requests.memory | string | `"1500Mi"` |  |
 | rabbitmq.auth.erlangCookie | string | `"ylY79lOdNUWsJEwAGdVQnhjSazV4QZKO="` |  |
 | rabbitmq.auth.password | string | `"CHANGEME"` |  |
 | rabbitmq.auth.username | string | `"user"` |  |
-| rabbitmq.customLivenessProbe.exec.command[0] | string | `"rabbitmq-diagnostics"` |  |
-| rabbitmq.customLivenessProbe.exec.command[1] | string | `"status"` |  |
-| rabbitmq.customLivenessProbe.initialDelaySeconds | int | `60` |  |
-| rabbitmq.customLivenessProbe.periodSeconds | int | `60` |  |
-| rabbitmq.customLivenessProbe.timeoutSeconds | int | `15` |  |
-| rabbitmq.customReadinessProbe.exec.command[0] | string | `"rabbitmq-diagnostics"` |  |
-| rabbitmq.customReadinessProbe.exec.command[1] | string | `"ping"` |  |
-| rabbitmq.customReadinessProbe.initialDelaySeconds | int | `20` |  |
-| rabbitmq.customReadinessProbe.periodSeconds | int | `60` |  |
 | rabbitmq.enabled | bool | `true` |  |
 | rabbitmq.extraPlugins | string | `""` |  |
-| rabbitmq.livenessProbe.enabled | bool | `false` |  |
-| rabbitmq.persistence.accessMode | string | `"ReadWriteOnce"` |  |
-| rabbitmq.persistence.storageClass | string | `nil` |  |
-| rabbitmq.readinessProbe.enabled | bool | `false` |  |
 | rabbitmq.resources.limits.memory | string | `"1500Mi"` |  |
 | rabbitmq.resources.requests.memory | string | `"1500Mi"` |  |
 | setup-acs-script-job.enabled | bool | `true` |  |