Skip to content

features domain broken on Tenant helm chart #1354

Jump to bottom
Allan Roger Reid edited this page Jan 19, 2023 · 12 revisions

Status: Open

Pull Request

Parse .spec.features.domains.minio array elements by allanrogerr · Pull Request #39 · allanrogerr/operator

Issues

features domain broken on Tenant helm chart · Issue #1354 · minio/operator

Steps

Reproduce using operator 4.5.8

1a. Clone https://github.com/allanrogerr/operator.git Checkout parse-tenant-domain and build Comment out "destroy_kind" from testing/deploy-tenant.sh Run testing/deploy-tenant.sh Once running, add to minio-operator deployment at .spec.template.spec.containers.image:

k -n minio-operator edit deployment/minio-operator

image: minio/operator:v4.5.8
image

b. Edit tenant under .spec.features

  features:
    domains:
      console: https://localhost:9443
      minio:
      - http://minio.default.svc.cluster.local
      - minio1.default.svc.cluster.local
      - https://test.default.svc.cluster.local
      - old.default.svc.cluster.local:9090
      - minio.default.svc.cluster.local
      - rbitrary.default.svc.cluster.local
      - default.svc.cluster.local

c. Run testing/deploy-tenant.sh

d. Observe

  • no overlapping domains are reported. default.svc.cluster.local should cause an overlap
  • strings with prefix https? are missing Tenant
  features:
    domains:
      console: https://localhost:9443
      minio:
      - http://minio.default.svc.cluster.local
      - minio1.default.svc.cluster.local
      - https://test.default.svc.cluster.local
      - old.default.svc.cluster.local:9090
      - minio.default.svc.cluster.local
      - rbitrary.default.svc.cluster.local
      - default.svc.cluster.local

Statefulset

        - name: MINIO_BROWSER_REDIRECT_URL
          value: https://localhost:9443
        - name: MINIO_DOMAIN
          value: ',minio.default.svc.cluster.local,test.default.svc.cluster.local,,,,'
        - name: MINIO_SERVER_URL
          value: default.svc.cluster.local

e. Observe pods do not start:

➜  operator git:(parse-tenant-domain) ✗ k -n tenant-lite get pods                            
NAME                                           READY   STATUS             RESTARTS       AGE
storage-lite-log-0                             1/1     Running            0              40m
storage-lite-log-search-api-5685f95857-xx9t5   1/1     Running            4 (39m ago)    40m
storage-lite-pool-0-0                          1/1     Running            0              15m
storage-lite-pool-0-1                          1/1     Running            0              15m
storage-lite-pool-0-2                          0/1     CrashLoopBackOff   5 (106s ago)   4m47s
storage-lite-pool-0-3                          0/1     CrashLoopBackOff   5 (2m6s ago)   4m51s
storage-lite-prometheus-0                      2/2     Running            0              38m
➜  operator git:(parse-tenant-domain) ✗ k -n tenant-lite logs pod/storage-lite-pool-0-2
ERROR Invalid MINIO_SERVER_URL value in environment variable: unexpected scheme found

f. Remove minio domain modification. Observe pods start. Observe console accessible on reported urls:

API: https://minio.tenant-lite.svc.cluster.local 
Console: https://localhost:9443
image

➜ operator git:(parse-tenant-domain) ✗ k -n tenant-lite get pods
NAME READY STATUS RESTARTS AGE storage-lite-log-0 1/1 Running 0 55m storage-lite-log-search-api-5685f95857-xx9t5 1/1 Running 4 (55m ago) 55m storage-lite-pool-0-0 1/1 Running 0 2m42s storage-lite-pool-0-1 1/1 Running 0 2m45s storage-lite-pool-0-2 1/1 Running 0 2m48s storage-lite-pool-0-3 1/1 Running 0 2m51s storage-lite-prometheus-0 2/2 Running 0 53m

##Fix

Fix using operator https://github.com/allanrogerr/operator/tree/parse-tenant-domain

1a. Edit minio-operator deployment at .spec.template.spec.containers.image:

k -n minio-operator edit deployment/minio-operator

image: minio/operator:noop
image

b. Edit tenant under .spec.features

  features:
    domains:
      console: localhost:9443
      minio:
      - http://minio.default.svc.cluster.local
      - minio1.default.svc.cluster.local
      - https://test.default.svc.cluster.local
      - old.default.svc.cluster.local:9090
      - minio.default.svc.cluster.local
      - rbitrary.default.svc.cluster.local
      - default.svc.cluster.local

c. Observe

  • overlapping domains are reported. default.svc.cluster.local caused an overlap. This entry was then removed.
image
  • strings with prefix https? are available without their prefixes Tenant
  features:
    domains:
      console: localhost:9443
      minio:
      - http://minio.default.svc.cluster.local
      - minio1.default.svc.cluster.local
      - https://test.default.svc.cluster.local
      - old.default.svc.cluster.local:9090
      - minio.default.svc.cluster.local
      - rbitrary.default.svc.cluster.local

Statefulset

        - name: MINIO_BROWSER_REDIRECT_URL
          value: https://localhost:9443
        - name: MINIO_DOMAIN
          value: minio.default.svc.cluster.local,minio1.default.svc.cluster.local,test.default.svc.cluster.local,old.default.svc.cluster.local,minio.default.svc.cluster.local,rbitrary.default.svc.cluster.local
        - name: MINIO_SERVER_URL
          value: https://minio.tenant-lite.svc.cluster.local:443

d. Observe pods start:

➜  operator git:(parse-tenant-domain) ✗ k -n tenant-lite get pods  
NAME                                           READY   STATUS    RESTARTS      AGE
storage-lite-log-0                             1/1     Running   0             73m
storage-lite-log-search-api-5685f95857-xx9t5   1/1     Running   4 (72m ago)   73m
storage-lite-pool-0-0                          1/1     Running   0             5m45s
storage-lite-pool-0-1                          1/1     Running   0             5m48s
storage-lite-pool-0-2                          1/1     Running   0             5m52s
storage-lite-pool-0-3                          1/1     Running   0             5m56s
storage-lite-prometheus-0                      2/2     Running   0             71m

e. Validate that the urls are accessible with

k -n tenant-lite exec -it storage-lite-pool-0-0 -- /bin/sh
curl https://localhost:9443
curl https://minio.tenant-lite.svc.cluster.local:443
image

f. Validate tenant console accessible

k port-forward service/storage-lite-console -n tenant-lite 9443

Navigate to https://localhost:9443 image

Clone this wiki locally