Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rework glance config process #295

Merged
merged 1 commit into from
Aug 22, 2023

Conversation

fmount
Copy link
Contributor

@fmount fmount commented Aug 20, 2023

This patch represents an implementation of the proposal [1] and aligns the glance-operator with the work already done in Cinder and the other operators.
There are a few relevant changes in the bootstrap process of Glance, in particular:

  • It stops using an InitContainer to generate the snippet files that configure each GlanceAPI service. The logic that was previously implemented in the InitContainer has been moved to the Controllers, where config files are generated and stored in k8s Secrets. InitContainers are fully removed from the bootstrap process;
  • It uses Kolla to copy the generated content to the target directory /etc/glance/glance.conf.d, which is used by each API service to run;
  • The relevant content, including scripts, previously stored in a ConfigMap, are now stored in a corresponding k8s Secret, which is mounted to the GlanceAPI deployment Pods (hence no additional calls to GetVolume/GetVolumeMounts are required);
  • A total of 4 config snippet files are generated:
    • 00-config.conf contains global settings that are common to every GlanceAPI Pod, including ones that are derived from deployment secrets (e.g. database password, etc.)
    • 01-config.conf contains the global customServiceConfig settings that apply to every GlanceAPI service.
    • 02-config.conf contains the customServiceConfig settings that are specific to each service.
    • 03-config.conf contains secrets specified by each service's customServiceConfigSecrets.
  • glance-image-import.conf has been removed, and the default config has been added to 00-config.conf, which is the common file generated by the umbrella Glance controller;
  • logging.conf has been removed as it's no longer required in the switch to a side container approach for logging purposes;
  • functional tests are aligned to the use of k8s Secrets instead of the old pattern based on ConfigMaps;
  • kuttl tests are updated and the initContainer has been removed.
  • DBsync now mounts only the required files (a minimal 00-config.conf) and a db-sync-config.json containing the command run through kolla.

[1] openstack-k8s-operators/dev-docs#31

@fmount
Copy link
Contributor Author

fmount commented Aug 20, 2023

/retest

This patch represents an implementation of the proposal [1] and aligns
the glance-operator with the work already done in Cinder and the other
storage operators. There are a few relevant changes in the bootstrap
process of Glance, in particular:

- It stops using an init container to generate the snippet files that
  configure each glanceAPI service. The logic that was previously
  implemented in the Init Container has been moved to the Controllers,
  where config files are generated and stored in k8s Secrets. Init
  Containers are fully removed from the bootstrap process;
- It uses Kolla to copy the generated content to the target directory
  /etc/glance/glance.conf.d, which is used by each API service to run;
- The relevant content, including scripts, previously stored in a
  ConfigMap, are now stored in a corresponding k8s Secret, which is
  mounted to the GlanceAPI deployment Pods (hence no additional calls
  to GetVolume/GetVolumeMounts are required);
- A total of 4 config snippet files are generated:
  - 00-config.conf contains global settings that are common to every
    GlanceAPI Pod, including ones that are derived from deployment
    secrets (e.g. database password, etc.)
  - 01-config.conf contains the global customServiceConfig settings
    that apply to every GlanceAPI service.
  - 02-config.conf contains the customServiceConfig settings that
    are specific to each service.
  - 03-config.conf contains secrets specified by each service's
    customServiceConfigSecrets.
- glance-image-import.conf has been removed, and the default config has
  been added to 00-config.conf, which is the common file generated by
  the umbrella Glance controller;
- logging.conf has been removed as it's no longer required in the switch
  to a side container approach for logging purposes;
- functional tests are aligned to the use of k8s Secrets instead of the
  old pattern based on ConfigMaps;
- kuttl tests are updated and the 'initContainer' has been removed.
- dbsync now mounts only the required files (a minimal 00-config.conf)
  and a db-sync-config.json containing the command run through kolla.

[1] openstack-k8s-operators/dev-docs#31

Signed-off-by: Francesco Pantano <[email protected]>
@fmount
Copy link
Contributor Author

fmount commented Aug 22, 2023

@konan-abhi worked as expected:

Secrets [1]

Pods are mounting what we defined in the config [2]:

  Mounts:
      /etc/ceph from ceph (ro)
      /var/lib/config-data/default from config-data (ro)
      /var/lib/glance from lib-data (rw)
      /var/lib/kolla/config_files/config.json from config-data (ro,path="glance-api-config.json")
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-lbxk5 (ro)

[1] https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/origin-ci-test/pr-logs/pull/openstack-k8s-operators_glance-operator/295/pull-ci-openstack-k8s-operators-glance-operator-main-glance-operator-build-deploy-kuttl/1693866122480193536/artifacts/glance-operator-build-deploy-kuttl/openstack-k8s-operators-gather/artifacts/must-gather/quay-io-openstack-k8s-operators-openstack-must-gather-sha256-4bc3ccfa15317dc06b06988da2227028fb4c03aee8d52f05f0dfe4d2640d72aa/namespaces/openstack/secrets/glance/
[2] https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/origin-ci-test/pr-logs/pull/openstack-k8s-operators_glance-operator/295/pull-ci-openstack-k8s-operators-glance-operator-main-glance-operator-build-deploy-kuttl/1693866122480193536/artifacts/glance-operator-build-deploy-kuttl/openstack-k8s-operators-gather/artifacts/must-gather/quay-io-openstack-k8s-operators-openstack-must-gather-sha256-4bc3ccfa15317dc06b06988da2227028fb4c03aee8d52f05f0dfe4d2640d72aa/namespaces/openstack/pods/glance-external-api-7bdd85dff7-cjpkn/glance-external-api-7bdd85dff7-cjpkn-describe.log

@fmount fmount requested a review from konan-abhi August 22, 2023 07:03
Copy link
Contributor

@konan-abhi konan-abhi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, verified and satisfied with the change.

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Aug 22, 2023

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: fmount, konan-abhi

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-merge-robot openshift-merge-robot merged commit 132a5c3 into openstack-k8s-operators:main Aug 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants