Propose Service Bootstrap config design #31
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
fmount
requested review from
ASBishop,
stuggi,
gibizer,
dprince,
jlarriba,
abays,
konan-abhi and
xek
abays
reviewed
May 24, 2023
gibizer
reviewed
May 24, 2023
| services, and other potential requirements that are not the same across the | ||
| board. | ||
|
|
||
|
|
There was a problem hiding this comment.
Does this proposal targets service config for services running on the EDPM nodes or only targeting services running in k8s pods?
There was a problem hiding this comment.
No, the proposal is for services in the ctlplane: I think we can follow up and expand to the EDPM world as long as we're in agreement on this side.
There was a problem hiding this comment.
Sure. Let's state that this proposal only targets services running in k8s pods.
There was a problem hiding this comment.
part of the inital feedback i provided is that we should not try and split this along those lines since there is disagreement on what is the control plane.
just because its on an edpm host does not mean its not part fo the control plane.
with that said if other service s want have different config generation for depm nodes that is there choice but it would be explicitly against one of our early design goals which was to have no config generation in ansible.
for the nova operator anything we implement for openshift will also be used for the edpm nodes
we already excplivly generate service configuration for nova-comptue and libvirt using golang templates.
ovn/neutron however are currently split in how they are configured
hopefully they will see value in converging on this. proposal for there edpm configs too.
There was a problem hiding this comment.
@SeanMooney I understand your concern, and given we have still an ongoing conversation on the workflow on the EDPM front, I'd like to address it in a follow up patch. I also assume that you or @gibizer can propose a follow up to this patch when the concerns on the EDPM side are addressed and the flow/options are well defined.
All of this to say that right now I'd like to explicitly state that this doc applies to the services deployed in the OpenStack controlplane: this doesn't mean we can't extend it to cover edpm in a follow up.
There was a problem hiding this comment.
If it helps settle this sooner then I agree to scope it to services in k8s pods only. While I agree with @SeanMooney that the same logic described here should be applied to services running on EDPM nodes, I prefer settle this with a smaller scope sooner and extending it later, than holding the whole thing up due to the EDPM architectural discussion.
There was a problem hiding this comment.
+1 I think we're on the same page here.
gibizer
reviewed
May 24, 2023
SeanMooney
reviewed
May 24, 2023
stuggi
reviewed
May 25, 2023
service_config.md
Outdated
| ## Conclusion | ||
|
|
||
| The model described here allows to reach many goals: | ||
| - remove (entirely) the `initContainer` and the related scripts that are no |
There was a problem hiding this comment.
as from my comment above there might be still a use case for init container, if e.g. IP information is required to setup the service/service-config.
There was a problem hiding this comment.
+1, I'm going to change the wording here, and as per your suggestion in the previous comment I'm going to state that there might be cases where we'd like to keep the init container to properly handle the service start (e.g. you need to know the IP address on the Multus network, which is known when the Pod is created and started).
Akrog
reviewed
May 25, 2023
This was referenced Jun 13, 2023
Closed
|
@gibizer @abays @stuggi except the last comment that I still discussing w/ gibi, do you have additional concern of we can approve this document at this point? |
I have nothing further :) |
The patch represents a proposal to find a common pattern for services bootstrap. Signed-off-by: Francesco Pantano <[email protected]>
I think I am good, thanks. lgtm |
|
/lgtm |
|
Looks like we have enough votes to move forward: I'm merging the patch at this point. |
fmount
added a commit
to fmount/glance-operator
that referenced
this pull request
Aug 20, 2023
This patch represents an implementation of the proposal [1] and aligns
the glance-operator with the work already done in Cinder and the other
storage operators. There are a few relevant changes in the bootstrap
process of the glance service, in particular:
- Stop using an init container to generate the snippet files that
configure each glanceAPI service. The logic that was previously
implemented in the init container has been moved to the controller,
where config files are generated and stored in secrets. init
Containers are fully removed;
- Kolla is still used to move the generated content to
/etc/glance/glance.conf.d, which represents the targed directory used
by each API service;
- The relevant content, including scripts, previously stored in a
ConfigMap are now stored in a corresponding secret, which is mounted
to the GlanceAPI deployment (hence no additional
GetVolume/GetVolumeMounts functions are required);
- A total of 4 config snippet files are generated:
- 00-config.conf contains global settings that are common to every
GlanceAPI service, including ones that are derived from deployment
secrets (e.g. transport URL, database password, etc.)
- 01-config.conf contains the global customServiceConfig settings
that apply to every glanceAPI service.
- 02-config.conf contains the customServiceConfig settings that
are specific to each service.
- 03-config.conf contains secrets specified by each service's
customServiceConfigSecrets.
- glance-image-import.conf has been removed, and the default config has
been added to 00-config.conf, which is the common file generated by
the umbrella Glance controller;
- logging.conf has been removed as it's no longer required in the switch
to a sidecontainer for logging purposed;
- functional tests are aligned to the use of Secrets instead of
ConfigMaps;
- kuttl tests are updated and the initContainer has been removed.
- dbsync now mounts only the required files (a minimal 00-config.conf)
and a db-sync-config.json containing the command run through kolla has
been added.
[1] openstack-k8s-operators/dev-docs#31
Signed-off-by: Francesco Pantano <[email protected]>
fmount
added a commit
to fmount/glance-operator
that referenced
this pull request
Aug 20, 2023
This patch represents an implementation of the proposal [1] and aligns
the glance-operator with the work already done in Cinder and the other
storage operators. There are a few relevant changes in the bootstrap
process of Glance, in particular:
- It stops using an init container to generate the snippet files that
configure each glanceAPI service. The logic that was previously
implemented in the Init Container has been moved to the Controllers,
where config files are generated and stored in k8s Secrets. Init
Containers are fully removed from the bootstrap process;
- It uses Kolla to copy the generated content to the target directory
/etc/glance/glance.conf.d, which is used by each API service to run;
- The relevant content, including scripts, previously stored in a
ConfigMap, are now stored in a corresponding k8s Secret, which is
mounted to the GlanceAPI deployment Pods (hence no additional calls
to GetVolume/GetVolumeMounts are required);
- A total of 4 config snippet files are generated:
- 00-config.conf contains global settings that are common to every
GlanceAPI Pod, including ones that are derived from deployment
secrets (e.g. database password, etc.)
- 01-config.conf contains the global customServiceConfig settings
that apply to every GlanceAPI service.
- 02-config.conf contains the customServiceConfig settings that
are specific to each service.
- 03-config.conf contains secrets specified by each service's
customServiceConfigSecrets.
- glance-image-import.conf has been removed, and the default config has
been added to 00-config.conf, which is the common file generated by
the umbrella Glance controller;
- logging.conf has been removed as it's no longer required in the switch
to a side container approach for logging purposes;
- functional tests are aligned to the use of k8s Secrets instead of the
old pattern based on ConfigMaps;
- kuttl tests are updated and the 'initContainer' has been removed.
- dbsync now mounts only the required files (a minimal 00-config.conf)
and a db-sync-config.json containing the command run through kolla.
[1] openstack-k8s-operators/dev-docs#31
Signed-off-by: Francesco Pantano <[email protected]>
fmount
added a commit
to fmount/glance-operator
that referenced
this pull request
Aug 21, 2023
This patch represents an implementation of the proposal [1] and aligns
the glance-operator with the work already done in Cinder and the other
storage operators. There are a few relevant changes in the bootstrap
process of Glance, in particular:
- It stops using an init container to generate the snippet files that
configure each glanceAPI service. The logic that was previously
implemented in the Init Container has been moved to the Controllers,
where config files are generated and stored in k8s Secrets. Init
Containers are fully removed from the bootstrap process;
- It uses Kolla to copy the generated content to the target directory
/etc/glance/glance.conf.d, which is used by each API service to run;
- The relevant content, including scripts, previously stored in a
ConfigMap, are now stored in a corresponding k8s Secret, which is
mounted to the GlanceAPI deployment Pods (hence no additional calls
to GetVolume/GetVolumeMounts are required);
- A total of 4 config snippet files are generated:
- 00-config.conf contains global settings that are common to every
GlanceAPI Pod, including ones that are derived from deployment
secrets (e.g. database password, etc.)
- 01-config.conf contains the global customServiceConfig settings
that apply to every GlanceAPI service.
- 02-config.conf contains the customServiceConfig settings that
are specific to each service.
- 03-config.conf contains secrets specified by each service's
customServiceConfigSecrets.
- glance-image-import.conf has been removed, and the default config has
been added to 00-config.conf, which is the common file generated by
the umbrella Glance controller;
- logging.conf has been removed as it's no longer required in the switch
to a side container approach for logging purposes;
- functional tests are aligned to the use of k8s Secrets instead of the
old pattern based on ConfigMaps;
- kuttl tests are updated and the 'initContainer' has been removed.
- dbsync now mounts only the required files (a minimal 00-config.conf)
and a db-sync-config.json containing the command run through kolla.
[1] openstack-k8s-operators/dev-docs#31
Signed-off-by: Francesco Pantano <[email protected]>
fmount
added a commit
to fmount/glance-operator
that referenced
this pull request
Aug 21, 2023
This patch represents an implementation of the proposal [1] and aligns
the glance-operator with the work already done in Cinder and the other
storage operators. There are a few relevant changes in the bootstrap
process of Glance, in particular:
- It stops using an init container to generate the snippet files that
configure each glanceAPI service. The logic that was previously
implemented in the Init Container has been moved to the Controllers,
where config files are generated and stored in k8s Secrets. Init
Containers are fully removed from the bootstrap process;
- It uses Kolla to copy the generated content to the target directory
/etc/glance/glance.conf.d, which is used by each API service to run;
- The relevant content, including scripts, previously stored in a
ConfigMap, are now stored in a corresponding k8s Secret, which is
mounted to the GlanceAPI deployment Pods (hence no additional calls
to GetVolume/GetVolumeMounts are required);
- A total of 4 config snippet files are generated:
- 00-config.conf contains global settings that are common to every
GlanceAPI Pod, including ones that are derived from deployment
secrets (e.g. database password, etc.)
- 01-config.conf contains the global customServiceConfig settings
that apply to every GlanceAPI service.
- 02-config.conf contains the customServiceConfig settings that
are specific to each service.
- 03-config.conf contains secrets specified by each service's
customServiceConfigSecrets.
- glance-image-import.conf has been removed, and the default config has
been added to 00-config.conf, which is the common file generated by
the umbrella Glance controller;
- logging.conf has been removed as it's no longer required in the switch
to a side container approach for logging purposes;
- functional tests are aligned to the use of k8s Secrets instead of the
old pattern based on ConfigMaps;
- kuttl tests are updated and the 'initContainer' has been removed.
- dbsync now mounts only the required files (a minimal 00-config.conf)
and a db-sync-config.json containing the command run through kolla.
[1] openstack-k8s-operators/dev-docs#31
Signed-off-by: Francesco Pantano <[email protected]>
fmount
added a commit
to fmount/glance-operator
that referenced
this pull request
Aug 21, 2023
This patch represents an implementation of the proposal [1] and aligns
the glance-operator with the work already done in Cinder and the other
storage operators. There are a few relevant changes in the bootstrap
process of Glance, in particular:
- It stops using an init container to generate the snippet files that
configure each glanceAPI service. The logic that was previously
implemented in the Init Container has been moved to the Controllers,
where config files are generated and stored in k8s Secrets. Init
Containers are fully removed from the bootstrap process;
- It uses Kolla to copy the generated content to the target directory
/etc/glance/glance.conf.d, which is used by each API service to run;
- The relevant content, including scripts, previously stored in a
ConfigMap, are now stored in a corresponding k8s Secret, which is
mounted to the GlanceAPI deployment Pods (hence no additional calls
to GetVolume/GetVolumeMounts are required);
- A total of 4 config snippet files are generated:
- 00-config.conf contains global settings that are common to every
GlanceAPI Pod, including ones that are derived from deployment
secrets (e.g. database password, etc.)
- 01-config.conf contains the global customServiceConfig settings
that apply to every GlanceAPI service.
- 02-config.conf contains the customServiceConfig settings that
are specific to each service.
- 03-config.conf contains secrets specified by each service's
customServiceConfigSecrets.
- glance-image-import.conf has been removed, and the default config has
been added to 00-config.conf, which is the common file generated by
the umbrella Glance controller;
- logging.conf has been removed as it's no longer required in the switch
to a side container approach for logging purposes;
- functional tests are aligned to the use of k8s Secrets instead of the
old pattern based on ConfigMaps;
- kuttl tests are updated and the 'initContainer' has been removed.
- dbsync now mounts only the required files (a minimal 00-config.conf)
and a db-sync-config.json containing the command run through kolla.
[1] openstack-k8s-operators/dev-docs#31
Signed-off-by: Francesco Pantano <[email protected]>
fmount
added a commit
to fmount/manila-operator
that referenced
this pull request
Aug 26, 2023
This patch represents an implementation of the proposal [1] and aligns
the manila-operator with the work already done in Cinder and the other
operators. There are a few relevant changes in the bootstrap process of
Manila, in particular:
1. It stops using an InitContainer to generate the snippet files that
configure each Manila service. The logic that was previously
implemented in the InitContainer has been moved to the Controllers,
where config files are generated and stored in k8s Secrets.
2. InitContainers are fully removed from the bootstrap process; It uses
to copy additional files (httpd and wsgi config in manila-api) to the
target directories, and mount the generated config (0{0,1,2,3}-config.conf
to /etc/manila/manila.conf.d, which is used by each service to run;
3. The relevant content, including scripts, previously stored in a
ConfigMap, are now stored in a corresponding k8s Secret, which is
mounted to the Service deployment Pods; A total of 4 config snippet files
are generated:
- 00-config.conf contains global settings that are common to every Manila
Pod, including ones that are derived from deployment secrets (e.g.
database password, etc.)
- 01-config.conf contains the global customServiceConfig settings that
apply to every Manila service.
- 02-config.conf contains the customServiceConfig settings that are
specific to each service.
- 03-config.conf contains secrets specified by each service's
customServiceConfigSecrets. logging.conf has been removed as it'
s no longer required in the switch to a side container approach for
logging purposes, which will be part of a follow up PR.
Finally, functional tests are aligned to the use of k8s Secrets instead of
the old pattern based on ConfigMaps; kuttl tests are updated and the
initContainer has been removed. DBsync now mounts only the required
files (a minimal 00-config.conf) and a db-sync-config.json
containing the command run through kolla.
[1] openstack-k8s-operators/dev-docs#31
Signed-off-by: Francesco Pantano <[email protected]>
fmount
added a commit
to fmount/manila-operator
that referenced
this pull request
Aug 26, 2023
This patch represents an implementation of the proposal [1] and aligns
the manila-operator with the work already done in Cinder and the other
operators. There are a few relevant changes in the bootstrap process of
Manila, in particular:
1. It stops using an InitContainer to generate the snippet files that
configure each Manila service. The logic that was previously
implemented in the InitContainer has been moved to the Controllers,
where config files are generated and stored in k8s Secrets.
2. InitContainers are fully removed from the bootstrap process; It uses
to copy additional files (httpd and wsgi config in manila-api) to the
target directories, and mount the generated config (0{0,1,2,3}-config.conf
to /etc/manila/manila.conf.d, which is used by each service to run;
3. The relevant content, including scripts, previously stored in a
ConfigMap, are now stored in a corresponding k8s Secret, which is
mounted to the Service deployment Pods; A total of 4 config snippet files
are generated:
- 00-config.conf contains global settings that are common to every Manila
Pod, including ones that are derived from deployment secrets (e.g.
database password, etc.)
- 01-config.conf contains the global customServiceConfig settings that
apply to every Manila service.
- 02-config.conf contains the customServiceConfig settings that are
specific to each service.
- 03-config.conf contains secrets specified by each service's
customServiceConfigSecrets. logging.conf has been removed as it'
s no longer required in the switch to a side container approach for
logging purposes, which will be part of a follow up PR.
Finally, functional tests are aligned to the use of k8s Secrets instead of
the old pattern based on ConfigMaps; kuttl tests are updated and the
initContainer has been removed. DBsync now mounts only the required
files (a minimal 00-config.conf) and a db-sync-config.json
containing the command run through kolla.
[1] openstack-k8s-operators/dev-docs#31
Signed-off-by: Francesco Pantano <[email protected]>
fmount
added a commit
to fmount/manila-operator
that referenced
this pull request
Aug 26, 2023
This patch represents an implementation of the proposal [1] and aligns
the manila-operator with the work already done in Cinder and the other
operators. There are a few relevant changes in the bootstrap process of
Manila, in particular:
1. It stops using an InitContainer to generate the snippet files that
configure each Manila service. The logic that was previously
implemented in the InitContainer has been moved to the Controllers,
where config files are generated and stored in k8s Secrets.
2. InitContainers are fully removed from the bootstrap process; It uses
to copy additional files (httpd and wsgi config in manila-api) to the
target directories, and mount the generated config (0{0,1,2,3}-config.conf
to /etc/manila/manila.conf.d, which is used by each service to run;
3. The relevant content, including scripts, previously stored in a
ConfigMap, are now stored in a corresponding k8s Secret, which is
mounted to the Service deployment Pods; A total of 4 config snippet files
are generated:
- 00-config.conf contains global settings that are common to every Manila
Pod, including ones that are derived from deployment secrets (e.g.
database password, etc.)
- 01-config.conf contains the global customServiceConfig settings that
apply to every Manila service.
- 02-config.conf contains the customServiceConfig settings that are
specific to each service.
- 03-config.conf contains secrets specified by each service's
customServiceConfigSecrets. logging.conf has been removed as it'
s no longer required in the switch to a side container approach for
logging purposes, which will be part of a follow up PR.
Finally, functional tests are aligned to the use of k8s Secrets instead of
the old pattern based on ConfigMaps; kuttl tests are updated and the
initContainer has been removed. DBsync now mounts only the required
files (a minimal 00-config.conf) and a db-sync-config.json
containing the command run through kolla.
[1] openstack-k8s-operators/dev-docs#31
Signed-off-by: Francesco Pantano <[email protected]>
fmount
added a commit
to fmount/manila-operator
that referenced
this pull request
Aug 26, 2023
This patch represents an implementation of the proposal [1] and aligns
the manila-operator with the work already done in Cinder and the other
operators. There are a few relevant changes in the bootstrap process of
Manila, in particular:
1. It stops using an InitContainer to generate the snippet files that
configure each Manila service. The logic that was previously
implemented in the InitContainer has been moved to the Controllers,
where config files are generated and stored in k8s Secrets.
2. InitContainers are fully removed from the bootstrap process; It uses
to copy additional files (httpd and wsgi config in manila-api) to the
target directories, and mount the generated config (0{0,1,2,3}-config.conf
to /etc/manila/manila.conf.d, which is used by each service to run;
3. The relevant content, including scripts, previously stored in a
ConfigMap, are now stored in a corresponding k8s Secret, which is
mounted to the Service deployment Pods; A total of 4 config snippet files
are generated:
- 00-config.conf contains global settings that are common to every Manila
Pod, including ones that are derived from deployment secrets (e.g.
database password, etc.)
- 01-config.conf contains the global customServiceConfig settings that
apply to every Manila service.
- 02-config.conf contains the customServiceConfig settings that are
specific to each service.
- 03-config.conf contains secrets specified by each service's
customServiceConfigSecrets. logging.conf has been removed as it'
s no longer required in the switch to a side container approach for
logging purposes, which will be part of a follow up PR.
Finally, functional tests are aligned to the use of k8s Secrets instead of
the old pattern based on ConfigMaps; kuttl tests are updated and the
initContainer has been removed. DBsync now mounts only the required
files (a minimal 00-config.conf) and a db-sync-config.json
containing the command run through kolla.
[1] openstack-k8s-operators/dev-docs#31
Signed-off-by: Francesco Pantano <[email protected]>
fmount
added a commit
to fmount/manila-operator
that referenced
this pull request
Sep 2, 2023
This patch represents an implementation of the proposal [1] and aligns
the manila-operator with the work already done in Cinder and the other
operators. There are a few relevant changes in the bootstrap process of
Manila, in particular:
1. It stops using an InitContainer to generate the snippet files that
configure each Manila service. The logic that was previously
implemented in the InitContainer has been moved to the Controllers,
where config files are generated and stored in k8s Secrets.
2. InitContainers are fully removed from the bootstrap process; It uses
to copy additional files (httpd and wsgi config in manila-api) to the
target directories, and mount the generated config (0{0,1,2,3}-config.conf
to /etc/manila/manila.conf.d, which is used by each service to run;
3. The relevant content, including scripts, previously stored in a
ConfigMap, are now stored in a corresponding k8s Secret, which is
mounted to the Service deployment Pods; A total of 4 config snippet files
are generated:
- 00-config.conf contains global settings that are common to every Manila
Pod, including ones that are derived from deployment secrets (e.g.
database password, etc.)
- 01-config.conf contains the global customServiceConfig settings that
apply to every Manila service.
- 02-config.conf contains the customServiceConfig settings that are
specific to each service.
- 03-config.conf contains secrets specified by each service's
customServiceConfigSecrets. logging.conf has been removed as it'
s no longer required in the switch to a side container approach for
logging purposes, which will be part of a follow up PR.
Finally, functional tests are aligned to the use of k8s Secrets instead of
the old pattern based on ConfigMaps; kuttl tests are updated and the
initContainer has been removed. DBsync now mounts only the required
files (a minimal 00-config.conf) and a db-sync-config.json
containing the command run through kolla.
[1] openstack-k8s-operators/dev-docs#31
Signed-off-by: Francesco Pantano <[email protected]>
fmount
added a commit
to fmount/manila-operator
that referenced
this pull request
Sep 7, 2023
This patch represents an implementation of the proposal [1] and aligns
the manila-operator with the work already done in Cinder and the other
operators. There are a few relevant changes in the bootstrap process of
Manila, in particular:
1. It stops using an InitContainer to generate the snippet files that
configure each Manila service. The logic that was previously
implemented in the InitContainer has been moved to the Controllers,
where config files are generated and stored in k8s Secrets.
2. InitContainers are fully removed from the bootstrap process; It uses
to copy additional files (httpd and wsgi config in manila-api) to the
target directories, and mount the generated config (0{0,1,2,3}-config.conf
to /etc/manila/manila.conf.d, which is used by each service to run;
3. The relevant content, including scripts, previously stored in a
ConfigMap, are now stored in a corresponding k8s Secret, which is
mounted to the Service deployment Pods; A total of 4 config snippet files
are generated:
- 00-config.conf contains global settings that are common to every Manila
Pod, including ones that are derived from deployment secrets (e.g.
database password, etc.)
- 01-config.conf contains the global customServiceConfig settings that
apply to every Manila service.
- 02-config.conf contains the customServiceConfig settings that are
specific to each service.
- 03-config.conf contains secrets specified by each service's
customServiceConfigSecrets. logging.conf has been removed as it'
s no longer required in the switch to a side container approach for
logging purposes, which will be part of a follow up PR.
Finally, functional tests are aligned to the use of k8s Secrets instead of
the old pattern based on ConfigMaps; kuttl tests are updated and the
initContainer has been removed. DBsync now mounts only the required
files (a minimal 00-config.conf) and a db-sync-config.json
containing the command run through kolla.
[1] openstack-k8s-operators/dev-docs#31
Signed-off-by: Francesco Pantano <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The patch represents a proposal to find a common pattern for services bootstrap.