This repository has been archived by the owner on Jun 25, 2024. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 47
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
WIP - Initial commit to generate EDPM certs
This is lightly tested so far (and so is still WIP), but it gives an idea of how EDPM certs
could be generated and mounted for each of the services that need them.
While I will test and refine, I'm hoping that this PR starts
conversations and questions so that we can decide on an implementation.
The basic idea is as follows:
Each service has a boolean parameter HasTLSCerts which determines
whether or not certs are expected for this service. If so, then we
expect that a secret with the name <nodeSet.Name>-<service.Name>-certs will
be created.
This secret contains the TLS certs, keys and cacerts for every node in
the nodeset. They are referenced as <nodeName>-tls.key, nodeName-tls.crt, nodeName-ca.crt
This secret will be mounted in the openstackAnsibleEE pod at
/var/lib/openstack/certs. It will be up to the ansible playbook to
move the appropriate certs, keys etc. for each node to the node, and do
any reconfiguration of the service.
More details are below:
1. After the DNS data is created in the openstackdataplanenodeset_controller,
a call is made to GenerateTLSCerts for each service that has HasTLSCerts: True.
2. GenerateTLSCerts right now only generates a single cert for each node in the
nodeset using the DNSNames for that node. As the cert contains the
DNSNames for all the node's interfaces, it should be usable on all
interfaces. This cert is stored in the secret "nodeName-cert". This
is the default cert - and it should only be created once - ie. the
first time this code is run.
3. Its likely that most services will be able to use this cert. If a
service needs a different kind of cert (maybe using ips - for ovn
for instance, or a different issuer), then code needs to be added to
the switch statement for that particular service.
4. The secret containing the cert will need to have an ownership
reference updated so that we can trigger a reconcile when the cert
changes or is renewed.- Loading branch information
Showing
13 changed files
with
250 additions
and
2 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -5,3 +5,4 @@ metadata: | |
| spec: | ||
| label: dataplane-deployment-libvirt | ||
| playbook: osp.edpm.libvirt | ||
| hasTLSCerts: True | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -7,3 +7,4 @@ spec: | |
| secrets: | ||
| - nova-cell1-compute-config | ||
| playbook: osp.edpm.nova | ||
| hasTLSCerts: True | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,176 @@ | ||
| /* | ||
| Copyright 2023. | ||
| Licensed under the Apache License, Version 2.0 (the "License"); | ||
| you may not use this file except in compliance with the License. | ||
| You may obtain a copy of the License at | ||
| http://www.apache.org/licenses/LICENSE-2.0 | ||
| Unless required by applicable law or agreed to in writing, software | ||
| distributed under the License is distributed on an "AS IS" BASIS, | ||
| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| See the License for the specific language governing permissions and | ||
| limitations under the License. | ||
| */ | ||
|
|
||
| package deployment | ||
|
|
||
| import ( | ||
| "context" | ||
| "fmt" | ||
| "strings" | ||
| "time" | ||
|
|
||
| corev1 "k8s.io/api/core/v1" | ||
| k8serrors "k8s.io/apimachinery/pkg/api/errors" | ||
| metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" | ||
| "k8s.io/apimachinery/pkg/types" | ||
|
|
||
| certmgrv1 "github.com/cert-manager/cert-manager/pkg/apis/certmanager/v1" | ||
| certmgrmetav1 "github.com/cert-manager/cert-manager/pkg/apis/meta/v1" | ||
| dataplanev1 "github.com/openstack-k8s-operators/dataplane-operator/api/v1beta1" | ||
| infranetworkv1 "github.com/openstack-k8s-operators/infra-operator/apis/network/v1beta1" | ||
| "github.com/openstack-k8s-operators/lib-common/modules/certmanager" | ||
| "github.com/openstack-k8s-operators/lib-common/modules/common/helper" | ||
| "github.com/openstack-k8s-operators/lib-common/modules/common/secret" | ||
| ) | ||
|
|
||
| // GenerateTLSCerts generates a secret containing all the certificates for the relevant service | ||
| // This secret will be mounted by the ansibleEE pod as an extra mount when the service is deployed. | ||
| func GenerateTLSCerts(ctx context.Context, helper *helper.Helper, | ||
| instance *dataplanev1.OpenStackDataPlaneNodeSet, | ||
| allIPSets map[string]infranetworkv1.IPSet, serviceName string) error { | ||
|
|
||
| certsData := map[string][]byte{} | ||
|
|
||
| // for each node in the nodeset, issue all the TLS certs needed based on the | ||
| // ips or DNS Names | ||
| for nodeName := range instance.Spec.Nodes { | ||
| var dnsNames []string | ||
| var secretName string | ||
| var certName string | ||
| var certSecret *corev1.Secret = nil | ||
| var err error | ||
|
|
||
| // TODO(alee) decide if we want to use other labels | ||
| // For now we just add the hostname so we can select all the certs on one node | ||
| labels := map[string]string{ | ||
| "hostname": nodeName, | ||
| } | ||
|
|
||
| ipSet, ok := allIPSets[nodeName] | ||
| if ok { | ||
| for _, res := range ipSet.Status.Reservation { | ||
| fqdnName := strings.Join([]string{nodeName, res.DNSDomain}, ".") | ||
| dnsNames = append(dnsNames, fqdnName) | ||
| } | ||
| } | ||
|
|
||
| switch serviceName { | ||
| default: | ||
| // The default case provides a cert with all the dns names for the host. | ||
| // This will probably be sufficient for most services. If a service needs | ||
| // a different kind of cert (for example, containing ips, or using a different | ||
| // issuer) then add a case for the service in this switch statement | ||
|
|
||
| secretName = "cert-" + nodeName | ||
| certSecret, _, err = secret.GetSecret(ctx, helper, secretName, instance.Namespace) | ||
| if err != nil { | ||
| if !k8serrors.IsNotFound(err) { | ||
| err = fmt.Errorf("Error retrieving secret %s - %w", secretName, err) | ||
| return err | ||
| } | ||
|
|
||
| certName = secretName | ||
| certSecret, err = IssueTLSCert(ctx, helper, certName, instance.Namespace, | ||
| secretName, nodeName, dnsNames, nil, RootCAIssuerInternalLabel, | ||
| RootCAIssuerInternalNamespaceLabel, labels) | ||
| if err != nil { | ||
| return err | ||
| } | ||
| } | ||
| } | ||
|
|
||
| // TODO(alee) Add an owner reference to the secret so it can be monitored | ||
| // We'll do this once stuggi adds a function to do this in libcommon | ||
|
|
||
| // To use this cert, add it to the relevant service data | ||
| for key, value := range certSecret.Data { | ||
| certsData[nodeName+"-"+key] = value | ||
| } | ||
| } | ||
|
|
||
| // create a secret to hold the certs for the service | ||
| serviceCertsSecret := &corev1.Secret{ | ||
| ObjectMeta: metav1.ObjectMeta{ | ||
| Name: GetServiceCertsSecretName(instance, serviceName), | ||
| Namespace: instance.Namespace, | ||
| }, | ||
| Data: certsData, | ||
| } | ||
| _, _, err := secret.CreateOrPatchSecret(ctx, helper, instance, serviceCertsSecret) | ||
| if err != nil { | ||
| err = fmt.Errorf("Error creating certs secret for %s - %w", serviceName, err) | ||
| return err | ||
| } | ||
|
|
||
| return nil | ||
| } | ||
|
|
||
| // GetServiceCertsSecretName - return name of secret to be mounted in ansibleEE which contains | ||
| // all the TLS certs for the relevant service | ||
| // The convention we use here is "<nodeset.name>-<service>-certs", so for example, | ||
| // openstack-epdm-nova-certs. | ||
| func GetServiceCertsSecretName(instance *dataplanev1.OpenStackDataPlaneNodeSet, serviceName string) string { | ||
| return fmt.Sprintf("%s-%s-certs", instance.Name, serviceName) | ||
| } | ||
|
|
||
| // IssueTLSCert - issues a cert with the relevant parameters and stores it in a secret | ||
| // It probably makes sense to put this function in lib-common | ||
| func IssueTLSCert(ctx context.Context, helper *helper.Helper, | ||
| name string, namespace string, secretName string, | ||
| hostname string, dnsnames []string, ips []string, | ||
| issuerName string, issuerNamespace string, | ||
| labels map[string]string) (*corev1.Secret, error) { | ||
| issuer := &certmgrv1.Issuer{} | ||
| err := helper.GetClient().Get(ctx, types.NamespacedName{Name: issuerName, Namespace: issuerNamespace}, issuer) | ||
| if err != nil { | ||
| err = fmt.Errorf("Error getting issuer %s/%s - %w", issuerName, issuerNamespace, err) | ||
| return nil, err | ||
| } | ||
| certReq := certmanager.Cert( | ||
| name, | ||
| namespace, | ||
| labels, | ||
| certmgrv1.CertificateSpec{ | ||
| CommonName: hostname, | ||
| DNSNames: dnsnames, | ||
| IPAddresses: ips, | ||
| Duration: &metav1.Duration{ | ||
| Duration: time.Hour * 24 * 365, | ||
| }, | ||
| IssuerRef: certmgrmetav1.ObjectReference{ | ||
| Name: issuer.Name, | ||
| Kind: issuer.Kind, | ||
| Group: issuer.GroupVersionKind().Group, | ||
| }, | ||
| SecretName: secretName, | ||
| }, | ||
| ) | ||
| // Note: 5 here is a timeout (in minutes?) | ||
| cert := certmanager.NewCertificate(certReq, 5) | ||
| _, err = cert.CreateOrPatch(ctx, helper) | ||
| if err != nil { | ||
| err = fmt.Errorf("Error issuing certificate %s - %w", name, err) | ||
| return nil, err | ||
| } | ||
|
|
||
| certSecret, _, err := secret.GetSecret(ctx, helper, secretName, namespace) | ||
| if err != nil { | ||
| err = fmt.Errorf("Error retrieving secret %s - %w", secretName, err) | ||
| return nil, err | ||
| } | ||
|
|
||
| return certSecret, nil | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters