Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[NovaAPI]Fix www_authenticate_uri configuration #535

Merged
merged 1 commit into from
Oct 2, 2023
Merged

[NovaAPI]Fix www_authenticate_uri configuration #535

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions api/bases/nova.openstack.org_novaapis.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,13 @@ spec:
in /etc/<service> .
type: object
keystoneAuthURL:
description: KeystoneAuthURL configures the keystone API endpoint
to be used by the service for authentication and authorization
type: string
keystonePublicAuthURL:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dumb question but can we look this up form the keystone catalog?

I'm not nessisarly against adding this just wondering if it is needed

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can. We can look for KeystoneEndpoint CRs (how the Nova controller does it to fill this value), or we can use the golang sdk and call keytone internal api to get the catalog and get the public URL. I went with the former in the nova controller as we already looking for KeystoneEndpoints there.

description: KeystonePublicAuthURL configures the public keystone
API endpoint. This can be different from KeystoneAuthURL. The service
uses this value to redirect unauthenticated users.
type: string
networkAttachments:
description: NetworkAttachments is a list of NetworkAttachment resource
Expand Down Expand Up @@ -383,6 +390,7 @@ spec:
- apiDatabaseHostname
- cell0DatabaseHostname
- keystoneAuthURL
- keystonePublicAuthURL
- registeredCells
- secret
- serviceAccount
Expand Down
8 changes: 8 additions & 0 deletions api/v1beta1/novaapi_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -100,8 +100,16 @@ type NovaAPISpec struct {
ServiceUser string `json:"serviceUser"`

// +kubebuilder:validation:Required
// KeystoneAuthURL configures the keystone API endpoint to be used
// by the service for authentication and authorization
KeystoneAuthURL string `json:"keystoneAuthURL"`

// +kubebuilder:validation:Required
// KeystonePublicAuthURL configures the public keystone API endpoint. This
// can be different from KeystoneAuthURL. The service uses this value
// to redirect unauthenticated users.
KeystonePublicAuthURL string `json:"keystonePublicAuthURL"`

// +kubebuilder:validation:Optional
// +kubebuilder:default="nova_api"
// APIDatabaseUser - username to use when accessing the API DB
Expand Down
8 changes: 8 additions & 0 deletions config/crd/bases/nova.openstack.org_novaapis.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,13 @@ spec:
in /etc/<service> .
type: object
keystoneAuthURL:
description: KeystoneAuthURL configures the keystone API endpoint
to be used by the service for authentication and authorization
type: string
keystonePublicAuthURL:
description: KeystonePublicAuthURL configures the public keystone
API endpoint. This can be different from KeystoneAuthURL. The service
uses this value to redirect unauthenticated users.
type: string
networkAttachments:
description: NetworkAttachments is a list of NetworkAttachment resource
Expand Down Expand Up @@ -383,6 +390,7 @@ spec:
- apiDatabaseHostname
- cell0DatabaseHostname
- keystoneAuthURL
- keystonePublicAuthURL
- registeredCells
- secret
- serviceAccount
Expand Down
43 changes: 27 additions & 16 deletions controllers/nova_controller.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -222,7 +222,8 @@ func (r *NovaReconciler) Reconcile(ctx context.Context, req ctrl.Request) (resul
return ctrl.Result{}, nil
}

keystoneAuthURL, err := r.getKeystoneAuthURL(ctx, h, instance)
keystoneInternalAuthURL, keystonePublicAuthURL, err := r.getKeystoneAuthURL(
ctx, h, instance)
if err != nil {
return ctrl.Result{}, err
}
Expand Down Expand Up @@ -430,7 +431,7 @@ func (r *NovaReconciler) Reconcile(ctx context.Context, req ctrl.Request) (resul
cell, status, err := r.ensureCell(
ctx, h, instance, cellName, cellTemplate,
cellDB.Database, apiDB, cellMQ.TransportURL,
keystoneAuthURL, secret,
keystoneInternalAuthURL, secret,
)
cells[cellName] = cell
switch status {
Expand Down Expand Up @@ -497,7 +498,8 @@ func (r *NovaReconciler) Reconcile(ctx context.Context, req ctrl.Request) (resul

result, err = r.ensureAPI(
ctx, h, instance, cell0Template,
cellDBs[novav1.Cell0Name].Database, apiDB, keystoneAuthURL,
cellDBs[novav1.Cell0Name].Database, apiDB,
keystoneInternalAuthURL, keystonePublicAuthURL,
topLevelSecretName,
)
if err != nil {
Expand All @@ -506,7 +508,7 @@ func (r *NovaReconciler) Reconcile(ctx context.Context, req ctrl.Request) (resul

result, err = r.ensureScheduler(
ctx, h, instance, cell0Template,
cellDBs[novav1.Cell0Name].Database, apiDB, keystoneAuthURL,
cellDBs[novav1.Cell0Name].Database, apiDB, keystoneInternalAuthURL,
topLevelSecretName,
)
if err != nil {
Expand All @@ -516,7 +518,7 @@ func (r *NovaReconciler) Reconcile(ctx context.Context, req ctrl.Request) (resul
if *instance.Spec.MetadataServiceTemplate.Enabled {
result, err = r.ensureMetadata(
ctx, h, instance, cell0Template,
cellDBs[novav1.Cell0Name].Database, apiDB, keystoneAuthURL,
cellDBs[novav1.Cell0Name].Database, apiDB, keystoneInternalAuthURL,
topLevelSecretName,
)
if err != nil {
Expand Down Expand Up @@ -815,7 +817,8 @@ func (r *NovaReconciler) ensureAPI(
cell0Template novav1.NovaCellTemplate,
cell0DB *database.Database,
apiDB *database.Database,
keystoneAuthURL string,
keystoneInternalAuthURL string,
keystonePublicAuthURL string,
secretName string,
) (ctrl.Result, error) {
// TODO(gibi): Pass down a narrowed secret that only hold
Expand All @@ -836,11 +839,12 @@ func (r *NovaReconciler) ensureAPI(
Resources: instance.Spec.APIServiceTemplate.Resources,
NetworkAttachments: instance.Spec.APIServiceTemplate.NetworkAttachments,
},
Override: instance.Spec.APIServiceTemplate.Override,
KeystoneAuthURL: keystoneAuthURL,
ServiceUser: instance.Spec.ServiceUser,
ServiceAccount: instance.RbacResourceName(),
RegisteredCells: instance.Status.RegisteredCells,
Override: instance.Spec.APIServiceTemplate.Override,
KeystoneAuthURL: keystoneInternalAuthURL,
KeystonePublicAuthURL: keystonePublicAuthURL,
ServiceUser: instance.Spec.ServiceUser,
ServiceAccount: instance.RbacResourceName(),
RegisteredCells: instance.Status.RegisteredCells,
}
api := &novav1.NovaAPI{
ObjectMeta: metav1.ObjectMeta{
Expand Down Expand Up @@ -1067,22 +1071,29 @@ func (r *NovaReconciler) getKeystoneAuthURL(
ctx context.Context,
h *helper.Helper,
instance *novav1.Nova,
) (string, error) {
) (string, string, error) {
// TODO(gibi): change lib-common to take the name of the KeystoneAPI as
// parameter instead of labels. Then use instance.Spec.KeystoneInstance as
// the name.
keystoneAPI, err := keystonev1.GetKeystoneAPI(ctx, h, instance.Namespace, map[string]string{})
if err != nil {
return "", err
return "", "", err
}
// NOTE(gibi): we use the internal endpoint as that is expected to be
// available on the external compute nodes as well and we want to keep
// thing consistent
authURL, err := keystoneAPI.GetEndpoint(endpoint.EndpointInternal)
internalAuthURL, err := keystoneAPI.GetEndpoint(endpoint.EndpointInternal)
if err != nil {
return "", "", err
}
// NOTE(gibi): but there is one case the www_authenticate_uri of nova-api
// the we need to configure the public keystone endpoint
publicAuthURL, err := keystoneAPI.GetEndpoint(endpoint.EndpointInternal)
if err != nil {
return "", err
return "", "", err
}
return authURL, nil

return internalAuthURL, publicAuthURL, nil
}

func (r *NovaReconciler) reconcileDelete(
Expand Down
7 changes: 5 additions & 2 deletions controllers/novaapi_controller.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -327,8 +327,11 @@ func (r *NovaAPIReconciler) generateConfigs(
) error {

templateParameters := map[string]interface{}{
"service_name": "nova-api",
"keystone_internal_url": instance.Spec.KeystoneAuthURL,
"service_name": "nova-api",
"keystone_internal_url": instance.Spec.KeystoneAuthURL,
// NOTE(gibi): As per the definition of www_authenticate_uri this
// always needs to point to the public keystone endpoint.
"www_authenticate_uri": instance.Spec.KeystonePublicAuthURL,
"nova_keystone_user": instance.Spec.ServiceUser,
"nova_keystone_password": string(secret.Data[ServicePasswordSelector]),
"api_db_name": instance.Spec.APIDatabaseUser, // fixme
Expand Down
4 changes: 3 additions & 1 deletion templates/nova.conf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -183,7 +183,9 @@ connection = mysql+pymysql://{{ .api_db_user }}:{{ .api_db_password }}@{{ .api_d
{{end}}

[keystone_authtoken]
www_authenticate_uri = {{ .keystone_internal_url }}
{{ if eq .service_name "nova-api"}}
www_authenticate_uri = {{ .www_authenticate_uri}}
{{end}}
auth_url = {{ .keystone_internal_url }}
auth_type = password
project_domain_name = {{ .default_project_domain }}
Expand Down
3 changes: 2 additions & 1 deletion test/functional/base_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,8 @@ func GetDefaultNovaAPISpec(novaNames NovaNames) map[string]interface{} {
"secret": novaNames.InternalTopLevelSecretName.Name,
"apiDatabaseHostname": "nova-api-db-hostname",
"cell0DatabaseHostname": "nova-cell0-db-hostname",
"keystoneAuthURL": "keystone-auth-url",
"keystoneAuthURL": "keystone-internal-auth-url",
"keystonePublicAuthURL": "keystone-public-auth-url",
"containerImage": ContainerImage,
"serviceAccount": "nova",
"registeredCells": map[string]string{},
Expand Down
2 changes: 2 additions & 0 deletions test/functional/novaapi_controller_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -179,6 +179,8 @@ var _ = Describe("NovaAPI controller", func() {
Expect(configData).Should(ContainSubstring("service_token_roles_required = true"))
Expect(configData).Should(ContainSubstring("enabled_apis=osapi_compute"))
Expect(configData).Should(ContainSubstring("osapi_compute_workers=1"))
Expect(configData).Should(ContainSubstring("auth_url = keystone-internal-auth-url"))
Expect(configData).Should(ContainSubstring("www_authenticate_uri = keystone-public-auth-url"))
Expect(configDataMap.Data).Should(HaveKey("02-nova-override.conf"))
extraData := string(configDataMap.Data["02-nova-override.conf"])
Expect(extraData).To(Equal("foo=bar"))
Expand Down
Loading