Clean up ./cloud/services

[devigned]

/kind cleanup

Describe the solution you'd like
A while back, the code used to interact with Azure under ./cloud/services was interfaced to allow some separation of concern between the controller / scopes and the types from the Azure SDK. With the advent of the azure.Service interface, Azure SDK / client code doesn't leak into the controllers. In fact, there should not longer be dependencies on the code that directly interacts with Azure. We should use this as an opportunity to crystallize this design goal of refactoring to azure.Service by cleaning up the service internals.

Current structure:

cloud/services/scalesets/
├── client.go
├── mock_scalesets
│   ├── client_mock.go
│   ├── doc.go
│   └── scalesets_mock.go
├── service.go
├── vmss.go
└── vmss_test.go

Proposed structure:

cloud/services/scalesets/
├── client.go  <-- don't export this; make it private
├── mock_scalesets
│   ├── client_mock.go
│   ├── doc.go
│   └── scaleset_mock.go
├── scaleset.go <-- includes service and scope
└── scaleset_test.go

The only thing exported from the new structure is the Service struct and the Scope interface for each service. This will ensure that the encapsulation will not be violated and guide future contributors.

[CecileRobertMichon]

/help

[k8s-ci-robot]

@CecileRobertMichon:
This request has been marked as needing help from a contributor.

Please ensure the request meets the requirements listed here.

If this request no longer meets these requirements, the label can be removed
by commenting with the /remove-help command.

In response to this:

/help

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[shysank]

/assign

[devigned]

@shysank please don't hesitate to ping me if you have any questions. Might be best to refactor one service in a PR before refactoring all at the same time.

[alexeldeib]

any thoughts on how to replace code like

cluster-api-provider-azure/exp/controllers/azuremangedcontrolplane_reconciler.go

Lines 219 to 263 in e92bba9

	func (r *azureManagedControlPlaneReconciler) reconcileEndpoint(ctx context.Context, scope *scope.ManagedControlPlaneScope, managedClusterSpec *managedclusters.Spec) error {
	// Fetch newly updated cluster
	managedClusterResult, err := r.managedClustersSvc.Get(ctx, managedClusterSpec)
	if err != nil {
	return err
	}
	managedCluster, ok := managedClusterResult.(containerservice.ManagedCluster)
	if !ok {
	return fmt.Errorf("expected containerservice ManagedCluster object")
	}
	old := scope.ControlPlane.DeepCopyObject()
	scope.ControlPlane.Spec.ControlPlaneEndpoint = clusterv1.APIEndpoint{
	Host: *managedCluster.ManagedClusterProperties.Fqdn,
	Port: 443,
	}
	if err := r.kubeclient.Patch(ctx, scope.ControlPlane, client.MergeFrom(old)); err != nil {
	return errors.Wrapf(err, "failed to set control plane endpoint")
	}
	return nil
	}
	func (r *azureManagedControlPlaneReconciler) reconcileKubeconfig(ctx context.Context, scope *scope.ManagedControlPlaneScope, managedClusterSpec *managedclusters.Spec) error {
	// Always fetch credentials in case of rotation
	data, err := r.managedClustersSvc.GetCredentials(ctx, managedClusterSpec.ResourceGroup, managedClusterSpec.Name)
	if err != nil {
	return errors.Wrapf(err, "failed to get credentials for managed cluster")
	}
	// Construct and store secret
	kubeconfig := makeKubeconfig(scope.Cluster, scope.ControlPlane)
	if _, err := controllerutil.CreateOrUpdate(ctx, r.kubeclient, kubeconfig, func() error {
	kubeconfig.Data = map[string][]byte{
	secret.KubeconfigDataName: data,
	}
	return nil
	}); err != nil {
	return errors.Wrapf(err, "failed to kubeconfig secret for cluster")
	}
	return nil
	}

?

notably, I don't see many other reconcilers which write data back into the CRD?

[alexeldeib]

that blurb is long -- referencing managedClustersSvc.Get and managedClustersSvc.GetCredentials to reconcile endpoint/kubeconfig

[alexeldeib]

I see:

cluster-api-provider-azure/cloud/services/virtualmachines/virtualmachines.go

Line 69 in e92bba9

s.Scope.SetAddresses(existingVM.Addresses)

[devigned]

Ahh... the code for *MachinePool and the rest in ./exp have not been refactored to the use the scope pattern. See #757.

I would focus on a service not used by the ./exp packages.

[alexeldeib]

I was thinking about how to refactor them in line with the same pattern (or if we should avoid that?) seems like the above solution would be one approach

[devigned]

We should refactor them. I've been remiss in not getting to them.

[shysank]

@devigned I'm trying to refactor disks service, and have a couple of questions:

1. Should we also make service struct private, and only export the scope interface, and the service initialization function?
2. Should we divide azure.Service into azure.Reconciler, and azure.Deleter to avoid the no-op in disks.Reconcile?

[devigned]

Great questions, @shysank.

1. I generally refrain from returning interfaces, but in this situation, it might be a good idea to ensure that a consumer of the API can only expect an azure.Service.
2. I think this will include a lot of work as you would then need to introduce disks.NewReconciler() azure.Reconciler and disks.NewDeleter azure.Deleter. I haven't thought through all of the implications of this change. I lean toward adding the no-op for now, returning an azure.Service, and possibly revisit in the future.

@CecileRobertMichon & @nader-ziada wdyt about this?

[nader-ziada]

Great questions, @shysank.

1. I generally refrain from returning interfaces, but in this situation, it might be a good idea to ensure that a consumer of the API can only expect an azure.Service.
2. I think this will include a lot of work as you would then need to introduce disks.NewReconciler() azure.Reconciler and disks.NewDeleter azure.Deleter. I haven't thought through all of the implications of this change. I lean toward adding the no-op for now, returning an azure.Service, and possibly revisit in the future.

@CecileRobertMichon & @nader-ziada wdyt about this?

what is the reason to return an interface? its usually not a best practice and I worry we might be complicating things by adding extra abstractions

[shysank]

what is the reason to return an interface? its usually not a best practice and I worry we might be complicating things by adding extra abstractions

I guess my main concern was to avoid code like this disks.Service{} for initialization, and only use the exported api's. But, I agree that it's not a good idea to return an interface, although in this case, we will still return the actual type except that it can only be inferred.

[devigned]

I was recommending an interface since at some point in the future, we will likely replace the cloud code to use Azure Service Operator to reconcile Azure resources rather than the cloud services directly.

As long as Reconcile and Delete are the only funcs exported, I'm 💯 behind return in a struct rather than an interface.

[alexeldeib]

we will likely replace the cloud code to use Azure Service Operator to reconcile Azure resources rather than the cloud services directly

Just a thought, I'm noticing several places where exporting Reconcile/Delete and handing that off to another implementer is not really sufficient. There are several places where the services write back into crds, several of which have bugs or awkward ergonomics right now. I would not expect ASO or another project to implement some of these things as part of reconcile a resource, since they are CAPI/CAPZ specific.

examples:

• cluster-api-provider-azure/cloud/services/networkinterfaces/networkinterfaces.go

  Line 93 in 272261b

  nicSpec.AcceleratedNetworking = &accelNet

  doesn't actually persist into the CRD

• cluster-api-provider-azure/cloud/services/virtualmachines/virtualmachines.go

  Lines 330 to 339 in 272261b

  	// enable ephemeral OS
  	if vmSpec.OSDisk.DiffDiskSettings != nil {
  	if !sku.HasCapability(resourceskus.EphemeralOSDisk) {
  	return nil, fmt.Errorf("vm size %s does not support ephemeral os. select a different vm size or disable ephemeral os", vmSpec.Size)
  	}
  	storageProfile.OsDisk.DiffDiskSettings = &compute.DiffDiskSettings{
  	Option: compute.DiffDiskOptions(vmSpec.OSDisk.DiffDiskSettings.Option),
  	}
  	}

  should be similar to above: when diffDiskSettings is nil, we should validate the VM size either has a large enough cache or temp disk, and enable ephemeral os by default (I think!). This would require writing back into the CRD, which will not work cleanly in the same way as acceleratednetworking.

• cluster-api-provider-azure/cloud/services/virtualnetworks/virtualnetworks.go

  Line 74 in 272261b

  existingVnet.DeepCopyInto(s.Scope.Vnet())

  won't work for managed control plane

• cluster-api-provider-azure/cloud/services/scalesets/vmss.go

  Line 81 in 272261b

  vmssSpec.AcceleratedNetworking = &accelNet

  might be broken same as for NIC. at the very least this should be done via setters or something.

other random opportunities for cleanup:

• vmspecs probably shouldn't be a slice:

  cluster-api-provider-azure/cloud/scope/machine.go

  Line 92 in 272261b

  func (m *MachineScope) VMSpecs() []azure.VMSpec {

• vnetspecs probably shouldn't be a slice:

  cluster-api-provider-azure/cloud/scope/cluster.go

  Line 206 in 272261b

  func (s *ClusterScope) VNetSpecs() []azure.VNetSpec {

[nader-ziada]

we still need to refactor the other services

/reopen

[k8s-ci-robot]

@nader-ziada: Reopened this issue.

In response to this:

we still need to refactor the other services

/reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[nader-ziada]

@shysank are you planning on working on the other services as well?

[shysank]

@nader-ziada Yeah, I'll take on the rest. Just caught up on some pending work in capi prs. Will start working on this once that is done.

[shysank]

/assign