Creating a cluster with a privateLink (ACR) will lead capz-controller-manager to crash loop on AzureCluster reconcile

[arsiesys]

/kind bug

What steps did you take and what happened:

• Create a new AzureCluster including a privateLink (to an ACR registry) will make the capz-controller-manager crash loop with the go trace below. The crash will happen after the controle-plane being created. It seems to happen at the reconcile.

│ I0919 18:33:44.407635       1 azuremachinepool_controller.go:271] controllers.AzureMachinePoolReconciler.reconcileNormal "msg"="Reconciling AzureMachinePool" "AzureMachinePool"={"name":"test-mp-0","namespace":"default"} "controller"="azuremachinepool" "controllerGroup"="infrastructure.cluster.x-k8s.io" "controllerKind"="AzureMachinePool" "na │
│ I0919 18:33:44.407682       1 azuremachinepool_controller.go:287] controllers.AzureMachinePoolReconciler.reconcileNormal "msg"="Cluster infrastructure is not ready yet" "AzureMachinePool"={"name":"test-mp-0","namespace":"default"} "controller"="azuremachinepool" "controllerGroup"="infrastructure.cluster.x-k8s.io" "controllerKind"="AzureMachi │
│ I0919 18:33:45.311988       1 azuremachine_controller.go:166] controllers.AzureMachineReconciler.Reconcile "msg"="Machine Controller has not yet set OwnerRef" "AzureMachine"={"name":"test-control-plane-qdzqk","namespace":"default"} "controller"="azuremachine" "controllerGroup"="infrastructure.cluster.x-k8s.io" "controllerKind"="AzureMachine" │
│ I0919 18:33:54.424712       1 azuremachinepool_controller.go:271] controllers.AzureMachinePoolReconciler.reconcileNormal "msg"="Reconciling AzureMachinePool" "AzureMachinePool"={"name":"test-mp-0","namespace":"default"} "controller"="azuremachinepool" "controllerGroup"="infrastructure.cluster.x-k8s.io" "controllerKind"="AzureMachinePool" "na │
│ I0919 18:33:54.424812       1 azuremachinepool_controller.go:293] controllers.AzureMachinePoolReconciler.reconcileNormal "msg"="Bootstrap data secret reference is not yet available" "AzureMachinePool"={"name":"test-mp-0","namespace":"default"} "controller"="azuremachinepool" "controllerGroup"="infrastructure.cluster.x-k8s.io" "controllerKind │
│ I0919 18:33:54.454408       1 azurecluster_controller.go:205] controllers.AzureClusterReconciler.reconcileNormal "msg"="Reconciling AzureCluster" "AzureCluster"={"name":"test","namespace":"default"} "controller"="azurecluster" "controllerGroup"="infrastructure.cluster.x-k8s.io" "controllerKind"="AzureCluster" "name"="test" "namespace"="defau │
│ I0919 18:33:55.328434       1 azuremachine_controller.go:243] controllers.AzureMachineReconciler.reconcileNormal "msg"="Reconciling AzureMachine" "AzureMachine"={"name":"test-control-plane-qdzqk","namespace":"default"} "controller"="azuremachine" "controllerGroup"="infrastructure.cluster.x-k8s.io" "controllerKind"="AzureMachine" "name"="test │
│ I0919 18:33:55.341220       1 machine.go:716] scope.MachineScope.GetVMImage "msg"="No image specified for machine, using default Linux Image" "AzureMachine"={"name":"test-control-plane-qdzqk","namespace":"default"} "controller"="azuremachine" "controllerGroup"="infrastructure.cluster.x-k8s.io" "controllerKind"="AzureMachine" "machine"="test- │
│ I0919 18:33:56.838191       1 controller.go:114]  "msg"="Observed a panic in reconciler: runtime error: invalid memory address or nil pointer dereference" "AzureCluster"={"name":"test","namespace":"default"} "controller"="azurecluster" "controllerGroup"="infrastructure.cluster.x-k8s.io" "controllerKind"="AzureCluster" "name"="test" "namespac │
│ panic: runtime error: invalid memory address or nil pointer dereference [recovered]                                                                                                                                                                                                                                                                     │
│     panic: runtime error: invalid memory address or nil pointer dereference                                                                                                                                                                                                                                                                             │
│ [signal SIGSEGV: segmentation violation code=0x1 addr=0x78 pc=0x1ba32e4]                                                                                                                                                                                                                                                                                │
│                                                                                                                                                                                                                                                                                                                                                         │
│ goroutine 595 [running]:                                                                                                                                                                                                                                                                                                                                │
│ sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile.func1()                                                                                                                                                                                                                                                                  │
│     /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:115 +0x1fa                                                                                                                                                                                                                                                 │
│ panic({0x21ffde0, 0x3cacae0})                                                                                                                                                                                                                                                                                                                           │
│     /usr/local/go/src/runtime/panic.go:884 +0x213                                                                                                                                                                                                                                                                                                       │
│ sigs.k8s.io/cluster-api-provider-azure/azure/services/privateendpoints.normalizePrivateEndpoint({0x0, 0xc000c8c710, 0xc000210950, 0xc000a9c9a0, 0xc000b8def0, 0xc000c8c7a0, 0xc000c8c700, 0xc0002108a0})                                                                                                                                                │
│     /workspace/azure/services/privateendpoints/spec.go:191 +0x144                                                                                                                                                                                                                                                                                       │
│ sigs.k8s.io/cluster-api-provider-azure/azure/services/privateendpoints.(*PrivateEndpointSpec).Parameters(0xc000c96cc0, {0x29b0a58?, 0xc00175dd10?}, {0x2509a80?, 0xc00133b940?})                                                                                                                                                                        │
│     /workspace/azure/services/privateendpoints/spec.go:166 +0x1059                                                                                                                                                                                                                                                                                      │
│ sigs.k8s.io/cluster-api-provider-azure/azure/services/async.(*Service[...]).CreateOrUpdateResource(0x29b0600, {0x29b0a20, 0xc00175dc50?}, {0x29b19e0?, 0xc000c96cc0}, {0x263ee07, 0x10})                                                                                                                                                                │
│     /workspace/azure/services/async/async.go:90 +0x490                                                                                                                                                                                                                                                                                                  │
│ sigs.k8s.io/cluster-api-provider-azure/azure/services/privateendpoints.(*Service).Reconcile(0xc0014b0540, {0x29b0a58?, 0xc001588120?})                                                                                                                                                                                                                  │
│     /workspace/azure/services/privateendpoints/privateendpoints.go:85 +0x186                                                                                                                                                                                                                                                                            │
│ sigs.k8s.io/cluster-api-provider-azure/controllers.(*azureClusterService).Reconcile(0xc001588090, {0x29b0a58?, 0xc0018c4570?})                                                                                                                                                                                                                          │
│     /workspace/controllers/azurecluster_reconciler.go:135 +0x19e                                                                                                                                                                                                                                                                                        │
│ sigs.k8s.io/cluster-api-provider-azure/controllers.(*AzureClusterReconciler).reconcileNormal(0xc00057d440, {0x29b0a58?, 0xc001340f30?}, 0xc00121b500)                                                                                                                                                                                                   │
│     /workspace/controllers/azurecluster_controller.go:221 +0x186                                                                                                                                                                                                                                                                                        │
│ sigs.k8s.io/cluster-api-provider-azure/controllers.(*AzureClusterReconciler).Reconcile(0xc00057d440, {0x29b0a58?, 0xc001340e40?}, {{{0xc0005db020?, 0xa?}, {0xc0005db018?, 0x0?}}})                                                                                                                                                                     │
│     /workspace/controllers/azurecluster_controller.go:198 +0xb2f                                                                                                                                                                                                                                                                                        │
│ sigs.k8s.io/cluster-api-provider-azure/pkg/coalescing.(*reconciler).Reconcile(0xc00057d580, {0x29b0a58?, 0xc001340cc0?}, {{{0xc0005db020?, 0x0?}, {0xc0005db018?, 0x40e1c7?}}})                                                                                                                                                                         │
│     /workspace/pkg/coalescing/reconciler.go:109 +0x3eb                                                                                                                                                                                                                                                                                                  │
│ sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile(0x29b0a58?, {0x29b0a58?, 0xc001340cc0?}, {{{0xc0005db020?, 0x20e29a0?}, {0xc0005db018?, 0x29924e8?}}})                                                                                                                                                                   │
│     /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:118 +0xc8                                                                                                                                                                                                                                                  │
│ sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler(0xc0003f75e0, {0x29b09b0, 0xc000654b90}, {0x237c700?, 0xc000af2160?})                                                                                                                                                                                             │
│     /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:314 +0x377                                                                                                                                                                                                                                                 │
│ sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem(0xc0003f75e0, {0x29b09b0, 0xc000654b90})                                                                                                                                                                                                                       │
│     /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:265 +0x1d9                                                                                                                                                                                                                                                 │
│ sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2()                                                                                                                                                                                                                                                                    │
│     /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:226 +0x85                                                                                                                                                                                                                                                  │
│ created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2                                                                                                                                                                                                                                                             │
│     /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:222 +0x587  

What did you expect to happen:
Once the controle-plane being setup, the other resources (Machine Pools) should be provisionned.

Anything else you would like to add:
AzureCluster ressource configuration:

apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: AzureCluster
metadata:
  name: test
  namespace: default
spec:
  identityRef:
    apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
    kind: AzureClusterIdentity
    name: cluster-identity
  location: eastus
  networkSpec:
    privateDNSZoneName: "test.xxxx.internal"
    apiServerLB:
      type: Internal
      frontendIPs:
      - name: cp
        privateIP: 10.1.0.100
    controlPlaneOutboundLB:
      frontendIPsCount: 1
    nodeOutboundLB:
      frontendIPsCount: 1
    subnets:
    - name: control-plane-subnet
      role: control-plane
      cidrBlocks:
        - 10.1.0.0/17
    - name: node-subnet
      role: node
      cidrBlocks:
        - 10.1.128.0/17
      privateEndpoints:
      - name: registry-test
        location: eastus
        privateLinkServiceConnections:
        - privateLinkServiceID: /subscriptions/xxxx/resourceGroups/registry/providers/Microsoft.ContainerRegistry/registries/xxxxx
          groupIDs:
          - registry
    vnet:
      name: test-vnet
      cidrBlocks:
        - 10.1.0.0/16
      peerings:
      - resourceGroup: bastion-rg
        remoteVnetName: mgmt-vnet
  resourceGroup: test
  subscriptionID: xxx

Environment:

• cluster-api-provider-azure version: 1.11.0 (tested in 1.10.4, no issue with it)
• Kubernetes version: (use kubectl version): 1.27.3 (kind)
• OS (e.g. from /etc/os-release): Ubuntu 22.04

[nojnhuh]

/assign