KCP does not report the right number of up-to-date machines during rollout

[fabriziopandini]

What steps did you take and what happened:
I created a 3CP control plane
I started a rolling upgrade and saw Rolling 1 replicas with outdated spec (2 replicas up to date) - tot 3 replicas - but as you can see below there are 4 machines in the cluster...

├─ControlPlane - KubeadmControlPlane/test1-control-plane  False  Warning   RollingUpdateInProgress  3m22s  Rolling 1 replicas with outdated spec (2 replicas up to date)
│ ├─Machine/test1-control-plane-2r4qf                     True                                      17m
│ │             ├─BootstrapReady                          True                                      18m
│ │             └─InfrastructureReady                     True                                      17m
│ ├─Machine/test1-control-plane-65tsl                     True                                      6m34s
│ │             ├─BootstrapReady                          True                                      7m25s
│ │             └─InfrastructureReady                     True                                      6m34s
│ ├─Machine/test1-control-plane-kqdr4                     True                                      10m
│ │             ├─BootstrapReady                          True                                      14m
│ │             └─InfrastructureReady                     True                                      10m
│ └─Machine/test1-control-plane-zd595                     True                                      2m40s
│               ├─BootstrapReady                          True                                      3m22s
│               └─InfrastructureReady                     True                                      2m40s

What did you expect to happen:
To see 3 replicas up to date instead on 2

├─ControlPlane - KubeadmControlPlane/test1-control-plane  False  Warning   RollingUpdateInProgress  3m22s  Rolling 1 replicas with outdated spec (3 replicas up to date)

/kind bug

[fabriziopandini]

/milestone v0.3.11

[fabriziopandini]

/assign
/lifecycle active

[CecileRobertMichon]

Could this be related to the issues we're seeing in the CAPZ KCP upgrade job kubernetes-sigs/cluster-api-provider-azure#932 (comment)? cc @nader-ziada

[nader-ziada]

Could this be related to the issues we're seeing in the CAPZ KCP upgrade job kubernetes-sigs/cluster-api-provider-azure#932 (comment)? cc @nader-ziada

it could be since this would make the upgrade process take longer, the part of the test where capz test gets stuck is getting a list of machines and checking that there are no machines with the the old version (upgrade from version).

[fabriziopandini]

#3900 is merged
/close

[k8s-ci-robot]

@fabriziopandini: Closing this issue.

In response to this:

#3900 is merged
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.