Unnecessary CA replacement run with custom CA

[scholzj]

When replacing custom CA or when migrating from Strimzi CA to a custom CA, the operator seems to do unnecessarily rolling update. Currently, the operator does the following:

• In CA Reconciler:
  • Sees from the changed cert and key generation annotations and identifies what is happening
  • When the key is replaced, postpones the CO secret update as it first needs to roll out the trust to the new public key based on the new CA
  • Rolls all components to trust the new CA
• In regular reconciliation:
  • Generates new server certs and updates the secrets and their generations
  • Rolls Pods to use the new generation
• In the next reconciliation in CA Reconciler
  • The CO Secret has not been updated initially so it still has the old generation
  • The CA detects that as CA replacement in progress and repeats the whole previous cycle (this time with the CA Secret update, so only one unnecessary cycle happens and not a neverending loop) instead of just updating the CO certs