Support disabling helm hooks on extraConfigMaps and extraSecrets

[ItaiMeushar]

Description

As mentioned in #17447:

Currently it is not possible to avoid the default Helm annotations on resources such as extraConfigMaps and extraSecrets and there's no way in ArgoCD to override / ignore those default hooks.

This issue might be related to ArgoCD’s behavior, though it is not certain, but it can result in pods entering CreateContainerConfigError state during updates like when the scheduler is trying to boot up new tasks or when new Airflow pods are created that depend on extra secrets / configMaps, which the hooks have just deleted.

Solution:

The fix in this PR takes after the same approach and conventions used in PR #18776 for the migrateDatabaseJob and createUserJob resources by using useHelmHooks flag to decide whether or not to add the Helm hooks annotations.

Changes

• Support disabling helm hooks by setting useHelmHooks=false (by default the annotations are added)
• Add helm tests
• Update Airflow Helm documentation

---

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

[boring-cyborg]

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide (https://github.com/apache/airflow/blob/main/contributing-docs/README.rst)
Here are some useful points:

• Pay attention to the quality of your code (ruff, mypy and type annotations). Our pre-commits will help you with that.
• In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
• Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
• Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
• Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
• Be sure to read the Airflow Coding style.
• Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
  Apache Airflow is a community-driven project and together we are making it better 🚀.
  In case of doubts contact the developers at:
  Mailing List: [email protected]
  Slack: https://s.apache.org/airflow-slack

[romsharon98]

Now, we don't get the default annotations because we dont have the key useHelmHooks in the values.yaml.
Can u add it please?

[ItaiMeushar]

Now, we don't get the default annotations because we dont have the key useHelmHooks in the values.json. Can u add it please?

Hey @romsharon98 🙂
What do you mean by adding to values.json? I've looked but didn't find this file.

Since its custom configMaps / secrets controlled by the users, I couldn't really add a default of useHelmHooks: true like we have on the jobs which is why I check if either the key is passed with true or if it doesn't exist ill add the annotations.

I've checked and also added tests for it, there's the test_extra_configmaps_secrets_additional_annotations that checks that the default helm annotations exist when the flag is not passed and I've also added the test_extra_configmaps_secrets_disable_helm_hooks to make sure the hooks are not added if the flag is passed with false.

I've also updated the values.schema.json with useHelmHooks defaults to true but you don't really need to pass anything cause by default it adds the hooks.

Am I missing anything? 🤔

[romsharon98]

Now, we don't get the default annotations because we dont have the key useHelmHooks in the values.json. Can u add it please?

Hey @romsharon98 🙂 What do you mean by adding to values.json? I've looked but didn't find this file.

Since its custom configMaps / secrets controlled by the users, I couldn't really add a default of useHelmHooks: true like we have on the jobs which is why I check if either the key is passed with true or if it doesn't exist ill add the annotations.

I've checked and also added tests for it, there's the test_extra_configmaps_secrets_additional_annotations that checks that the default helm annotations exist when the flag is not passed and I've also added the test_extra_configmaps_secrets_disable_helm_hooks to make sure the hooks are not added if the flag is passed with false.

I've also updated the values.schema.json with useHelmHooks defaults to true but you don't really need to pass anything cause by default it adds the hooks.

Am I missing anything? 🤔

Thanks for clarify it to me 😄. After your clarification this looks good!

[boring-cyborg]

Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions.