Status of testing of Apache Airflow Helm Chart 1.10.0rc1

[jedcunningham]

We are kindly requesting that contributors to Apache Airflow Helm Chart 1.10.0rc1 help test the RC.

Please let us know by commenting if the issue is addressed in the latest RC.

• Use template comments for the chart license header (#30569): @dnskr
• Support minAvailable property for PodDisruptionBudget (#30603): @dzhigimont
• fix(chart): webserver probes timeout and period. (#30609): @mikaeld
• Validate executor and config.core.executor match (#30693): @jedcunningham
• Adding loggroomer sidecar to dag processor (#30726): @wookiist
• Template extra volumes in helm chart (#30773): @nirroz93
• Remove Python 3.7 support (#30963): @potiuk - not relevant
• Chart: add custom volumemounts to dag processor waitForMigrations init container (#30990): @Aakcht
• feat(chart): add container security context (#31043): @NipunaPrashan @mikaeld
  Linked issues:
  • [Chart] Container security context is not defined (#27612)
• Add missing priorityClassName to K8S worker pod template (#31328): @hussein-awala @TarunParimi
  Linked issues:
  • Worker pod template file doesn't have option to add priorityClassName (#31311)
• Add missing waitForMigrations for workers (#31625): @grjones
• Cleanup Kubernetes < 1.23 support (#31847): @dnskr
• Align apiVersion and kind order in chart templates (#31850): @dnskr
• Chart: Do not propagate global security context to statsd and redis (#31865): @Aakcht
• Chart: Default airflow version to 2.6.2 (#31979): @eladkal

Thanks to all who contributed to the release (probably not a complete list!):
@potiuk @ephraimbuddy @wookiist @dzhigimont @hussein-awala @mikaeld @nirroz93 @eladkal @grjones @jedcunningham @TarunParimi @NipunaPrashan @Aakcht @dnskr

[Aakcht]

Tested #31865 - all good.

As for #30990 looks like it got mixed up with #30773 and the extraVolumeMounts of dagprocessor wait for migrations init container are not templated and extraVolumeMounts of other containers are templated. I added a PR to fix this: #32100

[pankajkoti]

I am wondering if we want to include #30963 here as the PR has an associated milestone of 2.7.0

If we want to include it here, I have a question next. When I deploy the helm chart from this RC, it runs fine and example DAGs are running fine. But when I exec into pods, and enter python shell it uses Python3.7, also example DAGs with Python operator seem to use Python3.7. What should be the expected Python version here?

[potiuk]

I am wondering if we want to include #30963 here as the PR has an associated milestone of 2.7.0

Python version is connected with Airflow version. It's not "chart" property it's actually "image" property. And according to our agreed policies, default versio of Airflow supported in 2.6 is 3.7.

From: https://github.com/apache/airflow/blob/main/README.md#support-for-python-and-kubernetes-versions

We drop support for Python and Kubernetes versions when they reach EOL. Except for Kubernetes, a version stays supported by Airflow if two major cloud providers still provide support for it. We drop support for those EOL versions in main right after EOL date, and it is effectively removed when we release the first new MINOR (Or MAJOR if there is no new MINOR version) of Airflow. For example, for Python 3.8 it means that we will drop support in main right after 27.06.2023, and the first MAJOR or MINOR version of Airflow released after will not have it.

I think there is a one mistake in README regarding 2.6.2 image after we bumped version numbers (I will fix it in a moment) but intention was that 3.7 will remain default and released version for all 2.6* airflow/image versions. The 3.8 will become default/minimum version supported when we release 2.7.0.

[potiuk]

#32103 -> to remove the confusing paragraph.

[dnskr]

• Use template comments for the chart license header #30569
  Looks good, all license header replacements are in place.

• Cleanup Kubernetes < 1.23 support #31847
  Checked. There are no kubeVersion references.

• Align apiVersion and kind order in chart templates #31850
  Looks good, all changes are in place.

[potiuk]

BTW @pankajkoti clarifying comment:

The reason why the PR to remove support was included in the list was this https://github.com/apache/airflow/pull/30963/files#diff-b3ffb75587232c802c4c65d8d27f30a07f72e1819e3a695ca5758d727bde3b31L2079

"description": "Enable triggerer (requires Python 3.7+).",-> "description": "Enable triggerer",

And well - this was actually an overdue change. This should have been removed already when we got rid of airflow 3.6 - I just removed it while removing python 3.7, because I looked for all the 3.7 occurences. So even if "Remove Python 3.7" support was part of the PRs in the chart, the actual removal of Python 3.7 had nothing to do with the chart

[potiuk]

Looks good !

[pankajkoti]

Thank you for clarifying in detail @potiuk and the quick fix. Yes, was wondering intially if the image itself needed some fix, but all clear now.

[jedcunningham]

1.10.0 is (almost) out! Thanks everyone for taking the time to test the RC!