KubernetesExecutor should default to template image if used #19484
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dstandish
requested review from
ashb,
dimberman,
kaxil and
XD-DENG
as code owners
boring-cyborg
bot
added
provider:cncf-kubernetes
dstandish
force-pushed
the
k8s-config-default-to-template
branch
from
a123a9d to
684a5a1
Compare
Currently, the user must specify image and tag in airflow.cfg, even when they are using a pod template file. If the pod template file specifies an image and tag, the user should not be forced to also specify this in airflow.cfg.
dstandish
force-pushed
the
k8s-config-default-to-template
branch
from
684a5a1 to
98e4a1a
Compare
dstandish
commented
Nov 8, 2021
Comment on lines
-41
to
-42
| class TestPodGenerator(unittest.TestCase): | ||
| def setUp(self): |
There was a problem hiding this comment.
converting to pytest is required when combining pytest params and unittest.mock
| @@ -45,7 +45,10 @@ def __init__(self): | |||
|
|
|||
| self.worker_container_repository = conf.get(self.kubernetes_section, 'worker_container_repository') | |||
| self.worker_container_tag = conf.get(self.kubernetes_section, 'worker_container_tag') | |||
| self.kube_image = f'{self.worker_container_repository}:{self.worker_container_tag}' | |||
| if self.worker_container_repository and self.worker_container_tag: | |||
There was a problem hiding this comment.
we could probably loosen this to allow user to specify image only (and not tag) but this is the current behavior. currently if either is not specified it won't work unless specified in executor_config
jedcunningham
approved these changes
Nov 9, 2021
github-actions
bot
added
the
full tests needed
|
The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease. |
|
looks like the one failure is unrelated |
kaxil
approved these changes
Nov 9, 2021
jedcunningham
pushed a commit
that referenced
this pull request
Dec 7, 2021
Currently, the user must specify image and tag in airflow.cfg, even when they are using a pod template file. If the pod template file specifies an image and tag, the user should not be forced to also specify this in airflow.cfg. (cherry picked from commit d18e2b0)
38 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently, the user must specify image and tag in airflow.cfg, even when they are using a pod template file. If the pod template file specifies an image and tag, the user should not be forced to also specify this in airflow.cfg.
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.