random_name_suffix is not working for sparkKubernetesOperator #43800
Labels
area:core
area:providers
kind:bug
This is a clearly a bug
needs-triage
label for new issues that we didn't triage yet
provider:apache-spark
provider:cncf-kubernetes
Kubernetes provider related issues
Comments
Flametaa
added
area:core
kind:bug
|
Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval. |
dosubot
bot
added
area:providers
provider:apache-spark
provider:cncf-kubernetes
mrk-andreev
added a commit
to mrk-andreev/airflow
that referenced
this issue
Nov 9, 2024
Prior to this change, `random_name_suffix` was only documented but not implemented as a configurable option. Passing this value as an argument had no effect. This commit introduces a `false` option for `random_name_suffix`, which prevents the generation of a random suffix for the pod name. For compatibility, the default value is set to `true`, ensuring the pod name will still conform to `MAX_LABEL_LEN = 63`. Fixes: apache#43800
mrk-andreev
added a commit
to mrk-andreev/airflow
that referenced
this issue
Nov 9, 2024
Prior to this change, `random_name_suffix` was only documented but not implemented as a configurable option. Passing this value as an argument had no effect. This commit introduces a `false` option for `random_name_suffix`, which prevents the generation of a random suffix for the pod name. For compatibility, the default value is set to `true`, ensuring the pod name will still conform to `MAX_LABEL_LEN = 63`. Fixes: apache#43800
ellisms
pushed a commit
to ellisms/airflow
that referenced
this issue
Nov 13, 2024
…che#43847) * Add random_name_suffix to SparkKubernetesOperator (apache#43800) Prior to this change, `random_name_suffix` was only documented but not implemented as a configurable option. Passing this value as an argument had no effect. This commit introduces a `false` option for `random_name_suffix`, which prevents the generation of a random suffix for the pod name. For compatibility, the default value is set to `true`, ensuring the pod name will still conform to `MAX_LABEL_LEN = 63`. Fixes: apache#43800 * Add random_name_suffix to SparkKubernetesOperator (apache#43800) Prior to this change, `random_name_suffix` was only documented but not implemented as a configurable option. Passing this value as an argument had no effect. This commit introduces a `false` option for `random_name_suffix`, which prevents the generation of a random suffix for the pod name. For compatibility, the default value is set to `true`, ensuring the pod name will still conform to `MAX_LABEL_LEN = 63`. Fixes: apache#43800
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Apache Airflow version
2.10.3
If "Other Airflow 2 version" selected, which one?
No response
What happened?
In sparkKuberneteOperator, it doesn't matter if you set
random_name_suffixor not as a random suffix is always added without checking this parameter (even though the documentation says that it should be controllable). https://github.com/apache/airflow/blob/providers-cncf-kubernetes/9.0.1/providers/src/airflow/providers/cncf/kubernetes/operators/spark_kubernetes.py#L166What you think should happen instead?
i think setting the parameter to False should result in the sparkApplication name being the same as the
nameHow to reproduce
SparkKubernetesOperator(random_name_suffix=False,...)Operating System
Linux
Versions of Apache Airflow Providers
No response
Deployment
Official Apache Airflow Helm Chart
Deployment details
No response
Anything else?
No response
Are you willing to submit PR?
Code of Conduct
The text was updated successfully, but these errors were encountered: