-
Notifications
You must be signed in to change notification settings - Fork 14.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
remove K8S 1.24 support #35214
remove K8S 1.24 support #35214
Conversation
|
ok , I remove the prefix "fix" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Airflow policy is "until at least 2 major ( USA 😅😅 ) cloud provider support it |
I think we should consider Alibaba Cloud as a major, no? Otherwise, we should replace "until at least 2 major" with "until at least 2 of (GCP, AWS, Azure)" |
If we include Alibaba (we did not when we set the policy), IMHO we should change to "at least 3 providers out of (GCP/AWS/Azure/Alibaba)" which does not change the outcome. PR maybe @hussein-awala explaining it ? |
Okay, let's keep them 2 of the top 3, but IMHO, we should wait for the next minor version to do that, removing the support in a patch version doesn't make sense. If you agree, I will update the doc to explain what are the major cloud providers and when we can remove the support. |
Absolutely Feel free and we can - I guess keep the discussion in PR and get few approvals for that. Re - which version we drop support, we did not even plan to remove it in Patchlevel release. From our docs:
|
we could just remove
since cloud providers are way more up to date with K8S versions and also Apache airflow is not astronomer and we could have the same policy that we have with python and postgres wdyt ? |
I do not think the rule we have is "Astronomer" rule - it's been proposed by @ashb quite long time ago https://lists.apache.org/thread/fr3tdy4nc8ywo3p0hpy9h04nyv1kldsy, and a few people agreed it makes sense back then without opposal (coincidentally they were all from Astronomer + me :D) . The discussion has (as I re-read it now) quite a good justification on why keep it until it is likely a number of people still use it. A lot of people keep they old clusters running for as long as they can and only forced migration will keep them to move. I believe k8s is a bit different than say "Postgres" - in the sense that it has many more internal components that might make things more difficult to migrate than say migrating the database. K8S is a conglomerate of about a hundred or so different - independently released components internally so there might be one small single thing that might prevent particular installation to migrate to a newer version without some serious pains. But I agree, it would be quite a bit more consistent to have "EOL" as a cut-off time for "software" we release - because we do not run "service". Maybe worth to restart the discussion on devlist and see what people think about it TODAY @raphaelauv ?. I'd support changing it to EOL only. Just personally - I am (for now) generally fine with both approaches but I'd prefer following the "EOL" route. I see why it is good to keep it a little longer, but also I see how strictly following EOL from K8S might actually drive people to update things. Putting my The landscape is changing a lot with Cyber Resilience Act and similar US legislations that all are going to be codified in 2024 and will likely have to be implemented 2/3 years from the moment they pass. It will be legally and financially costly for companies NOT to upgrade the software with latest security fixes and use software that is known it will not receive security fixes. So basically what it might mean (this is what is the spirit of the regulations at least - the regulations and standards are not yet set in stone) that if someneone finds a security bug in K8S that is EOL and we (Airflow) rely on it, we (Airflow/ASF) will have to (somehow) make sure that our users will get a fix. Making it clear that we do not support EOL software, removes that obligation from us. It is of course pretty messy with smaller libraries and their dependencies etc. But there, vendoring-in and relasing security fixes is much easier for us even if we need to, rather than advising our customers on how to patch their k8s. So explicitly marking EOL software as not supported in latest version of Airflow is simply smart thing to do to prevent any responsibilities we might get. I think we will see our industry shifting in the coming years from "costly to migrate so let's not think about it" to "costly to NOT migrate so let's actively manage it". But we will yet see what effect those regulations might have. But anticipating that - I think using EOL as cut-off day for us is a good idea. |
Yeah, both Amazon and Google have changed how they handle k8s versions more recently, so I'd be open to revisit this, but it should probably be an on-list discussion? |
@hussein-awala until rules update , we could already merge this ( since it follow current K8S policy ) |
Absolutely. |
+1 to discuss this on dev list |
yes, I'm ok with that |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
end of GKE 1.24 support in 4 days -> https://endoflife.date/google-kubernetes-engine