Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Keep Python up-to-date #174

Open
chicheng opened this issue Jun 2, 2021 · 43 comments
Open

Keep Python up-to-date #174

chicheng opened this issue Jun 2, 2021 · 43 comments

Comments

@chicheng
Copy link

chicheng commented Jun 2, 2021

Community Note

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
  • Please do not leave "+1" or "me too" comments. They generate extra noise for issue followers and do not help prioritize the request.

Tell us about your request
What do you want us to build?
Keep Python up-to-date

Is this request specific to an Elastic Beanstalk platform?
Python

Additional context
It seems Elastic Beanstalk is good at following upstream releases, except Python, albeit almost all Python .z releases contain a few security fixes.

e.g.

Language Version Upstream release Elastic Beanstalk release Interval
PHP 8.0.6, 7.4.19 2021-05-06 2021-06-01 4 weeks
Go 1.16.4 2021-05-06 2021-06-01 4 weeks
Go 1.16.3 2021-04-01 2021-05-03 5 weeks
Node 14.17.0 2021-05-11 2021-06-01 3 weeks
Docker 20.10.4 2021-02-26 2021-05-03 2 months 1 week
Docker Compose 1.29.2 2021-05-10 2021-06-01 3 weeks
.NET Core 5.0.6, 3.1.15, 2.1.28 2021-05-11 2021-06-01 3 weeks
.NET Core 5.0.5 2021-04-06 2021-05-03 4 weeks
.NET Core 3.1.14, 2.1.27 2021-04-13 2021-05-03 3 weeks
Java (Corretto) 11.0.11.9.1, 8.292.10.1 2021-04-20 2021-05-03 2 weeks
Ruby 2.7.3, 2.6.7 2021-04-05 2021-05-03 4 weeks
Python 3.8.6 ~ 3.8.14 Since 2020-09-25 or 2021-02-26? (EB Python 3.8 GA) ? More than 17 months, there are 20+ security tickets fixed since Python 3.8.5
Python 3.7.11 ~ 3.7.14 2021-06-29 ? More than 14 months, these are "security fix only" releases

PS. Don't check mod_wsgi, there are ~80 releases since AWS last time update it on Python 3.6 platform 7 years ago

How about new language versions?

Language Version Upstream release Elastic Beanstalk release Interval
PHP 8.1 2021-11-25 2022-08-08 8 months
PHP 8.0 2020-11-26 2021-03-31 4 months
Node 14 2020-04-22 2020-10-27 2021-02-26 10 months, 4 months since active LTS starts on 2020-10-27: "Production applications should only use Active LTS or Maintenance LTS releases"
.NET 5.0 2020-11-10 2020-11-20 10 days
Go 1.17 2021-08-16 2021-07-02 17 days
Go 1.16 2021-02-16 2021-02-26 10 days
Go 1.15 2020-08-11 2020-09-03 3 weeks
Ruby 2.7 2019-12-25 2020-04-30 (AL2 Ruby GA) 4 months
Ruby 3.0 2020-12-25 2021-11-19 11 months
Python 3.10 2021-10-04 ? 11 months and counting
Python 3.9 2020-10-05 ? 23 months and counting
Python 3.8 2019-10-14 2021-02-26 16 months
Python 3.7 2018-06-27 2020-04-08 (AL2 Python GA) 9 months
@dgagnon
Copy link

dgagnon commented Jun 2, 2021

We use EB in a compliant environment where security updates are required to be applied in 30 days. The current release cycle of python versions in EB is preventing it's usage in a secure environment.

@matthewdouglas
Copy link

@chicheng To be fair Ruby 3.0 (released 2020-12-25) belongs on the language version list as well.

Node 14.18.0 LTS release was 2021-09-28. ElasticBeanstalk had support for this just 5 days later on 2021-10-03.

Ruby 2.7.4 release was 2021-07-07. ElasticBeanstalk update from 2.7.3 was just 2 weeks later.

Go 1.16.6 release was 2021-07-12. ElasticBeanstalk update from 1.16.5 was 9 days later on 2021-07-21.

PHP 8.0.8 release was 2021-07-01. ElasticBeanstalk update from 8.0.6 was 20 days later on 2021-07-21. However I will note that 8.0.7 (release date 2021-06-03) was skipped.

Go 1.17.1 release was 2021-08-16. ElasticBeanstalk update from 1.17.0 was 48 days later on 2021-10-03.

Python 3.7.10 release was 2021-02-15. It took ElasticBeanstalk nearly 5 months to update from 3.7.9 on 2021-07-07.

Node 16 will go LTS later this month, and next month we're going to see .NET 6. I'm curious how long it will take for those.

@1tonyca 1tonyca removed the Proposed label Oct 21, 2021
@matthewdouglas
Copy link

I've noticed that Python 3.9.8 is included OOTB in the Amazon Linux 2022 technical preview that was just released (x86_64 AMI). I was also able to find packages for 3.8.12-1.amazon2022 and 3.7.12-1.amazon2022.

Would it be reasonable to assume that Amazon Linux 2022 platforms will be coming to Elastic Beanstalk near the same point in time as the GA of AL2022? It would be great to have a Python 3.9 AL2022 platform with faster package updates from the Fedora upstream.

@Alex3917
Copy link

I haven't historically cared a ton about Python usually being a couple versions behind on Beanstalk, but this is going to become really painful after Python 3.11 once each version of Python starts getting up to 2x faster.

@januszm
Copy link

januszm commented May 17, 2022

Python 3.9 was released in 2020 and it is still not available on Amazon Elastic Beanstalk.

Installing any version of Python is rather smooth on a server using pyenv / pipenv. AWS EB team, please let us know when new Python versions are available on this platform.

@jordanade
Copy link

Current patch version of PHP 8.0 is over three months out of date!

@jbrace-masschallenge
Copy link

It is also seemingly impossible right now to create a custom platform on ElasticBeanstalk due to using an old version of the AWS-SDK on the packer build AMI they use.

@MicBehrens
Copy link

Congratulations AWS.
You managed to NOT update Python to 3.9 or 3.10, before Python 3.8 came into the "security fixes only"-stage.

According to the release calendar specified in PEP 569, Python 3.8 is now in the "security fixes only" stage of its life cycle: 3.8 branch only accepts security fixes and releases of those are made irregularly in source-only form until October 2024. Python 3.8 isn't receiving regular bug fixes anymore, and binary installers are no longer provided for it. Python 3.8.10 was the last full bugfix release of Python 3.8 with binary installers.

I guess AWS Elasticbeanstalk is dying.

@pabloredigonda
Copy link

+1

@dfrdmn
Copy link

dfrdmn commented Sep 13, 2022

@1tonyca, it is now coming up on 1 year since this moved to Researching. Would you please provide some updates?

I imagine many people would at least like to know the likelihood of more recent Python versions so that we can plan to either wait or move to a different strategy/service. As @MicBehrens points out, Python 3.8 is very much fading away.

@davidxbuck
Copy link

I second all of the above. It is getting very difficult to plan anything at the moment. We want to move to a fully supported, modern, version of Python and you are neither able to provide this, nor are you engaging with the community. AWS are also dangling Linux2022 in front of us without even a draft release date. I'm pretty sure that no-one wants to release new apps on Python 3.8 or Linux 2, but we don't know if you'll announce them next week or next year.

@Briscoooe
Copy link

I have seen chatter online that AWS EB is on it's way out which would explain the slow development. Can anyone from AWS speak to the future of the service?

Regarding Python versioning, a workaround I have found is to use the Docker environment and then use whatever Python version image you want. It's not native of course but if you absolutely have to get a Python 3.9+ app on EB then this works.

@jaymegordo
Copy link

@Briscoooe What info do you have about EB being on the way out? Is there something we should be switching to as an alternative?

@pfcodes
Copy link

pfcodes commented Nov 3, 2022

Bump

@Briscoooe
Copy link

@jaymegordo

No official announcements really, just some comments from non-AWS people on Reddit threads, people don't seem too optimistic on it.

I actually recently set up a new EB environment using CDK and notice it being pretty light on feature set for configuration, the majority I had to use lots of raw strings to configure properties instead of dedicated properties or constants. Everything else on CDK was a breeze

The recommendation seems to be just setting up ECS/Fargate instead, both of which have rich configuration options with CDK.

@modohash
Copy link

Any updates on this or do we all just move away from EB?

@jonmooser
Copy link

Another +1.
I would be happy to try something other than EB for my Python Web App deployment, but AWS doesn't provide much in the way of documentation. All of the guides in the AWS docs gravitate to "Use Elastic Beanstalk."

So if EB is being sunset, Amazon is being quite hush about it.

@CraigWoolgar
Copy link

Another +1. I would be happy to try something other than EB for my Python Web App deployment, but AWS doesn't provide much in the way of documentation. All of the guides in the AWS docs gravitate to "Use Elastic Beanstalk."

So if EB is being sunset, Amazon is being quite hush about it.

AWS App Runner may be a possible solution for you. It runs from ECS Containers or Github hosted source code.

Also, another +1 from me.

@pfcodes
Copy link

pfcodes commented Jan 19, 2023

AWS App Runner may be a possible solution for you. It runs from ECS Containers or Github hosted source code.

Just a heads up to anyone thinking about using App Runner. It has some limitations you should know about mainly:

@Briscoooe
Copy link

ECS is very similar to EB and is extremely popular from what I can tell so I wouldn't worry about its future.

With ECS you create a cluster which has an autoscaling group, defined instance types, min/max number of instances, and defined VPC/subnets. It also handles rolling updates via Task Definitions. Like EB it also cost nothing as you just pay for the underlying EC2 instances, which you get full access to. I think the only thing missing is the automatic load balancer configuration.

It is definitely more work to set up but it has pretty close feature parity and appears to be future proof.

@jonmooser
Copy link

ECS is very similar to EB and is extremely popular from what I can tell so I wouldn't worry about its future.

Thanks @Briscoooe. I will check that out. I have a few problems with EB, the biggest one being that I can't move past python 3.8. If I can get over that and other issues, it would be worth the extra work.

@Briscoooe
Copy link

Briscoooe commented Jan 20, 2023

@jonmooser ECS is Docker so you can set Python to any version that you want. You could deploy Python 3.12 today if you wanted.

@Alex3917
Copy link

Alex3917 commented Jan 20, 2023

@Briscoooe How much more work is it to keep everything updated and locked down on ECS without the shared security model that EB provides? That's one of my main concerns with ECS, as compared with Beanstalk.

@Briscoooe
Copy link

@Alex3917 Do you mean the AWS-managed EB platform versions? I think in practical terms the security model is the same as both services simply launch regular EC2 instances with a specific AWS-managed AMI.

@Alex3917
Copy link

Alex3917 commented Jan 20, 2023

@Briscoooe That makes sense about the AWS-managed AMIs for EC2. But since AL2 only offers up to Python 3.8, switching from Beanstalk with AL2 to ECS with AL2 wouldn't really be an obvious advantage.

@Briscoooe
Copy link

@Alex3917 with ECS, the underlying Python version installed in the AMI is irrelevant. Your actual app code runs in a Docker container, not directly in the EC2 instance environment and the Docker container's Python version is completely independent of the EC2's version.

@swt2c
Copy link

swt2c commented Feb 2, 2023

Part of this issue seems to have been addressed. The last couple of Elastic Beanstalk updates have included Python version updates.

For example, the updates released yesterday (https://docs.aws.amazon.com/elasticbeanstalk/latest/relnotes/release-2023-02-01-linux.html) included Python 3.7.16 and 3.8.16 (the latest releases).

Adding support for 3.9, 3.10, 3.11 still remains though.

@modohash
Copy link

modohash commented Feb 3, 2023

@swt2c I agree this is helpful, but it doesn't excuse not supporting at least 3.9.

Python 3.9 is security fix only at this point and was released in October 2020! I like working with elastic beanstalk, I want it to keep going, but the ridiculous response time for releases will eventually force me off it.

@skoloCFD
Copy link

Helpful thread for sure, but there really ought to be python 3.9 and 3.10 support!

3.7 and 3.8 are both getting close to end of life. This seems like a trivial reason to essentially kick people off of the elastic service?

@ChristopherBanas
Copy link

Bump

@carlkibler
Copy link

Python support is now 3 releases out of date. Python 3.11 came out in December 2022 so I don't expect it to be supported yet. But it means 3.10 should be!

Really, most of us are looking for indications of whether EB is going to keep up to date or we need to plan a migration off it. If it's on life support as a service, please tell your users. I've enjoyed EB a lot but it has fallen way behind the Python ecosystem.

@januszm
Copy link

januszm commented Feb 27, 2023

I'm still a bit surprised that no one from AWS has replied to you here. I have talked to AWS (paid) customer support and at the moment the answer to my questions that I received suggests that the platform will be maintained but it will not receive any new features, support for new language versions should not appear either. This is sad from the customers point of view, but AWS seems to want to divert your attention to other products. Personally, I received a suggestion to migrate to the AppRunner platform and use the Copilot CLI.

@jonmooser
Copy link

I get that they are encouraging users to move to other products. But I wish it were more explicit: so much of the documentation is still written with EB in mind.
I'll be migrating off of EB, probably to ECS+Docker. But it's not quite urgent for me right now as I'm in a bit of a prototyping state.

@huksley
Copy link

huksley commented Mar 3, 2023

It's dead folks.

I wish I had known that before I spend hours trying to read through myriad of configuration options to setup Elastic BeanStalk instance properly :(

I started this page, feel free to contribute so people will not spend time using "dead but not officially dead" services.

https://github.com/huksley/aws-dead-services/

@goetzb
Copy link

goetzb commented May 5, 2023

Fyi - "Elastic Beanstalk launches Amazon Linux 2023 Python platform on May 04, 2023": https://docs.aws.amazon.com/elasticbeanstalk/latest/relnotes/release-2023-05-04-al2023.html

The new options:

  • Python 3.11 (3.11.2) on Amazon Linux 2023 (2023.0.20230503)
  • Python 3.9 (3.9.16) on Amazon Linux 2023 (2023.0.20230503)

I haven't yet found a migration guide to see what config changes might be needed from AL2 to AL2023. Maybe it's straightforward this time without too many adjustments.

@bobbyrenwick
Copy link

Thanks for the update @goetzb 🥳

@januszm
Copy link

januszm commented May 5, 2023

Thanks @goetzb , from what I see, two main changes are:

yum => dnf # but there's still an alias
no support for EPEL

which means that if you need extra packages from epel in your .ebextensions, you'll need to find replacement

Now waiting for the same update for the Ruby platform.

@goetzb
Copy link

goetzb commented May 5, 2023

which means that if you need extra packages from epel in your .ebextensions, you'll need to find replacement

@januszm yes, that’s what I thought too. Getting the right PostgreSQL client package always required some adjustments in the past, so I’ll have to look into how this will work with dnf - or maybe newer PostgreSQL packages are now already available as default options. On https://docs.aws.amazon.com/linux/al2023/release-notes/all-packages-al2023-20230501.html a package postgresql15 for PostgreSQL 15 is mentioned, for example.

@chicheng
Copy link
Author

chicheng commented May 7, 2023

I guess libraries for geodjango are still not available in AL2023? which means the platform does not even fully support the most popular python web framework.

@baiyli
Copy link

baiyli commented May 10, 2023

Hey, Elastic Beanstalk has included Comparing Amazon Linux 2 and Amazon Linux 2023 in the Release: Elastic Beanstalk launches Amazon Linux 2023 Python platform on May 04, 2023 for documenting the changes between AL2 and AL2023 platforms. Thanks.

@stewartsmith
Copy link

I guess libraries for geodjango are still not available in AL2023? which means the platform does not even fully support the most popular python web framework.

Are you able to clarify what OS level libraries this requires? We're happy to track adding additional packages to AL2023 over at https://github.com/amazonlinux/amazon-linux-2023/issues and having "we need package X for use case Y" is helpful in prioritizing such requests.

@stewartsmith
Copy link

For Python 3.12, we're tracking the addition of Python 3.12 to Amazon Linux 2023 over at amazonlinux/amazon-linux-2023#483

For people looking for Python 3.12, I'd love it if you could +1 our Amazon Linux issue as well so that we can get a better view as to the demand for it, and thus prioritize accordingly.

@dgagnon
Copy link

dgagnon commented Oct 16, 2023

I can confirm django 4.2 works on latest AL2023. We have this in production.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Researching
Development

No branches or pull requests