-
Notifications
You must be signed in to change notification settings - Fork 46
Upgrading to Python 3.8
Due to the possible complications with DNS records and SSL certificate management I recommend reading through these directions in full before proceeding.
Any code snippets below assume python is directed at a Python 3 executable, depending on your Python environment configuration you may need to substitute python3, python3.6, python3.8 etc.
Python has 3.6 hit end-of-support on Jan 1st, 2022. This is a normal part of the Python language release cadence. The reason you need to take special system administrator actions are because AWS has made substantial changes to the Elastic Beanstalk platform for Python 3.8.
- This operation consists of a new command in the cluster management launch script.
- You will need to have your credential files appropriately configured.
- You may be prompted in this operation to slightly update certain credential fields, this is due to added validation.
- You will need to run an
eb(awsebcli) deployment command, so make sure you are configured and credentialed. - You may need to swap the Git branch you are on more than once, particularly if the main branch is substantially ahead of your servers' code version.
- (It is conceivable that you will need to update your local Python environment from Python 3.6 to 3.8 to run the launch script; this is not the case at time of writing but could occur in the future.)
This is your eb (awsebcli) configuration file. This was accidentally added to the repository many years ago, it has finally been removed. Mostly it points the correct region and credential profile for the eb tool. You may need to restore the file or check-out changes to the file to swap between branches.
If you are still on the master branch we have swapped to using the name main for our production-ready branch. The production, development and master branch names have all been retired.
If you are substantially behind I recommend swapping to our python-3.6 branch and doing an eb deploy operation on your existing Python 3.6 environment in order to catch up on database operations that might otherwise confound the 3.8 upgrade. If you do so you should then swap back to the main branch after the deploy finishes. (The python-3.6 branch does not have the clone command used in later steps.)
3. Navigate into the cluster_management folder and simply run python launch_script.py to familiarize yourself with the commands you will be running below.
This does exactly what it says.
This operation takes the credentials for the cluster you specify, re-analyzes them in case there are any issues, makes a copy of them under the new name you choose, and then deploys a new (empty) Elastic Beanstalk Environment that is configured to connect to your old database and data on S3. The credentials for the old environment in environment_configuration will not be removed, you may do so at your own convenience.
Your AWS account's DNS and SSL configuration may not be trivial to update.
Many institutions impose strict requirements on SSL certificate management, and DNS entries may use a different DNS provider than the AWS Route53 service. Depending on these factors you may need to run this step out-of-order with a first deploy operation of the Python 3.8 platform, or you may want to delay a first deployment until DNS is configured. If DNS is totally out of your control you may want to request a shortened TTL (time to live) on relevant DNS entries ahead of time (several days), and/or run this step while in direct real-time contact with an individual with DNS access privilages.
Go to the AWS EC2 online console, navigate to the Load Balancer section on the left, and identify the load balancers for your old EB Environment, and for the new one. Select the existing one and view the Listeners section at the bottom. A correct configuration of a Beiwe environment will have an entry for HTTPS (port 443). If you click on "change" you will be presented with a dialog of various options to provide the SSL certificate. You will need to replicate the old configuration, or possibly acquire and upload a new certificate depending on any organizational policies that apply.
For AWS ACM (a.k.a. "Certificate Manager"), on the new load balancer click edit, click add, select HTTPS, and forward port 443 to port 80. For the certificate click change, select "Choose a certificate from ACM", and select the certificate based on the desired domain name.
If you are using a custom cert or an existing uploaded cert select that option instead
If you are configuring last-leg encryption you will have to configure that anew for the Python 3.8 environment, and forward to port 443. (Old methods of configuring last-leg encryption are not compatible with the new server stack.)
If you are customizing the Cipher suite (also called the Security Policy) please read step 8.
If your DNS entries for both EB Environments are contained within Route53 on the same account you can use the URL swap operation on the EB online console. You can find a full direction and description here: https://docs.aws.amazon.com/whitepapers/latest/blue-green-deployments/swap-the-environment-of-an-elastic-beanstalk-application.html ("blue-green" is a DevOps phrase, you can ignore it.)
If your DNS configuration is external you will need to manually change the target of the DNS record. You can find details for this here: https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/routing-to-beanstalk-environment.html
Reminder: AWS has a magic DNS hack that they call an "alias". It handles certain configuration complexities, like pointing an A-record DNS entry to a load balancer; non-AWS DNS providers probably will not have this capability.
This step is super simple, just run eb deploy my_new_environment_name_here from inside the root of the beiwe-backend repo.
If you are substantially behind the main branch and want to operate a "safer" first deploy you can swap to the Git branch python-3.8-redeploy. This has identical code state to the python-3.6 branch, with only the new file structures required for the Python 3.8 platform. After a successful deploy of that branch you can then swap back to main and do a final fully-updated deployment operation.
It can be useful to run the excellent SSL Test at https://www.ssllabs.com/ssltest/ This takes a minute or two to complete.
Mostly I want to draw your attention to the final "Handshake Simulation" section, where you will note the versions of Android and iOS that are compatible with your server. If you edited Cipher Suite in step 6 (or if oh-perhaps-maybe AWS silently changes their default Cipher Suite again) you may need to go back and adjust them in order to allow older devices to upload data.
(A full analysis of results from this SSL test are out-of-scope for this article. This is a complex and constantly evolving expert-level subject. Please do not post GitHub issues about SSL security unless you are absolutely positive that your issue will affect other deployments.)
Make sure you are on the same branch that you just deployed, cd into cluster_management, and run python manage.py -create-manager