Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for Py3.12; drop support for Py3.7 #1292

Merged
merged 2 commits into from
Jan 10, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
<!--- Provide a general summary of the issue in the Title above -->
## Context
<!--- Provide a more detailed introduction to the issue itself, and why you consider it to be a bug -->
<!--- Also, please make sure that you are running Zappa _from a virtual environment_ and are using Python 3.7/3.8/3.9/3.10/3.11 -->
<!--- Also, please make sure that you are running Zappa _from a virtual environment_ and are using Python 3.8/3.9/3.10/3.11/3.12 -->

## Expected Behavior
<!--- Tell us what should happen -->
Expand Down
2 changes: 1 addition & 1 deletion .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ Before you submit this PR, please make sure that you meet these criteria:

* Did you **make sure this code actually works on Lambda**, as well as locally?

* Did you test this code with all of **Python 3.7**, **Python 3.8**, **Python 3.9**, **Python 3.10** and **Python 3.11** ?
* Did you test this code with all of: **Python 3.8**, **Python 3.9**, **Python 3.10**, **Python 3.11**, and **Python 3.12**?

* Does this commit ONLY relate to the issue at hand and have your linter shit all over the code?

Expand Down
10 changes: 5 additions & 5 deletions .github/workflows/cd.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,14 +11,14 @@ on: # yamllint disable-line rule:truthy

jobs:
publish:
runs-on: ubuntu-20.04
runs-on: ubuntu-latest
steps:
- name: Checkout Code Repository
uses: actions/checkout@v3
- name: Set up Python 3.11
uses: actions/setup-python@v4
uses: actions/checkout@v4
- name: Set up Python 3.12
uses: actions/setup-python@v5
with:
python-version: "3.11"
python-version: "3.12"
- name: Install `pypa/build`
run: python -m pip install build
- name: Build sdist and wheel
Expand Down
13 changes: 7 additions & 6 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,15 +8,15 @@ on: # yamllint disable-line rule:truthy

jobs:
test:
runs-on: ubuntu-20.04
runs-on: ubuntu-latest
strategy:
matrix:
python: [3.7, 3.8, 3.9, "3.10", "3.11"]
python: ["3.8", "3.9", "3.10", "3.11", "3.12"]
steps:
- name: Checkout Code Repository
uses: actions/checkout@v3
uses: actions/checkout@v4
- name: Set up Python ${{ matrix.python }}
uses: actions/setup-python@v4
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python }}
- uses: actions/cache@v3
Expand All @@ -40,13 +40,14 @@ jobs:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
COVERALLS_FLAG_NAME: py${{ matrix.python }}
COVERALLS_PARALLEL: true
COVERALLS_SERVICE_JOB_ID: ${{ github.run_id }}

coverage:
needs: test
runs-on: ubuntu-20.04
runs-on: ubuntu-latest
steps:
- name: Set up Python
uses: actions/setup-python@v4
uses: actions/setup-python@v5
- name: Finalise Coverage
run: pip3 install --upgrade coveralls && coveralls --service=github --finish
env:
Expand Down
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -133,7 +133,7 @@ __Awesome!__

## Installation and Configuration

_Before you begin, make sure you are running Python 3.7/3.8/3.9/3.10/3.11 and you have a valid AWS account and your [AWS credentials file](https://blogs.aws.amazon.com/security/post/Tx3D6U6WSFGOK2H/A-New-and-Standardized-Way-to-Manage-Credentials-in-the-AWS-SDKs) is properly installed._
_Before you begin, make sure you are running Python 3.8/3.9/3.10/3.11/3.12 and you have a valid AWS account and your [AWS credentials file](https://blogs.aws.amazon.com/security/post/Tx3D6U6WSFGOK2H/A-New-and-Standardized-Way-to-Manage-Credentials-in-the-AWS-SDKs) is properly installed._

**Zappa** can easily be installed through pip, like so:

Expand Down Expand Up @@ -443,7 +443,7 @@ For instance, suppose you have a basic application in a file called "my_app.py",

Any remote print statements made and the value the function returned will then be printed to your local console. **Nifty!**

You can also invoke interpretable Python 3.7/3.8/3.9/3.10/3.11 strings directly by using `--raw`, like so:
You can also invoke interpretable Python 3.8/3.9/3.10/3.11/3.12 strings directly by using `--raw`, like so:

$ zappa invoke production "print(1 + 2 + 3)" --raw

Expand Down Expand Up @@ -985,7 +985,7 @@ to change Zappa's behavior. Use these at your own risk!
"role_name": "MyLambdaRole", // Name of Zappa execution role. Default <project_name>-<env>-ZappaExecutionRole. To use a different, pre-existing policy, you must also set manage_roles to false.
"role_arn": "arn:aws:iam::12345:role/app-ZappaLambdaExecutionRole", // ARN of Zappa execution role. Default to None. To use a different, pre-existing policy, you must also set manage_roles to false. This overrides role_name. Use with temporary credentials via GetFederationToken.
"route53_enabled": true, // Have Zappa update your Route53 Hosted Zones when certifying with a custom domain. Default true.
"runtime": "python3.11", // Python runtime to use on Lambda. Can be one of "python3.7", "python3.8", "python3.9", or "python3.10", or "python3.11". Defaults to whatever the current Python being used is.
"runtime": "python3.12", // Python runtime to use on Lambda. Can be one of: "python3.8", "python3.9", "python3.10", "python3.11", or "python3.12". Defaults to whatever the current Python being used is.
"s3_bucket": "dev-bucket", // Zappa zip bucket,
"slim_handler": false, // Useful if project >50M. Set true to just upload a small handler to Lambda and load actual project from S3 at runtime. Default false.
"settings_file": "~/Projects/MyApp/settings/dev_settings.py", // Server side settings file location,
Expand Down
8 changes: 4 additions & 4 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -44,15 +44,15 @@
"License :: OSI Approved :: Apache Software License",
"Operating System :: OS Independent",
"Programming Language :: Python",
"Programming Language :: Python :: 3.7",
"Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Framework :: Django",
"Framework :: Django :: 1.11",
"Framework :: Django :: 2.0",
"Framework :: Django :: 3.0",
"Framework :: Django :: 3.2",
"Framework :: Django :: 4.2",
"Framework :: Django :: 5.0",
"Topic :: Internet :: WWW/HTTP",
"Topic :: Internet :: WWW/HTTP :: Dynamic Content",
],
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
},
"FunctionName": "zappa-ttt888",
"FunctionArn": "arn:aws:lambda:us-east-1:004396165043:function:zappa-ttt888",
"Runtime": "python3.6",
"Runtime": "python3.8",
"Role": "arn:aws:iam::004396165043:role/zappa-ttt888-ZappaLambdaExecutionRole",
"Handler": "handler.lambda_handler",
"CodeSize": 16975308,
Expand Down
117 changes: 58 additions & 59 deletions tests/tests.py
Original file line number Diff line number Diff line change
Expand Up @@ -92,28 +92,12 @@ def test_disable_click_colors(self):
def test_create_lambda_package(self):
# mock the pkg_resources.WorkingSet() to include a known package in lambda_packages so that the code
# for zipping pre-compiled packages gets called
mock_installed_packages = {"psycopg2": "2.6.1"}
mock_installed_packages = {"psycopg": "3.1.17"}
with mock.patch(
"zappa.core.Zappa.get_installed_packages",
return_value=mock_installed_packages,
):
z = Zappa(runtime="python3.7")
path = z.create_lambda_zip(handler_file=os.path.realpath(__file__))
self.assertTrue(os.path.isfile(path))
os.remove(path)

def test_get_manylinux_python37(self):
z = Zappa(runtime="python3.7")
self.assertIsNotNone(z.get_cached_manylinux_wheel("psycopg2", "2.7.6"))
self.assertIsNone(z.get_cached_manylinux_wheel("derp_no_such_thing", "0.0"))

# mock with a known manylinux wheel package so that code for downloading them gets invoked
mock_installed_packages = {"psycopg2": "2.7.6"}
with mock.patch(
"zappa.core.Zappa.get_installed_packages",
return_value=mock_installed_packages,
):
z = Zappa(runtime="python3.7")
z = Zappa(runtime="python3.8")
path = z.create_lambda_zip(handler_file=os.path.realpath(__file__))
self.assertTrue(os.path.isfile(path))
os.remove(path)
Expand Down Expand Up @@ -199,17 +183,59 @@ def test_get_manylinux_python310(self):
self.assertTrue(os.path.isfile(path))
os.remove(path)

def test_verify_python37_does_not_download_2_24_manylinux_wheel(self):
z = Zappa(runtime="python3.7")
cached_wheels_dir = os.path.join(tempfile.gettempdir(), "cached_wheels")
expected_wheel_path = os.path.join(
cached_wheels_dir, "cryptography-35.0.0-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.whl"
)
def test_get_manylinux_python311(self):
z = Zappa(runtime="python3.11")
self.assertIsNotNone(z.get_cached_manylinux_wheel("psycopg2-binary", "2.9.7"))
self.assertIsNone(z.get_cached_manylinux_wheel("derp_no_such_thing", "0.0"))

# Check with known manylinux wheel package
actual_wheel_path = z.get_cached_manylinux_wheel("cryptography", "35.0.0")
self.assertEqual(actual_wheel_path, expected_wheel_path)
os.remove(actual_wheel_path)
# mock with a known manylinux wheel package so that code for downloading them gets invoked
mock_installed_packages = {"psycopg2-binary": "2.9.7"}
with mock.patch(
"zappa.core.Zappa.get_installed_packages",
return_value=mock_installed_packages,
):
z = Zappa(runtime="python3.11")
path = z.create_lambda_zip(handler_file=os.path.realpath(__file__))
self.assertTrue(os.path.isfile(path))
os.remove(path)

# same, but with an ABI3 package
mock_installed_packages = {"cryptography": "2.8"}
with mock.patch(
"zappa.core.Zappa.get_installed_packages",
return_value=mock_installed_packages,
):
z = Zappa(runtime="python3.11")
path = z.create_lambda_zip(handler_file=os.path.realpath(__file__))
self.assertTrue(os.path.isfile(path))
os.remove(path)

def test_get_manylinux_python312(self):
z = Zappa(runtime="python3.12")
self.assertIsNotNone(z.get_cached_manylinux_wheel("psycopg-binary", "3.1.17"))
self.assertIsNone(z.get_cached_manylinux_wheel("derp_no_such_thing", "0.0"))

# mock with a known manylinux wheel package so that code for downloading them gets invoked
mock_installed_packages = {"psycopg-binary": "3.1.17"}
with mock.patch(
"zappa.core.Zappa.get_installed_packages",
return_value=mock_installed_packages,
):
z = Zappa(runtime="python3.12")
path = z.create_lambda_zip(handler_file=os.path.realpath(__file__))
self.assertTrue(os.path.isfile(path))
os.remove(path)

# same, but with an ABI3 package
mock_installed_packages = {"cryptography": "41.0.7"}
with mock.patch(
"zappa.core.Zappa.get_installed_packages",
return_value=mock_installed_packages,
):
z = Zappa(runtime="python3.12")
path = z.create_lambda_zip(handler_file=os.path.realpath(__file__))
self.assertTrue(os.path.isfile(path))
os.remove(path)

def test_verify_downloaded_manylinux_wheel(self):
z = Zappa(runtime="python3.10")
Expand Down Expand Up @@ -260,33 +286,6 @@ def test_verify_manylinux_filename_is_lowered(self):
cached_pypi_info_dir = os.path.join(tempfile.gettempdir(), "cached_pypi_info")
os.remove(os.path.join(cached_pypi_info_dir, "markupsafe-2.1.3.json"))

def test_get_manylinux_python311(self):
z = Zappa(runtime="python3.11")
self.assertIsNotNone(z.get_cached_manylinux_wheel("psycopg2-binary", "2.9.7"))
self.assertIsNone(z.get_cached_manylinux_wheel("derp_no_such_thing", "0.0"))

# mock with a known manylinux wheel package so that code for downloading them gets invoked
mock_installed_packages = {"psycopg2-binary": "2.9.7"}
with mock.patch(
"zappa.core.Zappa.get_installed_packages",
return_value=mock_installed_packages,
):
z = Zappa(runtime="python3.11")
path = z.create_lambda_zip(handler_file=os.path.realpath(__file__))
self.assertTrue(os.path.isfile(path))
os.remove(path)

# same, but with an ABI3 package
mock_installed_packages = {"cryptography": "2.8"}
with mock.patch(
"zappa.core.Zappa.get_installed_packages",
return_value=mock_installed_packages,
):
z = Zappa(runtime="python3.11")
path = z.create_lambda_zip(handler_file=os.path.realpath(__file__))
self.assertTrue(os.path.isfile(path))
os.remove(path)

def test_get_exclude_glob__file_not_deleted(self):
z = Zappa(runtime="python3.11")
self.assertIsNotNone(z.get_cached_manylinux_wheel("psycopg2-binary", "2.9.7"))
Expand All @@ -309,7 +308,7 @@ def test_get_exclude_glob__file_not_deleted(self):
os.remove(path)

def test_getting_installed_packages(self, *args):
z = Zappa(runtime="python3.7")
z = Zappa(runtime="python3.8")

# mock pkg_resources call to be same as what our mocked site packages dir has
mock_package = collections.namedtuple("mock_package", ["project_name", "version", "location"])
Expand Down Expand Up @@ -344,7 +343,7 @@ def test_get_current_venv(self, *args):
self.assertEqual(current_venv, None)

def test_getting_installed_packages_mixed_case_location(self, *args):
z = Zappa(runtime="python3.7")
z = Zappa(runtime="python3.8")

# mock pip packages call to be same as what our mocked site packages dir has
mock_package = collections.namedtuple("mock_package", ["project_name", "version", "location"])
Expand All @@ -367,7 +366,7 @@ def test_getting_installed_packages_mixed_case_location(self, *args):
)

def test_getting_installed_packages_mixed_case(self, *args):
z = Zappa(runtime="python3.7")
z = Zappa(runtime="python3.8")

# mock pkg_resources call to be same as what our mocked site packages dir has
mock_package = collections.namedtuple("mock_package", ["project_name", "version", "location"])
Expand Down Expand Up @@ -2581,7 +2580,7 @@ def test_minor_version_only_check_when_in_docker(self, *_):
reload(zappa)

@mock.patch("os.getenv", return_value="True")
@mock.patch("sys.version_info", new_callable=partial(get_sys_versioninfo, 7))
@mock.patch("sys.version_info", new_callable=partial(get_sys_versioninfo, 8))
def test_no_runtimeerror_when_in_docker(self, *_):
from importlib import reload

Expand Down
4 changes: 2 additions & 2 deletions zappa/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,8 @@ def running_in_docker() -> bool:
return running_in_docker_flag


SUPPORTED_VERSIONS = [(3, 7), (3, 8), (3, 9), (3, 10), (3, 11)]
MINIMUM_SUPPORTED_MINOR_VERSION = 7
SUPPORTED_VERSIONS = [(3, 8), (3, 9), (3, 10), (3, 11), (3, 12)]
MINIMUM_SUPPORTED_MINOR_VERSION = 8

if not running_in_docker() and sys.version_info[:2] not in SUPPORTED_VERSIONS:
print(running_in_docker())
Expand Down
31 changes: 7 additions & 24 deletions zappa/core.py
Original file line number Diff line number Diff line change
Expand Up @@ -279,7 +279,7 @@ def __init__(
load_credentials=True,
desired_role_name=None,
desired_role_arn=None,
runtime="python3.7", # Detected at runtime in CLI
runtime="python3.8", # Detected at runtime in CLI
tags=(),
endpoint_urls={},
xray_tracing=False,
Expand All @@ -304,27 +304,10 @@ def __init__(

self.runtime = runtime

if self.runtime == "python3.7":
self.manylinux_suffix_start = "cp37m"
elif self.runtime == "python3.8":
# The 'm' has been dropped in python 3.8+ since builds with and without pymalloc are ABI compatible
# See https://github.com/pypa/manylinux for a more detailed explanation
self.manylinux_suffix_start = "cp38"
elif self.runtime == "python3.9":
self.manylinux_suffix_start = "cp39"
elif self.runtime == "python3.10":
self.manylinux_suffix_start = "cp310"
else:
self.manylinux_suffix_start = "cp311"

# AWS Lambda supports manylinux1/2010, manylinux2014, and manylinux_2_24
# Currently python3.7 lambda runtime does not support manylinux_2_24
# See https://github.com/zappa/Zappa/issues/1249 for more details
if self.runtime == "python3.7":
self.manylinux_suffixes = ("2014", "2010", "1")
else:
self.manylinux_suffixes = ("_2_24", "2014", "2010", "1")

# TODO: Support PEP600 properly (https://peps.python.org/pep-0600/)
self.manylinux_suffix_start = f"cp{self.runtime[6:].replace('.', '')}"
self.manylinux_suffixes = ("_2_24", "2014", "2010", "1")
# TODO: Support aarch64 architecture
self.manylinux_wheel_file_match = re.compile(
rf'^.*{self.manylinux_suffix_start}-(manylinux_\d+_\d+_x86_64[.])?manylinux({"|".join(self.manylinux_suffixes)})_x86_64[.]whl$' # noqa: E501
)
Expand Down Expand Up @@ -1100,7 +1083,7 @@ def create_lambda_function(
publish=True,
vpc_config=None,
dead_letter_config=None,
runtime="python3.7",
runtime="python3.8",
aws_environment_variables=None,
aws_kms_key_arn=None,
xray_tracing=False,
Expand Down Expand Up @@ -1285,7 +1268,7 @@ def update_lambda_configuration(
ephemeral_storage={"Size": 512},
publish=True,
vpc_config=None,
runtime="python3.7",
runtime="python3.8",
aws_environment_variables=None,
aws_kms_key_arn=None,
layers=None,
Expand Down
14 changes: 9 additions & 5 deletions zappa/utilities.py
Original file line number Diff line number Diff line change
Expand Up @@ -209,15 +209,19 @@ def get_runtime_from_python_version():
raise ValueError("Python 2.x is no longer supported.")
else:
if sys.version_info[1] <= 7:
return "python3.7"
elif sys.version_info[1] <= 8:
raise ValueError("Python 3.7 and below are no longer supported.")
elif sys.version_info[1] == 8:
return "python3.8"
elif sys.version_info[1] <= 9:
elif sys.version_info[1] == 9:
return "python3.9"
elif sys.version_info[1] <= 10:
elif sys.version_info[1] == 10:
return "python3.10"
else:
elif sys.version_info[1] == 11:
return "python3.11"
elif sys.version_info[1] == 12:
return "python3.12"
else:
raise ValueError(f"Python f{'.'.join(str(v) for v in sys.version_info[:2])} is not yet supported.")


##
Expand Down
Loading