Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PR-6096: Upgrade to Python 3.10 #842

Merged
merged 1 commit into from
Sep 20, 2024
Merged

PR-6096: Upgrade to Python 3.10 #842

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/deploy-docs-website.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ jobs:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: 3.9
python-version: "3.10"
- run: pip install -r requirements/requirements-docs.txt
- run: mkdocs gh-deploy --force

Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/lint.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ jobs:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: 3.8
python-version: "3.10"

- uses: TrueBrain/actions-flake8@v2
with:
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ jobs:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: 3.8
python-version: "3.10"

- name: Install dependencies
run: |
Expand Down
11 changes: 5 additions & 6 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ DOCKER := docker
# On Linux we need to do a bit of userid finagling so that the output files
# end up being owned by us and not by root. On Mac this works out of the box.
DOCKER_USER_ARG := --user "$(shell id -u):$(shell id -g)"
DOCKER_COMMAND = $(DOCKER) run --rm $(DOCKER_USER_ARG) -v "$$PWD":/var/task $(DOCKER_ARGS)
DOCKER_COMMAND = $(DOCKER) run --rm $(DOCKER_USER_ARG) -v "$$PWD":/var/task $(DOCKER_ARGS) --entrypoint ""

PYTHON := python3
BUILD_VENV := $(DIR)/.venv
Expand All @@ -57,7 +57,6 @@ Makefile.config:
include Makefile.config

ifdef DOCKER_COMMAND
DOCKER_LAMBDA_CI = $(DOCKER_COMMAND) lambci/lambda:build-python3.8
DOCKER_DEPENDENCY_BUILDER = $(DOCKER_COMMAND) tea-dependency-builder
endif

Expand All @@ -71,7 +70,7 @@ all: build ;

# Build everything
.PHONY: build
build: \
build: tea-dependency-builder \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is desired, this will rebuild the docker image every time you run make build.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW the GNU make documentation is pretty good:
https://www.gnu.org/software/make/manual/html_node/Phony-Targets.html

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If there is nothing new with the dockerfile, it takes half a second to run that portion. I don't see an issue with that.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we removed the line DOCKER_LAMBDA_CI = $(DOCKER_COMMAND) lambci/lambda:build-python3.8, and instead are using our own docker image, we need to build the docker image before we can run make build.

$(DIR)/thin-egress-app-code.zip \
$(DIR)/thin-egress-app-dependencies.zip \
$(DIR)/thin-egress-app.yaml \
Expand Down Expand Up @@ -106,7 +105,7 @@ $(BUILD_VENV): requirements/requirements-make.txt
$(DIR)/thin-egress-app-dependencies.zip: requirements/requirements.txt $(REQUIREMENTS_DEPS)
rm -rf $(DIR)/python
@mkdir -p $(DIR)/python
$(DOCKER_LAMBDA_CI) build/dependency_builder.sh "$(DIR)/thin-egress-app-dependencies.zip" "$(DIR)"
$(DOCKER_DEPENDENCY_BUILDER) build/dependency_builder.sh "$(DIR)/thin-egress-app-dependencies.zip" "$(DIR)"

.SECONDARY: $(DIST_MD_RESOURCES)
$(DIST_MD_RESOURCES): $(DIR)/code/%.html: %.md $(BUILD_VENV)
Expand Down Expand Up @@ -271,8 +270,8 @@ cleandeploy:
###############

.PHONY: tea-dependency-builder
tea-dependency-builder: build/lambda-ci.Dockerfile
$(DOCKER) build -f build/lambda-ci.Dockerfile -t tea-dependency-builder ./build
tea-dependency-builder: build/tea-dependency-builder.Dockerfile
$(DOCKER) build -f build/tea-dependency-builder.Dockerfile -t tea-dependency-builder ./build
@mkdir -p $(EMPTY)
@touch $@

Expand Down
2 changes: 1 addition & 1 deletion README.MD
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ releases see the [GitHub Actions README](.github/workflows/README.md).*
In order to build TEA, you will need to have a few tools installed:

- `make` for build automation
- `python3.8` for running tests and creating the CloudFormation template
- `python3.10` for running tests and creating the CloudFormation template
- `docker` for building the dependency layer
- `git` for installing rain-api-core
- `zip` for creating zip files
Expand Down
2 changes: 1 addition & 1 deletion build/dependency_builder.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ mkdir -p $BUILD_DIR/python
cd $BUILD_DIR/python || exit

echo "Installing ${REQUIREMENTS_FILE}"
python3.8 -m pip install \
python3 -m pip install \
--upgrade \
-r "$REQUIREMENTS_FILE" \
--target $BUILD_DIR/python \
Expand Down
3 changes: 0 additions & 3 deletions build/lambda-ci.Dockerfile
View file
Open in desktop

This file was deleted.

4 changes: 4 additions & 0 deletions build/tea-dependency-builder.Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
FROM public.ecr.aws/lambda/python:3.10

RUN yum install -y git zip
RUN pip install pip-tools
10 changes: 4 additions & 6 deletions cloudformation/thin-egress-app.yaml.j2
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -532,7 +532,7 @@ Resources:
vpcid: !Sub "${PrivateVPC}"
Timeout: !Ref LambdaTimeout
Handler: update_lambda.lambda_handler
Runtime: 'python3.8'
Runtime: 'python3.10'
Layers:
- !Ref EgressLambdaDependencyLayer
#MemorySize: 128
Expand All @@ -558,9 +558,7 @@ Resources:
Type: AWS::Lambda::LayerVersion
Properties:
CompatibleRuntimes:
- python3.6
- python3.7
- python3.8
- python3.10
Content:
S3Bucket: !Ref LambdaCodeS3Bucket
S3Key: !Ref LambdaCodeDependencyArchive
Expand Down Expand Up @@ -630,7 +628,7 @@ Resources:
OTEL_COLLECTOR_URL: !Ref OtCollectorURL
Timeout: !Ref LambdaTimeout
Handler: app.app
Runtime: 'python3.8'
Runtime: 'python3.10'
Layers:
- !Ref EgressLambdaDependencyLayer
- !Sub "arn:aws:lambda:${AWS::Region}:901920570463:layer:aws-otel-python-amd64-ver-1-25-0:1"
Expand Down Expand Up @@ -1110,7 +1108,7 @@ Resources:
BUILD_VERSION: "{{ build_version }}"
MATURITY: !Ref Maturity
Handler: tea_bumper.lambda_handler
Runtime: 'python3.8'
Runtime: 'python3.10'
Layers:
- !Ref EgressLambdaDependencyLayer

Expand Down
16 changes: 7 additions & 9 deletions requirements/requirements-dev.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
#
# This file is autogenerated by pip-compile with Python 3.8
# This file is autogenerated by pip-compile with Python 3.10
gjclark marked this conversation as resolved.
Show resolved Hide resolved
# by the following command:
#
# pip-compile requirements/requirements-dev.in
#
boto3==1.35.18
boto3==1.35.23
# via
# -r requirements/requirements-dev.in
# moto
botocore==1.35.18
botocore==1.35.23
# via
# -c requirements/requirements.txt
# boto3
Expand Down Expand Up @@ -40,12 +40,10 @@ docker==7.1.0
# via moto
exceptiongroup==1.2.2
# via pytest
idna==3.8
idna==3.10
# via requests
importlib-metadata==7.1.0
# via
# build
# opentelemetry-api
# via opentelemetry-api
iniconfig==2.0.0
# via pytest
jinja2==3.1.4
Expand Down Expand Up @@ -121,7 +119,7 @@ tomli==2.0.1
# coverage
# pip-tools
# pytest
urllib3==1.26.20
urllib3==2.2.3
# via
# -c requirements/requirements.txt
# botocore
Expand All @@ -140,7 +138,7 @@ wrapt==1.16.0
# opentelemetry-instrumentation
xmltodict==0.13.0
# via moto
zipp==3.20.1
zipp==3.20.2
# via importlib-metadata

# The following packages are considered to be unsafe in a requirements file:
Expand Down
25 changes: 8 additions & 17 deletions requirements/requirements-docs.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#
# This file is autogenerated by pip-compile with Python 3.8
# This file is autogenerated by pip-compile with Python 3.10
# by the following command:
#
# pip-compile requirements/requirements-docs.in
Expand All @@ -16,13 +16,8 @@ colorama==0.4.6
# via mkdocs-material
ghp-import==2.1.0
# via mkdocs
idna==3.8
idna==3.10
# via requests
importlib-metadata==8.4.0
# via
# markdown
# mkdocs
# mkdocs-get-deps
jinja2==3.1.4
# via
# mkdocs
Expand All @@ -45,11 +40,11 @@ mkdocs==1.6.1
# -r requirements/requirements-docs.in
# mkdocs-asf-theme
# mkdocs-material
mkdocs-asf-theme==0.3.2
mkdocs-asf-theme==0.4.0
# via -r requirements/requirements-docs.in
mkdocs-get-deps==0.2.0
# via mkdocs
mkdocs-material==9.5.34
mkdocs-material==9.5.35
# via mkdocs-asf-theme
mkdocs-material-extensions==1.3.1
# via mkdocs-material
Expand All @@ -59,16 +54,14 @@ paginate==0.5.7
# via mkdocs-material
pathspec==0.12.1
# via mkdocs
platformdirs==4.3.2
platformdirs==4.3.6
# via mkdocs-get-deps
pygments==2.18.0
# via mkdocs-material
pymdown-extensions==10.9
# via mkdocs-material
python-dateutil==2.9.0.post0
# via ghp-import
pytz==2024.1
# via babel
pyyaml==6.0.2
# via
# mkdocs
Expand All @@ -77,15 +70,13 @@ pyyaml==6.0.2
# pyyaml-env-tag
pyyaml-env-tag==0.1
# via mkdocs
regex==2024.7.24
regex==2024.9.11
# via mkdocs-material
requests==2.32.3
# via mkdocs-material
six==1.16.0
# via python-dateutil
urllib3==2.2.2
urllib3==2.2.3
# via requests
watchdog==4.0.2
watchdog==5.0.2
# via mkdocs
zipp==3.20.1
# via importlib-metadata
6 changes: 1 addition & 5 deletions requirements/requirements-make.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,9 @@
#
# This file is autogenerated by pip-compile with Python 3.8
# This file is autogenerated by pip-compile with Python 3.10
# by the following command:
#
# pip-compile requirements/requirements-make.in
#
importlib-metadata==8.4.0
# via markdown
jinja2==3.1.4
# via -r requirements/requirements-make.in
markdown==3.7
Expand All @@ -20,5 +18,3 @@ pymdown-extensions==10.9
# via -r requirements/requirements-make.in
pyyaml==6.0.2
# via pymdown-extensions
zipp==3.20.1
# via importlib-metadata
12 changes: 6 additions & 6 deletions requirements/requirements-test.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
#
# This file is autogenerated by pip-compile with Python 3.8
# This file is autogenerated by pip-compile with Python 3.10
# by the following command:
#
# pip-compile requirements/requirements-test.in
#
boto3==1.35.14
boto3==1.35.23
# via -r requirements/requirements-test.in
botocore==1.35.14
botocore==1.35.23
# via
# boto3
# s3transfer
Expand All @@ -16,7 +16,7 @@ charset-normalizer==3.3.2
# via requests
exceptiongroup==1.2.2
# via pytest
idna==3.8
idna==3.10
# via requests
iniconfig==2.0.0
# via pytest
Expand All @@ -28,7 +28,7 @@ packaging==24.1
# via pytest
pluggy==1.5.0
# via pytest
pytest==8.3.2
pytest==8.3.3
# via -r requirements/requirements-test.in
python-dateutil==2.9.0.post0
# via botocore
Expand All @@ -42,7 +42,7 @@ six==1.16.0
# via python-dateutil
tomli==2.0.1
# via pytest
urllib3==1.26.20
urllib3==2.2.3
# via
# botocore
# requests
6 changes: 3 additions & 3 deletions requirements/requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
#
# This file is autogenerated by pip-compile with Python 3.8
# This file is autogenerated by pip-compile with Python 3.10
# by the following command:
#
# pip-compile requirements/requirements.in
#
blessed==1.20.0
# via inquirer
botocore==1.35.18
botocore==1.35.23
# via chalice
cachetools==5.5.0
# via
Expand Down Expand Up @@ -57,7 +57,7 @@ six==1.16.0
# blessed
# chalice
# python-dateutil
urllib3==1.26.20
urllib3==2.2.3
# via
# botocore
# cfnresponse
Expand Down
2 changes: 1 addition & 1 deletion tea-cli/pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ packages = [{include = "tea_cli"}]
tea = "tea_cli:main.main"

[tool.poetry.dependencies]
python = "^3.8"
python = "^3.10"
boto3 = "^1.26.163"
cryptography = "^41.0.1"

Expand Down
2 changes: 1 addition & 1 deletion tests/test_tea_bumper.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ def test_lambda(client_iam, client_lambda, test_lambda_code_zip):

client_lambda.create_function(
FunctionName="test-lambda",
Runtime="python3.8",
Runtime="python3.10",
Role=role["Arn"],
Code={
"ZipFile": b64encode(test_lambda_code_zip)
Expand Down