Skip to content

Commit

Permalink
Remove Python 3.7 support (#30963)
Browse files Browse the repository at this point in the history
potiuk authored May 29, 2023

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
1 parent c55f9ea commit 9fa75aa
Showing 243 changed files with 1,424 additions and 1,680 deletions.
4 changes: 2 additions & 2 deletions .github/actions/breeze/action.yml
Original file line number Diff line number Diff line change
@@ -37,8 +37,8 @@ runs:
path: ~/.local/pipx
# README has the latest breeze's hash and python location is used to distinguish between
# different minor versions of python
key: "breeze-${{ env.pythonLocation }}-${{ hashFiles('dev/breeze/README.md') }}"
restore-keys: breeze-${{ env.pythonLocation }}
key: "breeze-3.8-${{ env.pythonLocation }}-${{ hashFiles('dev/breeze/README.md') }}"
restore-keys: breeze-3.8-${{ env.pythonLocation }}
- name: "Install Breeze"
shell: bash
run: ./scripts/ci/install_breeze.sh
2 changes: 1 addition & 1 deletion .github/actions/build-prod-images/action.yml
Original file line number Diff line number Diff line change
@@ -37,7 +37,7 @@ runs:
shell: bash
run: breeze ci-image pull --tag-as-latest
env:
PYTHON_MAJOR_MINOR_VERSION: "3.7"
PYTHON_MAJOR_MINOR_VERSION: "3.8"
- name: "Cleanup dist and context file"
shell: bash
run: rm -fv ./dist/* ./docker-context-files/*
2 changes: 1 addition & 1 deletion .readthedocs.yml
Original file line number Diff line number Diff line change
@@ -20,7 +20,7 @@ formats: []
sphinx:
configuration: docs/rtd-deprecation/conf.py
python:
version: "3.7"
version: "3.8"
install:
- method: pip
path: .
43 changes: 37 additions & 6 deletions BREEZE.rst
Original file line number Diff line number Diff line change
@@ -245,6 +245,17 @@ Run this command to install Breeze (make sure to use ``-e`` flag):
pipx install -e ./dev/breeze
.. note:: Note for Windows users

The ``./dev/breeze`` in command about is a PATH to sub-folder where breeze source packages are.
If you are on Windows, you should use Windows way to point to the ``dev/breeze`` sub-folder
of Airflow either as absolute or relative path. For example:

.. code-block:: bash
pipx install -e dev\breeze
Once this is complete, you should have ``breeze`` binary on your PATH and available to run by ``breeze``
command.

@@ -278,6 +289,26 @@ where it was installed.
You can run ``breeze setup version`` command to see where breeze installed from and what are the current sources
that Breeze works on

.. warning:: Upgrading from earlier Python version

If you used Breeze with Python 3.7 and when running it, it will complain that it needs Python 3.8. In this
case you should force-reinstall Breeze with ``pipx``:

.. code-block:: bash
pipx install --force -e ./dev/breeze
.. note:: Note for Windows users

The ``./dev/breeze`` in command about is a PATH to sub-folder where breeze source packages are.
If you are on Windows, you should use Windows way to point to the ``dev/breeze`` sub-folder
of Airflow either as absolute or relative path. For example:

.. code-block:: bash
pipx install --force -e dev\breeze
Running Breeze for the first time
---------------------------------

@@ -360,12 +391,12 @@ You can use additional ``breeze`` flags to choose your environment. You can spec
version to use, and backend (the meta-data database). Thanks to that, with Breeze, you can recreate the same
environments as we have in matrix builds in the CI.

For example, you can choose to run Python 3.7 tests with MySQL as backend and with mysql version 8
For example, you can choose to run Python 3.8 tests with MySQL as backend and with mysql version 8
as follows:

.. code-block:: bash
breeze --python 3.7 --backend mysql --mysql-version 8
breeze --python 3.8 --backend mysql --mysql-version 8
The choices you make are persisted in the ``./.build/`` cache directory so that next time when you use the
``breeze`` script, it could use the values that were used previously. This way you do not have to specify
@@ -526,15 +557,15 @@ When you are starting airflow from local sources, www asset compilation is autom

.. code-block:: bash
breeze --python 3.7 --backend mysql start-airflow
breeze --python 3.8 --backend mysql start-airflow
You can also use it to start any released version of Airflow from ``PyPI`` with the
``--use-airflow-version`` flag.

.. code-block:: bash
breeze start-airflow --python 3.7 --backend mysql --use-airflow-version 2.2.5
breeze start-airflow --python 3.8 --backend mysql --use-airflow-version 2.2.5
Those are all available flags of ``start-airflow`` command:

@@ -1422,10 +1453,10 @@ suffix and they need to also be paired with corresponding runtime dependency add
.. code-block:: bash
breeze prod-image build --python 3.7 --additional-dev-deps "libasound2-dev" \
breeze prod-image build --python 3.8 --additional-dev-deps "libasound2-dev" \
--additional-runtime-apt-deps "libasound2"
Same as above but uses python 3.7.
Same as above but uses python 3.8.
Building PROD image
...................
6 changes: 3 additions & 3 deletions CI.rst
Original file line number Diff line number Diff line change
@@ -145,7 +145,7 @@ have to be percent-encoded when you access them via UI (/ = %2F)
+--------------+----------------------------------------------------------+----------------------------------------------------------+

* <BRANCH> might be either "main" or "v2-*-test"
* <X.Y> - Python version (Major + Minor).Should be one of ["3.7", "3.8", "3.9"].
* <X.Y> - Python version (Major + Minor).Should be one of ["3.8", "3.9", "3.10", "3.11"].
* <COMMIT_SHA> - full-length SHA of commit either from the tip of the branch (for pushes/schedule) or
commit from the tip of the branch used for the PR.
@@ -523,9 +523,9 @@ For example knowing that the CI job was for commit ``cd27124534b46c9688a1d89e75f

.. code-block:: bash
docker pull ghcr.io/apache/airflow/main/ci/python3.7:cd27124534b46c9688a1d89e75fcd137ab5137e3
docker pull ghcr.io/apache/airflow/main/ci/python3.8:cd27124534b46c9688a1d89e75fcd137ab5137e3
docker run -it ghcr.io/apache/airflow/main/ci/python3.7:cd27124534b46c9688a1d89e75fcd137ab5137e3
docker run -it ghcr.io/apache/airflow/main/ci/python3.8:cd27124534b46c9688a1d89e75fcd137ab5137e3
But you usually need to pass more variables and complex setup if you want to connect to a database or
10 changes: 5 additions & 5 deletions CONTRIBUTING.rst
Original file line number Diff line number Diff line change
@@ -414,7 +414,7 @@ these guidelines:
- Run tests locally before opening PR.

- You can use any supported python version to run the tests, but the best is to check
if it works for the oldest supported version (Python 3.7 currently). In rare cases
if it works for the oldest supported version (Python 3.8 currently). In rare cases
tests might fail with the oldest version when you use features that are available in newer Python
versions. For that purpose we have ``airflow.compat`` package where we keep back-ported
useful features from newer versions.
@@ -840,7 +840,7 @@ from the PyPI package:
.. code-block:: bash
pip install apache-airflow[google,amazon,async]==2.2.5 \
--constraint "https://raw.githubusercontent.com/apache/airflow/constraints-2.2.5/constraints-3.7.txt"
--constraint "https://raw.githubusercontent.com/apache/airflow/constraints-2.2.5/constraints-3.8.txt"
The last one can be used to install Airflow in "minimal" mode - i.e when bare Airflow is installed without
extras.
@@ -852,15 +852,15 @@ requirements).
.. code-block:: bash
pip install -e . \
--constraint "https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-source-providers-3.7.txt"
--constraint "https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-source-providers-3.8.txt"
This works also with extras - for example:

.. code-block:: bash
pip install ".[ssh]" \
--constraint "https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-source-providers-3.7.txt"
--constraint "https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-source-providers-3.8.txt"
There are different set of fixed constraint files for different python major/minor versions and you should
@@ -872,7 +872,7 @@ If you want to update just airflow dependencies, without paying attention to pro
.. code-block:: bash
pip install . --upgrade \
--constraint "https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-no-providers-3.7.txt"
--constraint "https://raw.githubusercontent.com/apache/airflow/constraints-main/constraints-no-providers-3.8.txt"
The ``constraints-<PYTHON_MAJOR_MINOR_VERSION>.txt`` and ``constraints-no-providers-<PYTHON_MAJOR_MINOR_VERSION>.txt``
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
@@ -46,7 +46,7 @@ ARG AIRFLOW_USER_HOME_DIR=/home/airflow
# latest released version here
ARG AIRFLOW_VERSION="2.6.1"

ARG PYTHON_BASE_IMAGE="python:3.7-slim-bullseye"
ARG PYTHON_BASE_IMAGE="python:3.8-slim-bullseye"

ARG AIRFLOW_PIP_VERSION=23.1.2
ARG AIRFLOW_IMAGE_REPOSITORY="https://github.com/apache/airflow"
4 changes: 2 additions & 2 deletions Dockerfile.ci
Original file line number Diff line number Diff line change
@@ -16,7 +16,7 @@
#
# WARNING: THIS DOCKERFILE IS NOT INTENDED FOR PRODUCTION USE OR DEPLOYMENT.
#
ARG PYTHON_BASE_IMAGE="python:3.7-slim-bullseye"
ARG PYTHON_BASE_IMAGE="python:3.8-slim-bullseye"

##############################################################################################
# This is the script image where we keep all inlined bash scripts needed in other segments
@@ -615,7 +615,7 @@ chmod 1777 /tmp

AIRFLOW_SOURCES=$(cd "${IN_CONTAINER_DIR}/../.." || exit 1; pwd)

PYTHON_MAJOR_MINOR_VERSION=${PYTHON_MAJOR_MINOR_VERSION:=3.7}
PYTHON_MAJOR_MINOR_VERSION=${PYTHON_MAJOR_MINOR_VERSION:=3.8}

export AIRFLOW_HOME=${AIRFLOW_HOME:=${HOME}}

Loading

0 comments on commit 9fa75aa

Please sign in to comment.