Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(sdk): drop Python 3.7 for kfp sdk #10750

Merged
merged 29 commits into from
May 3, 2024
Merged
Show file tree
Hide file tree
Changes from 25 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
0a93e1a
chore(components): Drop GCPC Python 3.7
rickyxie0929 Apr 23, 2024
0dcc480
drop support python3.7
rickyxie0929 Apr 23, 2024
cb5ac53
fix test
rickyxie0929 Apr 23, 2024
0fa3562
fix test
rickyxie0929 Apr 24, 2024
7b986e2
fix test
rickyxie0929 Apr 24, 2024
dd0b55a
fix test
rickyxie0929 Apr 24, 2024
2c524d5
fix tes
rickyxie0929 Apr 24, 2024
15c066b
fix test
rickyxie0929 Apr 24, 2024
19b1528
fix test
rickyxie0929 Apr 24, 2024
0481ff3
Merge branch 'kubeflow:master' into python
rickyxie0929 Apr 25, 2024
60bd20f
remove kfp related
rickyxie0929 Apr 25, 2024
97cd571
remove kfp related
rickyxie0929 Apr 25, 2024
11e79d2
just kfp change
rickyxie0929 Apr 25, 2024
2a0a67b
only change in kubernetes_platform/python/setup.py
rickyxie0929 Apr 25, 2024
0203d08
only change in init.py
rickyxie0929 Apr 25, 2024
41bb5a3
change
rickyxie0929 Apr 25, 2024
926353e
change
rickyxie0929 Apr 25, 2024
584c101
Merge branch 'kubeflow:master' into drop
rickyxie0929 Apr 29, 2024
a120d8e
Merge branch 'kubeflow:master' into drop
rickyxie0929 Apr 30, 2024
2f9ad28
Update release
rickyxie0929 May 1, 2024
849b1ea
Update base image
rickyxie0929 May 2, 2024
e114d3e
change test python 3.7 to python 3.8
rickyxie0929 May 2, 2024
d399aa0
update golden snapshot
rickyxie0929 May 2, 2024
4902e7d
update yaml to python3.8 under kubernetes_platform/python/test/snapsh…
rickyxie0929 May 2, 2024
90122b5
update yaml python 3.8 for sdk/python/test_data/pipelines/parallelfor…
rickyxie0929 May 2, 2024
a67ee0e
update yaml to fix tests
rickyxie0929 May 2, 2024
0da6c6d
Update Readme
rickyxie0929 May 2, 2024
77a70b3
Update kubernetes_setup python version
rickyxie0929 May 2, 2024
20ab4d8
Update readme
rickyxie0929 May 2, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ deploymentSpec:
'
- "\nimport kfp\nfrom kfp import dsl\nfrom kfp.dsl import *\nfrom typing import\
\ *\n\ndef comp():\n pass\n\n"
image: python:3.7
image: python:3.8
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the files under kubernetes_platform/ are for kfp-kubernetes. Should we do that in a separate change?

pipelineInfo:
name: my-pipeline
root:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ deploymentSpec:
'
- "\nimport kfp\nfrom kfp import dsl\nfrom kfp.dsl import *\nfrom typing import\
\ *\n\ndef comp():\n pass\n\n"
image: python:3.7
image: python:3.8
pipelineInfo:
name: my-pipeline
root:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,7 @@ deploymentSpec:
\ *\n\ndef consumer() -> str:\n with open('/data/file.txt', 'r') as file:\n\
\ content = file.read()\n print(content)\n return content\n\
\n"
image: python:3.7
image: python:3.8
exec-createpvc:
container:
image: argostub/createpvc
Expand Down Expand Up @@ -150,7 +150,7 @@ deploymentSpec:
\ file.write('Hello world')\n with open('/data/file.txt', 'r')\
\ as file:\n content = file.read()\n print(content)\n return\
\ content\n\n"
image: python:3.7
image: python:3.8
pipelineInfo:
name: my-pipeline
root:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ deploymentSpec:
'
- "\nimport kfp\nfrom kfp import dsl\nfrom kfp.dsl import *\nfrom typing import\
\ *\n\ndef comp():\n pass\n\n"
image: python:3.7
image: python:3.8
exec-createpvc:
container:
image: argostub/createpvc
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,7 @@ deploymentSpec:
'
- "\nimport kfp\nfrom kfp import dsl\nfrom kfp.dsl import *\nfrom typing import\
\ *\n\ndef comp():\n pass\n\n"
image: python:3.7
image: python:3.8
exec-createpvc:
container:
image: argostub/createpvc
Expand Down Expand Up @@ -141,7 +141,7 @@ deploymentSpec:
'
- "\nimport kfp\nfrom kfp import dsl\nfrom kfp.dsl import *\nfrom typing import\
\ *\n\ndef get_pvc_name() -> str:\n return 'static-pvc-name'\n\n"
image: python:3.7
image: python:3.8
pipelineInfo:
name: my-pipeline
root:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ deploymentSpec:
'
- "\nimport kfp\nfrom kfp import dsl\nfrom kfp.dsl import *\nfrom typing import\
\ *\n\ndef comp():\n pass\n\n"
image: python:3.7
image: python:3.8
pipelineInfo:
name: my-pipeline
root:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ deploymentSpec:
'
- "\nimport kfp\nfrom kfp import dsl\nfrom kfp.dsl import *\nfrom typing import\
\ *\n\ndef comp():\n pass\n\n"
image: python:3.7
image: python:3.8
pipelineInfo:
name: my-pipeline
root:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ deploymentSpec:
'
- "\nimport kfp\nfrom kfp import dsl\nfrom kfp.dsl import *\nfrom typing import\
\ *\n\ndef comp():\n pass\n\n"
image: python:3.7
image: python:3.8
pipelineInfo:
name: my-pipeline
root:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ deploymentSpec:
'
- "\nimport kfp\nfrom kfp import dsl\nfrom kfp.dsl import *\nfrom typing import\
\ *\n\ndef comp():\n pass\n\n"
image: python:3.7
image: python:3.8
pipelineInfo:
name: my-pipeline
root:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ deploymentSpec:
'
- "\nimport kfp\nfrom kfp import dsl\nfrom kfp.dsl import *\nfrom typing import\
\ *\n\ndef comp():\n pass\n\n"
image: python:3.7
image: python:3.8
pipelineInfo:
name: my-pipeline
root:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ deploymentSpec:
'
- "\nimport kfp\nfrom kfp import dsl\nfrom kfp.dsl import *\nfrom typing import\
\ *\n\ndef comp():\n pass\n\n"
image: python:3.7
image: python:3.8
pipelineInfo:
name: my-pipeline
root:
Expand Down
2 changes: 1 addition & 1 deletion kubernetes_platform/python/test/snapshot/data/timeout.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ deploymentSpec:
'
- "\nimport kfp\nfrom kfp import dsl\nfrom kfp.dsl import *\nfrom typing import\
\ *\n\ndef comp():\n pass\n\n"
image: python:3.7
image: python:3.8
pipelineInfo:
name: my-pipeline
root:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ deploymentSpec:
'
- "\nimport kfp\nfrom kfp import dsl\nfrom kfp.dsl import *\nfrom typing import\
\ *\n\ndef comp():\n pass\n\n"
image: python:3.7
image: python:3.8
pipelineInfo:
name: my-pipeline
root:
Expand Down
1 change: 1 addition & 0 deletions sdk/RELEASE.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@

## Documentation updates
* Make full version dropdown visible on all KFP SDK docs versions [\#10577](https://github.com/kubeflow/pipelines/pull/10577)
* Drop support for Python 3.7 since it has reached end-of-life(part1). [\#10750](https://github.com/kubeflow/pipelines/pull/10750)
rickyxie0929 marked this conversation as resolved.
Show resolved Hide resolved

# 2.7.0

Expand Down
4 changes: 2 additions & 2 deletions sdk/python/kfp/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,9 @@
import sys
import warnings

rickyxie0929 marked this conversation as resolved.
Show resolved Hide resolved
if sys.version_info < (3, 8):
if sys.version_info < (3, 9):
warnings.warn(
('Python 3.7 has reached end-of-life. KFP will drop support for Python 3.7 on April 23, 2024. To use new versions of the KFP SDK after that date, you will need to upgrade to Python >= 3.8. See https://devguide.python.org/versions/ for more details.'
('KFP will drop support for Python 3.8 on Oct 1, 2024. To use new versions of the KFP SDK after that date, you will need to upgrade to Python >= 3.9. See https://devguide.python.org/versions/ for more details.'
),
FutureWarning,
stacklevel=2,
Expand Down
10 changes: 5 additions & 5 deletions sdk/python/kfp/cli/component_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -463,7 +463,7 @@ def test_docker_file_is_created_correctly(self):
textwrap.dedent('''\
# Generated by KFP.

FROM python:3.7
FROM python:3.8

WORKDIR /usr/local/src/kfp/components
COPY runtime-requirements.txt runtime-requirements.txt
Expand Down Expand Up @@ -492,7 +492,7 @@ def test_docker_file_is_created_correctly_with_one_url(self):
textwrap.dedent('''\
# Generated by KFP.

FROM python:3.7
FROM python:3.8

WORKDIR /usr/local/src/kfp/components
COPY runtime-requirements.txt runtime-requirements.txt
Expand Down Expand Up @@ -523,7 +523,7 @@ def test_docker_file_is_created_correctly_with_two_urls(self):
textwrap.dedent('''\
# Generated by KFP.

FROM python:3.7
FROM python:3.8

WORKDIR /usr/local/src/kfp/components
COPY runtime-requirements.txt runtime-requirements.txt
Expand Down Expand Up @@ -566,7 +566,7 @@ def test_existing_dockerfile_can_be_overwritten(self):
textwrap.dedent('''\
# Generated by KFP.

FROM python:3.7
FROM python:3.8

WORKDIR /usr/local/src/kfp/components
COPY runtime-requirements.txt runtime-requirements.txt
Expand Down Expand Up @@ -602,7 +602,7 @@ def test_dockerfile_can_contain_custom_kfp_package(self):
file_start = textwrap.dedent('''\
# Generated by KFP.

FROM python:3.7
FROM python:3.8

WORKDIR /usr/local/src/kfp/components
COPY runtime-requirements.txt runtime-requirements.txt
Expand Down
4 changes: 2 additions & 2 deletions sdk/python/kfp/dsl/component_factory.py
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@
from kfp.dsl.types import type_annotations
from kfp.dsl.types import type_utils

_DEFAULT_BASE_IMAGE = 'python:3.7'
_DEFAULT_BASE_IMAGE = 'python:3.8'
SINGLE_OUTPUT_NAME = 'Output'


Expand Down Expand Up @@ -537,7 +537,7 @@ def create_component_from_func(
if base_image is None:
base_image = _DEFAULT_BASE_IMAGE
warnings.warn(
("Python 3.7 has reached end-of-life. The default base_image used by the @dsl.component decorator will switch from 'python:3.7' to 'python:3.8' on April 23, 2024. To ensure your existing components work with versions of the KFP SDK released after that date, you should provide an explicit base_image argument and ensure your component works as intended on Python 3.8."
("The default base_image used by the @dsl.component decorator will switch from 'python:3.8' to 'python:3.9' on Oct 1, 2024. To ensure your existing components work with versions of the KFP SDK released after that date, you should provide an explicit base_image argument and ensure your component works as intended on Python 3.9."
),
FutureWarning,
stacklevel=2,
Expand Down
2 changes: 1 addition & 1 deletion sdk/python/kfp/dsl/component_factory_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -293,7 +293,7 @@ def test_default_base_image(self):

with self.assertWarnsRegex(
FutureWarning,
r"Python 3\.7 has reached end-of-life\. The default base_image used by the @dsl\.component decorator will switch from 'python:3\.7' to 'python:3\.8' on April 23, 2024\. To ensure your existing components work with versions of the KFP SDK released after that date, you should provide an explicit base_image argument and ensure your component works as intended on Python 3\.8\."
r"The default base_image used by the @dsl\.component decorator will switch from 'python:3\.8' to 'python:3\.9' on Oct 1, 2024\. To ensure your existing components work with versions of the KFP SDK released after that date, you should provide an explicit base_image argument and ensure your component works as intended on Python 3\.9\."
):

@dsl.component
Expand Down
2 changes: 1 addition & 1 deletion sdk/python/kfp/init_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ def test(self):

with self.assertWarnsRegex(
FutureWarning,
r'Python 3\.7 has reached end-of-life\. KFP will drop support for Python 3\.7 on April 23, 2024\. To use new versions of the KFP SDK after that date, you will need to upgrade to Python >= 3\.8\. See https:\/\/devguide\.python\.org\/versions\/ for more details\.'
r'KFP will drop support for Python 3\.8 on Oct 1, 2024\. To use new versions of the KFP SDK after that date, you will need to upgrade to Python >= 3\.9\. See https:\/\/devguide\.python\.org\/versions\/ for more details\.'
):
# simulate first import from kfp
importlib.reload(mod)
Expand Down
3 changes: 1 addition & 2 deletions sdk/python/setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,6 @@ def read_readme() -> str:
'Intended Audience :: Science/Research',
'License :: OSI Approved :: Apache Software License',
'Programming Language :: Python :: 3',
'Programming Language :: Python :: 3.7',
'Programming Language :: Python :: 3.8',
'Programming Language :: Python :: 3.9',
'Programming Language :: Python :: 3.10',
Expand All @@ -99,7 +98,7 @@ def read_readme() -> str:
'Topic :: Software Development :: Libraries',
'Topic :: Software Development :: Libraries :: Python Modules',
],
python_requires='>=3.7.0,<3.13.0',
python_requires='>=3.8.0,<3.13.0',
include_package_data=True,
entry_points={
'console_scripts': [
Expand Down
6 changes: 3 additions & 3 deletions sdk/python/test_data/components/add_numbers.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ deploymentSpec:
- -c
- "\nif ! [ -x \"$(command -v pip)\" ]; then\n python3 -m ensurepip ||\
\ python3 -m ensurepip --user || apt-get install python3-pip\nfi\n\nPIP_DISABLE_PIP_VERSION_CHECK=1\
\ python3 -m pip install --quiet --no-warn-script-location 'kfp==2.1.3'\
\ python3 -m pip install --quiet --no-warn-script-location 'kfp==2.7.0'\
\ '--no-deps' 'typing-extensions>=3.7.4,<5; python_version<\"3.9\"' && \"\
$0\" \"$@\"\n"
- sh
Expand All @@ -47,7 +47,7 @@ deploymentSpec:
'
- "\nimport kfp\nfrom kfp import dsl\nfrom kfp.dsl import *\nfrom typing import\
\ *\n\ndef add_numbers(a: int, b: int) -> int:\n return a + b\n\n"
image: python:3.7
image: python:3.8
pipelineInfo:
name: add-numbers
root:
Expand Down Expand Up @@ -83,4 +83,4 @@ root:
Output:
parameterType: NUMBER_INTEGER
schemaVersion: 2.1.0
sdkVersion: kfp-2.1.3
sdkVersion: kfp-2.7.0
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ deploymentSpec:
- -c
- "\nif ! [ -x \"$(command -v pip)\" ]; then\n python3 -m ensurepip ||\
\ python3 -m ensurepip --user || apt-get install python3-pip\nfi\n\nPIP_DISABLE_PIP_VERSION_CHECK=1\
\ python3 -m pip install --quiet --no-warn-script-location 'kfp==2.1.3'\
\ python3 -m pip install --quiet --no-warn-script-location 'kfp==2.7.0'\
\ '--no-deps' 'typing-extensions>=3.7.4,<5; python_version<\"3.9\"' && \"\
$0\" \"$@\"\n"
- sh
Expand All @@ -72,7 +72,7 @@ deploymentSpec:
\ as f:\n content_b = f.read()\n\n concatenated_string = content_a\
\ + content_b\n with open(out_dataset.path, 'w') as f:\n f.write(concatenated_string)\n\
\n return concatenated_string\n\n"
image: python:3.7
image: python:3.8
pipelineInfo:
name: dataset-joiner
root:
Expand Down Expand Up @@ -126,4 +126,4 @@ root:
description: The concatenated string.
parameterType: STRING
schemaVersion: 2.1.0
sdkVersion: kfp-2.1.3
sdkVersion: kfp-2.7.0
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ deploymentSpec:
- "\nif ! [ -x \"$(command -v pip)\" ]; then\n python3 -m ensurepip ||\
\ python3 -m ensurepip --user || apt-get install python3-pip\nfi\n\nPIP_DISABLE_PIP_VERSION_CHECK=1\
\ python3 -m pip install --quiet --no-warn-script-location --index-url https://pypi.org/simple\
\ --trusted-host https://pypi.org/simple 'kfp==2.1.3' '--no-deps' 'typing-extensions>=3.7.4,<5;\
\ --trusted-host https://pypi.org/simple 'kfp==2.7.0' '--no-deps' 'typing-extensions>=3.7.4,<5;\
\ python_version<\"3.9\"' && python3 -m pip install --quiet --no-warn-script-location\
\ --index-url https://pypi.org/simple --trusted-host https://pypi.org/simple\
\ 'yapf' && \"$0\" \"$@\"\n"
Expand All @@ -35,7 +35,7 @@ deploymentSpec:
- "\nimport kfp\nfrom kfp import dsl\nfrom kfp.dsl import *\nfrom typing import\
\ *\n\ndef component_with_pip_install():\n import yapf\n print(dir(yapf))\n\
\n"
image: python:3.7
image: python:3.8
pipelineInfo:
name: component-with-pip-install
root:
Expand All @@ -49,4 +49,4 @@ root:
taskInfo:
name: component-with-pip-install
schemaVersion: 2.1.0
sdkVersion: kfp-2.1.3
sdkVersion: kfp-2.7.0
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ deploymentSpec:
- -c
- "\nif ! [ -x \"$(command -v pip)\" ]; then\n python3 -m ensurepip ||\
\ python3 -m ensurepip --user || apt-get install python3-pip\nfi\n\nPIP_DISABLE_PIP_VERSION_CHECK=1\
\ python3 -m pip install --quiet --no-warn-script-location 'kfp==2.1.3'\
\ python3 -m pip install --quiet --no-warn-script-location 'kfp==2.7.0'\
\ '--no-deps' 'typing-extensions>=3.7.4,<5; python_version<\"3.9\"' && \"\
$0\" \"$@\"\n"
- sh
Expand All @@ -40,7 +40,7 @@ deploymentSpec:
- "\nimport kfp\nfrom kfp import dsl\nfrom kfp.dsl import *\nfrom typing import\
\ *\n\ndef exit_comp(status: dsl.PipelineTaskFinalStatus):\n print(status)\n\
\n"
image: python:3.7
image: python:3.8
pipelineInfo:
name: exit-comp
root:
Expand All @@ -63,4 +63,4 @@ root:
isOptional: true
parameterType: TASK_FINAL_STATUS
schemaVersion: 2.1.0
sdkVersion: kfp-2.1.3
sdkVersion: kfp-2.7.0
6 changes: 3 additions & 3 deletions sdk/python/test_data/components/concat_message.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ deploymentSpec:
- -c
- "\nif ! [ -x \"$(command -v pip)\" ]; then\n python3 -m ensurepip ||\
\ python3 -m ensurepip --user || apt-get install python3-pip\nfi\n\nPIP_DISABLE_PIP_VERSION_CHECK=1\
\ python3 -m pip install --quiet --no-warn-script-location 'kfp==2.1.3'\
\ python3 -m pip install --quiet --no-warn-script-location 'kfp==2.7.0'\
\ '--no-deps' 'typing-extensions>=3.7.4,<5; python_version<\"3.9\"' && \"\
$0\" \"$@\"\n"
- sh
Expand All @@ -48,7 +48,7 @@ deploymentSpec:
- "\nimport kfp\nfrom kfp import dsl\nfrom kfp.dsl import *\nfrom typing import\
\ *\n\ndef concat_message(message1: str, message2: str) -> str:\n return\
\ message1 + message2\n\n"
image: python:3.7
image: python:3.8
pipelineInfo:
name: concat-message
root:
Expand Down Expand Up @@ -84,4 +84,4 @@ root:
Output:
parameterType: STRING
schemaVersion: 2.1.0
sdkVersion: kfp-2.1.3
sdkVersion: kfp-2.7.0
2 changes: 1 addition & 1 deletion sdk/python/test_data/components/container_io.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -57,4 +57,4 @@ root:
output_path:
parameterType: STRING
schemaVersion: 2.1.0
sdkVersion: kfp-2.1.3
sdkVersion: kfp-2.7.0
2 changes: 1 addition & 1 deletion sdk/python/test_data/components/container_no_input.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -24,4 +24,4 @@ root:
taskInfo:
name: container-no-input
schemaVersion: 2.1.0
sdkVersion: kfp-2.1.3
sdkVersion: kfp-2.7.0
Original file line number Diff line number Diff line change
Expand Up @@ -79,4 +79,4 @@ root:
model_config_path:
parameterType: STRING
schemaVersion: 2.1.0
sdkVersion: kfp-2.1.3
sdkVersion: kfp-2.7.0
Original file line number Diff line number Diff line change
Expand Up @@ -73,4 +73,4 @@ root:
output_path:
parameterType: STRING
schemaVersion: 2.1.0
sdkVersion: kfp-2.1.3
sdkVersion: kfp-2.7.0
Loading