Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ci] [python-package] Add CI job testing compatibility with oldest possible versions #5936

Merged
merged 16 commits into from
Jul 6, 2023
Merged

[ci] [python-package] Add CI job testing compatibility with oldest possible versions #5936

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
3fd1abc
more changes
jameslamb Jun 21, 2023
92e239b
add CI job
jameslamb Jun 21, 2023
7427793
revert build-python change
jameslamb Jun 21, 2023
413d959
Merge branch 'master' into ci/old-versions
jameslamb Jun 23, 2023
af94ad9
install docker
jameslamb Jun 23, 2023
9df09cd
no sudo
jameslamb Jun 23, 2023
f417967
apt-get update
jameslamb Jun 23, 2023
3dd2c89
hmmmmm
jameslamb Jun 23, 2023
be64783
run directly on VM
jameslamb Jun 23, 2023
634d337
no -it
jameslamb Jun 23, 2023
c2d1602
more changes
jameslamb Jun 23, 2023
16d67a2
revert setup.cfg changes
jameslamb Jun 23, 2023
381b047
Merge branch 'master' into ci/old-versions
jameslamb Jun 23, 2023
e56b022
empty commit
jameslamb Jun 23, 2023
f79f17a
Merge branch 'master' into ci/old-versions
jameslamb Jun 30, 2023
483fe83
Merge branch 'master' into ci/old-versions
jameslamb Jul 4, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 49 additions & 0 deletions .ci/test-python-oldest.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
#!/bin/bash

# oldest versions of dependencies published after
# minimum supported Python version's first release
#
# see https://devguide.python.org/versions/
#
echo "installing lightgbm's dependencies"
pip install \
'numpy==1.12.0' \
'pandas==0.24.0' \
'scikit-learn==0.18.2' \
'scipy==0.19.0' \
|| exit -1
echo "done installing lightgbm's dependencies"
jmoralez marked this conversation as resolved.
Show resolved Hide resolved

echo "installing lightgbm"
pip install --no-deps dist/*.whl || exit -1
echo "done installing lightgbm"

echo "installed package versions:"
pip freeze

echo ""
echo "checking that examples run without error"

# run a few examples to test that Python package minimally works
echo ""
echo "--- advanced_example.py ---"
echo ""
python ./examples/python-guide/advanced_example.py || exit -1

echo ""
echo "--- logistic_regression.py ---"
echo ""
python ./examples/python-guide/logistic_regression.py || exit -1

echo ""
echo "--- simple_example.py ---"
echo ""
python ./examples/python-guide/simple_example.py || exit -1

echo ""
echo "--- sklearn_example.py ---"
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why only these examples? Why not any dask ones or the plotting one?

numpy, scikit-learn, pandas, and scipy introduce very few additional dependencies. For example, from the CI logs:

installed package versions:
lightgbm @ file:///opt/lgb-build/dist/lightgbm-3.3.5.99-py3-none-manylinux_2_28_x86_64.whl
numpy==1.12.0
pandas==0.24.0
python-dateutil==2.8.2
pytz==2023.3
scikit-learn==0.18.2
scipy==0.19.0
six==1.16.0

(build link)

dask, on the other hands, introduces many more. With dask[array,dataframe,distributed]==2.0.0 installed in that same environment, here's what pip freeze shows:

click==8.0.4
cloudpickle==2.2.1
dask==2.0.0
distributed==2.5.1
HeapDict==1.0.1
importlib-metadata==4.8.3
locket==1.0.0
msgpack==1.0.5
numpy==1.13.0
pandas==0.24.0
partd==1.2.0
psutil==5.9.5
python-dateutil==2.8.2
pytz==2023.3
PyYAML==6.0
scikit-learn==0.18.2
scipy==0.19.0
six==1.16.0
sortedcontainers==2.4.0
tblib==1.7.0
toolz==0.12.0
tornado==6.1
typing_extensions==4.1.1
zict==2.1.0
zipp==3.6.0

I don't want to take on the maintenance burden of making all of those things continue working on Python 3.6.

And as for the plotting examples... similar concern. They rely on some sometimes-hard-to-build graphics libraries like graphviz and tkinter.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not running the whole test suite? We would need a couple more dependencies like cloudpickle and psutil and some minor changes to some tests but it'd work. That'd allow us to prevent introducing a breaking change for py36.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not running the whole test suite?

This is exactly what I meant when I said in #5759 (comment) that "I don't want to take on any more support than [a minimal CI job] for Python 3.6". In this project that's already struggling from a severe lack of maintainer attention and availability, I don't think we should be taking on the maintenance burden of trying to keep all the tests working with Python 3.6, which also implies keeping them working with:

  • older versions of pytest
  • older versions of all the recursive dependencies of the libraries used by lightgbm + in its tests

In this PR's current state, we get a new CI job that'll confirm that on every commit, the Python package...

  • is syntactically-valid Python 3.6 code
  • runs without error for the most heavily-used parts of the LightGBM API

I think that's an incremental improvement over the current state, and honestly good enough. I don't support reverting #4891 and trying to say this project fully and completely supports Python 3.6.

If you feel really strongly that we should go farther than this PR does, how about this... could we merge this as-is, and then you could put up a follow-up PR that adds running the unit tests?

According to https://pypistats.org/packages/lightgbm, only around 3-6% of lightgbm installations from PyPI are for Python 3.6 these days.

image

I don't know the equivalent number for conda, but it must be lower, given that both Anaconda and conda-forge have stopped supporting Python 3.6 (and even Python 3.7 already!).

Per https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-python.html

Conda supports Python 3.8, 3.9, 3.10, and 3.11.

And per https://conda-forge.org/docs/user/announcements.html#dropping-python-3-7 and conda-forge/conda-forge-pinning-feedstock#2623 conda-forge dropped support for Python 3.7 in August 2022 (10 months ago).

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that's an incremental improvement over the current state, and honestly good enough.

Totally agree.

that both Anaconda and conda-forge have stopped supporting Python 3.6 (and even Python 3.7 already!).

This is for publishing new versions of packages, right? You can still create an env with python 3.6 from conda-forge.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can still create an env with python 3.6 from conda-forge.

Yeah I think you're right. I just tried that on my Mac with mambaforge:

mamba create --name py36 \
    numpy \
    pandas \
    python=3.6 \
    scikit-learn \
    scipy

And it worked, albeit with some fairly old versions:

  + ca-certificates  2023.5.7  h8857fd0_0          conda-forge/osx-64     Cached
  + joblib              1.3.0  pyhd8ed1ab_0        conda-forge/noarch     Cached
  + libblas             3.9.0  17_osx64_openblas   conda-forge/osx-64     Cached
  + libcblas            3.9.0  17_osx64_openblas   conda-forge/osx-64     Cached
  + libcxx             16.0.6  hd57cbcb_0          conda-forge/osx-64     Cached
  + libffi              3.4.2  h0d85af4_5          conda-forge/osx-64     Cached
  + libgfortran         5.0.0  11_3_0_h97931a8_31  conda-forge/osx-64     Cached
  + libgfortran5       12.2.0  he409387_31         conda-forge/osx-64     Cached
  + liblapack           3.9.0  17_osx64_openblas   conda-forge/osx-64     Cached
  + libopenblas        0.3.23  openmp_h429af6e_0   conda-forge/osx-64     Cached
  + libsqlite          3.42.0  h58db7d2_0          conda-forge/osx-64     Cached
  + libzlib            1.2.13  h8a1eda9_5          conda-forge/osx-64     Cached
  + llvm-openmp        16.0.6  hff08bdf_0          conda-forge/osx-64     Cached
  + ncurses               6.4  hf0c8a7f_0          conda-forge/osx-64     Cached
  + numpy              1.19.5  py36h08b5fde_2      conda-forge/osx-64     Cached
  + openssl            1.1.1u  h8a1eda9_0          conda-forge/osx-64     Cached
  + pandas              1.1.5  py36h2be6da3_0      conda-forge/osx-64     Cached
  + pip                21.3.1  pyhd8ed1ab_0        conda-forge/noarch     Cached
  + python             3.6.15  haf480d7_0_cpython  conda-forge/osx-64     Cached
  + python-dateutil     2.8.2  pyhd8ed1ab_0        conda-forge/noarch     Cached
  + python_abi            3.6  2_cp36m             conda-forge/osx-64     Cached
  + pytz               2023.3  pyhd8ed1ab_0        conda-forge/noarch     Cached
  + readline              8.2  h9e318b2_1          conda-forge/osx-64     Cached
  + scikit-learn       0.24.2  py36h979b75d_1      conda-forge/osx-64     Cached
  + scipy               1.5.3  py36h4f136de_1      conda-forge/osx-64     Cached
  + setuptools         58.0.4  py36h79c6626_2      conda-forge/osx-64      983kB
  + six                1.16.0  pyh6c4a22f_0        conda-forge/noarch     Cached
  + sqlite             3.42.0  h2b0dec6_0          conda-forge/osx-64     Cached
  + threadpoolctl       3.1.0  pyh8a188c0_0        conda-forge/noarch     Cached
  + tk                 8.6.12  h5dbffcc_0          conda-forge/osx-64     Cached
  + wheel              0.37.1  pyhd8ed1ab_0        conda-forge/noarch     Cached
  + xz                  5.2.6  h775f41a_0          conda-forge/osx-64     Cached

echo ""
python ./examples/python-guide/sklearn_example.py || exit -1

echo ""
echo "done testing on oldest supported Python version"
28 changes: 27 additions & 1 deletion .github/workflows/python_package.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,10 +76,36 @@ jobs:
export PATH=${CONDA}/bin:${PATH}
$GITHUB_WORKSPACE/.ci/setup.sh || exit -1
$GITHUB_WORKSPACE/.ci/test.sh || exit -1
test-oldest-versions:
name: Python - oldest supported versions (ubuntu-latest)
runs-on: ubuntu-latest
timeout-minutes: 60
steps:
- name: Checkout repository
uses: actions/checkout@v3
with:
fetch-depth: 5
submodules: true
- name: Create wheel
run: |
docker run \
--rm \
-v $(pwd):/opt/lgb-build \
-w /opt/lgb-build \
lightgbm/vsts-agent:manylinux_2_28_x86_64 \
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the same image we use for building the wheels published to PyPI.

LightGBM/.vsts-ci.yml

Lines 21 to 22 in ac57d5a

- container: linux-artifact-builder
image: lightgbm/vsts-agent:manylinux_2_28_x86_64

LightGBM/.vsts-ci.yml

Lines 36 to 38 in ac57d5a

PRODUCES_ARTIFACTS: 'true'
pool: sh-ubuntu
container: linux-artifact-builder

It's also not possible, as of #5759, to run sh build-python.sh in Python < 3.7, since:

  • scikit-build-core has a floor of Python >= 3.7
  • the oldest supported setuptools for Python 3.6, v59.6.0 (December 2021), doesn't support pyproject.toml

So as of #5759, Python 3.6 users will have to use the published wheels. Customizations like --cuda will require building the wheel in a Python 3.7+ environment and then distributing it into wherever that Python 3.6 code is running.

I think Python 3.6 users should be expect to face issues for many packages (not just LightGBM) as a result of the changes in the Python packaging ecosystem described in #5061.

/bin/bash -c 'PATH=/opt/miniforge/bin:$PATH sh ./build-python.sh bdist_wheel --nomp'
- name: Test compatibility
run: |
docker run \
--rm \
-v $(pwd):/opt/lgb-build \
-w /opt/lgb-build \
python:3.6 \
/bin/bash ./.ci/test-python-oldest.sh
all-python-package-jobs-successful:
if: always()
runs-on: ubuntu-latest
needs: [test]
needs: [test, test-oldest-versions]
steps:
- name: Note that all tests succeeded
uses: re-actors/[email protected]
Expand Down