Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use Python 3.10 in Continuous Integration tests #1577

Merged
merged 28 commits into from
Dec 27, 2021
Merged

Use Python 3.10 in Continuous Integration tests #1577

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
eec37af
Update continuous integration workflows to use Python 3.10
weiji14 Oct 8, 2021
e197bde
Put quotes around 3.10 so that it doesn't get read as Python 3.1
weiji14 Oct 8, 2021
d73da3f
Merge branch 'main' into ci/python-3.10
weiji14 Oct 12, 2021
c7bca7d
Revert ci_docs and ci_tests to use Python 3.9
weiji14 Oct 12, 2021
2b9abf7
Manually install Python 3.10 on ci_tests_dev.yaml
weiji14 Oct 12, 2021
76bbf2c
Merge branch 'main' into ci/python-3.10
weiji14 Oct 16, 2021
00cafd7
Use pip to install dvc on ci_tests_dev workflow
weiji14 Oct 16, 2021
36ce2d3
Merge branch 'main' into ci/python-3.10
weiji14 Oct 21, 2021
f0e8e76
Merge branch 'main' into ci/python-3.10
weiji14 Oct 25, 2021
2ce685e
Use pip to install geopandas on ci_tests_dev workflow
weiji14 Oct 26, 2021
2a4f15a
Try with newer version of conda/mamba
weiji14 Oct 26, 2021
63cd1d0
Revert "Try with newer version of conda/mamba"
weiji14 Oct 26, 2021
57d42cc
Try flexible channel priority
weiji14 Oct 26, 2021
56d044b
Temporarily run GMT Dev Tests on pull request pushes
weiji14 Oct 26, 2021
bfd8527
Install numpy before other deps since it is a build dependency for some
weiji14 Oct 26, 2021
c32e6c9
Merge branch 'main' into ci/python-3.10
weiji14 Nov 5, 2021
20854ee
Prefer older binary pip packages over newer pip packages
weiji14 Nov 5, 2021
a462adc
Revert "Try flexible channel priority"
weiji14 Oct 26, 2021
31e8908
Install gdal Python package in addition to libgdal from conda-forge
weiji14 Nov 5, 2021
69d4e67
Install fiona instead of gdal
weiji14 Nov 5, 2021
0a9007d
Merge branch 'main' into ci/python-3.10
weiji14 Dec 4, 2021
0604c95
Revert "Use pip to install geopandas on ci_tests_dev workflow"
weiji14 Dec 4, 2021
f692874
Merge branch 'main' into ci/python-3.10
weiji14 Dec 7, 2021
b5a5ab5
Merge branch 'main' into ci/python-3.10
weiji14 Dec 20, 2021
a4b10af
Revert "Temporarily run GMT Dev Tests on pull request pushes"
weiji14 Oct 26, 2021
fca7a68
Remove quotes around python version numbers
weiji14 Dec 20, 2021
0772260
Merge branch 'main' into ci/python-3.10
weiji14 Dec 20, 2021
afed7c9
Set minimum Python version to 3.8 in README.rst
weiji14 Dec 20, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci_docs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ jobs:
strategy:
fail-fast: false
matrix:
python-version: [3.9]
python-version: ['3.9']
os: [ubuntu-latest, macOS-latest, windows-latest]
# Is it a draft Pull Request (true or false)?
isDraft:
Expand Down
16 changes: 9 additions & 7 deletions .github/workflows/ci_tests_dev.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ jobs:
strategy:
fail-fast: false
matrix:
python-version: [3.9]
python-version: ['3.10']
os: [ubuntu-latest, macOS-11.0, windows-2022]
gmt_git_ref: [master]
timeout-minutes: 30
Expand Down Expand Up @@ -77,7 +77,7 @@ jobs:
uses: conda-incubator/[email protected]
with:
activate-environment: pygmt
python-version: ${{ matrix.python-version }}
# python-version: ${{ matrix.python-version }}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any special reason to comment out this line?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The setup-miniconda action won't work with Python 3.10, because there isn't a miniconda or mambaforge version with Python 3.10 yet (see https://repo.anaconda.com/miniconda/ and https://github.com/conda-forge/miniforge/releases). The Python 3.10 install step happens later below.

channels: conda-forge,nodefaults
channel-priority: strict
miniforge-version: latest
Expand All @@ -88,11 +88,13 @@ jobs:
# Install dependencies from conda-forge
- name: Install dependencies
run: |
mamba install ninja cmake libblas libcblas liblapack fftw gdal geopandas \
ghostscript libnetcdf hdf5 zlib curl pcre make dvc
pip install --pre numpy pandas xarray netCDF4 packaging \
ipython pytest-cov pytest-mpl pytest>=6.0 sphinx-gallery \
tomli
mamba install python=${{ matrix.python-version }} \
ninja cmake libblas libcblas liblapack fftw libgdal \
geopandas ghostscript libnetcdf hdf5 zlib curl pcre make
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think we should install geopandas using pip instead?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally yes, and I did try it (see 69d4e67), but pip install geopandas doesn't work on Python 3.10 because the fiona dependency doesn't have Python 3.10 wheels yet and the install is not functional:

E           ImportError: the 'to_file' method requires the 'fiona' package, but it is not installed or does not import correctly.
E           Importing fiona resulted in: /usr/share/miniconda3/envs/pygmt/lib/python3.10/site-packages/fiona/ogrext.cpython-310-x86_64-linux-gnu.so: undefined symbol: _PyGen_Send

So I had to revert back to using conda install geopandas in 0604c95.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI, Python 3.10 support in fiona is being worked on at Toblerity/Fiona#1049. Seems like the Python 3.10 wheels might take a while to come out, so probably shouldn't wait.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we're waiting for the wheels to come out, why shouldn't we wait to upgrade to Python 3.10?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we're waiting for the wheels to come out, why shouldn't we wait to upgrade to Python 3.10?

If I'm reading your sentence correctly, no, we don't have to wait for the fiona Python 3.10 wheels to come out. The current state on the main branch is that fiona is installed indirectly via conda install geopandas. @seisman's suggestion was that we use pip install --pre geopandas instead, but that doesn't work because fiona doesn't have Python 3.10 wheels.

Since geopandas/fiona is an optional dependency for PyGMT, I think we can just push ahead with this PR as is (i.e. stick with conda install geopandas), but happy to wait too if there's a good reason to.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems the new fiona release will take at least another month (Toblerity/Fiona#1053). I think we don't need to wait for it.

pip install --pre --prefer-binary \
numpy pandas xarray netCDF4 packaging \
dvc ipython 'pytest>=6.0' pytest-cov \
pytest-mpl sphinx-gallery tomli

# Build and install latest GMT from GitHub
- name: Install GMT ${{ matrix.gmt_git_ref }} branch (Linux/macOS)
Expand Down
2 changes: 1 addition & 1 deletion README.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -218,7 +218,7 @@ Compatibility with GMT/Python/NumPy versions
* - `Dev <https://github.com/GenericMappingTools/pygmt/milestone/9>`_ (upcoming release)
- `Dev Documentation <https://www.pygmt.org/dev>`_ (reflects `main branch <https://github.com/GenericMappingTools/pygmt>`_)
- >=6.3.0
- >=3.7
- >=3.8
- >=1.19
* - `v0.5.0 <https://github.com/GenericMappingTools/pygmt/releases/tag/v0.5.0>`_ (latest release)
- `v0.5.0 Documentation <https://www.pygmt.org/v0.5.0>`_
Expand Down