Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NEP29: Set minimum required versions to NumPy 1.17+ and Python 3.7+ #1074

Merged
merged 20 commits into from
Mar 24, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
78c3d2a
NEP29: Set minimum required versions to NumPy 1.17+ and Python 3.7+
weiji14 Mar 18, 2021
9e641c2
Modify CI test matrix to test on Numpy 1.17 and 1.20, Python 3.7 and 3.9
weiji14 Mar 18, 2021
fcb6bad
Setup test matrix combinations (OS/Python/NumPy) explicitly
weiji14 Mar 18, 2021
5919ef8
Try alternative way of excluding certain matrix builds during draft PR
weiji14 Mar 18, 2021
06b0022
Include NumPy version in matrix build name
weiji14 Mar 18, 2021
1205006
Try add isDraft variable to ubuntu-latest matrix builds too
weiji14 Mar 18, 2021
2d293bb
Fancy ternary operator to pair Py3.7 with NumPy1.17 & Py3.9 with NumP…
weiji14 Mar 18, 2021
dd8863d
Use a less fancy way of setting min/max NEP29 versions for Python/NumPy
weiji14 Mar 18, 2021
a8355b5
Add NumPy version to codecov reporting and disable Linux py3.7 on draft
weiji14 Mar 18, 2021
69823ba
Merge branch 'master' into adopt_nep29
weiji14 Mar 18, 2021
20d83c2
Try fix error when evaluating 'runs-on' for job 'test' in Github Actions
weiji14 Mar 18, 2021
5239c7a
Update website links of PyGMT deps to use https and official sources
weiji14 Mar 20, 2021
5e903af
Add NEP29 policy language to installation page
weiji14 Mar 20, 2021
ed32c5b
Revert changes to line wrapping
weiji14 Mar 22, 2021
2510e56
Update MAINTENANCE.md to mention two Linux CI jobs are run in draft PRs
weiji14 Mar 23, 2021
7ee4b77
Merge branch 'master' into adopt_nep29
weiji14 Mar 23, 2021
5db0bfc
Place NEP29 dependencies policy in MAINTENANCE.md
weiji14 Mar 23, 2021
376b41d
List NumPy in compatibility version list on main README.rst page
weiji14 Mar 23, 2021
52fdcb7
Merge branch 'master' into adopt_nep29
weiji14 Mar 23, 2021
1016331
Remove hardcoded Python/NumPy versions in MAINTENANCE.md docs
weiji14 Mar 23, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 16 additions & 11 deletions .github/workflows/ci_tests.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -25,12 +25,12 @@ on:

jobs:
test:
name: ${{ matrix.os }} - Python ${{ matrix.python-version }}
name: ${{ matrix.os }} - Python ${{ matrix.python-version }} / NumPy ${{ matrix.numpy-version }}
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
python-version: [3.7, 3.8, 3.9]
python-version: [3.7, 3.9]
os: [ubuntu-latest, macOS-latest, windows-latest]
# Is it a draft Pull Request (true or false)?
isDraft:
Expand All @@ -41,20 +41,24 @@ jobs:
isDraft: true
- os: windows-latest
isDraft: true
- os: ubuntu-latest
python-version: 3.7
isDraft: true
- os: ubuntu-latest
python-version: 3.8
isDraft: true
# - os: ubuntu-latest
# python-version: 3.7
# isDraft: true
Comment on lines +44 to +46
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For some reason, these lines can't be uncommented so that only one workflow (Ubuntu - Py3.9) runs. It fails with Error when evaluating 'runs-on' for job 'test'. (Line: 29, Col: 14): Unexpected value '' (see e.g. https://github.com/GenericMappingTools/pygmt/actions/runs/663292971), though that Line 29 appears totally unrelated.

Suggested change
# - os: ubuntu-latest
# python-version: 3.7
# isDraft: true
# - os: ubuntu-latest
# python-version: 3.7
# isDraft: true

Might have something to do with the 'include:' lines below? But maybe that's ok, we could just run 2 Linux tests (Python 3.7/NumPy 1.17 and Python 3.9/NumPy 1.20) in draft mode (i.e. +1 extra test compared to #906).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tried to solve the problem but failed. Two Linux jobs are OK to me.

Copy link
Member Author

@weiji14 weiji14 Mar 23, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah it's a tricky problem 😅 I'll leave it as it is and revise the MAINTENANCE.md docs to mention that two jobs are run instead of just one. Edit: done at 2510e56

# Pair Python 3.7 with NumPy 1.17 and Python 3.9 with NumPy 1.20
include:
- python-version: 3.7
numpy-version: '1.17'
- python-version: 3.9
numpy-version: '1.20'
Comment on lines +47 to +52
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The NumPy versions are paired with Python versions here using 'environment variables' according to https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#using-environment-variables-in-a-matrix.

defaults:
run:
shell: bash -l {0}

# environmental variables used in coverage
# Environment variables used by codecov
env:
OS: ${{ matrix.os }}
PYTHON: ${{ matrix.python-version }}
NUMPY: ${{ matrix.numpy-version }}

steps:
# Cancel previous runs that are not completed
Expand Down Expand Up @@ -82,7 +86,8 @@ jobs:
# Install GMT and other required dependencies from conda-forge
- name: Install dependencies
run: |
conda install gmt=6.1.1 numpy pandas xarray netCDF4 packaging \
conda install gmt=6.1.1 numpy=${{ matrix.numpy-version }} \
pandas xarray netCDF4 packaging \
codecov coverage[toml] dvc ipython make \
pytest-cov pytest-mpl pytest>=6.0 \
sphinx-gallery
Expand Down Expand Up @@ -138,5 +143,5 @@ jobs:
uses: codecov/[email protected]
with:
file: ./coverage.xml # optional
env_vars: OS,PYTHON
env_vars: OS,PYTHON,NUMPY
fail_ci_if_error: false
22 changes: 20 additions & 2 deletions MAINTENANCE.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ If you want to make a contribution to the project, see the
* [Reviewing and Merging Pull Requests](#reviewing-and-merging-pull-requests)
* [Continuous Integration](#continuous-integration)
* [Continuous Documentation](#continuous-documentation)
* [Dependencies Policy](#dependencies-policy)
* [Making a Release](#making-a-release)
- [Updating the Changelog](#updating-the-changelog)
- [Check the README Syntax](#check-the-readme-syntax)
Expand Down Expand Up @@ -77,8 +78,9 @@ There are 9 configuration files located in `.github/workflows`:

This is run on every commit to the *master* and Pull Request branches.
It is also scheduled to run daily on the *master* branch.
In draft Pull Requests, only one job (Linux + Python latest)
is triggered to save on Continuous Integration resources.
In draft Pull Requests, only two jobs on Linux (minimum NEP29 Python/NumPy versions
and latest Python/NumPy versions) are triggered to save on Continuous Integration
resources.

3. `ci_docs.yml` (Build documentation on Linux/macOS/Windows)

Expand Down Expand Up @@ -135,6 +137,22 @@ The actual script `package.json` is used by Vercel to install the necessary pack
build the documentation, copy the files to a 'public' folder and deploy that to the web,
see https://vercel.com/docs/build-step.


## Dependencies Policy

PyGMT has adopted [NEP29](https://numpy.org/neps/nep-0029-deprecation_policy)
alongside the rest of the Scientific Python ecosystem, and therefore supports:

* All minor versions of Python released 42 months prior to the project,
and at minimum the two latest minor versions.
* All minor versions of NumPy released in the 24 months prior to the project,
and at minimum the last three minor versions.

In `setup.py`, the `python_requires` variable should be set to the minimum
supported version of Python. Minimum Python and NumPy version support should be
adjusted upward on every major and minor release, but never on a patch release.
Comment on lines +151 to +153
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't add the line "All supported minor versions of Python should be in the test matrix and have binary artifacts built for the release." as in https://numpy.org/neps/nep-0029-deprecation_policy#implementation because:

  1. We're not testing the middle Python version (Python 3.8)
  2. We're using no-arch builds on conda-forge and a single PyGMT Python 3 wheel on PyPI, instead of having specific binary artifacts for each Python version.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good to me.



## Making a Release

We try to automate the release process as much as possible.
Expand Down
13 changes: 7 additions & 6 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -229,14 +229,15 @@ Documentation for other versions
* `v0.1.0 <https://www.pygmt.org/v0.1.0>`__
* `v0.0.1a0 <https://www.pygmt.org/0.0.1a0>`__

Compatibility with Python and GMT versions
------------------------------------------
Compatibility with GMT and Python/NumPy versions
------------------------------------------------

======= ========== =========
PyGMT GMT Python
======= ========== =========
======= ========== ========= =========
PyGMT GMT Python NumPy
======= ========== ========= =========
0.4.x >=6.2.0 >=3.7 >=1.17.0
Comment on lines +232 to +238
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it a good idea to add this here now, or should we do it during the v0.4.0 release process instead?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should add it when we release v0.4.0, but I'm afraid we may forget it during the release process. I think it's fine to add this here.

0.3.x >=6.1.1 >=3.7
0.2.1 >=6.1.1 >=3.6
0.2.0 >=6.1.1 3.6 - 3.8
0.1.x >=6.0.0 3.6 - 3.8
======= ========== =========
======= ========== ========= =========
19 changes: 8 additions & 11 deletions doc/install.rst
Original file line number Diff line number Diff line change
Expand Up @@ -46,8 +46,7 @@ Start by looking at the tutorials on our sidebar, good luck!
Which Python?
-------------

PyGMT is tested to run on **Python 3.7 or greater**. Older Python versions may
work, but there is no guarantee that PyGMT will behave as expected.
PyGMT is tested to run on **Python 3.7 or greater**.

We recommend using the `Anaconda <https://www.anaconda.com/distribution>`__
Python distribution to ensure you have all dependencies installed and the
Expand Down Expand Up @@ -79,19 +78,17 @@ We recommend following the instructions further on to install GMT 6.
Dependencies
------------

PyGMT requires the following libraries:
PyGMT requires the following libraries to be installed:

* `numpy <http://www.numpy.org/>`__
* `pandas <https://pandas.pydata.org/>`__
* `xarray <http://xarray.pydata.org/>`__
* `netCDF4 <https://github.com/Unidata/netcdf4-python>`__
* `packaging <https://pypi.org/project/packaging/>`__
* `numpy <https://numpy.org>`__ (>= 1.17)
* `pandas <https://pandas.pydata.org>`__
* `xarray <https://xarray.pydata.org>`__
* `netCDF4 <https://unidata.github.io/netcdf4-python>`__
* `packaging <https://packaging.pypa.io>`__

The following are optional (but recommended) dependencies:

* `IPython <https://ipython.org/>`__: For embedding the figures in Jupyter
notebooks.

* `IPython <https://ipython.org>`__: For embedding the figures in Jupyter notebooks.

Installing GMT and other dependencies
-------------------------------------
Expand Down
2 changes: 1 addition & 1 deletion environment.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ dependencies:
# Required dependencies
- pip
- gmt=6.1.1
- numpy
- numpy>=1.17
- pandas
- xarray
- netCDF4
Expand Down
2 changes: 1 addition & 1 deletion requirements.txt
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Required packages
numpy
numpy>=1.17
pandas
xarray
netCDF4
Expand Down
6 changes: 4 additions & 2 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,10 +30,11 @@
"Programming Language :: Python :: 3.7",
"Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.9",
"License :: OSI Approved :: {}".format(LICENSE),
f"License :: OSI Approved :: {LICENSE}",
]
PLATFORMS = "Any"
INSTALL_REQUIRES = ["numpy", "pandas", "xarray", "netCDF4", "packaging"]
PYTHON_REQUIRES = ">=3.7"
INSTALL_REQUIRES = ["numpy>=1.17", "pandas", "xarray", "netCDF4", "packaging"]
# Configuration for setuptools-scm
SETUP_REQUIRES = ["setuptools_scm"]
USE_SCM_VERSION = {"local_scheme": "node-and-date", "fallback_version": "unknown"}
Expand All @@ -57,6 +58,7 @@
package_data=PACKAGE_DATA,
classifiers=CLASSIFIERS,
keywords=KEYWORDS,
python_requires=PYTHON_REQUIRES,
install_requires=INSTALL_REQUIRES,
setup_requires=SETUP_REQUIRES,
)