Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adopt ruff as the central tool for linting, formatting, and import sorting #702

Merged
merged 12 commits into from
Oct 3, 2024
Merged
32 changes: 13 additions & 19 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,33 +4,27 @@ fail_fast: true

repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.4.0
rev: v4.6.0
hooks:
- id: trailing-whitespace
- id: end-of-file-fixer
- id: check-yaml

- repo: https://github.com/psf/black
rev: 23.3.0
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.6.8
hooks:
- id: black

- repo: https://github.com/timothycrosley/isort
rev: 5.12.0
hooks:
- id: isort

# Need to use flake8 GitHub mirror due to CentOS git issue with GitLab
# https://github.com/pre-commit/pre-commit/issues/1206
- repo: https://github.com/pycqa/flake8
rev: 6.0.0
hooks:
- id: flake8
args: ["--config=setup.cfg"]
additional_dependencies: [flake8-isort]
# Sort the imports
- id: ruff
name: ruff-sort-imports
args: [--select, I, --fix]
# Run the linter.
- id: ruff
args: [--fix]
# Run the formatter.
- id: ruff-format

- repo: https://github.com/pre-commit/mirrors-mypy
rev: v1.4.0
rev: v1.11.2
hooks:
- id: mypy
args: ["--config=pyproject.toml"]
Expand Down
9 changes: 2 additions & 7 deletions .vscode/xcdat.code-workspace
Original file line number Diff line number Diff line change
Expand Up @@ -20,17 +20,12 @@
// ===================
"[python]": {
// editor.rulers: [comments, max line length, wrap line length],
// Black does not wrap comments.
"editor.rulers": [80, 88, 120],
"editor.wordWrap": "wordWrapColumn",
"editor.wordWrapColumn": 120,
"editor.defaultFormatter": "ms-python.black-formatter"
"editor.defaultFormatter": "charliermarsh.ruff"
},
"black-formatter.importStrategy": "fromEnvironment",
// Code Formatting and Linting
// ---------------------------
"flake8.args": ["--config=setup.cfg"],
"flake8.importStrategy": "fromEnvironment",
"ruff.importStrategy": "fromEnvironment",
// Type checking
// ---------------------------
"mypy-type-checker.args": ["--config=pyproject.toml"],
Expand Down
14 changes: 5 additions & 9 deletions CONTRIBUTING.rst
Original file line number Diff line number Diff line change
Expand Up @@ -246,7 +246,7 @@ Here's a simple checklist for PRs:
- Write new tests if needed.
- Test the code using `Pytest`_. Running all tests (type ``make test`` or ``pytest`` in the root directory) takes a while, so feel free to only run the tests you think are needed based on your PR (example: ``pytest tests/test_dataset.py``). CI will catch any failing tests.

- **Properly format your code** and verify that it passes the formatting guidelines set by `Black`_ and `Flake8`_.
- **Properly format your code** and verify that it passes the formatting guidelines set by `Ruff`_.
You can use `pre-commit`_ to run these automatically on each commit.

- Run ``pre-commit run --all-files`` in the root directory. This may modify some files. Confirm and commit any formatting changes.
Expand All @@ -261,10 +261,8 @@ Code formatting

xCDAT uses several tools to ensure a consistent code format throughout the project:

- `Black`_ for standardized code formatting
- `Flake8`_ for code linting
- `isort`_ for standardized order of imports
- `mypy`_ for static type checking on `type hints`_
- `Ruff`_ for standardized code formatting, linting, and ordering of imports.
- `MyPy`_ for static type checking on `type hints`_

We highly recommend that you setup `pre-commit hooks`_ to automatically run all the
above tools every time you make a git commit. Check out the :ref:`install pre-commit
Expand All @@ -277,10 +275,8 @@ hooks` section above for instructions.
.. _pre-commit: https://pre-commit.com/
.. _pre-commit hooks: https://pre-commit.com/
.. _Pytest: http://doc.pytest.org/en/latest/
.. _Black: https://black.readthedocs.io/en/stable/
.. _Flake8: https://flake8.pycqa.org/en/latest/
.. _isort: https://pycqa.github.io/isort/
.. _mypy: http://mypy-lang.org/
.. _Ruff: https://docs.astral.sh/ruff/
.. _MyPy: http://MyPy-lang.org/
.. _type hints: https://docs.python.org/3/library/typing.html

Testing with continuous integration
Expand Down
8 changes: 6 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -60,8 +60,12 @@ clean-test: ## remove test and coverage artifacts
pre-commit: # run pre-commit quality assurance checks
pre-commit run --all-files

lint: ## check style with flake8
flake8 xcdat tests
lint: ## check style ruff
ruff check --select I --fix
ruff check --fix

format: ## format code using ruff
ruff format

test: ## run tests quickly with the default Python and produces code coverage report
pytest
Expand Down
8 changes: 3 additions & 5 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
+--------------------+------------------------------------------------------+
| DevOps | |CI/CD Build Workflow| |codecov| |docs| |
+--------------------+------------------------------------------------------+
| Quality Assurance | |pre-commit| |black| |flake8| |mypy| |
| Quality Assurance | |pre-commit| |ruff| |mypy| |
+--------------------+------------------------------------------------------+

.. raw:: html
Expand All @@ -46,10 +46,8 @@
:target: https://codecov.io/gh/xCDAT/xcdat
.. |pre-commit| image:: https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit&logoColor=white
:target: https://github.com/pre-commit/pre-commit
.. |black| image:: https://img.shields.io/badge/code%20style-black-000000.svg
:target: https://github.com/psf/black
.. |flake8| image:: https://img.shields.io/badge/flake8-enabled-green
:target: https://github.com/PyCQA/flake8
.. |ruff| image:: https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json
:target: https://github.com/astral-sh/ruff
.. |mypy| image:: http://www.mypy-lang.org/static/mypy_badge.svg
:target: http://mypy-lang.org/

Expand Down
7 changes: 2 additions & 5 deletions conda-env/dev.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,12 +36,9 @@ dependencies:
# ==================
- types-python-dateutil
# NOTE: If the tools below are updated, also update their 'rev' in `.pre-commit.config.yaml`
- black=23.3.0
- flake8=6.0.0
- flake8-isort=6.0.0
- isort=5.12.0
- mypy=1.4.0
- pre-commit=3.2.0
- ruff=0.6.8
- mypy=1.11.2
# Testing
# ==================
- pytest
Expand Down
81 changes: 51 additions & 30 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,34 +1,55 @@
[tool.black]
line-length = 88
target-version = ['py36']
include = '\.pyi?$'
exclude = '''
(
/(
\.eggs
| \.git
| \.hg
| \.mypy_cache
| \.tox
| \.idea
| venv
| _build
| buck-out
| build
| dist
| docs
| config
)/
)
'''
[tool.ruff]
# Exclude a variety of commonly ignored directories.
exclude = [
".bzr",
".direnv",
".eggs",
".git",
".git-rewrite",
".hg",
".ipynb_checkpoints",
".mypy_cache",
".nox",
".pants.d",
".pyenv",
".pytest_cache",
".pytype",
".ruff_cache",
".svn",
".tox",
".venv",
".vscode",
"__pypackages__",
"_build",
"buck-out",
"build",
"dist",
"docs",
"node_modules",
"site-packages",
"venv",
]

[tool.isort]
# Docs: https://pycqa.github.io/isort/docs/configuration/options.html#example-pyprojecttoml_4
multi_line_output = 3
include_trailing_comma = true
force_grid_wrap = 0
use_parentheses = true
line_length = 88
[tool.ruff.lint]
# E4 - whitespace
# E7 - multiple-statements
# E9 - trailing-whitespace
# F - Enable Pyflakes
# B - Enable flake8-bugbear
# W - Enable pycodestyle
# C901 - complex-structure
# D - Enable flake8-docstrings
select = ["E4", "E7", "E9", "F", "B", "W", "C901"]

# E501 - line-too-long
ignore = ["E501"]

[tool.ruff.lint.mccabe]
# Flag errors (`C901`) whenever the complexity level exceeds 5.
max-complexity = 10

[tool.ruff.lint.pydocstyle]
convention = "numpy"

[tool.pytest.ini_options]
# Docs: https://docs.pytest.org/en/7.2.x/reference/customize.html#configuration
Expand Down
29 changes: 0 additions & 29 deletions setup.cfg

This file was deleted.

1 change: 1 addition & 0 deletions tests/__init__.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
"""Unit test package for xcdat."""

from xarray.core.options import set_options
from xarray.tests import requires_dask # noqa: F401

Expand Down
1 change: 1 addition & 0 deletions tests/fixtures.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
"""This module stores reusable test fixtures."""

from typing import Literal

import cftime
Expand Down
2 changes: 1 addition & 1 deletion tests/test_bounds.py
Original file line number Diff line number Diff line change
Expand Up @@ -847,7 +847,7 @@ def test_add_monthly_bounds_for_end_of_month_set_to_true(self):
# Get time axis and create new axis with time set to first day of month.
time = ds_with_bnds.time
new_time = []
for i, t in enumerate(time.values):
for _, t in enumerate(time.values):
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

y = t.year
m = t.month
nt = cftime.DatetimeGregorian(y, m, 1, 0)
Expand Down
13 changes: 9 additions & 4 deletions tests/test_regrid.py
Original file line number Diff line number Diff line change
Expand Up @@ -204,7 +204,12 @@ def test_methods(self):
xgcm.XGCMRegridder(self.ds, self.output_grid, method="linear", target_data=None)

with pytest.raises(ValueError, match="'dummy' is invalid, possible choices"):
xgcm.XGCMRegridder(self.ds, self.output_grid, method="dummy", target_data=None) # type: ignore
xgcm.XGCMRegridder(
self.ds,
self.output_grid,
method="dummy", # type: ignore
target_data=None,
)

def test_missing_input_z_coord(self):
ds = fixtures.generate_dataset(
Expand Down Expand Up @@ -663,7 +668,7 @@ def test_map_latitude_coarse_to_fine(self):
np.testing.assert_allclose(x, y)

for x2, y2 in zip(weights, expected_weigths):
np.testing.assert_allclose(x, y)
np.testing.assert_allclose(x2, y2)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


def test_map_latitude_fine_to_coarse(self):
mapping, weights = regrid2._map_latitude(
Expand Down Expand Up @@ -1254,7 +1259,7 @@ def test_grid(self):
ValueError,
match=r".*lon\d?.*lon\d?.*",
):
ds_multi.regridder.grid
ds_multi.regridder.grid # noqa: B018
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


def test_grid_raises_error_when_dataset_has_multiple_dims_for_an_axis(self):
ds_bounds = fixtures.generate_dataset(
Expand All @@ -1265,7 +1270,7 @@ def test_grid_raises_error_when_dataset_has_multiple_dims_for_an_axis(self):
)

with pytest.raises(ValueError):
ds_bounds.regridder.grid
ds_bounds.regridder.grid # noqa: B018

@mock.patch("xcdat.regridder.accessor._get_input_grid")
def test_horizontal_tool_check(self, _get_input_grid):
Expand Down
1 change: 1 addition & 0 deletions xcdat/__init__.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
"""Top-level package for xcdat."""

from xcdat.axis import ( # noqa: F401
center_times,
get_dim_coords,
Expand Down
1 change: 1 addition & 0 deletions xcdat/_logger.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
"""Logger module for setting up a logger."""

import logging
import logging.handlers

Expand Down
17 changes: 12 additions & 5 deletions xcdat/bounds.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,11 @@
"""Bounds module for functions related to coordinate bounds."""

from __future__ import annotations

import collections
import datetime
import warnings
from typing import Dict, List, Literal, Optional, Union
from typing import Dict, List, Literal, Optional, Tuple, Union

import cf_xarray as cfxr # noqa: F401
import cftime
Expand Down Expand Up @@ -124,7 +127,7 @@ def keys(self) -> List[str]:
)

def add_missing_bounds( # noqa: C901
self, axes: List[CFAxisKey] = ["X", "Y", "T"]
self, axes: List[CFAxisKey] | Tuple[CFAxisKey, ...] = ("X", "Y", "T")
) -> xr.Dataset:
Comment on lines 129 to 131
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"""Adds missing coordinate bounds for supported axes in the Dataset.

Expand Down Expand Up @@ -153,9 +156,9 @@ def add_missing_bounds( # noqa: C901

Parameters
----------
axes : List[str]
axes : List[CFAxesKey] | Tuple[CFAxisKey, ...]
List of CF axes that function should operate on, by default
["X", "Y", "T"]. Options include "X", "Y", "T", or "Z".
("X", "Y", "T"). Options include "X", "Y", "T", or "Z".

Returns
-------
Expand Down Expand Up @@ -614,7 +617,11 @@ def _create_time_bounds( # noqa: C901
hrs = diff.seconds / 3600
daily_subfreq = int(24 / hrs) # type: ignore

time_bnds = self._create_daily_time_bounds(timesteps, obj_type, freq=daily_subfreq) # type: ignore
time_bnds = self._create_daily_time_bounds(
timesteps,
obj_type,
freq=daily_subfreq, # type: ignore
)

# Create the bounds data array
da_time_bnds = xr.DataArray(
Expand Down
Loading
Loading