Skip to content

Commit

Permalink
Use ruff instead of black and flake8
Browse files Browse the repository at this point in the history
  • Loading branch information
timothy-nunn committed Jan 8, 2025
1 parent 7844aa2 commit de73c85
Show file tree
Hide file tree
Showing 5 changed files with 64 additions and 77 deletions.
15 changes: 0 additions & 15 deletions .flake8

This file was deleted.

12 changes: 5 additions & 7 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,14 +9,12 @@ repos:
- id: check-docstring-first
- id: check-merge-conflict
- id: debug-statements
- repo: https://github.com/psf/black
rev: 22.3.0
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.8.0
hooks:
- id: black
- repo: https://github.com/PyCQA/flake8
rev: 4.0.1
hooks:
- id: flake8
- id: ruff
args: [--fix]
- id: ruff-format
- repo: https://github.com/jumanjihouse/pre-commit-hook-yamlfmt
rev: 0.2.3
hooks:
Expand Down
16 changes: 7 additions & 9 deletions documentation/proc-pages/development/ci-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,11 @@ Our GitHub actions Continuous Integration (CI) pipeline serves to ensure each br

| Name | Functionality |
| ---- | ------------- |
| docker | Checks if the `process-ci` Docker container is up-to-date and builds it if not. Only runs on the **main** branch. |
| make-py38 | Builds and archives the PROCESS build artefacts |
| unit-py38 | Installs PROCESS and runs the unit tests. The job will fail if any of the unit tests fail. |
| integration-py38 | Installs PROCESS and runs the integration tests. The job will fail if any of the integration tests fail. |
| regression-py38 | Installs PROCESS and runs the regression tests with a 0% and 5% tolerance, respectively. The job will fail if any of the regression tests fail. |
| large-tokamak-py38 | Installs PROCESS and runs the `large-tokamak` input file, archiving the output MFILE. Only runs on the **main** branch. |
| flake8 | Runs the flake8 Python linter and fails if any lint errors occur. |
| black | Runs the black Python formatter and fails if any formatting issues are detected. |
| tracking | Collects MFILEs for input files of interest and creates a dashboard of changes in key values over time (one datapoint for each commit on main). |
| make | Builds and archives the PROCESS build artefacts |
| unit-test | Installs PROCESS and runs the unit tests. The job will fail if any of the unit tests fail. |
| integration-test | Installs PROCESS and runs the integration tests. The job will fail if any of the integration tests fail. |
| regression-test | Installs PROCESS and runs the regression tests with a 0% and 5% tolerance, respectively. The job will fail if any of the regression tests fail. |
| run-tracking-inputs | Installs PROCESS and runs the regression test input files, archiving the output MFILEs. Only runs on the **main** branch. |
| tracking | Collects MFILEs for input files of interest and creates a dashboard of changes in key values over time (one datapoint for each commit on main). Only runs on the **main** branch. |
| pre-commit-quality-check | ensures the pushed code meets our standards as defined in `.pre-commit-config.yaml`. |
| docs | Builds and deploys the documentation onto GitHub pages. |
67 changes: 21 additions & 46 deletions documentation/proc-pages/development/pre-commit.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@ the commit will not be made. On a failure, one of two things can happen:
1. Pre-commit plugins will rectify the mistakes made. This will happen with code formatters
(whose job it is to edit your files to the correct style). The files the plugins have changed
can then be `git add`ed again and the `git commit` command re-issued.
2. A pre-commit plugin will identify the mistake but will NOT fix it. This will happen with
`flake8` which is a linter and warns of code mistakes but does not correct them. You will need
2. A pre-commit plugin will identify the mistake but will NOT fix it. This could happen with
`ruff` which is a linter and warns of code mistakes but does not correct them. You will need
to fix these mistakes manually. Now, the changed files can be `git add`ed and the `git commit` command re-issued.

!!! Info "VSCode GUI users"
Expand Down Expand Up @@ -58,63 +58,44 @@ fixed and you will need to re-add the files that pre-commit has changed.

!!! example "Adding two files"
Consider that two files are being `git add`ed.
One of the files, `foo.py` has stylistic changes which **Black** objects to.
One of the files, `foo.py` has stylistic changes which **ruff** objects to.

```
> git add foo.py bar.py
> git commit -m "Adding foo and bar"
Trim Trailing Whitespace.................................................Passed
Check for merge conflicts................................................Passed
Debug Statements (Python)................................................Passed
black....................................................................Failed
- hook id: black
ruff.....................................................................Failed
- hook id: ruff-format
- exit code: 1
- files were modified by this hook

Fixing foo.py
Format YAML files....................................(no files to check)Skipped

> git add foo.py # since black has modified foo.py
> git add foo.py # since rufd has modified foo.py
> git commit -m "Adding foo and bar"
```

To avoid the need to re-add files a second time you could run `black .` which will do the
formatting (of Python code) that pre-commit would do.

!!! example "black won't fix all flake8 issues"
Flake8 (as has been stressed on this documentation) is a linter and not a formatter. This means
flake8 will never make any changes to your Python source code.

Consider the following file very simple file, `example.py`:

```python
from process.fortran import tfcoil_variables, fwbs_variables

def get_whttf():
return tfcoil_variables.whttf
```

Flake8 will return the following trace for this file:
```
example.py:1:47 F401 Module fwbs_variables imported but never used
```
because `fwbs_variables` is imported, but never used. However, this is not a style issue, it is
a semantic issue. Therefore, `black` will not fix this issue. It is up to the developer to
rectify this issue manually, `git add` the fixed file, and finally re-do the `git commit` command.

## Pre-commit and the `quality` CI stage
The Process continuous integration system (used to check Process builds and passes tests) also has
a `quality` stage. This is where several jobs will run to ensure the quality of your code. If all
your commits pass through pre-commit, then these jobs should not fail as your code will be of a high quality.

## Using black with VSCode
Although not required, the `black` VSCode extension will ensure that all the Python files you save
will be black-compliant and, as such, won't need to modified by pre-commit.

In VSCode, use `Ctrl+,` (`Command+,` for Mac users) to open the settings page. From here, use the
search bar to find **"Editor: Format On Save"** and tick the box. Next, search for
**"Python > Formatting: Provider"** and set it to `black`. Now, upon saving a Python file, the
black formatter will run over it.
## Using ruff with VSCode
Although not required, the `ruff` VSCode extension will ensure that all the Python files you save
will be ruff-compliant and, as such, won't need to modified by pre-commit.

Open or create the file `.vscode/settings.sh` and add/modify the following settings:
```json
{
"[python]": {
"editor.defaultFormatter": "charliermarsh.ruff",
"editor.formatOnSave": true
}
}
```

## What does pre-commit check for?

Expand All @@ -131,13 +112,7 @@ Pre-commit performs a few checks on each and every file you add, regardless of t

Because Process is becoming increasingly Pythonised, pre-commit performs many Python style checks.

* [`black`](https://black.readthedocs.io/en/stable/) has already been discussed on this page. It is
an industry-standard Python code formatter that enforces a "one-way is right" style. This ensures
all of our Python is of the same, black-correct, style. **This plugin will automatically fix any mistakes it finds**.
* [`flake8`](https://flake8.pycqa.org/en/latest/) is the linter of choice on Process. A linter
checks common errors in code. Flake8, for instance, can check for `import` statements that are
unused or variables that are declared but never used. Black, a formatter, will not remove these
"mistakes" as it will never change the semantic meaning of code. **This plugin will NOT automatically fix any mistakes it finds**.
* [`ruff`](https://github.com/astral-sh/ruff) formats and lints code. It will identify formatting and stylistic errors in Python code. **This plugin will automatically fix any mistakes it finds**.
* `check-docstring-first` will check that a function/class docstring comes before any of the body
(ie the docstring must be at the top). **This plugin will NOT automatically fix any mistakes it finds**.
* `debug-statements` will check that debug statements (those using the built-in `pdb` module)
Expand All @@ -146,4 +121,4 @@ Because Process is becoming increasingly Pythonised, pre-commit performs many Py
### Other checks

* [`yamlfmt`](https://github.com/jumanjihouse/pre-commit-hook-yamlfmt) formats YAML code (similar
to what `black` does for Python code). **This plugin will automatically fix any mistakes it finds**.
to what `ruff` does for Python code). **This plugin will automatically fix any mistakes it finds**.
31 changes: 31 additions & 0 deletions ruff.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
extend-exclude = ["env", ".env"]
target-version = "py310"

[lint]
extend-select = [
"I",
"INT",
"YTT",
"ASYNC",
"COM",
"T10",
"FA",
"LOG",
"PYI",
"Q",
"RSE",
"SLOT",
"TID",
"PGH",
"FLY",
"F",
"W",
]

ignore = ["COM812", "FBT", "G004"]

[lint.per-file-ignores]
"tests/*" = ["ARG"]

[format]
preview = true

0 comments on commit de73c85

Please sign in to comment.