Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replacing isort with Ruff #10912

Merged
merged 10 commits into from
Oct 28, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 0 additions & 5 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,6 @@ repos:
hooks:
- id: black
language_version: python3.10
- repo: https://github.com/pycqa/isort
rev: 5.12.0 # must match requirements-tests.txt
hooks:
- id: isort
name: isort (python)
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.1.0 # must match requirements-tests.txt and tests.yml
hooks:
Expand Down
9 changes: 5 additions & 4 deletions .vscode/extensions.json
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@
"editorconfig.editorconfig",
"ms-python.black-formatter",
"ms-python.flake8",
"ms-python.isort",
"ms-python.mypy-type-checker",
"ms-python.python",
"ms-python.vscode-pylance",
Expand All @@ -16,17 +15,19 @@
],
"unwantedRecommendations": [
/*
* Don't recommend by default for this workspace
* Don't recommend by default for this workspace
*/
"christian-kohler.npm-intellisense",
/*
* Must disable in this workspace
* https://github.com/microsoft/vscode/issues/40239
* Must disable in this workspace
* https://github.com/microsoft/vscode/issues/40239
*/
// even-better-toml has format on save
"bungcip.better-toml",
// Don't use two mypy extensions simultaneously
"matangover.mypy",
// Use Ruff instead
"ms-python.isort",
// We use Black
"ms-python.autopep8",
// Not using pylint
Expand Down
8 changes: 3 additions & 5 deletions .vscode/settings.default.json
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,6 @@
"python.linting.prospectorEnabled": false,
"python.linting.pylamaEnabled": false,
"python.linting.pylintEnabled": false,
// Not using bandit
"python.linting.banditEnabled": false,
// python.analysis is Pylance (pyright) configurations
"python.analysis.fixAll": [
Expand Down Expand Up @@ -109,13 +108,12 @@
"--config=.flake8"
],
"flake8.importStrategy": "fromEnvironment",
"isort.check": true,
"isort.importStrategy": "fromEnvironment",
"black-formatter.importStrategy": "fromEnvironment",
// Using Ruff instead of isort
"isort.check": false,
"ruff.importStrategy": "fromEnvironment",
"ruff.fixAll": true,
// Conflict between Ruff and isort
"ruff.organizeImports": false,
"ruff.organizeImports": true,
"evenBetterToml.formatter.alignComments": false,
"evenBetterToml.formatter.alignEntries": false,
"evenBetterToml.formatter.allowedBlankLines": 1,
Expand Down
9 changes: 3 additions & 6 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,8 +30,7 @@ Typeshed runs continuous integration (CI) on all pull requests. This means that
if you file a pull request (PR), our full test suite
-- including our linter, [Flake8](https://github.com/PyCQA/flake8) --
is run on your PR. It also means that bots will automatically apply
changes to your PR (using [Black](https://github.com/psf/black),
[isort](https://github.com/PyCQA/isort) and
changes to your PR (using [Black](https://github.com/psf/black) and
[Ruff](https://github.com/astral-sh/ruff)) to fix any formatting issues.
This frees you up to ignore all local setup on your side, focus on the
code and rely on the CI to fix everything, or point you to the places that
Expand Down Expand Up @@ -87,8 +86,7 @@ terminal to install all non-pytype requirements:

## Code formatting

The code is formatted using [`Black`](https://github.com/psf/black)
and [`isort`](https://github.com/PyCQA/isort).
The code is formatted using [`Black`](https://github.com/psf/black).
Various other autofixes are
also performed by [`Ruff`](https://github.com/astral-sh/ruff).

Expand All @@ -101,8 +99,7 @@ That being said, if you *want* to run the checks locally when you commit,
you're free to do so. Either run the following manually...

```bash
(.venv)$ isort .
(.venv)$ ruff .
(.venv)$ ruff check .
(.venv)$ black .
```

Expand Down
21 changes: 9 additions & 12 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -7,15 +7,13 @@ skip_magic_trailing_comma = true
# for just these files, but doesn't seem possible yet.
force-exclude = ".*_pb2.pyi"

[tool.isort]
profile = "black"
combine_as_imports = true
line_length = 130
skip = [".git", ".github", ".venv"]
extra_standard_library = [
[tool.ruff.isort]
split-on-trailing-comma = false
combine-as-imports = true
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I recommend also split-on-trailing-comma = false, for closer isort-like behavior.

Otherwise Ruff doesn't collapse multiline imports (even if they do fit on one line).

Copy link
Collaborator Author

@Avasam Avasam Oct 19, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Regardless of intentions for the default value, I'm down to keep imports on a single line when possible anyway. Especially since we have skip_magic_trailing_comma = true configured for Black

Thanks for letting me know of this difference.

extra-standard-library = [
"_typeshed",
"typing_extensions",
# Extra modules not recognized by isort
# Extra modules not recognized by Ruff/isort
"_ast",
"_bisect",
"_bootlocale",
Expand Down Expand Up @@ -56,7 +54,7 @@ extra_standard_library = [
"opcode",
"pyexpat",
]
known_first_party = ["parse_metadata", "utils"]
known-first-party = ["parse_metadata", "utils"]

[tool.ruff]
line-length = 130
Expand All @@ -77,12 +75,11 @@ exclude = [
".venv",
"env",
]

# Only enable rules that have safe autofixes;
# only enable rules that are relevant to stubs
select = [
"F401", # Remove unused imports
"FA", # flake8-future-annotations
"I", # isort
# Only enable rules that have safe autofixes:
"F401", # Remove unused imports
"PYI009", # use `...`, not `pass`, in empty class bodies
"PYI010", # function bodies must be empty
"PYI012", # class bodies must not contain `pass`
Expand Down
1 change: 0 additions & 1 deletion requirements-tests.txt
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@ flake8==6.1.0 # must match .pre-commit-config.yaml
flake8-bugbear==23.9.16 # must match .pre-commit-config.yaml
flake8-noqa==1.3.2 # must match .pre-commit-config.yaml
flake8-pyi==23.10.0 # must match .pre-commit-config.yaml
isort==5.12.0 # must match .pre-commit-config.yaml
mypy==1.6.1
pre-commit-hooks==4.5.0 # must match .pre-commit-config.yaml
pytype==2023.10.17; platform_system != "Windows" and python_version < "3.12"
Expand Down
12 changes: 3 additions & 9 deletions scripts/create_baseline_stubs.py
Original file line number Diff line number Diff line change
Expand Up @@ -65,14 +65,9 @@ def run_black(stub_dir: str) -> None:
subprocess.run(["black", stub_dir])


def run_isort(stub_dir: str) -> None:
print(f"Running isort: isort {stub_dir}")
subprocess.run([sys.executable, "-m", "isort", stub_dir])


def run_ruff(stub_dir: str) -> None:
print(f"Running Ruff: ruff {stub_dir}")
subprocess.run([sys.executable, "-m", "ruff", stub_dir])
print(f"Running Ruff: ruff check {stub_dir} --fix-only")
subprocess.run([sys.executable, "-m", "ruff", "check", stub_dir, "--fix-only"])


async def get_project_urls_from_pypi(project: str, session: aiohttp.ClientSession) -> dict[str, str]:
Expand Down Expand Up @@ -189,7 +184,7 @@ def add_pyright_exclusion(stub_dir: str) -> None:
def main() -> None:
parser = argparse.ArgumentParser(
description="""Generate baseline stubs automatically for an installed pip package
using stubgen. Also run Black, isort and Ruff. If the name of
using stubgen. Also run Black and Ruff. If the name of
the project is different from the runtime Python package name, you may
need to use --package (example: --package yaml PyYAML)."""
)
Expand Down Expand Up @@ -239,7 +234,6 @@ def main() -> None:
run_stubdefaulter(stub_dir)

run_ruff(stub_dir)
run_isort(stub_dir)
run_black(stub_dir)

create_metadata(project, stub_dir, version)
Expand Down
4 changes: 2 additions & 2 deletions scripts/generate_proto_stubs.sh
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ PYTHON_PROTOBUF_DIR="protobuf-$PYTHON_PROTOBUF_VERSION"
VENV=venv
python3 -m venv "$VENV"
source "$VENV/bin/activate"
pip install -r "$REPO_ROOT/requirements-tests.txt" # for Black and isort
pip install -r "$REPO_ROOT/requirements-tests.txt" # for Black and Ruff

# Install mypy-protobuf
pip install mypy-protobuf=="$MYPY_PROTOBUF_VERSION"
Expand Down Expand Up @@ -73,7 +73,7 @@ PROTO_FILES=$(grep "GenProto.*google" $PYTHON_PROTOBUF_DIR/python/setup.py | \
# shellcheck disable=SC2086
protoc_install/bin/protoc --proto_path="$PYTHON_PROTOBUF_DIR/src" --mypy_out="relax_strict_optional_primitives:$REPO_ROOT/stubs/protobuf" $PROTO_FILES

isort "$REPO_ROOT/stubs/protobuf"
ruff check "$REPO_ROOT/stubs/protobuf" --fix-only
black "$REPO_ROOT/stubs/protobuf"

sed --in-place="" \
Expand Down
4 changes: 1 addition & 3 deletions scripts/runtests.py
Original file line number Diff line number Diff line change
Expand Up @@ -83,9 +83,7 @@ def main() -> None:

# Run formatters first. Order matters.
print("\nRunning Ruff...")
subprocess.run([sys.executable, "-m", "ruff", path])
print("\nRunning isort...")
subprocess.run([sys.executable, "-m", "isort", path])
subprocess.run([sys.executable, "-m", "ruff", "check", path])
print("\nRunning Black...")
black_result = subprocess.run([sys.executable, "-m", "black", path])
if black_result.returncode == 123:
Expand Down
2 changes: 1 addition & 1 deletion scripts/sync_tensorflow_protobuf_stubs.sh
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ rm tensorflow/compiler/xla/service/hlo_execution_profile_data_pb2.pyi \
tensorflow/core/protobuf/worker_service_pb2.pyi \
tensorflow/core/util/example_proto_fast_parsing_test_pb2.pyi

isort "$REPO_ROOT/stubs/tensorflow/tensorflow"
ruff check "$REPO_ROOT/stubs/tensorflow/tensorflow" --fix-only
black "$REPO_ROOT/stubs/tensorflow/tensorflow"

sed --in-place="" \
Expand Down
2 changes: 1 addition & 1 deletion stdlib/csv.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ from _csv import (
)

if sys.version_info >= (3, 12):
from _csv import QUOTE_STRINGS as QUOTE_STRINGS, QUOTE_NOTNULL as QUOTE_NOTNULL
from _csv import QUOTE_NOTNULL as QUOTE_NOTNULL, QUOTE_STRINGS as QUOTE_STRINGS
from _typeshed import SupportsWrite
from collections.abc import Collection, Iterable, Iterator, Mapping, Sequence
from typing import Any, Generic, TypeVar, overload
Expand Down
2 changes: 1 addition & 1 deletion tests/check_consistent.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@

# These type checkers and linters must have exact versions in the requirements file to ensure
# consistent CI runs.
linters = {"black", "flake8", "flake8-bugbear", "flake8-noqa", "flake8-pyi", "isort", "ruff", "mypy", "pytype"}
linters = {"black", "flake8", "flake8-bugbear", "flake8-noqa", "flake8-pyi", "ruff", "mypy", "pytype"}


def assert_consistent_filetypes(
Expand Down