Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ruff pre-commit hook #1502

Merged
merged 7 commits into from
Jan 18, 2024

Conversation

Skylion007
Copy link
Contributor

@Skylion007 Skylion007 commented Jan 15, 2024

Closes #1501

I mostly only enabled rules that had autofixes available (and were therefore easy to enable, did not change large portions of the codebase, and are rather uncontroversial). Many of these changes are semantically identical but run more efficiently in Python (set generator -> set comprehension). Other changes make the typing more correct with the flake8-PYI linter and so on.

Good rules to enable in future PRs:

  • Flake8-bugbear
  • Flake8-pytest
  • pyupgrade (must set minimum supported Python version)

@martindurant
Copy link
Member

@Skylion007
Copy link
Contributor Author

@martindurant Nope, I was just conservative with it so it exactly match flake8. I am about to add a bunch more rules and changes in a few seconds.

@Skylion007
Copy link
Contributor Author

@martindurant Okay, I enabled a bunch of new rules and added a ton of fixes.

@Skylion007 Skylion007 force-pushed the skylion007/add-ruff-precommit branch 4 times, most recently from 4e29445 to 410891c Compare January 15, 2024 20:35
@Skylion007
Copy link
Contributor Author

This PR also only enables the linter, not the formatter.

@Skylion007 Skylion007 force-pushed the skylion007/add-ruff-precommit branch from 410891c to a0d0115 Compare January 15, 2024 20:37
Copy link
Member

@martindurant martindurant left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some questions - maybe you know why ruff is preferring to make these changes

@@ -19,7 +15,7 @@ class AbstractCacheMapper(abc.ABC):
def __call__(self, path: str) -> str:
...

def __eq__(self, other: Any) -> bool:
def __eq__(self, other: object) -> bool:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So lon we've been told to the the opposite, no?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://docs.astral.sh/ruff/rules/any-eq-ne-annotation/ Happy to revert this rule too and disable if necessary, just let me know.

@@ -251,7 +251,7 @@ async def _get_file(
if isfilelike(lpath):
outfile = lpath
else:
outfile = open(lpath, "wb")
outfile = open(lpath, "wb") # noqa: ASYNC101
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What would be the problem here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://docs.astral.sh/ruff/rules/open-sleep-or-subprocess-in-async-function/ Not sure there is a good way around it without relying the aiofiles dependency though so I just suppressed it

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What a strange rule! Just about any normal function call blocks for some time, and open() ought to be fast. I don't think there's a big reason to switch to aiofiles right now, maybe later :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I just left it because it seemed useful and we can noqa the nonsense suggestions like this. 👍

fsspec/implementations/tests/test_cached.py Show resolved Hide resolved
fsspec/implementations/tests/test_tar.py Outdated Show resolved Hide resolved
fsspec/implementations/tests/test_zip.py Outdated Show resolved Hide resolved
@@ -123,6 +123,7 @@ def test_chmod(mount_local):
["cp", str(mount_dir / "text"), str(mount_dir / "new")],
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
check=False,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why the extra arg

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://docs.astral.sh/ruff/rules/subprocess-run-without-check/ Makes you think about whether you should enable to reraise an exception if the process fails (instead of having it fail silently). Happy to disable that if you prefer though.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose check=True is reasonable, then. In a test like this, we would otherwise have the failure a couple of lines later. I'm not really sure why we're using subprocess at all in this case.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@martindurant Check=True made the test fail so I am just leaving it as is for now.

@Skylion007
Copy link
Contributor Author

@martindurant the one failure looks like a flake.

@martindurant
Copy link
Member

I have yet to determine why SMB fails to start sometimes

@martindurant
Copy link
Member

@Skylion007 - merging this now. I may ping you if future PRs show odd ruff failures I don't understand :).

@martindurant martindurant merged commit ec13d27 into fsspec:master Jan 18, 2024
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add ruff linter as pre-commit hook
2 participants