Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[flake8_comprehensions] add sum/min/max to unnecessary comprehension check (C419) #10759

Merged
merged 8 commits into from
Apr 3, 2024

Conversation

carljm
Copy link
Contributor

@carljm carljm commented Apr 3, 2024

Fixes #3259

Summary

Renames UnnecessaryComprehensionAnyAll to UnnecessaryComprehensionInCall and extends the check to sum, min, and max, in addition to any and all.

Test Plan

Updated snapshot test.

Built docs locally and verified the docs for this rule still render correctly.

@zanieb
Copy link
Member

zanieb commented Apr 3, 2024

I'm confused about the docs parse error.

@carljm
Copy link
Contributor Author

carljm commented Apr 3, 2024

I'm confused about the docs parse error.

I left in a stray unmatched bracket in a code example in the docs. I checked docs rendering locally, but I guess checking syntactic validity of docs code examples is a separate step I didn't run locally. Fixed now.

@zanieb
Copy link
Member

zanieb commented Apr 3, 2024

Oh nice. Yay #10484

Copy link
Contributor

github-actions bot commented Apr 3, 2024

ruff-ecosystem results

Linter (stable)

✅ ecosystem check detected no linter changes.

Linter (preview)

ℹ️ ecosystem check detected linter changes. (+41 -0 violations, +0 -0 fixes in 6 projects; 38 projects unchanged)

apache/airflow (+1 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --output-format concise --preview --select ALL

+ airflow/providers/weaviate/hooks/weaviate.py:1045:39: C419 Unnecessary list comprehension

commaai/openpilot (+8 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --output-format concise --preview

+ selfdrive/boardd/tests/test_boardd_loopback.py:95:17: C419 Unnecessary list comprehension
+ selfdrive/car/interfaces.py:58:10: C419 Unnecessary list comprehension
+ selfdrive/car/tests/test_lateral_limits.py:91:27: C419 Unnecessary list comprehension
+ selfdrive/debug/test_fw_query_on_routes.py:101:25: C419 Unnecessary list comprehension
+ selfdrive/locationd/helpers.py:31:16: C419 Unnecessary list comprehension
+ system/qcomgpsd/qcomgpsd.py:341:33: C419 Unnecessary list comprehension
+ system/qcomgpsd/qcomgpsd.py:353:41: C419 Unnecessary list comprehension
+ system/tests/test_logmessaged.py:55:19: C419 Unnecessary list comprehension

demisto/content (+23 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --output-format concise --preview

+ Packs/Akamai_SIEM/Integrations/Akamai_SIEM/Akamai_SIEM.py:90:34: C419 Unnecessary list comprehension
+ Packs/Base/Scripts/DBotFindSimilarIncidentsByIndicators/DBotFindSimilarIncidentsByIndicators.py:95:20: C419 Unnecessary list comprehension
+ Packs/Base/Scripts/DBotFindSimilarIncidentsByIndicators/DBotFindSimilarIncidentsByIndicators.py:96:13: C419 Unnecessary list comprehension
+ Packs/Base/Scripts/DBotMLFetchData/DBotMLFetchData.py:241:59: C419 Unnecessary list comprehension
+ Packs/Base/Scripts/DBotMLFetchData/DBotMLFetchData.py:244:67: C419 Unnecessary list comprehension
+ Packs/Base/Scripts/DBotMLFetchData/DBotMLFetchData.py:539:39: C419 Unnecessary list comprehension
+ Packs/Base/Scripts/DBotMLFetchData/DBotMLFetchData.py:540:32: C419 Unnecessary list comprehension
+ Packs/Base/Scripts/DBotMLFetchData/DBotMLFetchData.py:541:32: C419 Unnecessary list comprehension
+ Packs/Base/Scripts/DBotMLFetchData/DBotMLFetchData.py:543:9: C419 Unnecessary list comprehension
+ Packs/Base/Scripts/DBotMLFetchData/DBotMLFetchData.py:545:32: C419 Unnecessary list comprehension
+ Packs/Base/Scripts/DBotMLFetchData/DBotMLFetchData.py:546:37: C419 Unnecessary list comprehension
+ Packs/Base/Scripts/DBotMLFetchData/DBotMLFetchData.py:548:9: C419 Unnecessary list comprehension
+ Packs/Base/Scripts/DBotMLFetchData/DBotMLFetchData.py:553:40: C419 Unnecessary list comprehension
+ Packs/Campaign/Scripts/ShowCampaignLastIncidentOccurred/ShowCampaignLastIncidentOccurred.py:39:28: C419 Unnecessary list comprehension
+ Packs/CofenseVision/Scripts/ConvertDictOfListToListOfDict/ConvertDictOfListToListOfDict.py:16:23: C419 Unnecessary list comprehension
+ Packs/CommonScripts/Scripts/SSDeepReputation/SSDeepReputation.py:71:21: C419 Unnecessary list comprehension
+ Packs/CyberArkIdentity/Integrations/CyberArkIdentityEventCollector/CyberArkIdentityEventCollector.py:160:30: C419 Unnecessary list comprehension
+ Packs/JamfProtect/Integrations/JamfProtectEventCollector/JamfProtectEventCollector.py:329:31: C419 Unnecessary list comprehension
+ Packs/JamfProtect/Integrations/JamfProtectEventCollector/JamfProtectEventCollector.py:375:31: C419 Unnecessary list comprehension
+ Packs/RedCanary/Integrations/RedCanary/RedCanary.py:582:26: C419 Unnecessary list comprehension
+ Packs/SCADAfence_CNM/Integrations/SCADAfenceCNM/SCADAfenceCNM.py:58:12: C419 Unnecessary list comprehension
+ Utils/test_upload_flow/create_test_branch.py:190:35: C419 Unnecessary list comprehension
+ Utils/test_upload_flow/verify_bucket.py:164:24: C419 Unnecessary list comprehension

mlflow/mlflow (+7 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --output-format concise --preview

+ docs/source/llms/openai/notebooks/openai-code-helper.ipynb:cell 28:13:21: C419 Unnecessary list comprehension
+ mlflow/openai/api_request_parallel_processor.py:201:37: C419 Unnecessary list comprehension
+ mlflow/openai/api_request_parallel_processor.py:214:24: C419 Unnecessary list comprehension
+ mlflow/recipes/utils/execution.py:339:32: C419 Unnecessary list comprehension
+ mlflow/sentence_transformers/__init__.py:548:13: C419 Unnecessary list comprehension
+ mlflow/store/model_registry/sqlalchemy_store.py:662:28: C419 Unnecessary list comprehension
+ mlflow/utils/file_utils.py:1079:31: C419 Unnecessary list comprehension

pypa/pip (+1 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --output-format concise --preview

+ src/pip/_internal/commands/search.py:144:17: C419 Unnecessary list comprehension

rotki/rotki (+1 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --output-format concise --preview

+ rotkehlchen/tests/api/test_nfts.py:398:29: C419 Unnecessary list comprehension

Changes by rule (1 rules affected)

code total + violation - violation + fix - fix
C419 41 41 0 0 0

@zanieb
Copy link
Member

zanieb commented Apr 3, 2024

I'd suggest looking through the ecosystem checks before merging. Let me know if you have any questions about them.

@carljm
Copy link
Contributor Author

carljm commented Apr 3, 2024

I looked at all the new hits, they all look like valid and intended applications of the rule. Clearly this will cause new linter errors in stable on existing code though, not sure how we feel about that. I'm guessing in order to keep this in preview only we'd have to make it a new separate code?

@zanieb
Copy link
Member

zanieb commented Apr 3, 2024

Ah yeah we should probably gate this with preview since it's a significant increase in scope. You don't need a separate code, you can just gate the additional calls with a check for preview.

Copy link
Member

@AlexWaygood AlexWaygood left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple of optional docs nits:

@carljm
Copy link
Contributor Author

carljm commented Apr 3, 2024

Ecosystem results look good (same as before, but now preview only; stable is unchanged); merging.

@carljm carljm merged commit 5e24828 into main Apr 3, 2024
17 checks passed
@carljm carljm deleted the cjm/c419 branch April 3, 2024 20:44
@zanieb zanieb added the preview Related to preview mode features label Apr 3, 2024
@Skylion007
Copy link
Contributor

Nice tool, but will there be an config option to match the old C419 behavior at some point? Right now, this is low-stake since it's enabled behind a preview-flag, but users may want to leave the old behavior if it results in a performance penalty of using a generator over a list comp. Would be good to leave it configurable or give it's own error code IMO.

@carljm
Copy link
Contributor Author

carljm commented Apr 4, 2024

@Skylion007 Thanks for the feedback, it's a reasonable point that perhaps this should be two separate codes, since the tradeoffs look different for any/all vs sum/min/max, and it's reasonable to want to enable the former but disable the latter.

Would you be willing to open a new issue with a request for these to be two separate codes? Then we can take some time to collect more feedback (that's what preview is for, after all!) and make a decision before this goes stable.

Glyphack pushed a commit to Glyphack/ruff that referenced this pull request Apr 12, 2024
…check (C419) (astral-sh#10759)

Fixes astral-sh#3259 

## Summary

Renames `UnnecessaryComprehensionAnyAll` to
`UnnecessaryComprehensionInCall` and extends the check to `sum`, `min`,
and `max`, in addition to `any` and `all`.

## Test Plan

Updated snapshot test.

Built docs locally and verified the docs for this rule still render
correctly.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
preview Related to preview mode features
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Unnecessary list comprehension in sum, min, max
4 participants