Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CodeStyle] add pre-commit hook remove-tabs for python files #46290

Merged
merged 23 commits into from
Sep 27, 2022
Merged

[CodeStyle] add pre-commit hook remove-tabs for python files #46290

Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
6f1f5e0
[CodeStyle] add pre-commit hook `remove-tabs` for python files
SigureMo Sep 20, 2022
4145030
exclude files from thirdparty and unittest required tabs
SigureMo Sep 20, 2022
3ee8a15
empty commit, test=document_fix
SigureMo Sep 20, 2022
91fbc2f
Merge branch 'develop' into tabs/config/py
SigureMo Sep 21, 2022
469bd7c
empty commit, test=document_fix
SigureMo Sep 21, 2022
1343f2d
Merge branch 'develop' into tabs/config/py
SigureMo Sep 21, 2022
99775d4
remove W191 and E101 in .flake8
SigureMo Sep 21, 2022
7df65ba
empty commit, test=document_fix
SigureMo Sep 21, 2022
c6ce328
exclude fluid/ in flake8 config
SigureMo Sep 21, 2022
f4807a0
add comment
SigureMo Sep 21, 2022
f694c03
empty commit, test=document_fix
SigureMo Sep 21, 2022
aef8fdf
remove E124 and F404
SigureMo Sep 23, 2022
3b51a96
empty commit, test=document_fix
SigureMo Sep 23, 2022
5da1589
Merge branch 'develop' into tabs/config/py
SigureMo Sep 23, 2022
b9825a5
add comment
SigureMo Sep 23, 2022
02380df
add comment
SigureMo Sep 23, 2022
2bf08b2
empty commit, test=document_fix
SigureMo Sep 23, 2022
ed17d0f
update regex
SigureMo Sep 24, 2022
7d39482
empty commit, test=document_fix
SigureMo Sep 24, 2022
b98a5bf
fix glob
SigureMo Sep 25, 2022
5d08226
empty commit, test=document_fix
SigureMo Sep 25, 2022
9e87dc1
remove some error code
SigureMo Sep 25, 2022
daf98d8
empty commit, test=document_fix
SigureMo Sep 25, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 10 additions & 3 deletions .flake8
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,13 @@
[flake8]
select = C,E,F,W
exclude = ./build
exclude =
./build,
# A trick to exclude fluid/ but keep fluid/tests/
./python/paddle/fluid/[!t]**,
./python/paddle/fluid/transpiler/
Copy link
Member Author

@SigureMo SigureMo Sep 21, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

根据 #46288 (comment) 的讨论，目前 Flake8 忽略 python/paddle/fluid 整个目录，但需要单独保留 python/paddle/fluid/tests。

Flake8 路径相关使用的是 glob 语法（见 https://en.wikipedia.org/wiki/Glob_(programming) ），glob 语法并不能很好地直接实现这样的需求，Stack Overflow 上也有很多问题说明了这一问题（如 https://stackoverflow.com/questions/20638040/glob-exclude-pattern ）

这里先使用 ./python/paddle/fluid/[!t]** 以排除所有 ./python/paddle/fluid/ 下第一个字母不是 t 的路径，这样的路径包含了 tests/ 子目录、transpiler/ 子目录、trainer_desc.py 和 trainer_factory.py，因此实际上其余以 transpiler/、trainer_desc.py 和 trainer_factory.py 也被「豁免」了。为了解决这一问题，使用额外一项排除掉这样的路径，这样通过两条 glob 实现了上述的需求。

ignore =
# E, see https://pycodestyle.pycqa.org/en/latest/intro.html#error-codes
E101,E121,E122,E123,E124,E125,E126,E127,E128,E129,E131,
E121,E122,E123,E124,E125,E126,E127,E128,E129,E131,
E201,E202,E203,E221,E225,E226,E228,E231,E241,E251,E261,E262,E265,E266,E271,E272,E275,
E301,E302,E303,E305,E306,
E401,E402,
Expand All @@ -17,6 +21,9 @@ ignore =
F811,F821,F823,F841,

# W, see https://pycodestyle.pycqa.org/en/latest/intro.html#error-codes
W191,
W503,W504
W601,W605
per-file-ignores =
# These files need tabs for testing.
python/paddle/fluid/tests/unittests/dygraph_to_static/test_error.py:E101,W191
python/paddle/fluid/tests/unittests/collective/fleet/test_hdfs1.py:E101,W191
11 changes: 10 additions & 1 deletion .pre-commit-config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,8 @@ exclude: |
(?x)^(
patches/.+|
paddle/fluid/framework/fleet/heter_ps/cudf/.+|
paddle/fluid/distributed/ps/thirdparty/round_robin.h
paddle/fluid/distributed/ps/thirdparty/round_robin.h|
python/paddle/utils/gast/.+
SigureMo marked this conversation as resolved.
Show resolved Hide resolved
)$
repos:
- repo: https://github.com/Lucas-C/pre-commit-hooks.git
Expand All @@ -12,6 +13,14 @@ repos:
- id: remove-tabs
files: \.(c|cc|cxx|cpp|cu|h|hpp|hxx|xpu|kps)$
args: [--whitespaces-count, '2']
- id: remove-tabs
files: (.*\.(py|bzl)|BUILD|.*\.BUILD|WORKSPACE)$
args: [--whitespaces-count, '4']
exclude: |
(?x)^(
python/paddle/fluid/tests/unittests/collective/fleet/test_hdfs1.py|
python/paddle/fluid/tests/unittests/dygraph_to_static/test_error.py
)$
- repo: https://github.com/google/yapf
rev: v0.32.0
hooks:
Expand Down