Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

make lint is not clean in ctsm5.1.dev152 #2255

Closed
ekluzek opened this issue Nov 15, 2023 · 6 comments · Fixed by #2354 or #2382
Closed

make lint is not clean in ctsm5.1.dev152 #2255

ekluzek opened this issue Nov 15, 2023 · 6 comments · Fixed by #2354 or #2382
Assignees
@samsrabin @slevis-lmwg @TeaganKing
Labels
bug something is working incorrectly code health improving internal code structure to make easier to maintain (sustainability)

Comments

@ekluzek
Copy link
Collaborator

ekluzek commented Nov 15, 2023

Brief summary of bug

We were aware of this when this came in. But, pylint is not clean with ctsm5.1.dev152 with #2156. This is something that @TeaganKing and others will work on in upcoming tags. The issues should just be in run_neon python code. It is black clean however at this point.

General bug information

CTSM version you are using: ctsm5.1.dev152
Does this bug cause significantly incorrect results in the model's science? No

Configurations affected:

Details of bug

Important details of your setup / configuration so we can reproduce the bug

./py_env_create
conda activate ctsm_pylib  
cd python
make lint

Important output or errors that show the problem

(ctsm_pylib) cheyenne2 main_dev/python> make lint
pylint -j 4 --rcfile=ctsm/.pylintrc ctsm
************* Module ctsm.test.test_unit_modify_singlept_site_neon
ctsm/test/test_unit_modify_singlept_site_neon.py:21:0: C0413: Import "from ctsm.path_utils import path_to_ctsm_root" should be placed at the top of the module (wrong-import-position)
************* Module ctsm.test.test_sys_modify_singlept_site_neon
ctsm/test/test_sys_modify_singlept_site_neon.py:61:8: W0612: Unused variable 'parser' (unused-variable)
************* Module ctsm.test.test_sys_run_neon
ctsm/test/test_sys_run_neon.py:56:8: W0612: Unused variable 'parser' (unused-variable)
************* Module ctsm.site_and_regional.modify_singlept_site_neon
ctsm/site_and_regional/modify_singlept_site_neon.py:181:38: C0103: Variable name "f" doesn't conform to snake_case naming style (invalid-name)
ctsm/site_and_regional/modify_singlept_site_neon.py:277:4: C0103: Variable name "f1" doesn't conform to snake_case naming style (invalid-name)
ctsm/site_and_regional/modify_singlept_site_neon.py:306:0: C0103: Argument name "nc" doesn't conform to snake_case naming style (invalid-name)
ctsm/site_and_regional/modify_singlept_site_neon.py:390:8: W0612: Unused variable 'index' (unused-variable)
ctsm/site_and_regional/modify_singlept_site_neon.py:416:4: C0103: Variable name "df" doesn't conform to snake_case naming style (invalid-name)
ctsm/site_and_regional/modify_singlept_site_neon.py:417:4: C0103: Variable name "df" doesn't conform to snake_case naming style (invalid-name)
ctsm/site_and_regional/modify_singlept_site_neon.py:444:11: W0703: Catching too general exception Exception (broad-except)
ctsm/site_and_regional/modify_singlept_site_neon.py:436:34: C0103: Variable name "f" doesn't conform to snake_case naming style (invalid-name)
ctsm/site_and_regional/modify_singlept_site_neon.py:447:29: E1101: Instance of 'Exception' has no 'code' member (no-member)
ctsm/site_and_regional/modify_singlept_site_neon.py:450:0: C0103: Argument name "f2" doesn't conform to snake_case naming style (invalid-name)
ctsm/site_and_regional/modify_singlept_site_neon.py:478:0: C0116: Missing function or method docstring (missing-function-docstring)
ctsm/site_and_regional/modify_singlept_site_neon.py:512:4: C0103: Variable name "df" doesn't conform to snake_case naming style (invalid-name)
ctsm/site_and_regional/modify_singlept_site_neon.py:516:4: C0103: Variable name "f1" doesn't conform to snake_case naming style (invalid-name)
ctsm/site_and_regional/modify_singlept_site_neon.py:560:-1: W0105: String statement has no effect (pointless-string-statement)
ctsm/site_and_regional/modify_singlept_site_neon.py:563:4: C0103: Variable name "f2" doesn't conform to snake_case naming style (invalid-name)
ctsm/site_and_regional/modify_singlept_site_neon.py:665:4: C0103: Variable name "f2" doesn't conform to snake_case naming style (invalid-name)
ctsm/site_and_regional/modify_singlept_site_neon.py:504:4: W0612: Unused variable 'current_dir' (unused-variable)
ctsm/site_and_regional/modify_singlept_site_neon.py:478:0: R0915: Too many statements (89/50) (too-many-statements)
************* Module ctsm.site_and_regional.run_neon
ctsm/site_and_regional/run_neon.py:354:0: C0325: Unnecessary parens after 'in' keyword (superfluous-parens)
ctsm/site_and_regional/run_neon.py:420:0: C0301: Line too long (122/100) (line-too-long)
ctsm/site_and_regional/run_neon.py:425:0: C0301: Line too long (121/100) (line-too-long)
ctsm/site_and_regional/run_neon.py:539:0: C0301: Line too long (126/100) (line-too-long)
ctsm/site_and_regional/run_neon.py:544:0: C0301: Line too long (125/100) (line-too-long)
ctsm/site_and_regional/run_neon.py:700:0: C0301: Line too long (151/100) (line-too-long)
ctsm/site_and_regional/run_neon.py:660:28: W1401: Anomalous backslash in string: '\d'. String constant might be missing an r prefix. (anomalous-backslash-in-string)
ctsm/site_and_regional/run_neon.py:660:30: W1401: Anomalous backslash in string: '\d'. String constant might be missing an r prefix. (anomalous-backslash-in-string)
ctsm/site_and_regional/run_neon.py:660:32: W1401: Anomalous backslash in string: '\d'. String constant might be missing an r prefix. (anomalous-backslash-in-string)
ctsm/site_and_regional/run_neon.py:660:34: W1401: Anomalous backslash in string: '\d'. String constant might be missing an r prefix. (anomalous-backslash-in-string)
ctsm/site_and_regional/run_neon.py:660:37: W1401: Anomalous backslash in string: '\d'. String constant might be missing an r prefix. (anomalous-backslash-in-string)
ctsm/site_and_regional/run_neon.py:660:39: W1401: Anomalous backslash in string: '\d'. String constant might be missing an r prefix. (anomalous-backslash-in-string)
ctsm/site_and_regional/run_neon.py:660:42: W1401: Anomalous backslash in string: '\d'. String constant might be missing an r prefix. (anomalous-backslash-in-string)
ctsm/site_and_regional/run_neon.py:660:44: W1401: Anomalous backslash in string: '\d'. String constant might be missing an r prefix. (anomalous-backslash-in-string)
ctsm/site_and_regional/run_neon.py:660:48: W1401: Anomalous backslash in string: '\d'. String constant might be missing an r prefix. (anomalous-backslash-in-string)
ctsm/site_and_regional/run_neon.py:660:50: W1401: Anomalous backslash in string: '\d'. String constant might be missing an r prefix. (anomalous-backslash-in-string)
ctsm/site_and_regional/run_neon.py:660:52: W1401: Anomalous backslash in string: '\d'. String constant might be missing an r prefix. (anomalous-backslash-in-string)
ctsm/site_and_regional/run_neon.py:660:54: W1401: Anomalous backslash in string: '\d'. String constant might be missing an r prefix. (anomalous-backslash-in-string)
ctsm/site_and_regional/run_neon.py:660:56: W1401: Anomalous backslash in string: '\d'. String constant might be missing an r prefix. (anomalous-backslash-in-string)
ctsm/site_and_regional/run_neon.py:760:52: W1401: Anomalous backslash in string: '\d'. String constant might be missing an r prefix. (anomalous-backslash-in-string)
ctsm/site_and_regional/run_neon.py:760:54: W1401: Anomalous backslash in string: '\d'. String constant might be missing an r prefix. (anomalous-backslash-in-string)
ctsm/site_and_regional/run_neon.py:760:56: W1401: Anomalous backslash in string: '\d'. String constant might be missing an r prefix. (anomalous-backslash-in-string)
ctsm/site_and_regional/run_neon.py:760:58: W1401: Anomalous backslash in string: '\d'. String constant might be missing an r prefix. (anomalous-backslash-in-string)
ctsm/site_and_regional/run_neon.py:760:61: W1401: Anomalous backslash in string: '\d'. String constant might be missing an r prefix. (anomalous-backslash-in-string)
ctsm/site_and_regional/run_neon.py:760:63: W1401: Anomalous backslash in string: '\d'. String constant might be missing an r prefix. (anomalous-backslash-in-string)
ctsm/site_and_regional/run_neon.py:69:0: E0401: Unable to import 'CIME' (import-error)
ctsm/site_and_regional/run_neon.py:69:0: C0413: Import "from CIME import build" should be placed at the top of the module (wrong-import-position)
ctsm/site_and_regional/run_neon.py:70:0: E0401: Unable to import 'CIME.case' (import-error)
ctsm/site_and_regional/run_neon.py:70:0: C0413: Import "from CIME.case import Case" should be placed at the top of the module (wrong-import-position)
ctsm/site_and_regional/run_neon.py:71:0: E0401: Unable to import 'CIME.utils' (import-error)
ctsm/site_and_regional/run_neon.py:71:0: C0413: Import "from CIME.utils import safe_copy, expect, symlink_force" should be placed at the top of the module (wrong-import-position)
ctsm/site_and_regional/run_neon.py:73:0: C0413: Import "from ctsm.path_utils import path_to_ctsm_root" should be placed at the top of the module (wrong-import-position)
ctsm/site_and_regional/run_neon.py:74:0: C0413: Import "from ctsm.utils import parse_isoduration" should be placed at the top of the module (wrong-import-position)
ctsm/site_and_regional/run_neon.py:75:0: C0413: Import "from ctsm.download_utils import download_file" should be placed at the top of the module (wrong-import-position)
ctsm/site_and_regional/run_neon.py:77:0: C0413: Import "from ctsm import add_cime_to_path" should be placed at the top of the module (wrong-import-position)
ctsm/site_and_regional/run_neon.py:79:0: E0401: Unable to import 'standard_script_setup' (import-error)
ctsm/site_and_regional/run_neon.py:79:0: W0401: Wildcard import standard_script_setup (wildcard-import)
ctsm/site_and_regional/run_neon.py:79:0: C0413: Import "from standard_script_setup import *" should be placed at the top of the module (wrong-import-position)
ctsm/site_and_regional/run_neon.py:92:4: E0602: Undefined variable 'CIME' (undefined-variable)
ctsm/site_and_regional/run_neon.py:276:11: E0602: Undefined variable 'CIME' (undefined-variable)
ctsm/site_and_regional/run_neon.py:330:0: R0902: Too many instance attributes (8/7) (too-many-instance-attributes)
ctsm/site_and_regional/run_neon.py:384:20: W1202: Use lazy % formatting in logging functions (logging-format-interpolation)
ctsm/site_and_regional/run_neon.py:385:20: W1202: Use lazy % formatting in logging functions (logging-format-interpolation)
ctsm/site_and_regional/run_neon.py:437:12: C0103: Variable name "t0" doesn't conform to snake_case naming style (invalid-name)
ctsm/site_and_regional/run_neon.py:439:12: C0103: Variable name "t1" doesn't conform to snake_case naming style (invalid-name)
ctsm/site_and_regional/run_neon.py:449:8: C0103: Variable name "d1" doesn't conform to snake_case naming style (invalid-name)
ctsm/site_and_regional/run_neon.py:450:8: C0103: Variable name "d2" doesn't conform to snake_case naming style (invalid-name)
ctsm/site_and_regional/run_neon.py:453:4: R0201: Method could be a function (no-self-use)
ctsm/site_and_regional/run_neon.py:536:42: E0602: Undefined variable 'compset' (undefined-variable)
ctsm/site_and_regional/run_neon.py:558:31: W1202: Use lazy % formatting in logging functions (logging-format-interpolation)
ctsm/site_and_regional/run_neon.py:564:31: W1202: Use lazy % formatting in logging functions (logging-format-interpolation)
ctsm/site_and_regional/run_neon.py:467:4: R0915: Too many statements (86/50) (too-many-statements)
ctsm/site_and_regional/run_neon.py:648:16: W1202: Use lazy % formatting in logging functions (logging-format-interpolation)
ctsm/site_and_regional/run_neon.py:660:12: C0103: Variable name "m" doesn't conform to snake_case naming style (invalid-name)
ctsm/site_and_regional/run_neon.py:664:20: W1202: Use lazy % formatting in logging functions (logging-format-interpolation)
ctsm/site_and_regional/run_neon.py:666:27: W1202: Use lazy % formatting in logging functions (logging-format-interpolation)
ctsm/site_and_regional/run_neon.py:704:45: C0103: Variable name "fd" doesn't conform to snake_case naming style (invalid-name)
ctsm/site_and_regional/run_neon.py:378:8: W0201: Attribute 'base_case_root' defined outside __init__ (attribute-defined-outside-init)
ctsm/site_and_regional/run_neon.py:740:4: C0103: Variable name "df" doesn't conform to snake_case naming style (invalid-name)
ctsm/site_and_regional/run_neon.py:746:4: C0103: Variable name "df" doesn't conform to snake_case naming style (invalid-name)
ctsm/site_and_regional/run_neon.py:749:4: C0103: Variable name "df" doesn't conform to snake_case naming style (invalid-name)
ctsm/site_and_regional/run_neon.py:785:12: W1201: Use lazy % formatting in logging functions (logging-not-lazy)
ctsm/site_and_regional/run_neon.py:786:25: W1202: Use lazy % formatting in logging functions (logging-format-interpolation)
ctsm/site_and_regional/run_neon.py:787:25: W1202: Use lazy % formatting in logging functions (logging-format-interpolation)
ctsm/site_and_regional/run_neon.py:788:25: W1202: Use lazy % formatting in logging functions (logging-format-interpolation)
ctsm/site_and_regional/run_neon.py:789:25: W1202: Use lazy % formatting in logging functions (logging-format-interpolation)
ctsm/site_and_regional/run_neon.py:790:25: W1202: Use lazy % formatting in logging functions (logging-format-interpolation)
ctsm/site_and_regional/run_neon.py:753:13: W0612: Unused variable 'item' (unused-variable)
ctsm/site_and_regional/run_neon.py:832:8: W1201: Use lazy % formatting in logging functions (logging-not-lazy)
ctsm/site_and_regional/run_neon.py:860:24: W1202: Use lazy % formatting in logging functions (logging-format-interpolation)
ctsm/site_and_regional/run_neon.py:77:0: W0611: Unused add_cime_to_path imported from ctsm (unused-import)
ctsm/site_and_regional/run_neon.py:79:0: C0411: third party import "from standard_script_setup import *" should be placed before "from ctsm.path_utils import path_to_ctsm_root" (wrong-import-order)
************* Module ctsm.site_and_regional.neon_surf_wrapper
ctsm/site_and_regional/neon_surf_wrapper.py:89:4: C0103: Variable name "e" doesn't conform to snake_case naming style (invalid-name)
ctsm/site_and_regional/neon_surf_wrapper.py:104:8: W0612: Unused variable 'i' (unused-variable)

-------------------------------------------------------------------
Your code has been rated at 9.74/10 (previous run: 10.00/10, -0.26)

Makefile:47: recipe for target 'lint' failed
make: *** [lint] Error 30
@ekluzek ekluzek added code health improving internal code structure to make easier to maintain (sustainability) bug something is working incorrectly labels Nov 15, 2023
@ekluzek ekluzek mentioned this issue Nov 15, 2023
Merged
@slevis-lmwg
Copy link
Contributor

#2253 shortens the above list to:

(ctsm_pylib) single_point/python> make lint
pylint -j 4 --rcfile=ctsm/.pylintrc ctsm
************* Module ctsm.test.test_unit_modify_singlept_site_neon
ctsm/test/test_unit_modify_singlept_site_neon.py:21:0: C0413: Import "from ctsm.path_utils import path_to_ctsm_root" should be placed at the top of the module (wrong-import-position)
************* Module ctsm.site_and_regional.run_neon
ctsm/site_and_regional/run_neon.py:65:0: E0401: Unable to import 'standard_script_setup' (import-error)
ctsm/site_and_regional/run_neon.py:65:0: W0401: Wildcard import standard_script_setup (wildcard-import)
ctsm/site_and_regional/run_neon.py:71:0: E0401: Unable to import 'CIME' (import-error)
ctsm/site_and_regional/run_neon.py:71:0: C0413: Import "from CIME import build" should be placed at the top of the module (wrong-import-position)
ctsm/site_and_regional/run_neon.py:72:0: E0401: Unable to import 'CIME.case' (import-error)
ctsm/site_and_regional/run_neon.py:72:0: C0413: Import "from CIME.case import Case" should be placed at the top of the module (wrong-import-position)
ctsm/site_and_regional/run_neon.py:73:0: E0401: Unable to import 'CIME.utils' (import-error)
ctsm/site_and_regional/run_neon.py:73:0: C0413: Import "from CIME.utils import safe_copy, expect, symlink_force" should be placed at the top of the module (wrong-import-position)
ctsm/site_and_regional/run_neon.py:75:0: C0413: Import "from ctsm.path_utils import path_to_ctsm_root" should be placed at the top of the module (wrong-import-position)
ctsm/site_and_regional/run_neon.py:76:0: C0413: Import "from ctsm.utils import parse_isoduration" should be placed at the top of the module (wrong-import-position)
ctsm/site_and_regional/run_neon.py:77:0: C0413: Import "from ctsm.download_utils import download_file" should be placed at the top of the module (wrong-import-position)
ctsm/site_and_regional/run_neon.py:90:4: E0602: Undefined variable 'CIME' (undefined-variable)
ctsm/site_and_regional/run_neon.py:274:11: E0602: Undefined variable 'CIME' (undefined-variable)
ctsm/site_and_regional/run_neon.py:328:0: R0902: Too many instance attributes (8/7) (too-many-instance-attributes)
ctsm/site_and_regional/run_neon.py:520:42: E0602: Undefined variable 'compset' (undefined-variable)
ctsm/site_and_regional/run_neon.py:539:38: E1101: Instance of 'NeonSite' has no 'get_batch_query' member (no-member)
ctsm/site_and_regional/run_neon.py:616:30: E1101: Instance of 'NeonSite' has no 'get_batch_query' member (no-member)
ctsm/site_and_regional/run_neon.py:451:4: R0915: Too many statements (86/50) (too-many-statements)
ctsm/site_and_regional/run_neon.py:374:8: W0201: Attribute 'base_case_root' defined outside __init__ (attribute-defined-outside-init)
************* Module ctsm.site_and_regional.modify_singlept_site_neon
ctsm/site_and_regional/modify_singlept_site_neon.py:443:11: W0703: Catching too general exception Exception (broad-except)
ctsm/site_and_regional/modify_singlept_site_neon.py:446:29: E1101: Instance of 'Exception' has no 'code' member (no-member)
ctsm/site_and_regional/modify_singlept_site_neon.py:557:-1: W0105: String statement has no effect (pointless-string-statement)
ctsm/site_and_regional/modify_singlept_site_neon.py:475:0: R0915: Too many statements (88/50) (too-many-statements)

------------------------------------------------------------------
Your code has been rated at 9.87/10 (previous run: 9.87/10, +0.00)

Makefile:47: recipe for target 'lint' failed
make: *** [lint] Error 30

@samsrabin
Copy link
Collaborator

samsrabin commented Nov 21, 2023 •
edited
Loading

Thanks for the work on this, Sam! Some untested thoughts:

  • I think the errors related to from standard_script_setup import * can be resolved by changing that to from CIME.Tools.standard_script_setup import *
  • I think the "should be placed at the top of the module" errors for CIME imports are unavoidable because of the need to sys.path.insert(1, _CTSM_PYTHON). If so, those import lines should have # pylint: disable=wrong-import-position appended.
  • … Although I guess we could add a separate module whose sole purpose is to do that sys.path.insert(), then import that.

@slevis-lmwg
Copy link
Contributor

make lint complains about many new problems again in a later version of #2253.
I think this comes from updates in #2080.

@slevis-lmwg slevis-lmwg mentioned this issue Jan 22, 2024
Closed
3 tasks
@samsrabin
Copy link
Collaborator

I'm working on the crop-calendar related stuff.

@samsrabin samsrabin self-assigned this Jan 29, 2024
This was referenced Feb 8, 2024
Merged
Closed
@samsrabin samsrabin mentioned this issue Feb 22, 2024
Merged
@slevis-lmwg
Copy link
Contributor

Thank you for finishing this @samsrabin !

@TeaganKing
Copy link
Contributor

Yes, thank you @samsrabin !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@samsrabin samsrabin

@slevis-lmwg slevis-lmwg

@TeaganKing TeaganKing

Labels
bug something is working incorrectly code health improving internal code structure to make easier to maintain (sustainability)
Projects
None yet
Milestone
No milestone
4 participants
@samsrabin @ekluzek @slevis-lmwg @TeaganKing