Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(mypy): complete mypy support for the entire codebase #943

Merged
merged 12 commits into from
Jan 31, 2022

Conversation

mploski
Copy link
Contributor

@mploski mploski commented Jan 10, 2022

Issue #, if available:
aws-powertools/powertools-lambda#3

Description of changes:

  • Fix all typing errors discovered by mypy
  • Enable mypy in CI

Checklist

Breaking change checklist

RFC issue #:

  • Migration process documented
  • Implement warnings (if it can live side by side)

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@boring-cyborg boring-cyborg bot added area/metrics internal Maintenance changes labels Jan 10, 2022
@pull-request-size pull-request-size bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Jan 10, 2022
@mploski mploski requested a review from heitorlessa January 10, 2022 10:47
@mploski mploski marked this pull request as draft January 10, 2022 10:52
@mploski
Copy link
Contributor Author

mploski commented Jan 11, 2022

@heitorlessa github ci failing on the last error: https://github.com/mploski/aws-lambda-powertools-python/runs/4759867427?check_suite_focus=true.

I’d like to discuss a potential solution to this. In general mypy complains on subclass overwriting superclass get method with different signatures, which I believe breaks Liskov Substitution Principle. Other errors were solved already.

@mploski
Copy link
Contributor Author

mploski commented Jan 16, 2022

@heitorlessa github ci failing on the last error: https://github.com/mploski/aws-lambda-powertools-python/runs/4759867427?check_suite_focus=true.

I’d like to discuss a potential solution to this. In general mypy complains on subclass overwriting superclass get method with different signatures, which I believe breaks Liskov Substitution Principle. Other errors were solved already.

Since any change to this method's spec may be breaking to users and we cannot also move get method declaration in Base class to subclass for the same reason decided to silent it and accept potential risk

@boring-cyborg boring-cyborg bot added area/event_handlers dependencies Pull requests that update a dependency file labels Jan 16, 2022
@mploski mploski force-pushed the chore/full-mypy-support branch from 55b177b to bbb18b4 Compare January 16, 2022 21:51
Michal Ploski added 2 commits January 16, 2022 22:52
Since any change to this method spec may be breaking to users and we cannot also move get method declaration in Base class we need to silent it and accept potential risk
@mploski mploski force-pushed the chore/full-mypy-support branch 2 times, most recently from 630441e to 8d23b2d Compare January 16, 2022 21:56
@codecov-commenter
Copy link

codecov-commenter commented Jan 16, 2022

Codecov Report

Merging #943 (82d93fc) into develop (c43a5e8) will increase coverage by 0.00%.
The diff coverage is 100.00%.

Impacted file tree graph

@@           Coverage Diff            @@
##           develop     #943   +/-   ##
========================================
  Coverage    99.96%   99.96%           
========================================
  Files          119      119           
  Lines         5320     5337   +17     
  Branches       613      608    -5     
========================================
+ Hits          5318     5335   +17     
  Partials         2        2           
Impacted Files Coverage Δ
aws_lambda_powertools/event_handler/api_gateway.py 100.00% <100.00%> (ø)
aws_lambda_powertools/metrics/metrics.py 100.00% <100.00%> (ø)
aws_lambda_powertools/shared/functions.py 100.00% <100.00%> (ø)
...ws_lambda_powertools/utilities/batch/exceptions.py 100.00% <100.00%> (ø)
aws_lambda_powertools/utilities/batch/sqs.py 100.00% <100.00%> (ø)
...da_powertools/utilities/idempotency/idempotency.py 100.00% <100.00%> (ø)
...wertools/utilities/idempotency/persistence/base.py 99.36% <100.00%> (-0.01%) ⬇️
aws_lambda_powertools/utilities/parameters/base.py 100.00% <100.00%> (ø)
aws_lambda_powertools/utilities/parameters/ssm.py 100.00% <100.00%> (ø)
...bda_powertools/utilities/parser/envelopes/apigw.py 100.00% <100.00%> (ø)
... and 15 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c43a5e8...82d93fc. Read the comment docs.

@mploski mploski force-pushed the chore/full-mypy-support branch from 8d23b2d to be81184 Compare January 16, 2022 22:05
@mploski mploski force-pushed the chore/full-mypy-support branch from be81184 to 8bcb3fd Compare January 16, 2022 22:08
@mploski mploski requested a review from to-mc January 18, 2022 08:21
@mploski mploski force-pushed the chore/full-mypy-support branch from 5d57d5a to 0196883 Compare January 18, 2022 08:24
@mploski mploski marked this pull request as ready for review January 18, 2022 08:27
Copy link
Contributor

@heitorlessa heitorlessa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot @mploski - it covers a lot of ground and tech debt.

I'd like to ask you to consider two key aspects:

  • Consider cast your last resource. cast is gonna increase the execution time for end-customers, and it slowly litters the code (similar to type coercion in other languages). Sometimes propagating the type, a generic, or creating another variable and an explicit type is a solution.
  • Consider documenting beyond typing changes. I've noticed that we seem to be introducing a new bug in Idempotency by fixing a type, as we're changing a branching logic too. There are major block changes in Parameters too. For quick notes, use the PR description, and for larger ones like parameters comment on why this is necessary. Mypy can't see beyond the types, and while we have multiple checks in place we might slip a regression here. By changing that code, it'd be great to check whether additional tests are necessary too as we might have not covered that.

aws_lambda_powertools/utilities/batch/sqs.py Outdated Show resolved Hide resolved
aws_lambda_powertools/utilities/parser/envelopes/sns.py Outdated Show resolved Hide resolved
pyproject.toml Outdated Show resolved Hide resolved
@mploski mploski force-pushed the chore/full-mypy-support branch from 284bde2 to 0196883 Compare January 21, 2022 22:06
@mploski mploski force-pushed the chore/full-mypy-support branch from 66f2a0c to 862c67e Compare January 27, 2022 21:25
@mploski mploski requested a review from heitorlessa January 27, 2022 21:26
@boring-cyborg boring-cyborg bot added the tests label Jan 27, 2022
@mploski mploski force-pushed the chore/full-mypy-support branch from 20ce752 to a82e959 Compare January 27, 2022 22:09
@mploski
Copy link
Contributor Author

mploski commented Jan 27, 2022

@heitorlessa Added few tests to fix tests coverage

@heitorlessa heitorlessa changed the title chore: fix mypy errors feat(mypy): complete mypy support for the entire codebase Jan 31, 2022
@github-actions github-actions bot added the feature New feature or functionality label Jan 31, 2022
@heitorlessa heitorlessa removed the internal Maintenance changes label Jan 31, 2022
@heitorlessa
Copy link
Contributor

Added the last bits - thank you so so much again @mploski ;) A new era for Powertools. We must also keep a close eye after the release to see if this accidentally introduced any new bug that our checks + peer review wasn't able to detect.

@heitorlessa heitorlessa merged commit dea696e into aws-powertools:develop Jan 31, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dependencies Pull requests that update a dependency file feature New feature or functionality size/L Denotes a PR that changes 100-499 lines, ignoring generated files. tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants