Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

types/Coin: compile and reuse Regexps to reduce massive RAM+CPU burn #8001

Merged
merged 3 commits into from
Nov 24, 2020

Conversation

alessio
Copy link
Contributor

@alessio alessio commented Nov 21, 2020

This patch brings a client code breaking change. As such it requires a Stable Release Exception to be granted by the Stable Release Managers. @clevinson @ethanfrey

From: #7989
Closes: #7986

Co-authored-by: Federico Kunze [email protected]
Co-authored-by: Alessio Treglia [email protected]

Impact

The changes introduced in #7450 to support denoms custom validation
caused a significance performance hit due to the use of closures
instead of compiled regular expressions. As the reporter described in #7986,
this represents a security threat since many ParseCoin invocations made
concurrently could cause the client application to run out of RAM.

Test Case

The bug becomes visible by invoking ParseCoin concurrently 1000 times.
A benchmark test case is included in this PR.

Regression Potential

Regressions could manifest if the CoinDenomRegex was left publicly exposed
as any modification to it by the client would not trigger the recompilation of the
regular expressions that depend on it. The original patch addresses potential
regressions by replacing CoinDenomRegex with the accessor SetCoinDenomRegex()
in the the types package public interface.


Before we can merge this PR, please make sure that all the following items have been
checked off. If any of the checklist items are not applicable, please leave them but
write a little note why.

  • Targeted PR against correct branch (see CONTRIBUTING.md)
  • Linked to Github issue with discussion and accepted design OR link to spec that describes this work.
  • Code follows the module structure standards.
  • Wrote unit and integration tests
  • Updated relevant documentation (docs/) or specification (x/<module>/spec/)
  • Added relevant godoc comments.
  • Added a relevant changelog entry to the Unreleased section in CHANGELOG.md
  • Re-reviewed Files changed in the Github PR explorer
  • Review Codecov Report in the comment section below once CI passes

…7989)

From: #7989
Closes: #7986

Co-authored-by: Federico Kunze <[email protected]>
Co-authored-by: Alessio Treglia <[email protected]>
@alessio alessio requested a review from okwme November 21, 2020 11:03
@alessio alessio requested a review from odeke-em November 21, 2020 11:03
@codecov
Copy link

codecov bot commented Nov 21, 2020

Codecov Report

Merging #8001 (72c4d1c) into launchpad/backports (5be42d9) will increase coverage by 0.00%.
The diff coverage is 100.00%.

Impacted file tree graph

@@                 Coverage Diff                  @@
##           launchpad/backports    #8001   +/-   ##
====================================================
  Coverage                50.29%   50.30%           
====================================================
  Files                      338      338           
  Lines                    17570    17572    +2     
====================================================
+ Hits                      8837     8839    +2     
  Misses                    7940     7940           
  Partials                   793      793           
Impacted Files Coverage Δ
types/coin.go 91.21% <100.00%> (+0.07%) ⬆️
types/dec_coin.go 77.82% <100.00%> (ø)

Copy link
Collaborator

@odeke-em odeke-em left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @alessio for working this, LGTM, but deferring to the release managers and also to others.

Copy link
Member

@jackzampolin jackzampolin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great catch!

types/coin.go Show resolved Hide resolved
types/coin.go Show resolved Hide resolved
types/coin.go Outdated Show resolved Hide resolved
Co-authored-by: Alessio Treglia <[email protected]>
Copy link
Contributor

@ethanfrey ethanfrey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the previous change that this modifies was not part of a tagged release, and the default usage remains the same, I see no issues with this.

Ack

@alessio alessio merged commit 2727c14 into launchpad/backports Nov 24, 2020
@alessio alessio deleted the 7989-performance-improv branch November 24, 2020 06:15
Copy link

@alain2sf alain2sf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. @odeke-em nice example of profiling 👍🏻

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants