Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix performance of address page for validators #2908

Merged
merged 4 commits into from
Jan 4, 2020
Merged

Conversation

vbaranov
Copy link
Member

@vbaranov vbaranov commented Dec 3, 2019

#2306

Motivation

The issue with the performance of address's page is related only to validators and the slowest part is the query to get block rewards for validators. The problem is that some validators changed their payout keys. It means that once upon a time they have been getting rewards on their mining key and after the change, they began to get rewards to their payout key. Thus, opening the address page for mining key of validator somewhere deeply in block_rewards table we have rows for the rewards for that mining key but since we ordering by block numbers in descending order, the query works very slow to get 50 rows. I think we should get rewards only for the latest blocks. Let's say the last 2000 blocks - it is enough to keep track of rewards on the first 3 pages of chain with 20 validators. I doubt someone will look deeper than 2nd page looking for rewards.

Changelog

The query that slows down address page:

SELECT b0."address_type", b0."reward", b0."address_hash", b0."block_hash", b0."inserted_at", b0."updated_at"
FROM "block_rewards" AS b0
INNER JOIN "blocks" AS b1
ON b1."hash" = b0."block_hash"
WHERE (b0."address_hash" = '\x5ee341ac44d344ade1ca3a771c59b98eb2a77df2')
ORDER BY b1."number" DESC LIMIT 1;

it is suggested to change it to:

SELECT b0."address_type", b0."reward", b0."address_hash", b0."block_hash", b0."inserted_at", b0."updated_at"
FROM "block_rewards" AS b0 
INNER JOIN (SELECT b0.* FROM "blocks" AS b0 ORDER BY b0."number" 
DESC LIMIT 1000) AS s1 ON s1."hash" = b0."block_hash"
WHERE (b0."address_hash" = '\x5ee341ac44d344ade1ca3a771c59b98eb2a77df2')
ORDER BY s1."number" DESC LIMIT 1

Before:
Screenshot 2019-12-03 at 20 54 07

After:
Screenshot 2019-12-03 at 20 57 57

UPDATE 2019-12-12

I found a way how to increase the performance of the validators' address' page even without limiting the number of taking to account rewards. The idea is to get the range of blocks where transactions found for this page of the transactions' list in the pagination and show rewards, that are in the block's range. Even despite that fact that page will wait retrieving of this blocks' range first to proceed with rewards query, page works dramatically faster.

Checklist for your Pull Request (PR)

@vbaranov vbaranov changed the title Increase performance of block rewards query Fix performance of address page Dec 3, 2019
@vbaranov vbaranov self-assigned this Dec 3, 2019
@vbaranov vbaranov added the ready for review This PR is ready for reviews. label Dec 3, 2019
@coveralls
Copy link

coveralls commented Dec 3, 2019

Pull Request Test Coverage Report for Build c4f55590-f3fc-42d0-b14b-d3adfdbb3946

  • 21 of 28 (75.0%) changed or added relevant lines in 2 files are covered.
  • 2 unchanged lines in 1 file lost coverage.
  • Overall coverage decreased (-0.03%) to 75.218%

Changes Missing Coverage Covered Lines Changed/Added Lines %
apps/explorer/lib/explorer/chain.ex 16 19 84.21%
apps/explorer/lib/explorer/chain/block/reward.ex 5 9 55.56%
Files with Coverage Reduction New Missed Lines %
apps/block_scout_web/lib/block_scout_web/controllers/chain/market_history_chart_controller.ex 2 71.43%
Totals Coverage Status
Change from base Build 43c5f8fd-0d8e-4455-831a-a15f9933ffea: -0.03%
Covered Lines: 5351
Relevant Lines: 7114

💛 - Coveralls

@vbaranov vbaranov changed the title Fix performance of address page Fix performance of address page for validators Dec 4, 2019
@vbaranov vbaranov force-pushed the vb-block-rewards branch 5 times, most recently from 0e7ac1b to a268c0f Compare December 12, 2019 14:25
@vbaranov
Copy link
Member Author

@ayrat555 I found a way how to improve performance of the page for validators without limits:

UPDATE 2019-12-12
I found a way how to increase the performance of the validators' address' page even without limiting the number of taking to account rewards. The idea is to get the range of blocks where transactions found for this page of the transactions' list in the pagination and show rewards, that are in the block's range. Even despite that fact that page will wait retrieving of this blocks' range first to proceed with rewards query, page works dramatically faster.

need to be reviewed again

@vbaranov vbaranov added this to the 3.0.0 milestone Dec 18, 2019
@vbaranov vbaranov merged commit 0b60078 into master Jan 4, 2020
@vbaranov vbaranov deleted the vb-block-rewards branch January 4, 2020 19:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
performance ready for review This PR is ready for reviews.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants