Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speedup of blocks_without_reward_query #2915

Merged
merged 2 commits into from
Dec 20, 2019

Conversation

vbaranov
Copy link
Member

@vbaranov vbaranov commented Dec 10, 2019

Motivation

The query https://github.com/poanetwork/blockscout/blob/8c62be450d9a78b4a52ae6c3900f6b383d2b8382/apps/explorer/lib/explorer/chain/block.ex#L118 executes more than 5 hours on ETH Mainnet instance.

This PR aims to decrease the execution time.

Changelog

  1. Add partial index on block_rewards table block_hash column where address type is validator
  2. Decrease subsets of tables before joins, not after.

Before the changes:

8740 | 05:36:11.251542 |SELECT b0."hash", b0."consensus", b0."difficulty", b0."gas_limit", b0."gas_used", b0."nonce", b0."number", b0."size", b0."timestamp", b0."total_difficulty", b0."internal_transactions_indexed_at", b0."refetch_needed", b0."inserted_at", b0."updated_at", b0."miner_hash", b0."parent_hash" FROM "blocks" AS b0 LEFT OUTER JOIN "block_rewards" AS b1 ON b1."block_hash" = b0."hash" WHERE (b1."block_hash" IS NULL AND (b0."consensus" = TRUE))                                     | active

EXPLAIN ANALYZE on a small amount of the data:

Planning Time: 5.111 ms
Execution Time: 105251.312 ms

After the changes:

SELECT b0.* FROM
(SELECT * from "blocks" WHERE blocks."consensus" = TRUE) AS b0
LEFT OUTER JOIN
(SELECT * FROM "block_rewards"
WHERE block_rewards.address_type='validator') AS b1
ON b1."block_hash" = b0."hash"
WHERE b1."block_hash" IS NULL;

EXPLAIN ANALYZE on the same small set of the data, that it was before the changes:

Planning Time: 22.785 ms
Execution Time: 81535.140 ms

Checklist for your Pull Request (PR)

@vbaranov vbaranov added ready for review This PR is ready for reviews. performance labels Dec 10, 2019
@vbaranov vbaranov self-assigned this Dec 10, 2019
@vbaranov vbaranov force-pushed the vb-speedup-blocks_without_reward_query branch from 7aeafd7 to 7817fef Compare December 10, 2019 08:25
@coveralls
Copy link

coveralls commented Dec 10, 2019

Pull Request Test Coverage Report for Build 2f80627e-8d31-4ac7-b03b-9b8a7f1fd9a2

  • 2 of 2 (100.0%) changed or added relevant lines in 1 file are covered.
  • 2 unchanged lines in 2 files lost coverage.
  • Overall coverage increased (+0.007%) to 75.124%

Files with Coverage Reduction New Missed Lines %
apps/block_scout_web/lib/block_scout_web/controllers/chain/market_history_chart_controller.ex 1 78.57%
apps/explorer/test/support/factory.ex 1 92.97%
Totals Coverage Status
Change from base Build e3430a1c-af77-4433-8eea-9d8c50b2a66b: 0.007%
Covered Lines: 5303
Relevant Lines: 7059

💛 - Coveralls

@vbaranov vbaranov force-pushed the vb-speedup-blocks_without_reward_query branch from 7817fef to 223658a Compare December 12, 2019 17:03
@vbaranov vbaranov added this to the 3.0.0 milestone Dec 18, 2019
@vbaranov vbaranov merged commit 67f9a3c into master Dec 20, 2019
@vbaranov vbaranov deleted the vb-speedup-blocks_without_reward_query branch December 20, 2019 13:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
performance ready for review This PR is ready for reviews.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants