Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce execution time of stream_unfetched_token_instances query #2914

Merged
merged 3 commits into from
Dec 20, 2019

Conversation

vbaranov
Copy link
Member

@vbaranov vbaranov commented Dec 10, 2019

Motivation

The query https://github.com/poanetwork/blockscout/blob/da54650a4a559393d29a499a486b51d1477789ef/apps/explorer/lib/explorer/chain.ex#L2969 executes more than 1 hour on ETH Mainnet instance

This PR aims to increase the performance of this query.

Changelog

Decrease the subset before joining the tables

Before the changes query looked like this:

SELECT DISTINCT ON (t0."token_contract_address_hash", t0."token_id") t0."token_contract_address_hash", t0."token_id" FROM "token_transfers" AS t0
INNER JOIN "tokens" AS t1 ON t1."contract_address_hash" = t0."token_contract_address_hash"
LEFT OUTER JOIN "token_instances" AS t2 ON (t0."token_id" = t2."token_id")
AND (t0."token_contract_address_hash" = t2."token_contract_address_hash")
WHERE (((t1."type" = 'ERC-721')
AND t2."token_id" IS NULL) AND NOT (t0."token_id" IS NULL))
LIMIT 1;

EXPLAIN ANALYZE:

Planning Time: 0.520 ms
Execution Time: 0.851 ms

After the changes:

SELECT DISTINCT q."token_contract_address_hash", q."token_id" FROM (SELECT
  t0."token_contract_address_hash", t0."token_id" FROM "token_transfers" AS t0
INNER JOIN (SELECT * FROM "tokens" WHERE tokens."type" = 'ERC-721') AS t1 ON t1."contract_address_hash" = t0."token_contract_address_hash"
LEFT OUTER JOIN "token_instances" AS t2 ON (t0."token_id" = t2."token_id")
AND (t0."token_contract_address_hash" = t2."token_contract_address_hash")
AND t2."token_id" IS NULL AND NOT (t0."token_id" IS NULL)
) q;

EXPLAIN ANALYZE:

Planning Time: 0.423 ms
Execution Time: 0.592 ms

Checklist for your Pull Request (PR)

@vbaranov vbaranov added ready for review This PR is ready for reviews. performance labels Dec 10, 2019
@vbaranov vbaranov self-assigned this Dec 10, 2019
@vbaranov vbaranov force-pushed the vb-improve-stream_unfetched_token_instances-query branch from a759805 to 2d59ff4 Compare December 10, 2019 08:15
@coveralls
Copy link

coveralls commented Dec 10, 2019

Pull Request Test Coverage Report for Build 5d2a6440-9f2e-44ce-898b-8c1e50961ed9

  • 4 of 4 (100.0%) changed or added relevant lines in 1 file are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.01%) to 75.107%

Totals Coverage Status
Change from base Build 24fc869e-c854-4191-9ac9-0087b47c143a: 0.01%
Covered Lines: 5283
Relevant Lines: 7034

💛 - Coveralls

@ayrat555
Copy link
Contributor

@vbaranov tests are failing

@vbaranov
Copy link
Member Author

@ayrat555 it is again test ethsupply returns total supply from DB. It is failed through all branches except master.

@ayrat555
Copy link
Contributor

@vbaranov three tests related to token instances are also failing

@vbaranov
Copy link
Member Author

@ayrat555 thanks, column name was wrong in the distinct select from the subquery. Fixed a7d9f0f

@vbaranov vbaranov added this to the 3.0.0 milestone Dec 18, 2019
@vbaranov vbaranov merged commit f0de982 into master Dec 20, 2019
@vbaranov vbaranov deleted the vb-improve-stream_unfetched_token_instances-query branch December 20, 2019 13:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
performance ready for review This PR is ready for reviews.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants