Use trace_replayBlockTransactions API for faster tracing

[goodsoft]

Motivation

Currently indexer is using trace_replayTransaction JSONRPC method for fetching internal transaction.
This caused major slowdown of indexing and forced implementation of ad-hoc elimination rules (e.g. dropping simple token transfers).

Parity supports another API method: trace_replayBlockTransactions, which fetches internal transactions for the entire block.
Some utterly unscientific tests showed up to 20% speed boost in comparison with current approach, with the added bonus of fetching all transactions, including previously eliminated ones.

Also, previously realtime block fetcher needed to check, whether any particular transaction was a contract, which involved sending multiple eth_getCode API requests, which further slowed down new block arrival.
Some more unscientific benchmarks showed that employing trace_replayBlockTransactions method the average time of a new block arrival into database (inserted_at - timestamp) went down from ~2m15s to ~0m50s.

Changelog

• Add internal_transactions_indexed_at field to blocks and update InternalTransactions import runner to handle it.
• Replace Chain.stream_transactions_with_unfetched_internal_transactions with Chain.stream_blocks_with_unfetched_internal_transactions.
• Change EthereumJSONRPC.fetch_internal_transactions to accept list of block numbers and use trace_replayBlockTransactions API method
• Replace this method with a stub for Geth, as it has a corresponding method, but can't recover association between internal transaction and normal transaction, and has 128 block limit anyway.
• Increase timeout for JSONRPC API requests to 10 minutes, as some blocks from Ethereum DDoS times caused requests to take 1m30s and more.
• Update realtime, catchup and internal transactions fetchers to use the aforementioned methods, remove unneeded checks for simple token transfers.
• Update tests for these changes, and also fix some inconsistencies in test data, which old code was agnostic of.

Upgrading

Migration is added for adding internal_transactions_indexed_at to blocks table.
Theoretically, it is possible to update this field with some temporary value for blocks, which only have simple token transfer transactions and already fetched transactions to speed up indexing of interesting transactions, and then reset it back to nil for a second pass to index all remaining simple transactions.

[coveralls]

Pull Request Test Coverage Report for Build fd3cbba8-187f-4b4d-97a1-cb68011e2249

• 37 of 47 (78.72%) changed or added relevant lines in 7 files are covered.
• 26 unchanged lines in 5 files lost coverage.
• Overall coverage remained the same at 82.44%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
apps/ethereum_jsonrpc/lib/ethereum_jsonrpc/parity.ex	10	11	90.91%
apps/explorer/lib/explorer/chain/import/runner/internal_transactions.ex	9	10	90.0%
apps/indexer/lib/indexer/block/realtime/fetcher.ex	3	4	75.0%
apps/indexer/lib/indexer/block/catchup/fetcher.ex	3	6	50.0%
apps/indexer/lib/indexer/internal_transaction/fetcher.ex	9	13	69.23%

Files with Coverage Reduction	New Missed Lines	%
apps/ethereum_jsonrpc/lib/ethereum_jsonrpc/http.ex	1	73.47%
apps/ethereum_jsonrpc/lib/ethereum_jsonrpc.ex	1	87.5%
apps/ethereum_jsonrpc/lib/ethereum_jsonrpc/parity.ex	2	83.33%
apps/indexer/lib/indexer/block/realtime/fetcher.ex	5	30.93%
apps/indexer/lib/indexer/internal_transaction/fetcher.ex	17	57.14%

Totals
Change from base Build 8b2c4541-5b86-4250-856b-723ea36d43fb:	0.0%
Covered Lines:	4216
Relevant Lines:	5114

---

💛 - Coveralls

[ayrat555]

@goodsoft some chains are exclusively using only geth (for example, SafeChain) because they leverage features that only geth has. SafeChain are using geth because it has clique consensus algorithm. is there any way that we can fall back to trace_replayTransaction if geth is used?

[goodsoft]

We can do it like this:

• Split EthereumJSONRPC.fetch_transactions into two methods: the old one accepting list of transactions and the new one accepting list of blocks.
• Implement only the old one for geth and only new one for parity
• Bring back Explorer.Chain.stream_transactions_with_unfetched_internal_transactions for old API
• Switch between those combinations of APIs in InternalTransactions.Fetcher and Block.Realtime.Fetcher

Though, it'll probably be better to split InternalTransactions.Fetcher into two different modules as well, as it will need to handle these APIs differently.

Does this sound adequate to you?

[ayrat555]

@goodsoft sounds reasonable

[goodsoft]

The decrease in coverage is mostly caused by the internal transaction fetching tests being disabled for Geth. While the same code was being used for both Parity and Geth, it was properly exercised, but now Parity has a separate logic.

I'll leave it as-is, as figuring out what's up with internal transactions in Geth gets out of the scope of this issue.

[vbaranov]

@goodsoft could you change it also for prod?

[zachdaniel]

Switching on the variant here seems a little fragile. It might be a good idea to make this an application configuration, so that we could say if Application.get_env(...) then it becomes a bit more clear. Then the configuration could be placed in the adapter specific configurations like config/parity.exs.

[goodsoft]

It would make sense if both variants supported both APIs, so we could choose one in configuration.
Otherwise, there's exactly one valid value for this setting for each variant and it doesn't make sense to make it configurable at this point.

[ayrat555]

@goodsoft looks good. but what about my previous comment?

[goodsoft]

@ayrat555 if you mean the one about having fallback for Geth, I have implemented it in the second commit.