Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Internal transactions fetcher stops indexing after some time #2105

Closed
APshenkin opened this issue Jun 5, 2019 · 23 comments · Fixed by #2346
Closed

Internal transactions fetcher stops indexing after some time #2105

APshenkin opened this issue Jun 5, 2019 · 23 comments · Fixed by #2346
Assignees
@pasqu4le
Labels
waiting on feedback Waiting for original opener to respond if bug is fixed

Comments

@APshenkin
Copy link
Contributor

After launching blockscout it starts to sync internal transactions, but after some hours it's stops syncing them. Currently all blocks are in sync in our instance, but internal transactions is stil in progress. With this simple SQL we monitor amount of non-synced internal transactions blocks:

SELECT COUNT(*) FROM public.blocks t
     WHERE internal_transactions_indexed_at ISNULL and consensus = true;

On start up number of blocks decrease, but after several hours it start only increasing (new blocks inserted)

Environment

  • Elixir & Erlang/OTP versions (elixir -version):
Erlang/OTP 21 [erts-10.3.5] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [hipe]

Elixir 1.8.1 (compiled with Erlang/OTP 21)

  • Operating System: Alpine Linux v3.9

  • Blockscout version: 1.3.13

Steps to reproduce

Run blockscout and wait

Expected behaviour

internal transactions sync completes

Actual behaviour

after several hours it stops syncing and amount of non synced blocks increase (new blocks inserted)
@vbaranov
Copy link
Member

vbaranov commented Jun 6, 2019

@APshenkin what is in error log? it may be fat blocks, that cannot be processed in the allotted time. You can try to increase the timeout for indexer depending on your variant
https://github.com/poanetwork/blockscout/blob/55fd323fbc535a4b8a202ce37e3bb5c29ba90ece/apps/indexer/config/prod/geth.exs
https://github.com/poanetwork/blockscout/blob/55fd323fbc535a4b8a202ce37e3bb5c29ba90ece/apps/indexer/config/prod/parity.exs

or/and to decrease batch_size for internal transactions fetcher

https://github.com/poanetwork/blockscout/blob/55fd323fbc535a4b8a202ce37e3bb5c29ba90ece/apps/indexer/lib/indexer/fetcher/internal_transaction.ex#L22-L23

Also, if you use geth node I highly recommend using 1.3.15+ release, because of improvements in internal transactions indexer there.

@vbaranov vbaranov added the waiting on feedback Waiting for original opener to respond if bug is fixed label Jun 6, 2019
@APshenkin
Copy link
Contributor Author

I use parity archive node.

There is nothing special that I can see in logs. BTW after blockscout restarting it continues internal tx sync

@APshenkin
Copy link
Contributor Author

APshenkin commented Jun 7, 2019 •
edited
Loading

Found something in logs. They may be not in right order, because we send them to kibana from stdout

Here are huge logs 
iex(1)> 2019-06-06T17:39:30.777 fetcher=internal_transaction count=1 [error] Task #PID<0.24830.19> started from Indexer.Fetcher.InternalTransaction terminating

** (Postgrex.Error) ERROR 23503 (foreign_key_violation) insert or update on table "internal_transactions" violates foreign key constraint "internal_transactions_transaction_hash_fkey"
<<76, 94, 28, 174, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 207, 149, 63, 69, 2, 159, 165, 240, 76, 14, 217, 7, 113, 0, 188, 114, 17, 87, 114, 208>>}, inserted_at: #DateTime<2019-06-06 17:39:30.521792Z>, output: %Explorer.Chain.Data{bytes: <<0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 31, 94, 94, 59, 0, 21, 204, 222, 61, 248, 210, 8>>}, to_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<3, 197, 19, 255, 189, 189, 163, 75, 74, 102, 179, 5, 47, 165, 224, 230, 96, 110, 83, 27>>}, trace_address: [12, 4, 5, 1, 0], transaction_hash: %Explorer.Chain.Hash{byte_count: 32, bytes: <<252, 185, 10, 71, 131, 89, 23, 117, 68, 238, 250, 40, 38, 6, 80, 78, 93, 151, 51, 61, 154, 230, 62, 56, 154, 241, 22, 105, 86, 97, 52, 54>>}, transaction_index: 137, type: :call, updated_at: #DateTime<2019-06-06 17:39:30.521792Z>, value: #Explorer.Chain.Wei<0>}, %{block_number: 7907050, call_type: :delegatecall, from_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<4, 171, 237, 162, 1, 133, 10, 192, 18, 65, 97, 240, 55, 239, 215, 12, 116, 221, 199, 76>>}, gas: #Decimal<30427>, gas_used: #Decimal<669>, index: 48, input: %Explorer.Chain.Data{bytes: <<76, 94, 28, 174, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 207, 149, 63, 69, 2, 159, 165, 240, 76, 14, 217, 7, 113, 0, 188, 114, 17, 87, 114, 208>>}, inserted_at: #DateTime<2019-06-06 17:39:30.521792Z>, output: %Explorer.Chain.Data{bytes: <<0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 31, 94, 94, 59, 0, 21, 204, 222, 61, 248, 210, 8>>}, to_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<3, 197, 19, 255, 189, 189, 163, 75, 74, 102, 179, 5, 47, 165, 224, 230, 96, 110, 83, 27>>}, trace_address: [12, 4, 5, 1, 1], transaction_hash: %Explorer.Chain.Hash{byte_count: 32, bytes: <<252, 185, 10, 71, 131, 89, 23, 117, 68, 238, 250, 40, 38, 6, 80, 78, 93, 151, 51, 61, 154, 230, 62, 56, 154, 241, 22, 105, 86, 97, 52, 54>>}, transaction_index: 137, type: :call, updated_at: #DateTime<2019-06-06 17:39:30.521792Z>, value: #Explorer.Chain.Wei<0>}, %{block_number: 7907050, call_type: :delegatecall, from_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<4, 171, 237, 162, 1, 133, 10, 192, 18, 65, 97, 240, 55, 239, 215, 12, 116, 221, 199, 76>>}, gas: #Decimal<27894>, gas_used: #Decimal<5896>, index: 49, input: %Explorer.Chain.Data{bytes: <<171, 81, 123, 79, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 207, 149, 63, 69, 2, 159, 165, 240, 76, 14, 217, 7, 113, 0, 188, 114, 17, 87, 114, 208, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 31, 94, 93, 165, 218, 73, 247, 35, 137, 217, 217, 252>>}, inserted_at: #DateTime<2019-06-06 17:39:30.521792Z>, output: %Explorer.Chain.Data{bytes: <<0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1>>}, to_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<3, 197, 19, 255, 189, 189, 163, 75, 74, 102, 179, 5, 47, 165, 224, 230, 96, 110, 83, 27>>}, trace_address: [12, 4, 5, 1, 2], transaction_hash: %Explorer.Chain.Hash{byte_count: 32, bytes: <<252, 185, 10, 71, 131, 89, 23, 117, 68, 238, 250, 40, 38, 6, 80, 78, 93, 151, 51, 61, 154, 230, 62, 56, 154, 241, 22, 105, 86, 97, 52, 54>>}, transaction_index: 137, type: :call, updated_at: #DateTime<2019-06-06 17:39:30.521792Z>, value: #Explorer.Chain.Wei<0>}, %{block_number: 7907050, call_type: :delegatecall, from_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<4, 171, 237, 162, 1, 133, 10, 192, 18, 65, 97, 240, 55, 239, 215, 12, 116, 221, 199, 76>>}, gas: #Decimal<20317>, gas_used: #Decimal<669>, index: 50, input: %Explorer.Chain.Data{bytes: <<76, 94, 28, 174, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 92, 119, 140, 144, 117, 251, 131, 202, 223, 233, 69, 110, 243, 195, 186, 205, 59, 222, 254, 113>>}, inserted_at: #DateTime<2019-06-06 17:39:30.521792Z>, output: %Explorer.Chain.Data{bytes: <<0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 46, 214, 208, 61, 204, 194, 130, 200, 159, 205, 132>>}, to_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<3, 197, 19, 255, 189, 189, 163, 75, 74, 102, 179, 5, 47, 165, 224, 230, 96, 110, 83, 27>>}, trace_address: [12, 4, 5, 1, 3], transaction_hash: %Explorer.Chain.Hash{byte_count: 32, bytes: <<252, 185, 10, 71, 131, 89, 23, 117, 68, 238, 250, 40, 38, 6, 80, 78, 93, 151, 51, 61, 154, 230, 62, 56, 154, 241, 22, 105, 86, 97, 52, 54>>}, transaction_index: 137, type: :call, updated_at: #DateTime<2019-06-06 17:39:30.521792Z>, value: #Explorer.Chain.Wei<0>}, %{block_number: 7907050, call_type: :delegatecall, from_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<4, 171, 237, 162, 1, 133, 10, 192, 18, 65, 97, 240, 55, 239, 215, 12, 116, 221, 199, 76>>}, gas: #Decimal<17782>, gas_used: #Decimal<5896>, index: 51, input: %Explorer.Chain.Data{bytes: <<171, 81, 123, 79, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 92, 119, 140, 144, 117, 251, 131, 202, 223, 233, 69, 110, 243, 195, 186, 205, 59, 222, 254, 113, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 46, 215, 101, 99, 152, 152, 61, 124, 190, 197, 144>>}, inserted_at: #DateTime<2019-06-06 17:39:30.521792Z>, output: %Explorer.Chain.Data{bytes: <<0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1>>}, to_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<3, 197, 19, 255, 189, 189, 163, 75, 74, 102, 179, 5, 47, 165, 224, 230, 96, 110, 83, 27>>}, trace_address: [12, 4, 5, 1, 4], transaction_hash: %Explorer.Chain.Hash{byte_count: 32, bytes: <<252, 185, 10, 71, 131, 89, 23, 117, 68, 238, 250, 40, 38, 6, 80, 78, 93, 151, 51, 61, 154, 230, 62, 56, 154, 241, 22, 105, 86, 97, 52, 54>>}, transaction_index: 137, type: :call, updated_at: #DateTime<2019-06-06 17:39:30.521792Z>, value: #Explorer.Chain.Wei<0>}, %{block_number: 7907050, call_type: :call, from_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<106, 0, 209, 189, 2, 157, 183, 44, 204, 167, 215, 40, 243, 228, 109, 239, 231, 153, 125, 44>>}, gas: #Decimal<52100>, gas_used: #Decimal<14422>, index: 0, input: %Explorer.Chain.Data{bytes: <<169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 63, 92, 229, 251, 254, 62, 154, 243, 151, 29, 216, 51, 210, 107, 169, 181, 201, 54, 240, 190, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 28, 6, 208, 248, 209, 12, 244, 0, 0>>}, inserted_at: #DateTime<2019-06-06 17:39:30.521792Z>, output: %Explorer.Chain.Data{bytes: <<0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1>>}, to_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<250, 5, 167, 63, 254, 120, 239, 143, 26, 115, 148, 115, 228, 98, 197, 75, 174, 101, 103, 217>>}, trace_address: [], transaction_hash: %Explorer.Chain.Hash{byte_count: 32, bytes: <<252, 190, 110, 130, 54, 191, 116, 117, 204, 196, 10, 206, 50, 104, 122, 150, 69, 187, 59, 106, 22, 88, 84, 87, 118, 134, 58, 93, 22, 114, 4, 186>>}, transaction_index: 39, type: :call, updated_at: #DateTime<2019-06-06 17:39:30.521792Z>, value: #Explorer.Chain.Wei<0>}, %{block_number: 7907050, call_type: :call, from_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<148, 102, 25, 144, 112, 20, 242, 57, 244, 75, 150, 45, 51, 10, 151, 11, 69, 118, 30, 175>>}, gas: #Decimal<0>, gas_used: #Decimal<0>, index: 0, input: %Explorer.Chain.Data{bytes: ""}, inserted_at: #DateTime<2019-06-06 17:39:30.521792Z>, output: %Explorer.Chain.Data{bytes: ""}, to_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<108, 131, 157, 113, 204, 105, 22, 22, 18, 134, 180, 140, 184, 53, 96, 188, 92, 205, 77, 57>>}, trace_address: [], transaction_hash: %Explorer.Chain.Hash{byte_count: 32, bytes: <<252, 212, 135, 206, 126, 154, 40, 128, 219, 155, 228, 101, 116, 45, 68, 252, 102, 150, 168, 88, 92, 228, 89, 28, 197, 219, 86, 145, 207, 225, 21, 85>>}, transaction_index: 146, type: :call, updated_at: #DateTime<2019-06-06 17:39:30.521792Z>, value: #Explorer.Chain.Wei<2533576000000000>}, %{block_number: 7907050, call_type: :call, from_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<60, 156, 83, 13, 104, 74, 153, 204, 57, 56, 140, 100, 79, 33, 235, 238, 182, 28, 40, 57>>}, gas: #Decimal<121496>, gas_used: #Decimal<53628>, index: 0, input: %Explorer.Chain.Data{bytes: <<94, 131, 180, 99, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 58, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 120, 167, 176, 255, 60, 162, 53, 89, 84, 11, 125, 91, 169, 108, 68, 232, 107, 3, 146, 129, 185, 235, 43, 163, 137, 4, 254, 104, 117, 81, 228, 179, 141, 105, 85, 50, 123, 120, 40, 174, 226, 28, 77, 91, 182, 200, 111, 96, 143, 232, 106, 161, 43, 166, 82, 115, 168, 139, 228, 61, 67, 224, 48, 128, 44, 182, 62, 188, 68, 147, 244, 21, 216, 161, 240, 93, 11, 190, 73, 60, 81, 165, 9, 169, 175, 184, 130, 140, 12, 33, 82, 181, 22, 9, 35, 72, 103, 42, 249>>}, inserted_at: #DateTime<2019-06-06 17:39:30.521792Z>, output: %Explorer.Chain.Data{bytes: ""}, to_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<209, 206, 238, 238, 232, 63, 139, 207, 59, 237, 173, 67, 114, 2, 182, 21, 78, 159, 84, 5>>}, trace_address: [], transaction_hash: %Explorer.Chain.Hash{byte_count: 32, bytes: <<253, 67, 230, 191, 114, 210, 176, 113, 65, 130, 119, 34, 101, 236, 64, 101, 94, 39, 142, 185, 135, 50, 205, 129, 184, 77, 119, 18, 79, 1, 159, 222>>}, transaction_index: 8, type: :call, updated_at: #DateTime<2019-06-06 17:39:30.521792Z>, value: #Explorer.Chain.Wei<220000000000000000>}, %{block_number: 7907050, call_type: :call, from_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<89, 6, 197, 218, 115, 58, 180, 242, 230, 105, 181, 18, 53, 152, 175, 54, 173, 195, 53, 12>>}, gas: #Decimal<92768>, gas_used: #Decimal<14867>, index: 0, input: %Explorer.Chain.Data{bytes: <<169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 69, 113, 20, 55, 232, 1, 133, 225, 210, 0, 136, 93, 241, 203, 192, 220, 186, 216, 130, 227, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 6, 98, 54, 203, 151, 231, 112, 0>>}, inserted_at: #DateTime<2019-06-06 17:39:30.521792Z>, output: %Explorer.Chain.Data{bytes: <<0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1>>}, to_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<23, 75, 250, 102, 0, 191, 144, 200, 133, 199, 192, 28, 112, 49, 56, 158, 209, 70, 26, 185>>}, trace_address: [], transaction_hash: %Explorer.Chain.Hash{byte_count: 32, bytes: <<254, 100, 142, 42, 211, 84, 79, 106, 123, 4, 144, 117, 235, 186, 144, 21, 205, 30, 60, 27, 156, 48, 121, 229, 200, 72, 95, 98, 8, 4, 18, 204>>}, transaction_index: 43, type: :call, updated_at: #DateTime<2019-06-06 17:39:30.521792Z>, value: #Explorer.Chain.Wei<0>}, %{block_number: 7907050, call_type: :call, error: "Reverted", from_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<247, 158, 243, 204, 23, 136, 6, 26, 193, 3, 247, 195, 245, 37, 59, 204, 102, 85, 177, 0>>}, gas: #Decimal<178024>, index: 0, input: %Explorer.Chain.Data{bytes: <<228, 132, 155, 50, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4, 236, 150, 247, 87, 244, 36, 65, 176>>}, inserted_at: #DateTime<2019-06-06 17:39:30.521792Z>, to_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<189, 115, 230, 117, 225, 250, 61, 96, 163, 2, 199, 151, 223, 92, 130, 229, 88, 218, 124, 225>>}, trace_address: [], transaction_hash: %Explorer.Chain.Hash{byte_count: 32, bytes: <<254, 189, 233, 19, 45, 27, 92, 0, 212, 45, 36, 119, 241, 176, 217, 54, 239, 26, 149, 249, 235, 223, 82, 202, 109, 105, 216, 60, 125, 239, 98, 67>>}, transaction_index: 138, type: :call, updated_at: #DateTime<2019-06-06 17:39:30.521792Z>, value: #Explorer.Chain.Wei<0>}, %{block_number: 7907050, call_type: :call, from_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<128, 240, 156, 161, 247, 52, 8, 235, 63, 155, 230, 59, 72, 73, 11, 103, 96, 124, 202, 245>>}, gas: #Decimal<50376>, gas_used: #Decimal<13528>, index: 0, input: %Explorer.Chain.Data{bytes: <<169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 63, 92, 229, 251, 254, 62, 154, 243, 151, 29, 216, 51, 210, 107, 169, 181, 201, 54, 240, 190, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 8, 150, 125, 175, 62, 102, 174, 204, 0, 0>>}, inserted_at: #DateTime<2019-06-06 17:39:30.521792Z>, output: %Explorer.Chain.Data{bytes: <<0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1>>}, to_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<167, 68, 118, 68, 49, 25, 169, 66, 222, 73, 133, 144, 254, 31, 36, 84, 215, 212, 172, 13>>}, trace_address: [], transaction_hash: %Explorer.Chain.Hash{byte_count: 32, bytes: <<255, 139, 35, 74, 14, 118, 198, 12, 252, 172, 220, 252, 79, 0, 221, 11, 227, 43, 141, 117, 140, 92, 138, 235, 216, 192, 63, 210, 179, 146, 49, 166>>}, transaction_index: 57, type: :call, updated_at: #DateTime<2019-06-06 17:39:30.521792Z>, value: #Explorer.Chain.Wei<0>}]


[conflict_target: [:transaction_hash, :index], on_conflict: #Ecto.Query<from i0 in Explorer.Chain.InternalTransaction, where: fragment("(EXCLUDED.call_type, EXCLUDED.created_contract_address_hash, EXCLUDED.created_contract_code, EXCLUDED.error, EXCLUDED.from_address_hash, EXCLUDED.gas, EXCLUDED.gas_used, EXCLUDED.init, EXCLUDED.input, EXCLUDED.output, EXCLUDED.to_address_hash, EXCLUDED.trace_address, EXCLUDED.transaction_index, EXCLUDED.type, EXCLUDED.value) IS DISTINCT FROM (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)", i0.call_type, i0.created_contract_address_hash, i0.created_contract_code, i0.error, i0.from_address_hash, i0.gas, i0.gas_used, i0.init, i0.input, i0.output, i0.to_address_hash, i0.trace_address, i0.transaction_index, i0.type, i0.value), update: [set: [block_number: fragment("EXCLUDED.block_number"), call_type: fragment("EXCLUDED.call_type"), created_contract_address_hash: fragment("EXCLUDED.created_contract_address_hash"), created_contract_code: fragment("EXCLUDED.created_contract_code"), error: fragment("EXCLUDED.error"), from_address_hash: fragment("EXCLUDED.from_address_hash"), gas: fragment("EXCLUDED.gas"), gas_used: fragment("EXCLUDED.gas_used"), init: fragment("EXCLUDED.init"), input: fragment("EXCLUDED.input"), output: fragment("EXCLUDED.output"), to_address_hash: fragment("EXCLUDED.to_address_hash"), trace_address: fragment("EXCLUDED.trace_address"), transaction_index: fragment("EXCLUDED.transaction_index"), type: fragment("EXCLUDED.type"), value: fragment("EXCLUDED.value"), inserted_at: fragment("LEAST(?, EXCLUDED.inserted_at)", i0.inserted_at), updated_at: fragment("GREATEST(?, EXCLUDED.updated_at)", i0.updated_at)]]>, returning: [:transaction_hash, :index], timeout: 60000, timestamps: %{inserted_at: #DateTime<2019-06-06 17:39:30.521792Z>, updated_at: #DateTime<2019-06-06 17:39:30.521792Z>}]
--



, %{block_number: 7907050, call_type: :call, from_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<228, 54, 117, 30, 107, 127, 254, 31, 57, 18, 47, 45, 55, 241, 50, 74, 148, 208, 216, 117>>}, gas: #Decimal<51706>, gas_used: #Decimal<14257>, index: 0, input: %Explorer.Chain.Data{bytes: <<169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 63, 92, 229, 251, 254, 62, 154, 243, 151, 29, 216, 51, 210, 107, 169, 181, 201, 54, 240, 190, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 169, 221, 93, 154, 192>>}, inserted_at: #DateTime<2019-06-06 17:39:30.521792Z>, output: %Explorer.Chain.Data{bytes: <<0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1>>}, to_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<53, 151, 191, 213, 51, 169, 156, 154, 160, 131, 88, 123, 7, 68, 52, 230, 30, 176, 162, 88>>}, trace_address: [], transaction_hash: %Explorer.Chain.Hash{byte_count: 32, bytes: <<136, 6, 7, 114, 128, 214, 49, 19, 248, 25, 123, 0, 112, 250, 9, 110, 202, 203, 134, 196, 58, 168, 74, 187, 242, 192, 104, 57, 3, 178, 72, 33>>}, transaction_index: 9, type: :call, updated_at: #DateTime<2019-06-06 17:39:30.521792Z>, value: #Explorer.Chain.Wei<0>}, %{block_number: 7907050, call_type: :call, from_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<103, 33, 145, 174, 230, 229, 166, 240, 64, 81, 56, 122, 30, 53, 176, 155, 35, 127, 145, 155>>}, gas: #Decimal<51890>, gas_used: #Decimal<14285>, index: 0, input: %Explorer.Chain.Data{bytes: <<169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 63, 92, 229, 251, 254, 62, 154, 243, 151, 29, 216, 51, 210, 107, 169, 181, 201, 54, 240, 190, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 125, 179, 10, 0, 135, 28, 240, 59, 144, 0>>}, inserted_at: #DateTime<2019-06-06 17:39:30.521792Z>, output: %Explorer.Chain.Data{bytes: <<0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1>>}, to_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<108, 110, 229, 227, 29, 130, 141, 226, 65, 40, 43, 150, 6, 200, 233, 142, 164, 133, 38, 226>>}, trace_address: [], transaction_hash: %Explorer.Chain.Hash{byte_count: 32, bytes: <<136, 29, 161, 81, 201, 43, 57, 173, 218, 249, 5, 207, 141, 188, 105, 75, 112, 126, 58, 52, 43, 184, 218, 59, 58, 113, 93, 15, 82, 54, 210, 245>>}, transaction_index: 50, type: :call, updated_at: #DateTime<2019-06-06 17:39:30.521792Z>, value: #Explorer.Chain.Wei<0>}, %{block_number: 7907050, call_type: :call, from_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<100, 133, 252, 119, 190, 33, 134, 252, 96, 254, 234, 56, 208, 239, 67, 49, 216, 64, 75, 96>>}, gas: #Decimal<66680>, gas_used: #Decimal<14054>, index: 0, input: %Explorer.Chain.Data{bytes: <<169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 62, 198, 115, 38, 118, 219, 121, 150, 193, 179, 78, 100, 176, 80, 63, 148, 16, 37, 203, 99, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 100, 183, 140, 179, 233, 66, 196, 0, 0>>}, inserted_at: #DateTime<2019-06-06 17:39:30.521792Z>, output: %Explorer.Chain.Data{bytes: <<0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1>>}, to_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<254, 57, 230, 163, 42, 205, 42, 247, 149, 92, 179, 212, 6, 186, 43, 85, 201, 1, 242, 71>>}, trace_address: [], transaction_hash: %Explorer.Chain.Hash{byte_count: 32, bytes: <<136, 37, 228, 88, 11, 204, 30, 179, 71, 237, 109, 195, 179, 139, 145, 235, 201, 121, 148, 236, 111, 156, 84, 73, 251, 86, 6, 132, 161, 131, 5, 222>>}, transaction_index: 2, type: :call, updated_at: #DateTime<2019-06-06 17:39:30.521792Z>, value: #Explorer.Chain.Wei<0>}, %{block_number: 7907050, call_type: :call, from_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<0, 112, 98, 138, 176, 113, 61, 204, 122, 64, 142, 222, 129, 223, 29, 18, 20, 210, 178, 165>>}, gas: #Decimal<0>, gas_used: #Decimal<0>, index: 0, input: %Explorer.Chain.Data{bytes: ""}, inserted_at: #DateTime<2019-06-06 17:39:30.521792Z>, output: %Explorer.Chain.Data{bytes: ""}, to_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<233, 92, 209, 90, 231, 191, 227, 108, 198, 241, 48, 9, 14, 240, 220, 164, 143, 94, 199, 9>>}, trace_address: [], transaction_hash: %Explorer.Chain.Hash{byte_count: 32, bytes: <<137, 236, 25, 21, 218, 76, 90, 196, 197, 187, 253, 14, 152, 172, 94, 15, 151, 176, 255, 10, 44, 109, 133, 26, 47, 166, 34, 96, 113, 202, 64, 43>>}, transaction_index: 87, type: :call, updated_at: #DateTime<2019-06-06 17:39:30.521792Z>, value: #Explorer.Chain.Wei<874726260000000020>}, %{block_number: 7907050, call_type: :call, from_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<25, 224, 70, 106, 24, 229, 163, 117, 98, 27, 50, 124, 231, 129, 198, 75, 132, 68, 58, 27>>}, gas: #Decimal<326616>, gas_used: #Decimal<14653>, index: 0, input: %Explorer.Chain.Data{bytes: <<169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 166, 129, 62, 60, 236, 63, 0, 44, 195, 15, 174, 183, 130, 191, 11, 11, 147, 240, 13, 152, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 45, 243, 14, 234, 47, 67, 59, 229, 31>>}, inserted_at: #DateTime<2019-06-06 17:39:30.521792Z>, output: %Explorer.Chain.Data{bytes: <<0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1>>}, to_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<161, 92, 126, 190, 31, 7, 202, 246, 191, 240, 151, 216, 165, 137, 251, 138, 196, 154, 229, 179>>}, trace_address: [], transaction_hash: %Explorer.Chain.Hash{byte_count: 32, bytes: <<146, 63, 121, 116, 150, 206, 106, 228, 171, 126, 30, 67, 254, 214, 155, 149, 179, 214, 172, 59, 221, 1, 198, 4, 177, 131, 59, 17, 156, 248, 24, 64>>}, transaction_index: 101, type: :call, updated_at: #DateTime<2019-06-06 17:39:30.521792Z>, value: #Explorer.Chain.Wei<0>}, %{block_number: 7907050, call_type: :call, from_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<0, 69, 190, 187, 146, 72, 71, 148, 134, 38, 190, 58, 22, 41, 30, 205, 191, 238, 86, 52>>}, gas: #Decimal<69000>, gas_used: #Decimal<62826>, index: 0, input: %Explorer.Chain.Data{bytes: ""}, inserted_at: #DateTime<2019-06-06 17:39:30.521792Z>, output: %Explorer.Chain.Data{bytes: ""}, to_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<29, 70, 40, 17, 86, 44, 195, 103, 83, 117, 166, 208, 12, 22, 24, 163, 112, 194, 120, 54>>}, trace_address: [], transaction_hash: %Explorer.Chain.Hash{byte_count: 32, bytes: <<148, 225, 238, 63, 99, 215, 236, 72, 34, 225, 166, 147, 211, 21, 133, 204, 70, 158, 252, 233, 205, 214, 120, 201, 57, 133, 131, 245, 184, 131, 39, 61>>}, transaction_index: 170, type: :call, updated_at: #DateTime<2019-06-06 17:39:30.521792Z>, value: #Explorer.Chain.Wei<0>}, %{block_number: 7907050, call_type: :call, from_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<173, 185, 35, 107, 31, 119, 171, 119, 199, 102, 5, 108, 154, 241, 152, 231, 165, 129, 87, 137>>}, gas: #Decimal<93280>, gas_used: #Decimal<14867>, index: 0, input: %Explorer.Chain.Data{bytes: <<169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 203, 6, 236, 177, 226, 213, 206, 62, 176, 137, 128, 169, 212, 162, 136, 188, 109, 19, 132, 17, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 135, 223, 183, 63, 9, 28, 188, 0>>}, inserted_at: #DateTime<2019-06-06 17:39:30.521792Z>, output: %Explorer.Chain.Data{bytes: <<0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1>>}, to_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<23, 75, 250, 102, 0, 191, 144, 200, 133, 199, 192, 28, 112, 49, 56, 158, 209, 70, 26, 185>>}, trace_address: [], transaction_hash: %Explorer.Chain.Hash{byte_count: 32, bytes: <<150, 84, 120, 243, 35, 71, 133, 3, 183, 55, 218, 109, 179, 189, 16, 156, 124, 20, 11, 218, 211, 219, 55, 64, 153, 45, 60, 169, 181, 116, 229, 120>>}, transaction_index: 10, type: :call, updated_at: #DateTime<2019-06-06 17:39:30.521792Z>, value: #Explorer.Chain.Wei<0>}, %{block_number: 7907050, call_type: :call, from_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<25, 224, 70, 106, 24, 229, 163, 117, 98, 27, 50, 124, 231, 129, 198, 75, 132, 68, 58, 27>>}, gas: #Decimal<326552>, gas_used: #Decimal<14653>, index: 0, input: %Explorer.Chain.Data{bytes: <<169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 62, 158, 50, 144, 80, 10, 93, 138, 32, 45, 183, 52, 17, 244, 13, 58, 235, 240, 46, 149, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 46, 250, 173, 167, 252, 81, 147, 200, 125>>}, inserted_at: #DateTime<2019-06-06 17:39:30.521792Z>, output: %Explorer.Chain.Data{bytes: <<0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1>>}, to_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<161, 92, 126, 190, 31, 7, 202, 246, 191, 240, 151, 216, 165, 137, 251, 138, 196, 154, 229, 179>>}, trace_address: [], transaction_hash: %Explorer.Chain.Hash{byte_count: 32, bytes: <<150, 176, 3, 48, 63, 139, 244, 78, 44, 132, 22, 17, 55, 184, 102, 114, 140, 121, 11, 144, 160, 118, 33, 24, 206, 210, 52, 112, 202, 61, 165, 166>>}, transaction_index: 96, type: :call, updated_at: #DateTime<2019-06-06 17:39:30.521792Z>, value: #Explorer.Chain.Wei<0>}, %{block_number: 7907050, call_type: :call, from_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<147, 79, 107, 240, 249, 112, 0, 12, 133, 79, 142, 143, 18, 153, 214, 128, 211, 13, 109, 9>>}, gas: #Decimal<0>, gas_used: #Decimal<0>, index: 0, input: %Explorer.Chain.Data{bytes: ""}, inserted_at: #DateTime<2019-06-06 17:39:30.521792Z>, output: %Explorer.Chain.Data{bytes: ""}, to_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<108, 131, 157, 113, 204, 105, 22, 22, 18, 134, 180, 140, 184, 53, 96, 188, 92, 205, 77, 57>>}, trace_address: [], transaction_hash: %Explorer.Chain.Hash{byte_count: 32, bytes: <<151, 91, 77, 219, 16, 231, 11, 21, 224, 110, 64, 174, 151, 98, 193, 205, 205, 253, 60, 45, 204, 155, 95, 97, 105, 145, 167, 28, 120, 9, 5, 247>>}, transaction_index: 149, type: :call, updated_at: #DateTime<2019-06-06 17:39:30.521792Z>, value: #Explorer.Chain.Wei<2532190000000000>}, %{block_number: 7907050, call_type: :call, from_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<4, 52, 143, 6, 216, 225, 24, 235, 167, 240, 240, 67, 197, 182, 97, 29, 78, 178, 173, 19>>}, gas: #Decimal<93024>, gas_used: #Decimal<14867>, index: 0, input: %Explorer.Chain.Data{bytes: <<169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 162, 235, 185, 60, 57, 195, 231, 72, 156, 56, 132, 111, 180, 220, 204, 31, 189, 218, 176, 49, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 10, 251, 55, 215, 232, 252, 80, 0>>}, inserted_at: #DateTime<2019-06-06 17:39:30.521792Z>, output: %Explorer.Chain.Data{bytes: <<0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1>>}, to_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<23, 75, 250, 102, 0, 191, 144, 200, 133, 199, 192, 28, 112, 49, 56, 158, 209, 70, 26, 185>>}, trace_address: [], transaction_hash: %Explorer.Chain.Hash{byte_count: 32, bytes: <<153, 206, 189, 92, 211, 141, 236, 84, 147, 64, 23, 31, 42, 170, 23, 98, 40, 196, 129, 42, 23, 24, 140, 28, 109, 167, 112, 253, 179, 253, 145, 39>>}, transaction_index: 22, type: :call, updated_at: #DateTime<2019-06-06 17:39:30.521792Z>, value: #Explorer.Chain.Wei<0>}, %{block_number: 7907050, call_type: :call, error: "Reverted", from_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<28, 70, 46, 13, 146, 192, 8, 219, 53, 182, 21, 0, 64, 96, 161, 205, 209, 30, 42, 50>>}, gas: #Decimal<69000>, index: 0, input: %Explorer.Chain.Data{bytes: ""}, inserted_at: #DateTime<2019-06-06 17:39:30.521792Z>, to_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<29, 70, 40, 17, 86, 44, 195, 103, 83, 117, 166, 208, 12, 22, 24, 163, 112, 194, 120, 54>>}, trace_address: [], transaction_hash: %Explorer.Chain.Hash{byte_count: 32, bytes: <<159, 115, 247, 250, 102, 125, 215, 112, 152, 12, 142, 220, 130, 183, 96, 219, 126, 238, 22, 6, 75, 183, 154, 229, 101, 199, 112, 206, 144, 194, 73, 23>>}, transaction_index: 171, type: :call, updated_at: #DateTime<2019-06-06 17:39:30.521792Z>, value: #Explorer.Chain.Wei<0>}, %{block_number: 7907050, call_type: :call, from_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<25, 224, 70, 106, 24, 229, 163, 117, 98, 27, 50, 124, 231, 129, 198, 75, 132, 68, 58, 27>>}, gas: #Decimal<326552>, gas_used: #Decimal<14653>, index: 0, input: %Explorer.Chain.Data{bytes: <<169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 222, 172, 139, 39, 6, 33, 169, 167, 76, 184, 52, 113, 216, 115, 191, 88, 7, 51, 79, 106, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 46, 252, 17, 50, 187, 145, 161, 177, 34>>}, inserted_at: #DateTime<2019-06-06 17:39:30.521792Z>, output: %Explorer.Chain.Data{bytes: <<0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1>>}, to_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<161, 92, 126, 190, 31, 7, 202, 246, 191, 240, 151, 216, 165, 137, 251, 138, 196, 154, 229, 179>>}, trace_address: [], transaction_hash: %Explorer.Chain.Hash{byte_count: 32, bytes: <<161, 80, 112, 21, 166, 115, 65, 94, 6, 76, 225, 64, 2, 221, 182, 10, 58, 254, 24, 126, 232, 103, 136, 103, 64, 103, 121, 243, 141, 71, 14, 189>>}, transaction_index: 95, type: :call, updated_at: #DateTime<2019-06-06 17:39:30.521792Z>, value: #Explorer.Chain.Wei<0>}, %{block_number: 7907050, call_type: :call, from_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<27, 142, 21, 34, 159, 5, 43, 66, 193, 189, 255, 74, 125, 19, 31, 166, 0, 68, 121, 114>>}, gas: #Decimal<33896>, gas_used: #Decimal<14697>, index: 0, input: %Explorer.Chain.Data{bytes: <<163, 157, 113, 109>>}, inserted_at: #DateTime<2019-06-06 17:39:30.521792Z>, output: %Explorer.Chain.Data{bytes: ""}, to_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<209, 10, 147, 67, 160, 152, 28, 205, 1, 176, 99, 3, 162, 205, 136, 224, 137, 191, 29, 198>>}, trace_address: [], transaction_hash: %Explorer.Chain.Hash{byte_count: 32, bytes: <<161, 97, 21, 208, 114, 115, 228, 158, 116, 166, 112, 208, 100, 182, 132, 196, 22, 98, 168, 193, 146, 200, 233, 177, 235, 15, 194, 145, 15, 26, 7, 53>>}, transaction_index: 160, type: :call, updated_at: #DateTime<2019-06-06 17:39:30.521792Z>, value: #Explorer.Chain.Wei<0>}, %{block_number: 7907050, call_type: :call, from_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<209, 10, 147, 67, 160, 152, 28, 205, 1, 176, 99, 3, 162, 205, 136, 224, 137, 191, 29, 198>>}, gas: #Decimal<2300>, gas_used: #Decimal<0>, index: 1, input: %Explorer.Chain.Data{bytes: ""}, inserted_at: #DateTime<2019-06-06 17:39:30.521792Z>, output: %Explorer.Chain.Data{bytes: ""}, to_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<27, 142, 21, 34, 159, 5, 43, 66, 193, 189, 255, 74, 125, 19, 31, 166, 0, 68, 121, 114>>}, trace_address: [0], transaction_hash: %Explorer.Chain.Hash{byte_count: 32, bytes: <<161, 97, 21, 208, 114, 115, 228, 158, 116, 166, 112, 208, 100, 182, 132, 196, 22, 98, 168, 193, 146, 200, 233, 177, 235, 15, 194, 145, 15, 26, 7, 53>>}, transaction_index: 160, type: :call, updated_at: #DateTime<2019-06-06 17:39:30.521792Z>, value: #Explorer.Chain.Wei<3831159926113873>}, %{block_number: 7907050, call_type: :call, from_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<126, 209, 228, 105, 252, 179, 238, 25, 192, 54, 109, 130, 158, 41, 20, 81, 190, 99, 142, 89>>}, gas: #Decimal<42000>, gas_used: #Decimal<0>, index: 0, input: %Explorer.Chain.Data{bytes: ""}, inserted_at: #DateTime<2019-06-06 17:39:30.521792Z>, output: %Explorer.Chain.Data{bytes: ""}, to_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<73, 64, 119, 108, 44, 143, 247, 229, 223, 114, 185, 115, 111, 76, 189, 24, 94, 73, 0, 185>>}, trace_address: [], transaction_hash: %Explorer.Chain.Hash{byte_count: 32, bytes: <<163, 254, 146, 34, 88, 225, 105, 102, 103, 194, 140, 207, 42, 168, 154, 125, 241, 57, 160, 95, 233, 207, 58, 196, 75, 63, 3, 48, 178, 150, 93, 131>>}, transaction_index: 113, type: :call, updated_at: #DateTime<2019-06-06 17:39:30.521792Z>, value: #Explorer.Chain.Wei<280000000
--


-06-06 17:39:30.521792Z>, value: #Explorer.Chain.Wei<0>}, %{block_number: 7907050, call_type: :call, from_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<151, 100, 244, 22, 159, 179, 247, 51, 73, 154, 129, 31, 157, 110, 255, 70, 101, 61, 92, 188>>}, gas: #Decimal<0>, gas_used: #Decimal<0>, index: 0, input: %Explorer.Chain.Data{bytes: ""}, inserted_at: #DateTime<2019-06-06 17:39:30.521792Z>, output: %Explorer.Chain.Data{bytes: ""}, to_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<234, 41, 84, 183, 179, 123, 228, 151, 215, 210, 130, 56, 236, 82, 116, 37, 74, 196, 48, 192>>}, trace_address: [], transaction_hash: %Explorer.Chain.Hash{byte_count: 32, bytes: <<70, 253, 5, 250, 85, 86, 136, 196, 80, 95, 216, 150, 199, 248, 143, 121, 171, 129, 149, 176, 147, 129, 122, 68, 80, 214, 38, 169, 147, 225, 209, 32>>}, transaction_index: 140, type: :call, updated_at: #DateTime<2019-06-06 17:39:30.521792Z>, value: #Explorer.Chain.Wei<700000000000000>}, %{block_number: 7907050, call_type: :call, from_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<108, 77, 72, 181, 132, 214, 114, 181, 70, 81, 116, 80, 236, 161, 151, 72, 4, 24, 147, 49>>}, gas: #Decimal<69000>, gas_used: #Decimal<62826>, index: 0, input: %Explorer.Chain.Data{bytes: ""}, inserted_at: #DateTime<2019-06-06 17:39:30.521792Z>, output: %Explorer.Chain.Data{bytes: ""}, to_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<29, 70, 40, 17, 86, 44, 195, 103, 83, 117, 166, 208, 12, 22, 24, 163, 112, 194, 120, 54>>}, trace_address: [], transaction_hash: %Explorer.Chain.Hash{byte_count: 32, bytes: <<71, 20, 255, 125, 4, 198, 37, 74, 233, 127, 78, 54, 76, 121, 29, 10, 121, 152, 116, 195, 212, 144, 169, 109, 188, 152, 133, 158, 239, 220, 116, 63>>}, transaction_index: 169, type: :call, updated_at: #DateTime<2019-06-06 17:39:30.521792Z>, value: #Explorer.Chain.Wei<0>}, %{block_number: 7907050, call_type: :call, from_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<163, 93, 25, 251, 38, 230, 84, 68, 117, 135, 144, 233, 70, 199, 75, 131, 235, 173, 108, 225>>}, gas: #Decimal<75808>, gas_used: #Decimal<28772>, index: 0, input: %Explorer.Chain.Data{bytes: <<169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 166, 246, 17, 157, 225, 226, 227, 103, 206, 41, 222, 104, 198, 53, 0, 30, 186, 105, 1, 243, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 35, 171, 149, 153, 196, 63, 8, 0, 0>>}, inserted_at: #DateTime<2019-06-06 17:39:30.521792Z>, output: %Explorer.Chain.Data{bytes: ""}, to_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<26, 249, 120, 36, 167, 204, 211, 150, 59, 147, 133, 227, 126, 203, 244, 78, 206, 12, 115, 228>>}, trace_address: [], transaction_hash: %Explorer.Chain.Hash{byte_count: 32, bytes: <<72, 234, 30, 58, 153, 141, 18, 246, 154, 37, 209, 48, 222, 63, 187, 19, 213, 229, 179, 26, 117, 135, 89, 89, 14, 48, 5, 208, 1, 8, 189, 192>>}, transaction_index: 132, type: :call, updated_at: #DateTime<2019-06-06 17:39:30.521792Z>, value: #Explorer.Chain.Wei<0>}, %{block_number: 7907050, call_type: :call, from_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<25, 224, 70, 106, 24, 229, 163, 117, 98, 27, 50, 124, 231, 129, 198, 75, 132, 68, 58, 27>>}, gas: #Decimal<326552>, gas_used: #Decimal<14653>, index: 0, input: %Explorer.Chain.Data{bytes: <<169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 158, 33, 152, 181, 116, 70, 90, 188, 120, 78, 153, 43, 122, 36, 173, 17, 13, 243, 103, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 46, 235, 92, 150, 237, 230, 131, 92, 151>>}, inserted_at: #DateTime<2019-06-06 17:39:30.521792Z>, output: %Explorer.Chain.Data{bytes: <<0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1>>}, to_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<161, 92, 126, 190, 31, 7, 202, 246, 191, 240, 151, 216, 165, 137, 251, 138, 196, 154, 229, 179>>}, trace_address: [], transaction_hash: %Explorer.Chain.Hash{byte_count: 32, bytes: <<73, 255, 120, 227, 182, 93, 1, 3, 83, 159, 76, 239, 168, 90, 105, 67, 169, 176, 151, 100, 145, 222, 87, 93, 112, 185, 69, 60, 160, 224, 229, 187>>}, transaction_index: 97, type: :call, updated_at: #DateTime<2019-06-06 17:39:30.521792Z>, value: #Explorer.Chain.Wei<0>}, %{block_number: 7907050, call_type: :call, from_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<212, 194, 64, 78, 104, 83, 60, 7, 176, 13, 157, 166, 235, 42, 117, 145, 33, 53, 183, 8>>}, gas: #Decimal<31645>, gas_used: #Decimal<13345>, index: 0, input: %Explorer.Chain.Data{bytes: <<169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 165, 6, 117, 133, 68, 167, 25, 67, 181, 232, 114, 141, 45, 248, 236, 158, 114, 71, 58, 154, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 146, 116, 178, 89, 246, 84, 0, 0>>}, inserted_at: #DateTime<2019-06-06 17:39:30.521792Z>, output: %Explorer.Chain.Data{bytes: <<0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1>>}, to_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<95, 91, 23, 101, 83, 229, 17, 113, 130, 109, 26, 98, 229, 64, 188, 48, 66, 44, 119, 23>>}, trace_address: [], transaction_hash: %Explorer.Chain.Hash{byte_count: 32, bytes: <<76, 33, 220, 214, 181, 170, 215, 158, 130, 13, 105, 42, 21, 229, 169, 167, 49, 239, 178, 128, 130, 77, 211, 108, 234, 12, 140, 65, 113, 67, 157, 20>>}, transaction_index: 120, type: :call, updated_at: #DateTime<2019-06-06 17:39:30.521792Z>, value: #Explorer.Chain.Wei<0>}, %{block_number: 7907050, call_type: :call, from_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<2, 239, 39, 30, 123, 216, 92, 35, 215, 40, 100, 92, 36, 93, 249, 78, 108, 25, 105, 188>>}, gas: #Decimal<176680>, gas_used: #Decimal<28850>, index: 0, input: %Explorer.Chain.Data{bytes: <<169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 211, 134, 2, 172, 27, 243, 121, 141, 25, 75, 26, 177, 31, 79, 110, 95, 103, 237, 187, 103, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 13, 192, 58, 221, 103, 202, 62, 64, 0>>}, inserted_at: #DateTime<2019-06-06 17:39:30.521792Z>, output: %Explorer.Chain.Data{bytes: <<0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1>>}, to_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<228, 29, 36, 137, 87, 29, 50, 33, 137, 36, 109, 175, 165, 235, 222, 31, 70, 153, 244, 152>>}, trace_address: [], transaction_hash: %Explorer.Chain.Hash{byte_count: 32, bytes: <<76, 83, 57, 194, 73, 251, 78, 211, 76, 40, 45, 239, 3, 40, 73, 82, 245, 244, 58, 199, 88, 127, 133, 234, 102, 48, 77, 209, 156, 231, 211, 27>>}, transaction_index: 82, type: :call, updated_at: #DateTime<2019-06-06 17:39:30.521792Z>, value: #Explorer.Chain.Wei<0>}, %{block_number: 7907050, call_type: :call, from_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<25, 224, 70, 106, 24, 229, 163, 117, 98, 27, 50, 124, 231, 129, 198, 75, 132, 68, 58, 27>>}, gas: #Decimal<326552>, gas_used: #Decimal<14653>, index: 0, input: %Explorer.Chain.Data{bytes: <<169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 50, 122, 92, 197, 100, 3, 131, 235, 206, 164, 94, 254, 188, 175, 153, 232, 205, 219, 83, 61, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 45, 121, 174, 237, 117, 235, 109, 162, 123>>}, inserted_at: #DateTime<2019-06-06 17:39:30.521792Z>, output: %Explorer.Chain.Data{bytes: <<0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1>>}, to_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<161, 92, 126, 190, 31, 7, 202, 246, 191, 240, 151, 216, 165, 137, 251, 138, 196, 154, 229, 179>>}, trace_address: [], transaction_hash: %Explorer.Chain.Hash{byte_count: 32, bytes: <<77, 154, 33, 29, 104, 214, 223, 215, 149, 128, 246, 68, 101, 160, 108, 190, 147, 158, 81, 103, 88, 137, 103, 184, 45, 243, 140, 152, 145, 135, 100, 183>>}, transaction_index: 105, type: :call, updated_at: #DateTime<2019-06-06 17:39:30.521792Z>, value: #Explorer.Chain.Wei<0>}, %{block_number: 7907050, call_type: :call, from_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<0, 97, 146, 249, 146, 64, 41, 50, 214, 68, 71, 150, 49, 128, 238, 60, 194, 157, 238, 101>>}, gas: #Decimal<92424>, gas_used: #Decimal<32647>, index: 0, input: %Explorer.Chain.Data{bytes: <<26, 105, 82, 48, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 74, 0, 65, 153, 58, 108, 151, 247, 209, 206, 164, 98, 70, 146, 167, 81, 115, 124, 200, 97>>}, inserted_at: #DateTime<2019-06-06 17:39:30.521792Z>, output: %Explorer.Chain.Data{bytes: ""}, to_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<11, 149, 153, 58, 57, 163, 99, 217, 146, 128, 172, 149, 15, 94, 69, 54, 171, 92, 85, 102>>}, trace_address: [], transaction_hash: %Explorer.Chain.Hash{byte_count: 32, bytes: <<82, 239, 211, 55, 189, 100, 50, 209, 85, 148, 103, 254, 109, 191, 67, 6, 40, 211, 4, 63, 178, 255, 101, 206, 66, 190, 188, 247, 23, 20, 2, 52>>}, transaction_index: 55, type: :call, updated_at: #DateTime<2019-06-06 17:39:30.521792Z>, value: #Explorer.Chain.Wei<6000000000000000>}, %{block_number: 7907050, call_type: :call, from_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<11, 149, 153, 58, 57, 163, 99, 217, 146, 128, 172, 149, 15, 94, 69, 54, 171, 92, 85, 102>>}, gas: #Decimal<2300>, gas_used: #Decimal<0>, index: 1, input: %Explorer.Chain.Data{bytes: ""}, inserted_at: #DateTime<2019-06-06 17:39:30.521792Z>, output: %Explorer.Chain.Data{bytes: ""}, to_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<74, 0, 65, 153, 58, 108, 151, 247, 209, 206, 164, 98, 70, 146, 167, 81, 115, 124, 200, 97>>}, trace_address: [0], transaction_hash: %Explorer.Chain.Hash{byte_count: 32, bytes: <<82, 239, 211, 55, 189, 100, 50, 209, 85, 148, 103, 254, 109, 191, 67, 6, 40, 211, 4, 63, 178, 255, 101, 206, 66, 190, 188, 247, 23, 20, 2, 52>>}, transaction_index: 55, type: :call, updated_at: #DateTime<2019-06-06 17:39:30.521792Z>, value: #Explorer.Chain.Wei<6000000000000000>}, %{block_number: 7907050, call_type: :call, from_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<22, 1, 133, 228, 177, 137, 185, 114, 198, 121, 162, 148, 173, 6, 83, 193, 59, 38, 87, 106>>}, gas: #Decimal<93024>, gas_used: #Decimal<14867>, index: 0, input: %Explorer.Chain.Data{bytes: <<169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 64, 161, 178, 231, 55, 171, 186, 19, 199, 18, 245, 220, 56, 212, 69, 106, 160, 31, 148, 208, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 10, 199, 223, 160, 57, 171, 120, 0>>}, inserted_at: #DateTime<2019-06-06 17:39:30.521792Z>, output: %Explorer.Chain.Data{bytes: <<0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1>>}, to_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<23, 75, 250, 102, 0, 191, 144, 200, 133, 199, 192, 28, 112, 49, 56, 158, 209, 70, 26, 185>>}, trace_address: [], transaction_hash: %Explorer.Chain.Hash{byte_count: 32, bytes: <<83, 65, 1, 126, 40, 11, 203, 232, 169, 143, 159, 214, 8, 60, 2, 218, 107, 167, 26, 4, 216, 220, 203, 19, 67, 53, 65, 243, 235, 223, 193, 231>>}, transaction_index: 35, type: :call, updated_at: #DateTime<2019-06-06 17:39:30.521792Z>, value: #Explorer.Chain.Wei<0>}, %{block_number: 7907050, call_type: :call, error: "Reverted", from_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<39, 26, 194, 144, 226, 161, 164, 25, 138, 127, 244, 128, 229, 35, 207, 19, 13, 229, 149, 244>>}, gas: #Decimal<69000>, index: 0, input: %Explorer.Chain.Data{bytes: ""}, inserted_at: #DateTime<2019-06-06 17:39:30.521792Z>, to_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<29, 70, 40, 17, 86, 44, 195, 103, 83, 117, 166, 208, 12, 22, 24, 163, 112, 194, 120, 54>>}, trace_address: [], transaction_hash: %Explorer.Chain.Hash{byte_count: 32, bytes: <<83, 245, 73, 98, 18, 198, 183, 113, 255, 248, 179, 95, 145, 5, 59, 173, 122, 57, 83, 122, 36, 68, 8, 232, 56, 148, 53, 21, 228, 55, 237, 135>>}, transaction_index: 162, type: :call, updated_at: #DateTime<2019-06-06 17:39:30.521792Z>, value: #Explorer.Chain.Wei<0>}, %{block_number: 7907050, call_type: :call, from_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<140, 131, 138, 149, 155, 171, 104, 28, 153, 10, 136, 77, 249, 29, 175, 233, 30, 218, 230, 179>>}, gas: #Decimal<69000>, gas_used: #Decimal<62826>, index: 0, input: %Explorer.Chain.Data{bytes: ""}, inserted_at: #DateTime<2019-06-06 17:39:30.521792Z>, output: %Explorer.Chain.Data{bytes: ""}, to_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<29, 70, 40, 17, 86, 44, 195, 103, 83, 117, 166, 208, 12, 22, 24, 163, 112, 194, 120, 54>>}, trace_address: [], transaction_hash: %Explorer.Chain.Hash{byte_count: 32, bytes: <<84, 251, 245, 168, 236, 217, 48, 204, 42, 236, 242, 88, 187, 215, 214, 50, 59, 39, 6, 15, 72, 63, 197, 19, 183, 104, 73, 223, 233, 56, 79, 103>>}, transaction_index: 174, type: :call, updated_at: #DateTime<2019-06-06 17:39:30.521792Z>, value: #Explorer.Chain.Wei<0>}, %{block_number: 7907050, call_type: :call, from_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<226, 206, 8, 150, 160, 135, 13, 243, 95, 232, 233, 247, 233, 162, 226, 231, 197, 223, 92, 52>>}, gas: #Decimal<39057>, gas_used: #Decimal<28661>, index: 0, input: %Explorer.Chain.Data{bytes: <<169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 197, 223, 149, 125, 49, 240, 23, 115, 53, 50, 239, 62, 249, 67, 109, 30, 189, 34, 23, 153, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 64, 234, 27, 218, 60, 125, 188, 24, 0, 0>>}, inserted_at: #DateTime<2019-06-06 17:39:30.521792Z>, output: %Explorer.Chain.Data{bytes: <<0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1>>}, to_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<57, 236, 57, 246, 87, 50, 71, 239, 179, 149, 73, 116, 216, 97, 130, 23, 249, 128, 227, 160>>}, trace_address: [], transaction_hash: %Explorer.Chain.Hash{byte_count: 32, bytes: <<86, 252, 189, 178, 49, 95, 131, 125, 197, 40, 0, 100, 254, 227, 240, 194, 253, 112, 216, 70, 48, 134, 48, 19, 72, 28, 205, 50, 102, 103, 119, 156>>}, transaction_index: 81, type: :call, updated_at: #DateTime<2019-06-06 17:39:30.521792Z>, value: #Explorer.Chain.Wei<0>}, %{block_number: 7907050, call_type: :call, from_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<95, 80, 76, 69, 193, 214, 184, 58, 119, 126, 173, 10, 48, 58, 110, 77, 48, 197, 134, 36>>}, gas: #Decimal<14640>, gas_used: #Decimal<14640>, index: 0, input: %Explorer.Chain.Data{bytes: <<169, 5, 156, 187, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 52, 96, 99, 49, 140, 33, 15, 111, 11, 167, 39, 47, 142, 241, 6, 154, 220, 28, 55, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 120, 26, 245, 128>>}, inserted_at: #DateTime<2019-06-06 17:39:30.521792Z>, output: %Explorer.Chain.Data{bytes: <<0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1>>}, to_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<92, 137, 115, 110, 148, 84, 32, 1, 65, 184, 12, 55, 235, 40, 234, 206, 202, 44, 232, 203>>}, trace_address: [], transaction_hash: %Explorer.Chain.Hash{byte_count: 32, bytes: <<87, 0, 119, 152, 137, 29, 219, 63, 72, 244, 83, 58, 160, 42, 192, 217, 146, 248, 188, 129, 66, 227, 142, 134, 138, 159, 217, 173, 175, 108, 216, 106>>}, transaction_index: 176, type: :call, updated_at: #DateTime<2019-06-06 17:39:30.521792Z>, value: #Explorer.Chain.Wei<0>}, %{block_number: 7907050, call_type: :call, from_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<147, 173, 100, 153, 131, 180, 172, 146, 121, 174, 157, 118, 19, 126, 228, 54, 50, 111, 65, 151>>}, gas: #Decimal<0>, gas_used: #Decimal<0>, index: 0, input: %Explorer.Chain.Data{bytes: ""}, inserted_at: #DateTime<2019-06-06 17:39:30.521792Z>, output: %Explorer.Chain.Data{bytes: ""}, to_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<193, 43, 107, 105, 166, 189, 218, 169, 246, 43, 160, 226, 112, 54, 223, 196, 73, 133, 98, 56>>}, trace_address: [], transaction_hash: %Explorer.Chain.Hash{byte_count: 32, bytes: <<89, 85, 217, 171, 223, 73, 32, 223, 16, 150, 114, 122, 122, 46, 30, 142, 66, 66, 86, 131, 126, 86, 203, 69, 64, 2, 71, 159, 20, 58, 200, 255>>}, transaction_index: 77, type: :call, updated_at: #DateTi


some  other logs
iex(1)> 2019-06-07T03:31:24.598 application=explorer fetcher=internal_transaction count=2 import_id=5965059 transaction_id=3689924 [error] Could not insert all of chunk into Elixir.Explorer.Chain.InternalTransaction using options because of error.


[conflict_target: [:transaction_hash, :index], on_conflict: #Ecto.Query<from i0 in Explorer.Chain.InternalTransaction, where: fragment("(EXCLUDED.call_type, EXCLUDED.created_contract_address_hash, EXCLUDED.created_contract_code, EXCLUDED.error, EXCLUDED.from_address_hash, EXCLUDED.gas, EXCLUDED.gas_used, EXCLUDED.init, EXCLUDED.input, EXCLUDED.output, EXCLUDED.to_address_hash, EXCLUDED.trace_address, EXCLUDED.transaction_index, EXCLUDED.type, EXCLUDED.value) IS DISTINCT FROM (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)", i0.call_type, i0.created_contract_address_hash, i0.created_contract_code, i0.error, i0.from_address_hash, i0.gas, i0.gas_used, i0.init, i0.input, i0.output, i0.to_address_hash, i0.trace_address, i0.transaction_index, i0.type, i0.value), update: [set: [block_number: fragment("EXCLUDED.block_number"), call_type: fragment("EXCLUDED.call_type"), created_contract_address_hash: fragment("EXCLUDED.created_contract_address_hash"), created_contract_code: fragment("EXCLUDED.created_contract_code"), error: fragment("EXCLUDED.error"), from_address_hash: fragment("EXCLUDED.from_address_hash"), gas: fragment("EXCLUDED.gas"), gas_used: fragment("EXCLUDED.gas_used"), init: fragment("EXCLUDED.init"), input: fragment("EXCLUDED.input"), output: fragment("EXCLUDED.output"), to_address_hash: fragment("EXCLUDED.to_address_hash"), trace_address: fragment("EXCLUDED.trace_address"), transaction_index: fragment("EXCLUDED.transaction_index"), type: fragment("EXCLUDED.type"), value: fragment("EXCLUDED.value"), inserted_at: fragment("LEAST(?, EXCLUDED.inserted_at)", i0.inserted_at), updated_at: fragment("GREATEST(?, EXCLUDED.updated_at)", i0.updated_at)]]>, returning: [:transaction_hash, :index], timeout: 60000, timestamps: %{inserted_at: #DateTime<2019-06-07 03:31:23.167253Z>, updated_at: #DateTime<2019-06-07 03:31:23.167253Z>}]
--

@APshenkin
Copy link
Contributor Author

APshenkin commented Jun 7, 2019 •
edited
Loading

And issue is still present. After restart Blockscout sync internal transactions and works well but after some time (several hours) it stops syncing internal transactions

And after restart there is no any violates foreign key constraint errors (just sync all blocks)

@vbaranov
Copy link
Member

vbaranov commented Jun 7, 2019

If you updated existing BS instance with 1.3.13, that was previously partially indexed, it is recommended to run this query against db to fix "bad" blocks:
https://github.com/poanetwork/blockscout/blob/master/apps/explorer/priv/repo/migrations/scripts/20190326202921_lose_consensus_for_invalid_blocks.sql

@APshenkin
Copy link
Contributor Author

@vbaranov ok, will try it.

BTW our blockscout is already in sync (we set starting sync only from block 7878000)

So at the moment all blocks have synced internal transactions.

But during live sync (new generated blocks), syncing internal transactions stops after some time. Then we need to restart blockscout to run syncing again and it sync all blocks that were stucked

@APshenkin
Copy link
Contributor Author

So the issue is still present

@APshenkin
Copy link
Contributor Author

@vbaranov is there any updates?

Do you face the same issue? Or maybe we have to upgrade our blockscout version?

@vbaranov
Copy link
Member

@APshenkin no, we have no exactly your issue. We encounter timed out indexing for some blocks, that have a lot of transactions/internal transactions, but they finally processed later. Basically, what you can do is to manage memory usage, concurrency and batch size for internal txs indexing, as I recommended here #2105 (comment). Also, consider increasing the resources on your database instance. For example, here is the clarification of the size of our RDS instance to support ETH Mainnet #2013 (comment)

Upgrade to https://github.com/poanetwork/blockscout/releases/tag/v2.0.0-beta is also recommended to improve performance of the Blockscout pages

@APshenkin
Copy link
Contributor Author

APshenkin commented Jun 26, 2019 •
edited
Loading

@vbaranov
Have an update about this issue.

Today I was able to find an exception in logs. Log file is huge, so I pasted it to gist

https://gist.github.com/APshenkin/9d71fcf92f4d853bc84d8f88023e95e8

After this exception internal transaction fetching stops

@APshenkin
Copy link
Contributor Author

@vbaranov
Today I was able to catch more logs for this case

https://gist.githubusercontent.com/APshenkin/8931f07e5de98d570dea895e6998474a/raw/5a39fd7f030cf951964bd4ad858528150fd03924/log.log

@pasqu4le pasqu4le mentioned this issue Jul 11, 2019
Merged
4 tasks
@pasqu4le
Copy link
Contributor

Hey @APshenkin, I think I found the cause of your problem and that #2346 fixes it.

Basically there is a temporary fetcher that refetches blocks to check that they have the correct transactions.
The problem is that internal transactions get fetched for blocks that have not been refetched yet as well, so it can happen that their transaction (and so their foreign key) does not exist.

This problem normally should just disappear on its own, when the blocks and its transactions gets refetched, but I believe that what happens in your case it that it fails so many times that it stops before this can happen.

Could you please if possible confirm my suspicion and check that either

  • the block that raised an error still has refetch_needed set to true
    or
  • a transaction with the hash that did not exist is now in fact in the database

@APshenkin
Copy link
Contributor Author

APshenkin commented Jul 11, 2019 •
edited
Loading

Hi @pasqu4le

I'm not sure that this was the reason of issue, because I faced it before adding refetch_needed in blockscout.

Today we updgraded our blockscout to 2.0.1 and now we have a lot of blocks with refetch_needed: true

We will monitor this error after all refetch_needed will be completed

@pasqu4le
Copy link
Contributor

Hey @APshenkin, thanks for the feedback.

The fact that the issue predates refetch_needed does not exclude that it stems from the same problem, because that was implemented precisely to fix blocks (and their incorrect transactions).

Please keep me updated if the problem is still present after no block needs to be refetched anymore and reopen the issue if it gets closed before that.

@APshenkin
Copy link
Contributor Author

APshenkin commented Jul 11, 2019 •
edited
Loading

@pasqu4le
So refetch_needed is now 0 in our blockscout installation.

We restarted blockscout at 18 p.m to complete clean test.

After 1 hour there are 4 blocks, that have internal_transactions_indexed_at = null
And they don't try to insert (I don't see this in logs)

Their numbers are:
8131522
8131596
8131505
8131415

Also using pg_stat_activity I can't see any stucked inserts.

Here is full app log. Maybe it helps:
https://gist.github.com/APshenkin/34b10bfafc888c27271b171ab155b37c

FYI we launch sync only from block 7878000 and all blocks are sync

BTW thank you for help!

UPDATED: After blockscout restart these blocks are now synced for internal transactions
UPDATED: Hm, looks strange, but all this 4 blocks can be found in logs with the same error:

ex(1)> 2019-07-11T18:08:53.189 [error] Task #PID<0.23939.0> started from Indexer.Block.Realtime.Fetcher terminating
** (FunctionClauseError) no function clause matching in anonymous fn/1 in Indexer.Block.Fetcher.async_import_created_contract_codes/1
    (indexer) lib/indexer/block/fetcher.ex:240: anonymous fn(%Explorer.Chain.Transaction{to_address: #Ecto.Association.NotLoaded<association :to_address is not loaded>, token_transfers: #Ecto.Association.NotLoaded<association :token_transfers is not loaded>, forks: #Ecto.Association.NotLoaded<association :forks is not loaded>, old_block_hash: %Explorer.Chain.Hash{byte_count: 32, bytes: <<173, 240, 186, 108, 158, 22, 192, 102, 137, 242, 160, 137, 225, 87, 112, 220, 230, 83, 97, 79, 58, 22, 238, 198, 75, 186, 159, 13, 170, 234, 24, 28>>}, block_hash: %Explorer.Chain.Hash{byte_count: 32, bytes: <<173, 240, 186, 108, 158, 22, 192, 102, 137, 242, 160, 137, 225, 87, 112, 220, 230, 83, 97, 79, 58, 22, 238, 198, 75, 186, 159, 13, 170, 234, 24, 28>>}, uncles: #Ecto.Association.NotLoaded<association :uncles is not loaded>, logs: #Ecto.Association.NotLoaded<association :logs is not loaded>, internal_transactions: #Ecto.Association.NotLoaded<association :internal_transactions is not loaded>, input: %Explorer.Chain.Data{bytes: <<96, 128, 96, 64, 82, 96, 0, 96, 14, 85, 52, 128, 21, 98, 0, 0, 22, 87, 96, 0, 128, 253, 91, 80, 96, 64, 81, 96, 32, 128, 98, 0, 47, 205, 131, 57, 129, 1, 128, 96, ...>>}, hash: %Explorer.Chain.Hash{byte_count: 32, bytes: <<192, 126, 18, 7, 171, 227, 175, 182, 154, 77, 87, 78, 139, 234, 129, 24, 189, 215, 59, 47, 39, 190, 149, 18, 150, 181, 133, 165, 205, 206, 255, 244>>}, index: 74, created_contract_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<97, 143, 115, 69, 17, 197, 248, 94, 78, 170, 198, 116, 243, 87, 86, 216, 207, 221, 58, 43>>}, inserted_at: #DateTime<2019-07-11 18:08:27.412977Z>, nonce: 2, created_contract_address: #Ecto.Association.NotLoaded<association :created_contract_address is not loaded>, cumulative_gas_used: #Decimal<5891557>, __meta__: #Ecto.Schema.Metadata<:loaded, "transactions">, gas_price: #Explorer.Chain.Wei<20000000000>, to_address_hash: nil, error: nil, from_address: #Ecto.Association.NotLoaded<association :from_address is not loaded>, status: :ok, gas_used: #Decimal<3228377>, updated_at: #DateTime<2019-07-11 18:08:52.714582Z>, v: #Decimal<27>, created_contract_code_indexed_at: #DateTime<2019-07-11 18:08:52.714582Z>, s: #Decimal<8791980517531217237461665406055807904328326818125947028244376818612481547839>, gas: #Decimal<5000000>, value: #Explorer.Chain.Wei<0>, internal_transactions_indexed_at: nil, from_address_hash: %Explorer.Chain.Hash{byte_count: 20, bytes: <<93, 188, 105, 160, 231, 145, 117, 214, 43, 247, 3, 202, 92, 125, 218, 96, 17, ...>>}, block: #Ecto.Association.NotLoaded<association :block is not loaded>, earliest_processing_start: #DateTime<2019-07-11 18:08:26.985315Z>, block_number: 8131415, r: #Decimal<26201765788738853719051243285346394966496342377715264542841096940430505446071>}) in Indexer.Block.Fetcher.async_import_created_contract_codes/1
    (elixir) lib/enum.ex:2986: Enum.flat_map_list/2
    (elixir) lib/enum.ex:2987: Enum.flat_map_list/2
    (indexer) lib/indexer/block/fetcher.ex:240: Indexer.Block.Fetcher.async_import_created_contract_codes/1
    (indexer) lib/indexer/block/realtime/fetcher.ex:354: Indexer.Block.Realtime.Fetcher.async_import_remaining_block_data/3
    (indexer) lib/indexer/block/realtime/fetcher.ex:194: Indexer.Block.Realtime.Fetcher.import/2
    (indexer) lib/indexer/block/fetcher.ex:159: Indexer.Block.Fetcher.fetch_and_import_range/2
    (indexer) lib/indexer/block/realtime/fetcher.ex:262: Indexer.Block.Realtime.Fetcher.do_fetch_and_import_block/3
Function: &Indexer.Block.Realtime.Fetcher.fetch_and_import_block/3
    Args: [8131415, %Indexer.Block.Fetcher{broadcast: :realtime, callback_module: Indexer.Block.Realtime.Fetcher, json_rpc_named_arguments: [transport: EthereumJSONRPC.HTTP, transport_options: [http: EthereumJSONRPC.HTTP.HTTPoison, url: "http://10.136.20.6:8545", method_to_url: [eth_getBalance: "http://10.136.20.6:8545", trace_block: "http://10.136.20.6:8545", trace_replayTransaction: "http://10.136.20.6:8545"], http_options: [recv_timeout: 600000, timeout: 600000, hackney: [pool: :ethereum_jsonrpc]]], variant: EthereumJSONRPC.Parity], receipts_batch_size: 250, receipts_concurrency: 10}, false]

@pasqu4le
Copy link
Contributor

@APshenkin glad I could help, I'll check out this new error you got and see what I can do

@pasqu4le pasqu4le mentioned this issue Jul 17, 2019
Merged
4 tasks
@pasqu4le
Copy link
Contributor

Hey @APshenkin I have figured out what was the problem with your last error(s) and #2375 should get rid of it.

@APshenkin
Copy link
Contributor Author

@pasqu4le @vbaranov
I tried to apply these fixes on our enviroment and found that issue is not solved. I will send blockscout logs when all internal transactions will be synced (after several blockscout restarts)

@pasqu4le
Copy link
Contributor

Hey @APshenkin if you are referring to the first issue (the one with internal_transactions not inserting) we found out it can also happen for a different reason than the one that was solved.

I'm going to write an fix soon an I'll notify you on this thread when it is available.

@royki
Copy link
Contributor

royki commented Dec 3, 2021 •
edited
Loading

Hi @vbaranov @pasqu4le
Though the ticket is closed, we started receiving the same error. Any idea how to get rid of this?

  • The rds instance was full and When rds is up after extra storage, we started getting this error -
blockscout_1  | 
blockscout_1  | Exception:
blockscout_1  | 
blockscout_1  | ** (Postgrex.Error) ERROR 23503 (foreign_key_violation) insert or update on table "internal_transactions" violates foreign key constraint "internal_transactions_transaction_hash_fkey"
blockscout_1  | 
blockscout_1  |     table: internal_transactions
blockscout_1  |     constraint: internal_transactions_transaction_hash_fkey
blockscout_1  | 
blockscout_1  | Key (transaction_hash)=(\x091e8ec6422b5489d8302842cd67c6b201966a729fb8ab8c671913341ddcdbfc) is not present in table "transactions".
blockscout_1  |     (ecto_sql 3.5.3) lib/ecto/adapters/sql.ex:751: Ecto.Adapters.SQL.raise_sql_call_error/1
blockscout_1  |     (ecto_sql 3.5.3) lib/ecto/adapters/sql.ex:660: Ecto.Adapters.SQL.insert_all/8
blockscout_1  |     (ecto 3.5.5) lib/ecto/repo/schema.ex:54: Ecto.Repo.Schema.do_insert_all/6
blockscout_1  |     (explorer 0.0.1) lib/explorer/repo.ex:45: anonymous fn/5 in Explorer.Repo.safe_insert_all/3
blockscout_1  |     (elixir 1.11.4) lib/enum.ex:2193: Enum."-reduce/3-lists^foldl/2-0-"/3
blockscout_1  |     (explorer 0.0.1) lib/explorer/chain/import.ex:299: Explorer.Chain.Import.insert_changes_list/3
blockscout_1  |     (explorer 0.0.1) lib/explorer/chain/import/runner/internal_transactions.ex:148: Explorer.Chain.Import.Runner.InternalTransactions.insert/3
blockscout_1  |     (ecto 3.5.5) lib/ecto/multi.ex:646: Ecto.Multi.apply_operation/5
blockscout_1  | 
blockscout_1  | 2021-12-03T16:49:02.767 fetcher=internal_transaction count=10 [error] Task #PID<0.9842.10> started from Indexer.Fetcher.InternalTransaction terminating

@MaxTeiger
Copy link

Hello @royki !

I am facing the same issue, do you have any update on how you manage to fix this ?

@royki
Copy link
Contributor

royki commented Aug 17, 2022 •
edited
Loading

Hi @MaxTeiger , You need to update manually in the blocks table to make consensus = false of the blockNumber

From @vbaranov in discord -> https://discord.com/channels/502416149343109121/959785606051954720/960441819655708695

The transaction with hash 0xa7ecf90395b2208ed111347bf07547445056fd99903efd6de85c3240c65a2ab2 is not found in transactions table. You can find requesting the node in which block that transaction existed and try invalidate that block by setting consensus false for that block number in blocks table. After that Blockscout should re-import all transaction receipts for that block.

Here are the steps and sql, that I performed

  1. Get block number from RPC call, (you can use postman)
curl --location --request POST 'https://ARCHIVE_RPC_URL' \
--header 'Content-Type: application/json' \
--data-raw '{
	"jsonrpc":"2.0",
	"method":"trace_transaction",
	"params":["TRANSACTION_HASH"],
	"id":1
}'

This will return blockNumber.
2. In SQL query UPDATE public.blocks SET consensus = false WHERE number = 'blockNumber';
PS: You might need to update multiple blockNumbers depending on the logs. In our case, we had multiple blockNumbers to update to get rid of these error. I'm not sure if there is any other better way.

@MaxTeiger
Copy link

Thank you very much !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@pasqu4le pasqu4le

Labels
waiting on feedback Waiting for original opener to respond if bug is fixed
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Avoid fetching internal transactions of blocks that still need refetching blockscout/blockscout
5 participants
@royki @vbaranov @pasqu4le @APshenkin @MaxTeiger