Implementing Otterscan support #5414

naps62 · 2023-07-16T21:35:27Z

Motivation

The people want Etherscan for their anvil nodes.

Solution

Implements the Otterscan JSON RPC namespace, along with all the quirks that were found to be needed.
Some of the code is a bit messier than I'd like,

This is still a work in progress, as I have two endpoints to implement still, but they should be easy (famous last words) compared to a lot of the others, so I'm opening this ahead of time to gather feedback.
The code is also a bit messier than I'd like, but I was trying to touch as little as possible on Backend and other modules. maybe there's some patterns or utility function I missed in the codebase that could have helped, so feedback is much appreciated!

This was also a bit tricky for a couple of reasons:

Otterscan's endpoints are inherently hard to compute with existing data (otherwise they wouldn't be needed in the first place). They solve things that the original RPC spec didn't account for (most notably, listing historical transactions by address). For anvil, this is mostly a non-issue, as we can choose to traverse all the blocks & traces in-memory. Would be interesting to test this on an node with a heavy data-set though;
After having gone through the spec, it seems to not be as well though-out as it could, and in some cases outdated compared to their code. Some of the design decisions behind it are a bit awkward to implement (see Outdated spec? otterscan/otterscan#1081).

todo list

Adds anvil support for Otterscan's custom RPC endpoints. This is still a work in progress, as I have two endpoints to implement still, but they should be **easy** (famous last words) compared to a lot of the others, so I'm opening this ahead of time to gather feedback. This was a bit tricky for a couple of reasons: * Otterscan's endpoints are inherently hard to compute with existing data (otherwise they wouldn't be needed in the first place). They solve things that the original RPC spec didn't account for (most notably, listing historical transactions by address). For anvil, this is mostly a non-issue, as we can choose to traverse all the blocks & traces in-memory. Would be interesting to test this on an node with a heavy data-set though; * After having gone through the spec, it seems to not be as well though-out as it could, and in some cases outdated compared to their code. Some of the design decisions behind it are a bit awkward to implement (see otterscan/otterscan#1081).

naps62 · 2023-07-18T18:06:55Z

@Evalir couple questions:

I took a stab at one of the tests, but not sure I used the write approach (maybe there's a more appropriate test for this layer that I could base it on?)
I'm not yet sure about a couple of the endpoints. left a few TODOs in there where I still have questions. would appreciate feedback

Evalir

Hey hey! tysm for this—this is awesome.

i have some comments/clarifications so we can keep this going—i tried to check for correctness but the best way to keep testing is probably to keep testing directly with otterscan and see what it likes or doesn't like.

cc @mattsse beefy pr so ideally we'd have two eyes on this

Evalir · 2023-07-19T19:30:16Z

anvil/core/src/eth/mod.rs

+    /// Otterscan's `ots_getApiLevel` endpoint
+    #[cfg(feature = "otterscan")]
+    #[cfg_attr(feature = "serde", serde(rename = "erigon_getHeaderByNumber"))]
+    ErigonGetHeaderByNumber(


Should this be called so? If so, can we add a bit of context as to why it has erigon naming? not super familiar with otterscan so just wanna make sure we have this documented 😄

I documented the impl EthApi functions, but now that you mention, probably most of those comments belong here instead

As for this particular one, it's otterscan being a bit sneaky and making an erigon-specific call on load to "check if we're connected to an erigon node". In practice I think they could achieve the same result (show a proper error msg when the node isnt' supported) by using ay other ots_ endpoint, but oh well.

Perhaps I'll change that upstream later. for now it's reported here, waiting for input: otterscan/otterscan#1081

anvil/src/eth/api.rs

anvil/src/eth/mod.rs

anvil/src/eth/otterscan.rs

anvil/src/eth/api.rs

anvil/src/eth/otterscan.rs

anvil/tests/it/otterscan_api.rs

Evalir

another batch of comments, but this should be real close

@mattsse would love your eyes on this!

Evalir · 2023-07-25T19:08:56Z

anvil/src/eth/api.rs

+    ) -> Result<Option<Block<TxHash>>> {
+        node_info!("ots_getApiLevel");
+
+        // TODO: probably only return blocks from before the fork?


Did you determine what this should be? IMO it's fine if it's only post-fork.

anvil/src/eth/api.rs

anvil/src/eth/otterscan.rs

Co-authored-by: evalir <[email protected]>

mattsse

this is fantastic.

all of this looks very good already, only have a few needs re organization.

anvil/src/eth/api.rs

anvil/tests/it/otterscan.rs

naps62 · 2023-08-07T16:08:23Z

@Evalir @mattsse thanks for all the help.
Pretty sure now I handled all missing tests, and your feedback (including some comments about removing unwrap() that had gone outdated)

Only thing left is this now, which I can't seem to figure out. would appreciate a hand here whenever possible 🙏🏼

Evalir

This is beautiful—I'd just like docs for the things that don't have any.

Will look at solving the last todo as well!

anvil/src/eth/otterscan/types.rs

mattsse

amazing, this will be super useful

haven't reviewed the new handler impls too closely but all they do is add additional features so should be fine

anvil/tests/it/otterscan.rs

naps62 · 2023-08-08T16:52:14Z

@Evalir I believe everything's now done!

(I'll later create an issue on the book repo, and probably work on it as well)

Evalir

This is beautiful. Amazing work @naps62 — it was such a pleasure reviewing this!

Comply w/ foundry-rs/foundry#5414 Co-authored-by: Trajan0x <[email protected]>

adds a local otterscan block explorer for the anvil ethereum mainnet, helps to see what's happening on the base network. (they[ recently added support](foundry-rs/foundry#5414) for otterscan in anvil) the transactions/calls are not decoded, so right now it's primarily useful if we are using portal contracts - hard to tell what the rollup contracts are doing. sets the local otterscan at http://localhost:5100/ https://github.com/AztecProtocol/aztec-packages/assets/142251406/0dd83352-103b-4b75-8fb6-baff5f0eee6b

TangMonk · 2024-04-29T11:47:03Z

Otterscan works on anvil now?

naps62 mentioned this pull request Jul 17, 2023

feat(rpc): Add ots_ namespace and trait bindings for Otterscan Endpoints paradigmxyz/reth#3778

Merged

naps62 added 7 commits July 17, 2023 22:26

comments

7b6871b

code review

6205a0e

code review

34b966d

code review

93bcbd9

code review

e0dea78

Merge branch 'master' into ots

c80fab8

code review

88feee7

Evalir requested changes Jul 19, 2023

View reviewed changes

naps62 added 2 commits July 19, 2023 22:47

Some fixes

664b54a

code review

00e085e

Evalir requested changes Jul 25, 2023

View reviewed changes

naps62 and others added 2 commits July 26, 2023 15:14

Merge branch 'master' into ots

7350691

Update anvil/src/eth/otterscan.rs

25db0cc

Co-authored-by: evalir <[email protected]>

mattsse requested changes Jul 26, 2023

View reviewed changes

anvil/src/eth/api.rs Outdated Show resolved Hide resolved

anvil/src/eth/api.rs Outdated Show resolved Hide resolved

naps62 added 9 commits July 26, 2023 15:42

code review

0d38757

code review

52e03f4

ots_getBlockDetailsByHash

460d598

search endpoints tests

b10230a

tests for ots_getBlockTransactions

ac835bf

tests for ots_getBlockDetails

1d79382

code review

432d89f

code review

9b11ebf

code review

0520337

naps62 commented Jul 28, 2023

View reviewed changes

anvil/tests/it/otterscan.rs Outdated Show resolved Hide resolved

naps62 added 3 commits July 28, 2023 14:02

code review

3ad752e

code review

9d652cb

code review

92659cd

naps62 added 6 commits August 7, 2023 15:58

code review

751e88d

code review

2542ba8

code review

864e3b1

code review

3aba8d6

code review

9caf3fb

code review

52e94a8

Evalir approved these changes Aug 7, 2023

View reviewed changes

anvil/src/eth/otterscan/types.rs Show resolved Hide resolved

anvil/src/eth/otterscan/types.rs Show resolved Hide resolved

anvil/src/eth/otterscan/types.rs Show resolved Hide resolved

mattsse approved these changes Aug 7, 2023

View reviewed changes

mattsse reviewed Aug 7, 2023

View reviewed changes

anvil/tests/it/otterscan.rs Outdated Show resolved Hide resolved

Evalir marked this pull request as ready for review August 7, 2023 19:00

naps62 and others added 7 commits August 8, 2023 16:30

Merge branch 'master' into ots

293496e

code review

50cf28b

Update Cargo.toml

53dcf44

code review

4aafc9b

clippy

5444bc9

code review

9519f9f

code review

486c71d

Evalir approved these changes Aug 8, 2023

View reviewed changes

Evalir merged commit e4ac420 into foundry-rs:master Aug 8, 2023

naps62 mentioned this pull request Aug 9, 2023

Add ots_* namespace to anvil reference foundry-rs/book#973

Merged

trajan0x mentioned this pull request Aug 18, 2023

update foundry tag synapsecns/sanguine#1265

Merged

trajan0x added a commit to synapsecns/sanguine that referenced this pull request Aug 18, 2023

update foundry tag (#1265)

e7b05a1

Comply w/ foundry-rs/foundry#5414 Co-authored-by: Trajan0x <[email protected]>

dan-aztec mentioned this pull request Oct 3, 2023

feat: add otterscan to sandbox AztecProtocol/aztec-packages#2648

Merged

0xAlcibiades mentioned this pull request Mar 21, 2024

Add support for Otterscan JSON-RPC API extensions paradigmxyz/reth#3726

Open

naps62 deleted the ots branch April 29, 2024 16:23

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implementing Otterscan support #5414

Implementing Otterscan support #5414

naps62 commented Jul 16, 2023 •

edited

Loading

naps62 commented Jul 18, 2023

Evalir left a comment

Evalir Jul 19, 2023

naps62 Jul 19, 2023

Evalir left a comment

Evalir Jul 25, 2023

mattsse left a comment

naps62 commented Aug 7, 2023

Evalir left a comment

mattsse left a comment

naps62 commented Aug 8, 2023

Evalir left a comment

TangMonk commented Apr 29, 2024

Implementing Otterscan support #5414

Implementing Otterscan support #5414

Conversation

naps62 commented Jul 16, 2023 • edited Loading

Motivation

Solution

todo list

naps62 commented Jul 18, 2023

Evalir left a comment

Choose a reason for hiding this comment

Evalir Jul 19, 2023

Choose a reason for hiding this comment

naps62 Jul 19, 2023

Choose a reason for hiding this comment

Evalir left a comment

Choose a reason for hiding this comment

Evalir Jul 25, 2023

Choose a reason for hiding this comment

mattsse left a comment

Choose a reason for hiding this comment

naps62 commented Aug 7, 2023

Evalir left a comment

Choose a reason for hiding this comment

mattsse left a comment

Choose a reason for hiding this comment

naps62 commented Aug 8, 2023

Evalir left a comment

Choose a reason for hiding this comment

TangMonk commented Apr 29, 2024

naps62 commented Jul 16, 2023 •

edited

Loading