Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

all: live chain-aware tracing #27629

Closed
wants to merge 113 commits into from
Closed

all: live chain-aware tracing #27629

wants to merge 113 commits into from

Conversation

s1na
Copy link
Contributor

@s1na s1na commented Jun 30, 2023

This PR lets users run a long-running tracer. The tracer will run alongside normal chain processing operations. It will be aware of practically all observable changes to the chain. This allows people to write complete indexing solutions without needing a fork of geth with large footprint over sensitive and often-changing parts of the codebase. E.g. see the discussion at #27003. There are 2 major themes to this:

  • Configuring a tracer to run as part of normal blockchain operations
  • Extending the tracing interface to include high-level events such as BlockStart, BlockEnd, as well as minute details like OnNewAccount, OnBalanceChange, etc.

Given that the native tracers have access to the filesystem, it will be possible for example already to do a full sync and store all call traces in a file. However I plan to work on streamlining this and offering db persistence of traces directly in geth in the future. Then we can offer trace_filter-like APIs.

Backwards-compatibility

There has been a change in the hooks CaptureTxStart/CaptureTxEnd and CaptureStart/End/Enter/Exit. These hooks have been moved up slightly and will now capture tx/call validation errors. I.e. There will be e.g. CaptureEnter invocations even if the call fails validation (depth-check or value balance check), and consequently CaptureExit will be called with the error.

@joohhnnn
Copy link
Contributor

joohhnnn commented Jul 1, 2023

Greeting @s1na , I noticed that the StateLogger part of my pull request (#27617) aligns perfectly with what you have already implemented. I'm considering building upon your work. I have a few questions:

  1. Currently, it seems that only logs are output when onXXX is triggered. Is it appropriate to add a subscription mode here (event/feed/send())?

  2. I'm unsure if most nodes will enable the tracer module in the same way they enable the current subscription mode.

  3. I'd like to clarify that my planned extension for the subscription mode is designed through difflayer. Regarding the onXXXX tracer, my understanding is that it retrieves the existing transactions. When it comes to adding logs, would it be better to adding subscription functionality through the existing difflayer module directly (which overlaps a bit with question 1)?

@s1na
Copy link
Contributor Author

s1na commented Jul 3, 2023

Hey @joohhnnn. That is an interesting idea to push tracing notifications on a channel to be used with a subscription method. We could have something like eth_subscribe('traces', { tracer: 'prestateTracer' }). Then add a way for tracers to push notifications. However we have to figure out how to detect in FilterSystem which subscription a given trace belongs to.

Edit: We can totally simplify this at the cost of some flexibility. eth_subscribe('traces') would subscribe to events coming out of the live tracer, with no option to choose different tracers. I think this is a good direction.

@maoueh
Copy link
Contributor

maoueh commented Jul 4, 2023

@s1na Thanks for this initial PR, I'm going to start build up upon it to see how it looks like and report once I have played with it a little bit more.

cmd/utils/flags.go Outdated Show resolved Hide resolved
core/blockchain.go Outdated Show resolved Hide resolved
core/blockchain.go Outdated Show resolved Hide resolved
eth/tracers/printer.go Outdated Show resolved Hide resolved
core/state_processor.go Outdated Show resolved Hide resolved
core/state_processor.go Outdated Show resolved Hide resolved
core/blockchain.go Outdated Show resolved Hide resolved
core/blockchain.go Outdated Show resolved Hide resolved
type noop struct{}

func newNoopTracer(_ json.RawMessage) (core.BlockchainLogger, error) {
return &noop{}, nil
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

t := &myTracer{}
return &live.Tracer{
    CaptureStart: t.captureStart,
    CaptureEnd: r.captureEnd,
}

@@ -141,6 +142,9 @@ type Config struct {
// Enables tracking of SHA3 preimages in the VM
EnablePreimageRecording bool

// Enables VM tracing
VMTracer vm.EVMLogger
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is weird. This config object is decoded from the yaml config files. Shouldn't put objects like this here.

Comment on lines +609 to 614

if interpreter.evm.Config.Tracer != nil && returnGas > 0 {
interpreter.evm.Config.Tracer.OnGasChange(scope.Contract.Gas, scope.Contract.Gas+returnGas, GasChangeCallLeftOverRefunded)
}

scope.Contract.Gas += returnGas
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

combine all this into RefundGas

@holiman
Copy link
Contributor

holiman commented Mar 22, 2024

Computer says no

@holiman holiman closed this Mar 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants