Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extended tracer sei #15

Merged
merged 59 commits into from
Mar 19, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
59 commits
Select commit Hold shift + click to select a range
916d6a4
params: release Geth v1.15.5
karalabe Nov 14, 2023
7f8926e
Backporting `extended-tracer` branch
maoueh Feb 6, 2024
301839f
Merge branch 'extended-tracer-backport-1.13.5' into release/sei-1.x-f…
maoueh Feb 6, 2024
9f71824
Backported commits from `extended-tracer`
maoueh Feb 6, 2024
5456ce5
Merge branch 'extended-tracer-backport-1.13.5' into extended-tracer-s…
maoueh Feb 6, 2024
31b3b7d
rm OnNewAccount
maoueh Feb 8, 2024
7ff2ae6
indicate Known block
maoueh Feb 8, 2024
c5ebc52
Merge branch 'extended-tracer-backport-1.13.5' into extended-tracer-s…
maoueh Feb 8, 2024
bd651ce
Merge remote-tracking branch 'sei/master' into extended-tracer-sei
maoueh Feb 8, 2024
97e9a8b
fix blockchain -> logger circular import
s1na Feb 9, 2024
ee96dd2
live tracer err -> warn
s1na Feb 9, 2024
2264a9c
fix mdLogger
s1na Feb 9, 2024
04d3f64
Merge branch 'extended-tracer-backport-1.13.5' into extended-tracer-sei
maoueh Feb 9, 2024
39b25c9
Merge remote-tracking branch 'sei/master' into extended-tracer-sei
maoueh Feb 22, 2024
d0fd1bd
Fixed missing instrumentation for `txTrace` which was making a test fail
maoueh Feb 23, 2024
4952d19
Add method deny list in geth http server
yzang2019 Mar 7, 2024
8c577d5
Change error type to methodNotFoundError
yzang2019 Mar 12, 2024
c27c86e
Fix error code
yzang2019 Mar 12, 2024
318dc84
add config for tracers
s1na Feb 14, 2024
d17940d
minor fix
s1na Feb 14, 2024
b210cc7
remove onBeaconBlockRoot events
s1na Feb 15, 2024
8ec4560
refactor onBlockStart params, new skip method
s1na Feb 15, 2024
4df6904
use struct for tracing events
s1na Feb 16, 2024
4e43718
full struct-based tracing infra
s1na Feb 21, 2024
e0b9654
Add canceler interface
s1na Feb 21, 2024
6013241
fix memory copy util func
s1na Feb 27, 2024
f1027b7
fix goja scope object wrappers
s1na Feb 27, 2024
29ebfb3
fixes, rm t8ntool tracewriter
s1na Feb 27, 2024
e1c669b
moaar fixes
s1na Feb 28, 2024
b202e8a
renaming GetLogger
s1na Feb 28, 2024
39fc9dc
move logger interface to core/tracing
s1na Feb 28, 2024
d4ec634
rename GetLogger to Hooks
s1na Feb 28, 2024
688f0b9
rename scopeContext to opContext
s1na Feb 29, 2024
3253b8b
assign const values for gas change reasons
s1na Feb 29, 2024
e7fe236
a type for each hook
s1na Feb 29, 2024
a2ad953
rm old logger file
s1na Feb 29, 2024
25a3a74
rename Capture hooks to On
s1na Feb 29, 2024
03e48ee
rename tracer methods to On pattern
s1na Mar 6, 2024
7190602
rm embedding of noop tracer
s1na Mar 6, 2024
5c6fb31
rm OnKeccakPreimage
s1na Mar 6, 2024
ea02c9e
fix eth config
s1na Mar 6, 2024
94fc0b4
replace opcode with byte
s1na Mar 7, 2024
a476051
add fn refundGas
s1na Mar 7, 2024
d0a86c4
rm canceler
s1na Mar 7, 2024
77486e9
fix lint issue
s1na Mar 13, 2024
77a701c
rm extra param
s1na Mar 13, 2024
1770744
Call `OnBlockchainInit` before `OnGenesisBlock` (#22)
maoueh Mar 14, 2024
28a80d6
fix empty tracer config
s1na Mar 14, 2024
3834f13
Series of changes to get backport to 1.13.5 working
maoueh Mar 14, 2024
17fcdfa
Fixed incorrect `captureBeging/captureEnd` in EVM
maoueh Mar 15, 2024
f596461
Backported a bunch more of `eth/tracers`
maoueh Mar 15, 2024
45cb5cd
Merge branch 'extended-tracer-backport-1.13.5' into extended-tracer-sei
maoueh Mar 15, 2024
7ea867c
Reverted some changes that are not required anymore with the new trac…
maoueh Mar 15, 2024
7aaf4d2
Merge tag 'v1.13.5-sei-11' into extended-tracer-sei
maoueh Mar 15, 2024
313a43d
cmd/evm: fix tracing
holiman Mar 18, 2024
a44b99b
cmd/evm: add testcase for js-tracer
holiman Mar 18, 2024
8ae03dd
core/vm: documentation + minor refactoring
holiman Mar 18, 2024
c76d4f0
eth/tracers, cmd/evm: unexport JSONlogger
holiman Mar 18, 2024
9beae74
Merge branch 'extended-tracer-backport-1.13.5' into extended-tracer-sei
maoueh Mar 18, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion accounts/abi/bind/backends/simulated.go
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ import (
"github.com/ethereum/go-ethereum/core/bloombits"
"github.com/ethereum/go-ethereum/core/rawdb"
"github.com/ethereum/go-ethereum/core/state"
"github.com/ethereum/go-ethereum/core/tracing"
"github.com/ethereum/go-ethereum/core/types"
"github.com/ethereum/go-ethereum/core/vm"
"github.com/ethereum/go-ethereum/eth/filters"
Expand Down Expand Up @@ -681,7 +682,7 @@ func (b *SimulatedBackend) callContract(ctx context.Context, call ethereum.CallM

// Set infinite balance to the fake caller account.
from := stateDB.(*state.StateDB).GetOrNewStateObject(call.From)
from.SetBalance(math.MaxBig256)
from.SetBalance(math.MaxBig256, tracing.BalanceChangeUnspecified)

// Execute the call.
msg := &core.Message{
Expand Down
4 changes: 2 additions & 2 deletions cmd/evm/blockrunner.go
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ import (
"github.com/ethereum/go-ethereum/core"
"github.com/ethereum/go-ethereum/core/rawdb"
estate "github.com/ethereum/go-ethereum/core/state"
"github.com/ethereum/go-ethereum/core/vm"
"github.com/ethereum/go-ethereum/core/tracing"
"github.com/ethereum/go-ethereum/eth/tracers/logger"
"github.com/ethereum/go-ethereum/tests"
"github.com/urfave/cli/v2"
Expand All @@ -52,7 +52,7 @@ func blockTestCmd(ctx *cli.Context) error {
return errors.New("path-to-test argument required")
}

var tracer vm.EVMLogger
var tracer *tracing.Hooks
// Configure the EVM logger
if ctx.Bool(MachineFlag.Name) {
tracer = logger.NewJSONLogger(&logger.Config{
Expand Down
50 changes: 43 additions & 7 deletions cmd/evm/internal/t8ntool/execution.go
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,9 @@
package t8ntool

import (
"encoding/json"
"fmt"
"io"
"math/big"

"github.com/ethereum/go-ethereum/common"
Expand All @@ -28,9 +30,11 @@ import (
"github.com/ethereum/go-ethereum/core"
"github.com/ethereum/go-ethereum/core/rawdb"
"github.com/ethereum/go-ethereum/core/state"
"github.com/ethereum/go-ethereum/core/tracing"
"github.com/ethereum/go-ethereum/core/types"
"github.com/ethereum/go-ethereum/core/vm"
"github.com/ethereum/go-ethereum/crypto"
"github.com/ethereum/go-ethereum/eth/tracers/directory"
"github.com/ethereum/go-ethereum/ethdb"
"github.com/ethereum/go-ethereum/log"
"github.com/ethereum/go-ethereum/params"
Expand Down Expand Up @@ -117,7 +121,7 @@ type rejectedTx struct {
// Apply applies a set of transactions to a pre-state
func (pre *Prestate) Apply(vmConfig vm.Config, chainConfig *params.ChainConfig,
txIt txIterator, miningReward int64,
getTracerFn func(txIndex int, txHash common.Hash) (tracer vm.EVMLogger, err error)) (*state.StateDB, *ExecutionResult, []byte, error) {
getTracerFn func(txIndex int, txHash common.Hash) (*directory.Tracer, io.WriteCloser, error)) (*state.StateDB, *ExecutionResult, []byte, error) {
// Capture errors for BLOCKHASH operation, if we haven't been supplied the
// required blockhashes
var hashError error
Expand Down Expand Up @@ -220,11 +224,13 @@ func (pre *Prestate) Apply(vmConfig vm.Config, chainConfig *params.ChainConfig,
}
blobGasUsed += txBlobGas
}
tracer, err := getTracerFn(txIndex, tx.Hash())
tracer, traceOutput, err := getTracerFn(txIndex, tx.Hash())
if err != nil {
return nil, nil, nil, err
}
vmConfig.Tracer = tracer
if tracer != nil {
vmConfig.Tracer = tracer.Hooks
}
statedb.SetTxContext(tx.Hash(), txIndex)

var (
Expand All @@ -234,13 +240,24 @@ func (pre *Prestate) Apply(vmConfig vm.Config, chainConfig *params.ChainConfig,
)
evm := vm.NewEVM(vmContext, txContext, statedb, chainConfig, vmConfig)

if tracer != nil && tracer.OnTxStart != nil {
tracer.OnTxStart(evm.GetVMContext(), tx, msg.From)
}
// (ret []byte, usedGas uint64, failed bool, err error)
msgResult, err := core.ApplyMessage(evm, msg, gaspool)
if err != nil {
statedb.RevertToSnapshot(snapshot)
log.Info("rejected tx", "index", i, "hash", tx.Hash(), "from", msg.From, "error", err)
rejectedTxs = append(rejectedTxs, &rejectedTx{i, err.Error()})
gaspool.SetGas(prevGas)
if tracer != nil {
if tracer.OnTxEnd != nil {
tracer.OnTxEnd(nil, err)
}
if err := writeTraceResult(tracer, traceOutput); err != nil {
log.Warn("Error writing tracer output", "err", err)
}
}
continue
}
includedTxs = append(includedTxs, tx)
Expand Down Expand Up @@ -282,6 +299,12 @@ func (pre *Prestate) Apply(vmConfig vm.Config, chainConfig *params.ChainConfig,
//receipt.BlockNumber
receipt.TransactionIndex = uint(txIndex)
receipts = append(receipts, receipt)
if tracer != nil {
if tracer.Hooks.OnTxEnd != nil {
tracer.Hooks.OnTxEnd(receipt, nil)
}
writeTraceResult(tracer, traceOutput)
}
}

txIndex++
Expand All @@ -307,15 +330,15 @@ func (pre *Prestate) Apply(vmConfig vm.Config, chainConfig *params.ChainConfig,
reward.Sub(reward, new(big.Int).SetUint64(ommer.Delta))
reward.Mul(reward, blockReward)
reward.Div(reward, big.NewInt(8))
statedb.AddBalance(ommer.Address, reward)
statedb.AddBalance(ommer.Address, reward, tracing.BalanceIncreaseRewardMineUncle)
}
statedb.AddBalance(pre.Env.Coinbase, minerReward)
statedb.AddBalance(pre.Env.Coinbase, minerReward, tracing.BalanceIncreaseRewardMineBlock)
}
// Apply withdrawals
for _, w := range pre.Env.Withdrawals {
// Amount is in gwei, turn into wei
amount := new(big.Int).Mul(new(big.Int).SetUint64(w.Amount), big.NewInt(params.GWei))
statedb.AddBalance(w.Address, amount)
statedb.AddBalance(w.Address, amount, tracing.BalanceIncreaseWithdrawal)
}
// Commit block
root, err := statedb.Commit(vmContext.BlockNumber.Uint64(), chainConfig.IsEIP158(vmContext.BlockNumber))
Expand Down Expand Up @@ -358,7 +381,7 @@ func MakePreState(db ethdb.Database, accounts core.GenesisAlloc) *state.StateDB
for addr, a := range accounts {
statedb.SetCode(addr, a.Code)
statedb.SetNonce(addr, a.Nonce)
statedb.SetBalance(addr, a.Balance)
statedb.SetBalance(addr, a.Balance, tracing.BalanceIncreaseGenesisBalance)
for k, v := range a.Storage {
statedb.SetState(addr, k, v)
}
Expand Down Expand Up @@ -395,3 +418,16 @@ func calcDifficulty(config *params.ChainConfig, number, currentTime, parentTime
}
return ethash.CalcDifficulty(config, currentTime, parent)
}

func writeTraceResult(tracer *directory.Tracer, f io.WriteCloser) error {
defer f.Close()
result, err := tracer.GetResult()
if err != nil || result == nil {
return err
}
err = json.NewEncoder(f).Encode(result)
if err != nil {
return err
}
return nil
}
8 changes: 8 additions & 0 deletions cmd/evm/internal/t8ntool/flags.go
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,14 @@ var (
Name: "trace",
Usage: "Output full trace logs to files <txhash>.jsonl",
}
TraceTracerFlag = &cli.StringFlag{
Name: "trace.tracer",
Usage: "Configures the use of a custom tracer, e.g native or js tracers. Examples are callTracer and 4byteTracer. These tracers emit results into files as trace-<txIndex>-<txHash>.json",
}
TraceTracerConfigFlag = &cli.StringFlag{
Name: "trace.jsonconfig",
Usage: "The configurations for the custom tracer specified by --trace.tracer. If provided, must be in JSON format",
}
TraceDisableMemoryFlag = &cli.BoolFlag{
Name: "trace.nomemory",
Value: true,
Expand Down
48 changes: 29 additions & 19 deletions cmd/evm/internal/t8ntool/transition.go
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ import (
"encoding/json"
"errors"
"fmt"
"io"
"math/big"
"os"
"path"
Expand All @@ -33,6 +34,7 @@ import (
"github.com/ethereum/go-ethereum/core/state"
"github.com/ethereum/go-ethereum/core/types"
"github.com/ethereum/go-ethereum/core/vm"
"github.com/ethereum/go-ethereum/eth/tracers/directory"
"github.com/ethereum/go-ethereum/eth/tracers/logger"
"github.com/ethereum/go-ethereum/log"
"github.com/ethereum/go-ethereum/params"
Expand Down Expand Up @@ -87,12 +89,7 @@ func Transition(ctx *cli.Context) error {
glogger.Verbosity(slog.Level(ctx.Int(VerbosityFlag.Name)))
log.SetDefault(log.NewLogger(glogger))

var (
err error
tracer vm.EVMLogger
)
var getTracer func(txIndex int, txHash common.Hash) (vm.EVMLogger, error)

var getTracer = func(txIndex int, txHash common.Hash) (*directory.Tracer, io.WriteCloser, error) { return nil, nil, nil }
baseDir, err := createBasedir(ctx)
if err != nil {
return NewError(ErrorIO, fmt.Errorf("failed creating output basedir: %v", err))
Expand Down Expand Up @@ -124,20 +121,35 @@ func Transition(ctx *cli.Context) error {
prevFile.Close()
}
}()
getTracer = func(txIndex int, txHash common.Hash) (vm.EVMLogger, error) {
if prevFile != nil {
prevFile.Close()
}
getTracer = func(txIndex int, txHash common.Hash) (*directory.Tracer, io.WriteCloser, error) {
traceFile, err := os.Create(path.Join(baseDir, fmt.Sprintf("trace-%d-%v.jsonl", txIndex, txHash.String())))
if err != nil {
return nil, NewError(ErrorIO, fmt.Errorf("failed creating trace-file: %v", err))
return nil, nil, NewError(ErrorIO, fmt.Errorf("failed creating trace-file: %v", err))
}
logger := logger.NewJSONLogger(logConfig, traceFile)
tracer := &directory.Tracer{
Hooks: logger,
// jsonLogger streams out result to file.
GetResult: func() (json.RawMessage, error) { return nil, nil },
Stop: func(err error) {},
}
prevFile = traceFile
return logger.NewJSONLogger(logConfig, traceFile), nil
return tracer, traceFile, nil
}
} else {
getTracer = func(txIndex int, txHash common.Hash) (tracer vm.EVMLogger, err error) {
return nil, nil
} else if ctx.IsSet(TraceTracerFlag.Name) {
var config json.RawMessage
if ctx.IsSet(TraceTracerConfigFlag.Name) {
config = []byte(ctx.String(TraceTracerConfigFlag.Name))
}
getTracer = func(txIndex int, txHash common.Hash) (*directory.Tracer, io.WriteCloser, error) {
traceFile, err := os.Create(path.Join(baseDir, fmt.Sprintf("trace-%d-%v.json", txIndex, txHash.String())))
if err != nil {
return nil, nil, NewError(ErrorIO, fmt.Errorf("failed creating trace-file: %v", err))
}
tracer, err := directory.DefaultDirectory.New(ctx.String(TraceTracerFlag.Name), nil, config)
if err != nil {
return nil, nil, NewError(ErrorConfig, fmt.Errorf("failed instantiating tracer: %w", err))
}
return tracer, traceFile, nil
}
}
// We need to load three things: alloc, env and transactions. May be either in
Expand Down Expand Up @@ -176,9 +188,7 @@ func Transition(ctx *cli.Context) error {
}
prestate.Env = *inputData.Env

vmConfig := vm.Config{
Tracer: tracer,
}
vmConfig := vm.Config{}
// Construct the chainconfig
var chainConfig *params.ChainConfig
if cConf, extraEips, err := tests.GetChainConfig(ctx.String(ForknameFlag.Name)); err != nil {
Expand Down
5 changes: 3 additions & 2 deletions cmd/evm/runner.go
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ import (
"github.com/ethereum/go-ethereum/core"
"github.com/ethereum/go-ethereum/core/rawdb"
"github.com/ethereum/go-ethereum/core/state"
"github.com/ethereum/go-ethereum/core/tracing"
"github.com/ethereum/go-ethereum/core/vm"
"github.com/ethereum/go-ethereum/core/vm/runtime"
"github.com/ethereum/go-ethereum/eth/tracers/logger"
Expand Down Expand Up @@ -116,7 +117,7 @@ func runCmd(ctx *cli.Context) error {
}

var (
tracer vm.EVMLogger
tracer *tracing.Hooks
debugLogger *logger.StructLogger
statedb *state.StateDB
chainConfig *params.ChainConfig
Expand All @@ -130,7 +131,7 @@ func runCmd(ctx *cli.Context) error {
tracer = logger.NewJSONLogger(logconfig, os.Stdout)
} else if ctx.Bool(DebugFlag.Name) {
debugLogger = logger.NewStructLogger(logconfig)
tracer = debugLogger
tracer = debugLogger.Hooks()
} else {
debugLogger = logger.NewStructLogger(logconfig)
}
Expand Down
2 changes: 1 addition & 1 deletion cmd/evm/staterunner.go
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ func stateTestCmd(ctx *cli.Context) error {
cfg.Tracer = logger.NewJSONLogger(config, os.Stderr)

case ctx.Bool(DebugFlag.Name):
cfg.Tracer = logger.NewStructLogger(config)
cfg.Tracer = logger.NewStructLogger(config).Hooks()
}
// Load the test content from the input file
if len(ctx.Args().First()) != 0 {
Expand Down
Loading
Loading