Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

vms/platformvm: Verify txs before building a block #2359

Merged
merged 123 commits into from
Dec 12, 2023
Merged
Show file tree
Hide file tree
Changes from 107 commits
Commits
Show all changes
123 commits
Select commit Hold shift + click to select a range
b237933
`vms/platformvm`: Move `toEngine` channel to mempool
dhrubabasu Nov 17, 2023
cb81c85
nit
dhrubabasu Nov 17, 2023
03efd7e
short-circuit
dhrubabasu Nov 17, 2023
7ba8155
Merge branch 'dev' into move-engine-to-mempool
dhrubabasu Nov 17, 2023
df70afb
Merge branch 'dev' into move-engine-to-mempool
dhrubabasu Nov 17, 2023
86a559b
merge
dhrubabasu Nov 22, 2023
7eadabe
Refactor P-Chain block building
dhrubabasu Nov 22, 2023
bb05c02
check in
dhrubabasu Nov 24, 2023
1a2431f
Merge branch 'dev' into move-engine-to-mempool
dhrubabasu Nov 25, 2023
79d0607
Fix P-chain mempool tx count metric (#2361)
StephenButtolph Nov 22, 2023
d3e73d6
Update versions for v1.10.16 (#2353)
StephenButtolph Nov 23, 2023
42da99a
Remove Banff check from mempool verifier (#2360)
dhrubabasu Nov 23, 2023
fdfcbc7
Document storage growth in readme (#2364)
StephenButtolph Nov 24, 2023
a11cf08
Add metric for duration between block timestamp and block acceptance …
StephenButtolph Nov 24, 2023
d423d5c
Merge branch 'dev' into p-block-build-refactor
dhrubabasu Nov 27, 2023
43a9cb2
wip
dhrubabasu Nov 27, 2023
6eeb555
check err
dhrubabasu Nov 27, 2023
a42155b
`vms/platformvm`: Remove `HasTxs` and `PeekTxs` from `Mempool` interface
dhrubabasu Nov 27, 2023
dc95688
Merge branch 'dev' into remove-peek-txs
dhrubabasu Nov 27, 2023
9738f4a
Merge branch 'move-engine-to-mempool' into remove-peek-txs
dhrubabasu Nov 27, 2023
a55ce38
Merge branch 'dev' into move-engine-to-mempool
dhrubabasu Nov 27, 2023
27e7641
Merge branch 'move-engine-to-mempool' into remove-peek-txs
dhrubabasu Nov 27, 2023
d96cf85
fix len check
dhrubabasu Nov 27, 2023
f2b5822
nits
dhrubabasu Nov 27, 2023
3f0afe4
more
dhrubabasu Nov 27, 2023
de88d17
nit
dhrubabasu Nov 27, 2023
ae439bc
reduce diff
dhrubabasu Nov 27, 2023
8c794eb
Merge branch 'dev' into move-engine-to-mempool
dhrubabasu Nov 27, 2023
b053f85
Merge branch 'move-engine-to-mempool' into remove-peek-txs
dhrubabasu Nov 27, 2023
de4b995
remove defer
dhrubabasu Nov 27, 2023
035c2e6
nit
dhrubabasu Nov 27, 2023
06ee204
fix
dhrubabasu Nov 28, 2023
f3f2fbb
`vms/platformvm`: Remove double block building logic
dhrubabasu Nov 28, 2023
776db3e
nits
dhrubabasu Nov 28, 2023
eeb97d9
nits
dhrubabasu Nov 28, 2023
5624df8
merged
dhrubabasu Nov 28, 2023
b8bf721
merged
dhrubabasu Nov 28, 2023
6d5225a
merged
dhrubabasu Nov 28, 2023
efbb751
pr review
dhrubabasu Nov 28, 2023
903d83b
nit
dhrubabasu Nov 28, 2023
cd669b2
nit
dhrubabasu Nov 28, 2023
a25ef67
Merge branch 'dev' into stop-building-block-twice
dhrubabasu Nov 28, 2023
1d30f2b
Merge branch 'dev' into stop-building-block-twice
dhrubabasu Nov 29, 2023
e2c7817
merged
dhrubabasu Nov 29, 2023
b331747
restore enable and disable adding
dhrubabasu Nov 29, 2023
d469d88
reduce diff
dhrubabasu Nov 29, 2023
62424f2
reduce diff
dhrubabasu Nov 29, 2023
10b3004
merged
dhrubabasu Nov 29, 2023
f6a9548
reduce diff
dhrubabasu Nov 29, 2023
c0c04f2
add test for DropExpiredStakerTxs
dhrubabasu Nov 29, 2023
58f08ee
Merge branch 'dev' into stop-building-block-twice
dhrubabasu Nov 29, 2023
ff0f319
Merge branch 'dev' into stop-building-block-twice
dhrubabasu Nov 30, 2023
df125d5
pr review
dhrubabasu Nov 30, 2023
06b1bdc
cleanup
dhrubabasu Nov 30, 2023
80d00f2
add comments to test
dhrubabasu Dec 1, 2023
5b5b9c0
lint
dhrubabasu Dec 1, 2023
9ebb7f5
merge dev
dhrubabasu Dec 2, 2023
ef7cd6b
merge dev
dhrubabasu Dec 2, 2023
efc93fe
nit
dhrubabasu Dec 2, 2023
54edb7b
Merge branch 'stop-building-block-twice' into remove-peek-txs
dhrubabasu Dec 2, 2023
240867b
Merge branch 'dev' into stop-building-block-twice
dhrubabasu Dec 2, 2023
d2afdc3
Merge branch 'stop-building-block-twice' into remove-peek-txs
dhrubabasu Dec 2, 2023
e1bef55
Merge branch 'remove-peek-txs' into p-block-build-refactor
dhrubabasu Dec 2, 2023
980bd4d
wip
dhrubabasu Dec 3, 2023
cbb3868
nit
dhrubabasu Dec 3, 2023
9cd31a3
nit
dhrubabasu Dec 3, 2023
2266e7c
nit
dhrubabasu Dec 3, 2023
136b023
Merge branch 'stop-building-block-twice' into remove-peek-txs
dhrubabasu Dec 3, 2023
7e81ca7
Merge branch 'remove-peek-txs' into p-block-build-refactor
dhrubabasu Dec 3, 2023
f615b40
nit
dhrubabasu Dec 3, 2023
b5e4b25
generate mocks
dhrubabasu Dec 3, 2023
83ff092
nit
dhrubabasu Dec 3, 2023
645b989
merged
dhrubabasu Dec 4, 2023
16160b7
reduce diff
dhrubabasu Dec 4, 2023
5b70127
nits
dhrubabasu Dec 4, 2023
c1cff7a
wip
dhrubabasu Dec 4, 2023
5520c3e
wip
dhrubabasu Dec 4, 2023
8e19e4e
wip
dhrubabasu Dec 4, 2023
3faa32a
Merge branch 'stop-building-block-twice' into remove-peek-txs
dhrubabasu Dec 4, 2023
6361267
Merge branch 'dev' into stop-building-block-twice
dhrubabasu Dec 4, 2023
32f16f4
Merge branch 'stop-building-block-twice' into remove-peek-txs
dhrubabasu Dec 4, 2023
2ebfe1d
merged
dhrubabasu Dec 4, 2023
4b6e354
`vms/platformvm`: Remove `EnableAdding` and `DisableAdding`
dhrubabasu Dec 11, 2023
8d1b2b8
Merge branch 'dev' into slim-mempool-inft
dhrubabasu Dec 11, 2023
1745111
Merge branch 'dev' into slim-mempool-inft
dhrubabasu Dec 11, 2023
b741693
Remove `PeekTxs`
dhrubabasu Dec 11, 2023
32b3b81
reduce diff
dhrubabasu Dec 11, 2023
18b7822
Merge branch 'slim-mempool-inft' into remove-peek-txs
dhrubabasu Dec 11, 2023
8d5c0ab
merged
dhrubabasu Dec 11, 2023
8f3b855
reduce diff
dhrubabasu Dec 11, 2023
00a9303
reduce diff
dhrubabasu Dec 11, 2023
dad6150
nit
dhrubabasu Dec 11, 2023
fe7637c
merged
dhrubabasu Dec 11, 2023
25f9eb2
`vms/platformvm`: Surface `VerifyUniqueInputs` in the `Manager`
dhrubabasu Dec 11, 2023
c404e51
generate from source
dhrubabasu Dec 11, 2023
f31d532
nit
dhrubabasu Dec 11, 2023
1b215c1
Merge branch 'dev' into surface-verify-unique-inputs
dhrubabasu Dec 11, 2023
1934731
merged
dhrubabasu Dec 11, 2023
63cf84a
nit
dhrubabasu Dec 11, 2023
a8a4e7d
nit
dhrubabasu Dec 11, 2023
8d32f1d
nit
dhrubabasu Dec 11, 2023
b3bcc27
nit
dhrubabasu Dec 11, 2023
580bf33
nit
dhrubabasu Dec 11, 2023
959b566
remove tests for now
dhrubabasu Dec 11, 2023
3db339b
nit
dhrubabasu Dec 11, 2023
148f150
nit
dhrubabasu Dec 11, 2023
6eac64a
nit
dhrubabasu Dec 11, 2023
dc14bfd
packBlockTxs
dhrubabasu Dec 11, 2023
88e37d1
`vms/platformvm`: Add `TestBuildBlockShouldAdvanceTime` test
dhrubabasu Dec 11, 2023
c62a39b
`vms/platformvm`: Add `TestBuildBlockForceAdvanceTime` test
dhrubabasu Dec 11, 2023
3d84cc9
nit
dhrubabasu Dec 11, 2023
f74db4e
nit
dhrubabasu Dec 11, 2023
8df714b
nit
dhrubabasu Dec 12, 2023
0ab1ddc
merged
dhrubabasu Dec 12, 2023
36447af
merged
dhrubabasu Dec 12, 2023
8053fa1
merged
dhrubabasu Dec 12, 2023
0c9ac46
nits
dhrubabasu Dec 12, 2023
7e16518
txSize
dhrubabasu Dec 12, 2023
96e4250
merged
dhrubabasu Dec 12, 2023
a2ff9c1
nit
dhrubabasu Dec 12, 2023
2051ff1
ok stephen
dhrubabasu Dec 12, 2023
485051f
nits
dhrubabasu Dec 12, 2023
6c0271f
Merge branch 'dev' into p-block-build-refactor
dhrubabasu Dec 12, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion scripts/mocks.mockgen.txt
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,6 @@ github.com/ava-labs/avalanchego/vms/avm/state=Chain,State,Diff=vms/avm/state/moc
github.com/ava-labs/avalanchego/vms/avm/txs/mempool=Mempool=vms/avm/txs/mempool/mock_mempool.go
github.com/ava-labs/avalanchego/vms/components/avax=TransferableIn=vms/components/avax/mock_transferable_in.go
github.com/ava-labs/avalanchego/vms/components/verify=Verifiable=vms/components/verify/mock_verifiable.go
github.com/ava-labs/avalanchego/vms/platformvm/block/executor=Manager=vms/platformvm/block/executor/mock_manager.go
github.com/ava-labs/avalanchego/vms/platformvm/block=Block=vms/platformvm/block/mock_block.go
github.com/ava-labs/avalanchego/vms/platformvm/state=Chain,Diff,State,Versions=vms/platformvm/state/mock_state.go
github.com/ava-labs/avalanchego/vms/platformvm/state=StakerIterator=vms/platformvm/state/mock_staker_iterator.go
Expand Down
141 changes: 107 additions & 34 deletions vms/platformvm/block/builder/builder.go
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,12 @@ import (

"github.com/ava-labs/avalanchego/ids"
"github.com/ava-labs/avalanchego/snow/consensus/snowman"
"github.com/ava-labs/avalanchego/utils/set"
"github.com/ava-labs/avalanchego/utils/timer/mockable"
"github.com/ava-labs/avalanchego/utils/units"
"github.com/ava-labs/avalanchego/vms/platformvm/block"
"github.com/ava-labs/avalanchego/vms/platformvm/state"
"github.com/ava-labs/avalanchego/vms/platformvm/status"
"github.com/ava-labs/avalanchego/vms/platformvm/txs"
"github.com/ava-labs/avalanchego/vms/platformvm/txs/mempool"

Expand Down Expand Up @@ -192,16 +194,9 @@ func (b *builder) ShutdownBlockTimer() {
// This method removes the transactions from the returned
// blocks from the mempool.
func (b *builder) BuildBlock(context.Context) (snowman.Block, error) {
defer func() {
// If we need to advance the chain's timestamp in a standard block, but
// we build an invalid block, then we need to re-trigger block building.
//
// TODO: Remove once we are guaranteed to build a valid block.
b.ResetBlockTimer()
// If there are still transactions in the mempool, then we need to
// re-trigger block building.
b.Mempool.RequestBuildBlock(false /*=emptyBlockPermitted*/)
}()
// If there are still transactions in the mempool, then we need to
// re-trigger block building.
defer b.Mempool.RequestBuildBlock(false /*=emptyBlockPermitted*/)
dhrubabasu marked this conversation as resolved.
Show resolved Hide resolved

b.txExecutorBackend.Ctx.Log.Debug("starting to attempt to build a block")

Expand Down Expand Up @@ -237,6 +232,81 @@ func (b *builder) BuildBlock(context.Context) (snowman.Block, error) {
return b.blkManager.NewBlock(statelessBlk), nil
}

func buildBlockTxs(mempool mempool.Mempool, backend *txexecutor.Backend, manager blockexecutor.Manager, timestamp time.Time) ([]*txs.Tx, error) {
preferredID := manager.Preferred()
stateDiff, err := state.NewDiff(preferredID, manager)
if err != nil {
return nil, err
}

changes, err := txexecutor.AdvanceTimeTo(backend, stateDiff, timestamp)
if err != nil {
return nil, err
}
changes.Apply(stateDiff)
stateDiff.SetTimestamp(timestamp)

var (
blockTxs []*txs.Tx
inputs set.Set[ids.ID]
remainingSize = targetBlockSize
)

for {
tx, exists := mempool.Peek()
if !exists || len(tx.Bytes()) > remainingSize {
break
}
mempool.Remove([]*txs.Tx{tx})

// Invariant: [tx] has already been syntactically verified.

txDiff, err := wrapState(stateDiff)
if err != nil {
return nil, err
}

executor := &txexecutor.StandardTxExecutor{
Backend: backend,
State: txDiff,
Tx: tx,
}

err = tx.Unsigned.Visit(executor)
if err != nil {
txID := tx.ID()
mempool.MarkDropped(txID, err)
continue
}

if inputs.Overlaps(executor.Inputs) {
txID := tx.ID()
mempool.MarkDropped(txID, blockexecutor.ErrConflictingBlockTxs)
continue
}
err = manager.VerifyUniqueInputs(preferredID, inputs)
dhrubabasu marked this conversation as resolved.
Show resolved Hide resolved
dhrubabasu marked this conversation as resolved.
Show resolved Hide resolved
if err != nil {
txID := tx.ID()
mempool.MarkDropped(txID, err)
continue
}
inputs.Union(executor.Inputs)

txDiff.AddTx(tx, status.Committed)
err = txDiff.Apply(stateDiff)
if err != nil {
txID := tx.ID()
mempool.MarkDropped(txID, err)
continue
}
StephenButtolph marked this conversation as resolved.
Show resolved Hide resolved

remainingSize -= len(tx.Bytes())
dhrubabasu marked this conversation as resolved.
Show resolved Hide resolved
blockTxs = append(blockTxs, tx)
}

return blockTxs, nil
}

// [timestamp] is min(max(now, parent timestamp), next staker change time)
func buildBlock(
builder *builder,
Expand All @@ -259,38 +329,27 @@ func buildBlock(
return nil, fmt.Errorf("could not build tx to reward staker: %w", err)
}

blockTxs := []*txs.Tx{}
dhrubabasu marked this conversation as resolved.
Show resolved Hide resolved
// TODO: Cleanup post-Durango
if builder.txExecutorBackend.Config.IsDurangoActivated(timestamp) {
blockTxs, err = buildBlockTxs(builder.Mempool, builder.txExecutorBackend, builder.blkManager, timestamp)
dhrubabasu marked this conversation as resolved.
Show resolved Hide resolved
if err != nil {
return nil, fmt.Errorf("failed to build blockTxs: %w", err)
}
}

return block.NewBanffProposalBlock(
timestamp,
parentID,
height,
rewardValidatorTx,
[]*txs.Tx{}, // TODO: Populate with StandardBlock txs
)
}

// Clean out the mempool's transactions with invalid timestamps.
droppedStakerTxIDs := builder.Mempool.DropExpiredStakerTxs(timestamp.Add(txexecutor.SyncBound))
for _, txID := range droppedStakerTxIDs {
builder.txExecutorBackend.Ctx.Log.Debug("dropping tx",
zap.Stringer("txID", txID),
zap.Error(err),
blockTxs,
)
}

var (
blockTxs []*txs.Tx
remainingSize = targetBlockSize
)

for {
tx, exists := builder.Mempool.Peek()
if !exists || len(tx.Bytes()) > remainingSize {
break
}
builder.Mempool.Remove([]*txs.Tx{tx})

remainingSize -= len(tx.Bytes())
blockTxs = append(blockTxs, tx)
blockTxs, err := buildBlockTxs(builder.Mempool, builder.txExecutorBackend, builder.blkManager, timestamp)
if err != nil {
return nil, fmt.Errorf("failed to build blockTxs: %w", err)
}

// If there is no reason to build a block, don't.
Expand Down Expand Up @@ -342,3 +401,17 @@ func getNextStakerToReward(
}
return ids.Empty, false, nil
}

type stateGetter struct {
state state.Chain
}

func (s stateGetter) GetState(ids.ID) (state.Chain, bool) {
return s.state, true
}

func wrapState(parentState state.Chain) (state.Diff, error) {
return state.NewDiff(ids.Empty, stateGetter{
state: parentState,
})
}
dhrubabasu marked this conversation as resolved.
Show resolved Hide resolved
Loading
Loading