Deneb Support #564

avalonche · 2024-01-04T17:49:04Z

📝 Summary

mev-boost-relay changes for the deneb upgrade. See #489 for changes included in this branch. Summary:

Clean up of custom types in common/types.go and use attestant types instead
Default to PublishBlockV2 endpoint for block publishing
Deneb types and blob validation

📚 References

Specs

Builder Specs - https://github.com/ethereum/builder-specs around deneb types
Dropping blinded blob sidecar - Drop "blinded blob sidecar" concept after using signed header to authenticate blobs ethereum/builder-specs#90 - short Merkle proof from the signed header's body root to each KZG commitment gossiped by the CL via the PublishBlockV2 endpoint
Relay Specs - https://github.com/flashbots/relay-specs around the deneb block validation endpoint
Details on the 4844 upgrade - https://www.eip4844.com/.

✅ I have run these commands

make lint
make test-race
go mod tidy
I have seen and agree to CONTRIBUTING.md

codecov-commenter · 2024-01-04T19:14:19Z

Codecov Report

Attention: 778 lines in your changes are missing coverage. Please review.

Comparison is base (930f084) 33.24% compared to head (1ffdc1a) 35.66%.

Files	Patch %	Lines
common/types_spec.go	21.81%	211 Missing and 4 partials ⚠️
services/api/service.go	42.02%	136 Missing and 13 partials ⚠️
common/test_utils.go	8.52%	118 Missing ⚠️
common/utils.go	42.10%	51 Missing and 15 partials ⚠️
services/api/utils.go	18.64%	41 Missing and 7 partials ⚠️
datastore/redis.go	59.55%	29 Missing and 7 partials ⚠️
beaconclient/util.go	3.33%	28 Missing and 1 partial ⚠️
common/types.go	33.33%	16 Missing ⚠️
beaconclient/multi_beacon_client.go	0.00%	15 Missing ⚠️
beaconclient/prod_beacon_instance.go	11.76%	14 Missing and 1 partial ⚠️
... and 10 more

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #564      +/-   ##
==========================================
+ Coverage   33.24%   35.66%   +2.41%     
==========================================
  Files          24       24              
  Lines        5087     5227     +140     
==========================================
+ Hits         1691     1864     +173     
+ Misses       3177     3106      -71     
- Partials      219      257      +38

Flag	Coverage Δ
unittests	`35.66% <31.27%> (+2.41%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

* Remove bellatrix from wrapper types * remove test logs

* remove signed blinded beacon block wrapper type * linting

* remove submit block request wrapper types * fix tests * fix lint

* Upgrade go-boost-utils * pr comments * remove commented out code

* Make attestantio import names consistent * Fix linter errors & two comments

* Fix mistake in redis prefix for deneb * Fix typo in prefix

beaconclient/multi_beacon_client.go

beaconclient/prod_beacon_instance.go

common/test_utils.go

common/types_spec.go

jtraglia · 2024-01-19T18:53:46Z

common/utils.go

+		Value:                      value,
+		PrevRandao:                 prevRandao,
+		BlockNumber:                blockNumber,
+		Withdrawals:                withdrawals,


What about BlobGasUsed and ExcessBlobGas? The type doesn't have those fields.

not needed as those fields are validated at the builder, this is just an helper struct that grabs the common fields needed for validation at the relay

datastore/memcached_test.go

services/api/types_test.go

database/database_test.go

datastore/redis.go

jtraglia · 2024-01-19T21:28:27Z

common/types_spec.go

+type BuilderBlockValidationRequest struct {
+	*VersionedSubmitBlockRequest
+	RegisteredGasLimit    uint64       `json:"registered_gas_limit,string"`
+	ParentBeaconBlockRoot *phase0.Root `json:"parent_beacon_block_root,omitempty"`
 }


Will the validator function allow ParentBeaconBlockRoot to be missing? It doesn't appear to be an optional field in the builder's BuilderBlockValidationRequestV3 struct. Should BuilderBlockValidationRequest (here in the relay) have the omitempty qualifier?

I think the builder's BuilderBlockValidationRequestV3 struct will unmashal to a zero value (0 hash) as it does not validate that it is a required field.

README.md

beaconclient/multi_beacon_client.go

beaconclient/prod_beacon_instance.go

beaconclient/util.go

datastore/redis.go

services/api/service.go

michaelneuder

tiny nits, still wrapping my head around it. will take another pass when im more fresh tomorrow

common/types_spec.go

jtraglia · 2024-01-25T13:31:25Z

@avalonche I didn't realize that switch statement pattern (falllthrough) was used to appease exhaustive.

* backwards compatibility if no deneb schedule * Update services/api/service.go Co-authored-by: Chris Hager <[email protected]> --------- Co-authored-by: Chris Hager <[email protected]>

avalonche · 2024-01-25T19:54:18Z

@jtraglia yeah I didn't like having to list every version as the code only supports a small subset of the versions, so disabled the linter

metachris · 2024-01-25T20:03:55Z

PSA: We've deployed this PR (tip of deneb branch 1ffdc1a) to Flashbots mainnet relay.

* add test vectors * linting

avalonche changed the title ~~Deneb~~ Deneb Support Jan 4, 2024

avalonche force-pushed the deneb branch 2 times, most recently from 67cfe19 to 2f16414 Compare January 4, 2024 19:11

avalonche requested review from michaelneuder, metachris and jtraglia and removed request for metachris January 18, 2024 17:50

metachris added help wanted Extra attention is needed PSA labels Jan 19, 2024

avalonche and others added 19 commits January 19, 2024 20:44

Remove bellatrix from wrapper types (#475)

251af19

* Remove bellatrix from wrapper types * remove test logs

Remove get header wrapper types (#477)

c8f3b84

Remove signed blinded beacon block wrapper (#482)

b5a41c9

* remove signed blinded beacon block wrapper type * linting

remove signed beacon block wrapper types (#483)

3037d66

Remove submit block request wrapper (#485)

ece5601

* remove submit block request wrapper types * fix tests * fix lint

Upgrade go-boost-utils (#488)

d4bb90a

* Upgrade go-boost-utils * pr comments * remove commented out code

Add custom json marshalling for versioned structs (#493)

0d01945

Add deneb signature checking for block contents

45155e1

Add deneb support for type conversions

b555890

Add redis and database tests to store deneb payloads

7b6ee4d

Block submission to v3 validation endpoint

33cf57a

Update signed block conversions

0ee8e87

Replace some expectCont with expectOk (#509)

22df427

Allow fork epochs to be 0

0f3dd95

Make attestantio import names consistent (#510)

a4f4509

* Make attestantio import names consistent * Fix linter errors & two comments

Fix mistake in redis prefix name (#517)

7fdbbe3

* Fix mistake in redis prefix for deneb * Fix typo in prefix

rebase conflicts from main

f081c32

update submit block request

f72a2c5

bug fixes

632ab77

avalonche added 3 commits January 19, 2024 20:53

fix lint and tests

356b06d

switch to json encoding instead of ssz for block publishing v2

36a22c4

add blob logging

c1af95e

avalonche force-pushed the deneb branch from eb97dc3 to c1af95e Compare January 19, 2024 17:55

jtraglia reviewed Jan 19, 2024

View reviewed changes

database/database_test.go Outdated Show resolved Hide resolved

datastore/redis.go Outdated Show resolved Hide resolved

jtraglia reviewed Jan 19, 2024

View reviewed changes