Blobtx mining + brodcasting #2253

emailtovamos · 2024-03-05T10:07:04Z

Description

add a description of your changes here...

Rationale

tell us why we need these changes...

Example

add an example CLI or API response...

Changes

Notable changes:

add each change in a bullet point here
...

pool: switch ON blobpool+no reannotx

* chainconfig: use cancun fork for BSC; feat: fill WithdrawalsHash when BSC enable cancun fork; * rawdb: support to CRUD blobs; freezer: support to freeze block blobs; * freezer: support to freeze block blobs; * blockchain: add blob cache & blob query helper; * blockchain: add blob cache & blob query helper; * freezer: refactor addition table logic, add uts; * blobexpiry: add more extra expiry time, and logs; * ci: fix UT fails; * fix: fix some PR review comments; * parlia: implement IsDataAvailable function; * blob: refactor blob transfer logic; * blob: support config blob extra reserve;

core/types/block.go

eth/handler_eth.go

cmd/devp2p/internal/ethtest/conn.go

core/blockchain.go

eth/fetcher/block_fetcher.go

galaio · 2024-03-11T08:34:12Z

eth/protocols/eth/handlers.go

+	var packet BlockPacket
+
+	// First, try to decode as NewBlockPacket
+	nbp := new(NewBlockPacket)


Maybe we could add some UTs to check the decode is ok for the old msg and the new msg.

eth/handler.go

zzzckck · 2024-03-11T12:25:48Z

eth/protocols/eth/protocol.go

@@ -220,8 +220,29 @@ type NewBlockPacket struct {
 	TD    *big.Int
 }

+type NewBlockWithBlobPacket struct {


Is it possible that we don't add this new type, but to extend the NewBlockPacket, like:

type NewBlockPacket struct { Block *types.Block TD *big.Int Sidecars types.BlobTxSidecars `rlp:"optional"` }

Sidecars will be empty when broadcast block only. And Sidecars would contain payload if broadcast block with blobs

Removed it in this commit: 851ae4a
Please check.

eth/protocols/eth/handlers.go

galaio · 2024-03-12T04:52:28Z

eth/protocols/eth/handlers.go

@@ -352,8 +385,11 @@ func handleBlockBodies(backend Backend, msg Decoder, peer *Peer) error {
 			if body.Withdrawals != nil {
 				withdrawalHashes[i] = types.DeriveSha(types.Withdrawals(body.Withdrawals), hasher)
 			}
+			if body.Sidecars != nil {


I think the sidecarsHashes is useless, it cannot give help any validation. It can be removed. I think just modify DeliverBodies, and add blobs for fetchResult in reconstruct.

The importBlockResults in Downloader also need modify to append blobs.

for i, result := range results { blocks[i] = types.NewBlockWithHeader(result.Header).WithBody(result.Transactions, result.Uncles).WithWithdrawals(result.Withdrawals).WithBlobs(result.Blobs) }

NathanBSC · 2024-03-12T12:46:13Z

eth/protocols/eth/handlers.go

 	if err := ann.sanityCheck(); err != nil {
 		return err
 	}
+
 	if hash := types.CalcUncleHash(ann.Block.Uncles()); hash != ann.Block.UncleHash() {
 		log.Warn("Propagated block has invalid uncles", "have", hash, "exp", ann.Block.UncleHash())
 		return nil // TODO(karalabe): return error eventually, but wait a few releases


func handleBlockHeaders need some changes?

Header doesn't carry sidecar specific things. What change should be made?

sorry, I mean handleBlockBodies

NathanBSC · 2024-03-12T12:59:10Z

eth/protocols/eth/broadcast.go

@@ -37,12 +37,26 @@ type blockPropagation struct {
 	td    *big.Int
 }

+// blockPropagation is a block propagation event, waiting for its turn in the
+// broadcast queue.
+type blockAndBlobPropagation struct {


Is possible to reuse blockPropagation?
such things like

type blockPropagation struct { block *types.Block td *big.Int sidecars types.BlobTxSidecars `rlp:"optional"` }

same question, we can also remove AsyncSendNewBlockAndBlob(...) if we reuse blockPropagation

Removed AsyncSendNewBlockAndBlob() in this commit: 10cfa61

zzzckck · 2024-03-12T13:15:50Z

eth/protocols/eth/protocol.go

-	TD    *big.Int
+	Block    *types.Block
+	TD       *big.Int
+	Sidecars *types.BlobTxSidecars `rlp:"optional"`


not use pointer, but array?
Sidecars []*BlobTxSidecar

zzzckck

LGTM

* ci: temp enable blobtx branch ci run; * Switch ON blobpool & ensure Cancun hardfork can occur (#2223) * feat: support blob storage & miscs; (#2229) * chainconfig: use cancun fork for BSC; * feat: fill WithdrawalsHash when BSC enable cancun fork; * rawdb: support to CRUD blobs; * freezer: support to freeze block blobs; * blockchain: add blob cache & blob query helper; * freezer: refactor addition table logic, add uts; * blobexpiry: add more extra expiry time, and logs; * parlia: implement IsDataAvailable function; * blob: refactor blob transfer logic; * blob: support config blob extra reserve; * blockchian: support to import block with blob & blobGasFee; (#2260) * blob: implement min&max gas price logic; * blockchian: support import side chain; * blobpool: reject the banned address; * blockchain: add chasing head for DA check; * params: update blob related config; * blockchain: opt data available checking performance; * params: modify blob related params; * gasprice: support BEP-336 blob gas price calculate; * blobTx: mining + brodcasting (#2253) * blobtx mining pass (#2282) * Sidecar fetching changes for 4844 (#2283) * ci: temp enable blobtx branch ci run; * Switch ON blobpool & ensure Cancun hardfork can occur (#2223) * feat: support blob storage & miscs; (#2229) * chainconfig: use cancun fork for BSC; feat: fill WithdrawalsHash when BSC enable cancun fork; * rawdb: support to CRUD blobs; * freezer: support to freeze block blobs; * blockchain: add blob cache & blob query helper; * freezer: refactor addition table logic, add uts; * blobexpiry: add more extra expiry time, and logs; * parlia: implement IsDataAvailable function; * blob: refactor blob transfer logic; * blob: support config blob extra reserve; * blockchian: support to import block with blob & blobGasFee; (#2260) * blob: implement min&max gas price logic; * blockchian: support import side chain; * blobpool: reject the banned address; * blockchain: add chasing head for DA check; * params: update blob related config; * blockchain: opt data available checking performance; * params: modify blob related params; * gasprice: support BEP-336 blob gas price calculate; * fix failed check for WithdrawalsHash (#2276) * eth: include sidecars in fitering of body * core: refactor sidecars name * eth: sidecars type refactor * core: remove extra from bad merge * eth: fix handlenewblock test after merge * Implement eth_getBlobSidecars && eth_getBlobSidecarByTxHash (#2286) * execution: add blob gas fee reward to system; * syncing: support blob syncing & DA checking; * naming: rename blobs to sidecars; * fix the semantics of WithXXX (#2293) * config: reduce sidecar cache to 1024 and rename (#2297) * fix: Withdrawals turn into empty from nil when BlockBody has Sidecars (#2301) * internal/api_test: add test case for eth_getBlobSidecars && eth_getBlobSidecarByTxHash (#2300) * consensus/misc: rollback CalcBlobFee (#2306) * flags: add new flags to override blobs' params; * freezer: fix blob ancient save error; * blobsidecar: add new sidecar struct with metadata; (#2315) * core/rawdb: optimize write block with sidecars (#2318) * core: more check for validity of sidecars * mev: add TxIndex for mev bid (#2325) * remove useless Config() (#2326) * fix WithSidecars (#2327) * fix: fix mined block sidecar issue; (#2328) * fix WithSidecars (#2329) --------- Co-authored-by: GalaIO <[email protected]> Co-authored-by: buddho <[email protected]> Co-authored-by: Satyajit Das <[email protected]> Co-authored-by: Eric <[email protected]> Co-authored-by: zzzckck <[email protected]>

galaio and others added 7 commits February 27, 2024 10:40

ci: temp enable blobtx branch ci run;

468c53c

Switch ON blobpool & ensure Cancun hardfork can occur (#2223)

db55d1f

pool: switch ON blobpool+no reannotx

miner: set sidecar in block + todo

f135676

types: setsidecars

e8ecd22

eth: NewBlockWithBlobMsg+NewBlockWithBlobPacket

023e796

eth: AsyncSendNewBlockAndBlob

33305b1

galaio force-pushed the blobtx branch from 8e6bafe to 6498101 Compare March 6, 2024 08:23

galaio reviewed Mar 6, 2024

View reviewed changes

core/types/block.go Outdated Show resolved Hide resolved

eth/handler_eth.go Show resolved Hide resolved

emailtovamos added 3 commits March 6, 2024 12:42

eth: enqueue block with blobs to be inserted later

6eef58a

eth: remove NewBlockWithBlobMsg

59a33e9

eth: blockbodies todo

f0ed1d4

emailtovamos marked this pull request as ready for review March 7, 2024 12:21

emailtovamos requested a review from galaio March 7, 2024 12:22

emailtovamos added 3 commits March 8, 2024 08:04

eth: todos

123eab6

Merge branch 'blobtx' of https://github.com/bnb-chain/bsc into blobtx

4c12054

Merge branch 'blobtx' into blobtx-mining

a2b196f

galaio reviewed Mar 11, 2024

View reviewed changes

emailtovamos added 2 commits March 11, 2024 11:00

eth: blockBodiesMsg related changes part 1

653e210

params: version + remove unwanted comments

eca640b

zzzckck reviewed Mar 11, 2024

View reviewed changes

eth/handler.go Outdated Show resolved Hide resolved

zzzckck reviewed Mar 11, 2024

View reviewed changes

galaio reviewed Mar 12, 2024

View reviewed changes

emailtovamos added 7 commits March 12, 2024 09:46

eth: remove NewBlockWithBlobPacket

851ae4a

eth: pointer for sidecar

cfb5800

eth: remove version

978948c

Merge branch 'blobtx' of https://github.com/bnb-chain/bsc into blobtx

45ec15f

Merge branch 'blobtx' into blobtx-mining

c673b3e

eth: remove version

0effec5

eth: block bodies changes for downloader

153f603

emailtovamos added 2 commits March 12, 2024 11:47

eth: remove old todos

9a99cc5

core: remove blobversion

aaccf9a

emailtovamos requested review from galaio and zzzckck March 12, 2024 12:33

NathanBSC reviewed Mar 12, 2024

View reviewed changes

zzzckck reviewed Mar 12, 2024

View reviewed changes

eth: remove blockAndBlobPropagation

10cfa61

zzzckck approved these changes Mar 13, 2024

View reviewed changes

zzzckck merged commit 1bd1313 into blobtx Mar 13, 2024
4 of 7 checks passed

buddh0 pushed a commit that referenced this pull request Mar 13, 2024

blobTx: mining + brodcasting (#2253)

8c8eac0

buddh0 pushed a commit that referenced this pull request Mar 19, 2024

blobTx: mining + brodcasting (#2253)

6fecbdb

buddh0 pushed a commit that referenced this pull request Mar 21, 2024

blobTx: mining + brodcasting (#2253)

49e78bb

zzzckck deleted the blobtx-mining branch August 22, 2024 03:05

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Blobtx mining + brodcasting #2253

Blobtx mining + brodcasting #2253

emailtovamos commented Mar 5, 2024

galaio Mar 11, 2024

zzzckck Mar 11, 2024 •

edited

Loading

emailtovamos Mar 12, 2024

galaio Mar 12, 2024

NathanBSC Mar 12, 2024 •

edited

Loading

emailtovamos Mar 12, 2024

NathanBSC Mar 13, 2024 •

edited

Loading

NathanBSC Mar 12, 2024

zzzckck Mar 12, 2024

emailtovamos Mar 12, 2024

zzzckck Mar 12, 2024

zzzckck left a comment

Blobtx mining + brodcasting #2253

Blobtx mining + brodcasting #2253

Conversation

emailtovamos commented Mar 5, 2024

Description

Rationale

Example

Changes

Choose a reason for hiding this comment

zzzckck Mar 11, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

NathanBSC Mar 12, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

NathanBSC Mar 13, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zzzckck left a comment

Choose a reason for hiding this comment

zzzckck Mar 11, 2024 •

edited

Loading

NathanBSC Mar 12, 2024 •

edited

Loading

NathanBSC Mar 13, 2024 •

edited

Loading