QBFT + shanghai support #6353

matthew1001 · 2024-01-04T17:42:05Z

PR description

Implement shanghai support for QBFT and IBFT chains.

Main areas of change:

Previously the code made the assumption that BFT block creators and validators could be created ahead of time, on creation of a new QbftRound/IbftRound, since the next block number was known ahead of time. With number-based milestones such as london this is fine, but with timestamp-based milestones the type of EVM to use for a new round needs to be based on the time that the block is being created. As such, in many places a protocolSchedule is now passed down the stack instead of a blockValidator or a blockImporter to allow these to be created using the protocolSchedule when the timestamp is known.
The BftProtocolSchedule was only designed to allow a protocol spec to be requested by block number, so a new method getByBlockNumberAndTimestamp that takes both the block number and timestamp has replaced getByBlockNumber, and determines the fork spec based on whichever milestone is most recent.

I don't think any docs changes are required since we don't currently make any mention of shanghai in the docs today.

Fixed Issue(s)

Fixes #5446

github-actions · 2024-01-04T17:42:23Z

I thought about documentation and added the doc-change-required label to this PR if updates are required.
I thought about the changelog and included a changelog update if required.
If my PR includes database changes (e.g. KeyValueSegmentIdentifier) I have thought about compatibility and performed forwards and backwards compatibility tests

consensus/ibft/src/main/java/org/hyperledger/besu/consensus/ibft/statemachine/IbftRound.java

...us/ibft/src/main/java/org/hyperledger/besu/consensus/ibft/statemachine/IbftRoundFactory.java

...ft/src/main/java/org/hyperledger/besu/consensus/ibft/validation/MessageValidatorFactory.java

Signed-off-by: Matthew Whitehead <[email protected]>

…age validator creation Signed-off-by: Matthew Whitehead <[email protected]>

Signed-off-by: Matt Whitehead <[email protected]>

Signed-off-by: Matthew Whitehead <[email protected]>

jframe · 2024-01-30T04:44:17Z

...nsus/common/src/main/java/org/hyperledger/besu/consensus/common/bft/BftProtocolSchedule.java

   */
-  public ProtocolSpec getByBlockNumber(final long number) {
+  public ProtocolSpec getByBlockNumberAndTimestamp(final long number, final long timestamp) {


I think this getting a protocol spec by either the block number or timestamp. Better name would be getByBlockNumberOrTimestamp

Done - updated to getByBlockNumberOrTimestamp

jframe · 2024-01-30T04:53:49Z

consensus/ibft/src/main/java/org/hyperledger/besu/consensus/ibft/statemachine/IbftRound.java

@@ -312,6 +315,11 @@ private void importBlockToChain() {
          blockToImport.getHash());
    }
    LOG.trace("Importing block with extraData={}", extraData);
+    final BlockImporter blockImporter =
+        ((BftProtocolSchedule) protocolSchedule)
+            .getByBlockNumberAndTimestamp(


I think this could just use protocolSchedule.getByBlockHeader(blockToImport.getHeader()) instead. It will look up protocol schedules by both block timestamps and block numbers.

jframe · 2024-01-30T04:56:04Z

...nsus/ibft/src/main/java/org/hyperledger/besu/consensus/ibft/validation/MessageValidator.java

+
+    final BlockValidator blockValidator =
+        ((BftProtocolSchedule) protocolSchedule)
+            .getByBlockNumberAndTimestamp(


Similar to the IbftRound I think you could use the existing protocolSchedule.getByBlockHeader method here instead

jframe · 2024-01-30T05:00:18Z

consensus/qbft/src/main/java/org/hyperledger/besu/consensus/qbft/statemachine/QbftRound.java

@@ -342,6 +346,11 @@ private void importBlockToChain() {
          blockToImport.getHash());
    }
    LOG.trace("Importing proposed block with extraData={}", extraData);
+    final BlockImporter blockImporter =
+        ((BftProtocolSchedule) protocolSchedule)


Same in the IbftRound think we get away with using the protocolSchedule.getByBlockHeader method here

Signed-off-by: Matthew Whitehead <[email protected]>

Signed-off-by: Matt Whitehead <[email protected]>

Signed-off-by: Matthew Whitehead <[email protected]>

matthew1001 · 2024-01-30T17:02:16Z

Thanks for the latest comments @jframe. I think the latest commits resolve them - let me know what you think.

Signed-off-by: Matt Whitehead <[email protected]>

...nsus/ibft/src/main/java/org/hyperledger/besu/consensus/ibft/validation/MessageValidator.java

consensus/qbft/src/integration-test/resources/genesis_validator_contract_shanghai.json

...t/src/main/java/org/hyperledger/besu/consensus/qbft/validation/ProposalPayloadValidator.java

...sus/qbft/src/main/java/org/hyperledger/besu/consensus/qbft/validation/ProposalValidator.java

...rc/main/java/org/hyperledger/besu/consensus/qbft/validation/RoundChangeMessageValidator.java

Signed-off-by: Matthew Whitehead <[email protected]>

jframe · 2024-02-02T00:38:14Z

...sus/qbft/src/main/java/org/hyperledger/besu/consensus/qbft/validation/ProposalValidator.java

+    checkState(
+        protocolSchedule instanceof BftProtocolSchedule,
+        "Wrong class type for protocol schedule, requires BftProtocolSchedule");


nit: don't need this check anymore

Signed-off-by: Matthew Whitehead <[email protected]>

matthew1001 force-pushed the qbft-shanghai branch 11 times, most recently from 7bb70c0 to c1f726f Compare January 8, 2024 17:55

jframe reviewed Jan 9, 2024

View reviewed changes

matthew1001 force-pushed the qbft-shanghai branch 5 times, most recently from 0ba4bda to 3e4e682 Compare January 11, 2024 11:35

Refactored version of QBFT/shanghai

7cd7178

Signed-off-by: Matthew Whitehead <[email protected]>

matthew1001 force-pushed the qbft-shanghai branch from 3e4e682 to 7cd7178 Compare January 11, 2024 11:54

More refactoring

1498f91

Signed-off-by: Matthew Whitehead <[email protected]>

matthew1001 force-pushed the qbft-shanghai branch 5 times, most recently from f73e9e0 to 8195395 Compare January 16, 2024 11:06

Add equivalent IBFT fixes. Update QBFT and IBFT round tests

e08527b

Signed-off-by: Matthew Whitehead <[email protected]>

matthew1001 force-pushed the qbft-shanghai branch from 8195395 to e08527b Compare January 16, 2024 14:31

matthew1001 added 2 commits January 16, 2024 15:57

Update the change log

f284d5f

Signed-off-by: Matthew Whitehead <[email protected]>

Remove old comments

20c7d37

Signed-off-by: Matthew Whitehead <[email protected]>

matthew1001 force-pushed the qbft-shanghai branch 2 times, most recently from 7766e1a to 8665212 Compare January 17, 2024 17:23

Add Shanghai to the combined protocol schedule test, update IBFT mess…

41ab356

…age validator creation Signed-off-by: Matthew Whitehead <[email protected]>

matthew1001 force-pushed the qbft-shanghai branch from 8665212 to 41ab356 Compare January 17, 2024 17:34

matthew1001 marked this pull request as ready for review January 17, 2024 17:43

matthew1001 changed the title ~~WIP: QBFT + shanghai support~~ QBFT + shanghai support Jan 17, 2024

matthew1001 added 2 commits January 17, 2024 17:50

Merge branch 'main' into qbft-shanghai

a99e3e1

Signed-off-by: Matt Whitehead <[email protected]>

Fix merge

b9caf13

Signed-off-by: Matthew Whitehead <[email protected]>

matthew1001 force-pushed the qbft-shanghai branch from 10fe48f to b9caf13 Compare January 18, 2024 15:08

matthew1001 added 2 commits January 18, 2024 16:24

Update tests to use shanghai time > 0

6d9e841

Signed-off-by: Matthew Whitehead <[email protected]>

Merge branch 'main' into qbft-shanghai

4fb8d60

jframe reviewed Jan 30, 2024

View reviewed changes

matthew1001 and others added 5 commits January 30, 2024 13:44

Address PR comments

c76c6c1

Signed-off-by: Matthew Whitehead <[email protected]>

Merge branch 'main' into qbft-shanghai

6f00341

Signed-off-by: Matt Whitehead <[email protected]>

Refactor unit tests

0537e8a

Signed-off-by: Matthew Whitehead <[email protected]>

Refactor unit tests

e2fb007

Signed-off-by: Matthew Whitehead <[email protected]>

IbftRoundTests update

c832dc3

Signed-off-by: Matthew Whitehead <[email protected]>

Merge branch 'main' into qbft-shanghai

7d44693

Signed-off-by: Matt Whitehead <[email protected]>

matthew1001 mentioned this pull request Jan 31, 2024

Problem syncing (withdrawals) in besu 24.1.1 with QBFT y shanghai #6507

Closed

jframe reviewed Jan 31, 2024

View reviewed changes

matthew1001 and others added 2 commits February 1, 2024 10:42

Address PR comments

5707343

Signed-off-by: Matthew Whitehead <[email protected]>

Merge branch 'main' into qbft-shanghai

9be5ce5

jframe approved these changes Feb 2, 2024

View reviewed changes

matthew1001 and others added 2 commits February 2, 2024 08:51

Remove unnecessary class type check

e36fc94

Signed-off-by: Matthew Whitehead <[email protected]>

Merge branch 'main' into qbft-shanghai

97c5041

matthew1001 enabled auto-merge (squash) February 2, 2024 08:51

matthew1001 merged commit 03dd7f1 into hyperledger:main Feb 2, 2024
11 checks passed

matthew1001 mentioned this pull request Mar 19, 2024

QBFT/Shanghai blocks failing withdrawal validation on import #6760

Closed

matthew1001 mentioned this pull request May 16, 2024

BFT ignore withdrawals #7115

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

QBFT + shanghai support #6353

QBFT + shanghai support #6353

matthew1001 commented Jan 4, 2024 •

edited

Loading

github-actions bot commented Jan 4, 2024 •

edited by matthew1001

Loading

jframe Jan 30, 2024

matthew1001 Jan 30, 2024

jframe Jan 30, 2024

matthew1001 Jan 30, 2024

jframe Jan 30, 2024

matthew1001 Jan 30, 2024

jframe Jan 30, 2024

matthew1001 Jan 30, 2024

matthew1001 commented Jan 30, 2024

jframe Feb 2, 2024

QBFT + shanghai support #6353

QBFT + shanghai support #6353

Conversation

matthew1001 commented Jan 4, 2024 • edited Loading

PR description

Fixed Issue(s)

github-actions bot commented Jan 4, 2024 • edited by matthew1001 Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

matthew1001 commented Jan 30, 2024

Choose a reason for hiding this comment

matthew1001 commented Jan 4, 2024 •

edited

Loading

github-actions bot commented Jan 4, 2024 •

edited by matthew1001

Loading