Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sync kaleido-noderunner-dev with Besu 24.2.0 RC2 #6

Merged
merged 486 commits into from
Feb 28, 2024

Conversation

matthew1001
Copy link

Gabriel-Trintinalia and others added 30 commits November 1, 2023 14:13
There was a slight problem on the bonsai side because all account reads did not go through a single method. One of the two add the account to the cache but the other did not. This had two consequences:

Less good performance because certain accounts had to be read several times and also all account reads were not marked in the trielog. This will fix both problems.

Signed-off-by: Karim TAAM <[email protected]>
v13 of the official Ethereum Reference Tests.

Signed-off-by: Danno Ferrin <[email protected]>
Update the GraphQL and JSON-RPC endpoints to provide `yParity` instead
of `v` for non-legacy transactions.
Update the JSON-RPC tests to use the Hive data. Add tests for Shanghai
and Cancun Blocks.

Signed-off-by: Danno Ferrin <[email protected]>
* Revert "Log missing chain head as warning, not trace (hyperledger#6127)"
* leave chain head at warning, just revert the discard warning

Signed-off-by: garyschulte <[email protected]>
* Reverse added order and sequence number

Signed-off-by: Matthew Whitehead <[email protected]>

* Remove extraneous 'addedAt' check

Signed-off-by: Matthew Whitehead <[email protected]>

---------

Signed-off-by: Matthew Whitehead <[email protected]>
Signed-off-by: Fabio Di Fabio <[email protected]>
Co-authored-by: Sally MacFarlane <[email protected]>
…ution (hyperledger#6102)

* Delete leftPad when capturing the stack before and after the execution
* Still use leftPad when displaying the stack in the output (ex. for debug_traceTransaction)
* Fix integration test
* Use StringBuilder to left pad the hex representation of a 32 bytes

Signed-off-by: Ameziane H <[email protected]>
Fix the flakeiness in EthGetTransactionByHashTest as well as some other
sonar identified cleanup.

Signed-off-by: Danno Ferrin <[email protected]>
hyperledger#6146)

* Apply the same reverse sort order as hyperledger#6106 but to the base fee sorter

Signed-off-by: Matthew Whitehead <[email protected]>

* Fix unit tests

Signed-off-by: Matthew Whitehead <[email protected]>

* Update eviction unit tests to expect highest-sequence TXs be evicted first

Signed-off-by: Matthew Whitehead <[email protected]>

* Update change log

Signed-off-by: Matthew Whitehead <[email protected]>

* Spotless fixes

Signed-off-by: Matthew Whitehead <[email protected]>

---------

Signed-off-by: Matthew Whitehead <[email protected]>
…ame sender (hyperledger#6022)

* Add bool TX pool flag to allow TX selection to skip finding all TXs for the same sender

Signed-off-by: Matthew Whitehead <[email protected]>

* Implement one approach to eliminating sender TX grouping. Note - this is WIP for discussion with others. Not sure this is the implementation we want.

Signed-off-by: Matthew Whitehead <[email protected]>

* Make the no-sdr-grouping flag experimental, remove code no longer needed since hyperledger#6106 merged

Signed-off-by: Matthew Whitehead <[email protected]>

* Update tests, tidy up empty lines

Signed-off-by: Matthew Whitehead <[email protected]>

* Fix tests to be inline with recent sender-priority changes

Signed-off-by: Matthew Whitehead <[email protected]>

* Address PR comments

Signed-off-by: Matthew Whitehead <[email protected]>

* More unit test updates

Signed-off-by: Matthew Whitehead <[email protected]>

* Remove --Xtx-pool-disable-sender-grouping - the new legacy TX pool ordering removes the need for it

Signed-off-by: Matthew Whitehead <[email protected]>

* Fix test merge error

Signed-off-by: Matthew Whitehead <[email protected]>

---------

Signed-off-by: Matthew Whitehead <[email protected]>
* Synchronize access to block header

Signed-off-by: Matthew Whitehead <[email protected]>

* Add 'safe' version of getBlockHeader

Signed-off-by: Matthew Whitehead <[email protected]>

* Move retry with lock into getChainHeadBlockHeader()

Signed-off-by: Matthew Whitehead <[email protected]>

* Reinstate 'final' modifier

Signed-off-by: Matthew Whitehead <[email protected]>

---------

Signed-off-by: Matthew Whitehead <[email protected]>
* updates static integration test so txs are committed to using mainnet trusted setup
* new static vectors for precompile checks
* moves trusted setup to main

---------

Signed-off-by: Justin Florentine <[email protected]>
* removed parallelisation of ATs and use xl executor
* use machine executor for privacy ATs only
* removed small executor since it's not used
* moved privacy ATs to nightly
* restore parallelisation for ATs but reduce number

---------

Signed-off-by: Sally MacFarlane <[email protected]>
* basic test coverage
* blob caching on tx removal in legacy pool
* blob caching on tx removal in layered pool
* blob restoral in both legacy and layered implementations and test coverage of tx copy in builder
* refactors into reusable BlobCache, and rekeys on versioned hash

---------

Signed-off-by: Justin Florentine <[email protected]>
Co-authored-by: Fabio Di Fabio <[email protected]>
* adds upload links and hashes

Signed-off-by: Justin Florentine <[email protected]>

* reference test updates

Signed-off-by: Justin Florentine <[email protected]>

---------

Signed-off-by: Justin Florentine <[email protected]>
macfarla and others added 24 commits February 14, 2024 15:42
* test each profile
* removed duplicated line
* use non-deprecated name for sync mode

Signed-off-by: Sally MacFarlane <[email protected]>

---------

Signed-off-by: Sally MacFarlane <[email protected]>
* invalidate txs below blob base fee
* updates tests to provide blob prices
* correction to reftests


---------

Signed-off-by: Justin Florentine <[email protected]>
Co-authored-by: Fabio Di Fabio <[email protected]>
Signed-off-by: garyschulte <[email protected]>
…perledger#6540)

Add rocksdb x-stats command to print various stats for each column family

---------

Signed-off-by: Simon Dudley <[email protected]>
* snap by default to replace fast

Signed-off-by: Sally MacFarlane <[email protected]>

---------

Signed-off-by: Sally MacFarlane <[email protected]>
Signed-off-by: Gabriel-Trintinalia <[email protected]>
* default bonsai
* be explicit about config in tests
* default forest storage for fast sync tests
* added upcoming breaking changes section
* added db mismatch message

Signed-off-by: Sally MacFarlane <[email protected]>

---------

Signed-off-by: Sally MacFarlane <[email protected]>
Update tracing and evm tool
* Intrinsic gas is optional in EVMTool
* For call series, also charge the gas given to the next call level for debug_ series calls.

Signed-off-by: Danno Ferrin <[email protected]>
Update to new reference tests for cancun.
* Init KZG as part of ReferenceTestProtocolSchedules
* fix unsigned long errors
* block validation on borked blobGasUsed

Signed-off-by: Justin Florentine <[email protected]>
Signed-off-by: Danno Ferrin <[email protected]>
Signed-off-by: Sally MacFarlane <[email protected]>
Co-authored-by: Justin Florentine <[email protected]>
Co-authored-by: Sally MacFarlane <[email protected]>
* wildcard in place of task name in path, in case it gets changed.
* test splitting action is unreliable at these numbers of tests
* pin to new gradle action, increase runners
* found the missing tests, back to parity
* rename to avoid consolidation status collision with workflow name
* fixes javadoc checks at build time
* overrides project version on release, so embeds (startup info) will match release artifacts


---------

Signed-off-by: Justin Florentine <[email protected]>
Co-authored-by: RoboCopsGoneSock <[email protected]>
Improve flat trace generation performance

Signed-off-by: Ameziane H <[email protected]>
* mining parameter metrics

Signed-off-by: Brindrajsinh-Chauhan <[email protected]>

* update changelog

Signed-off-by: Brindrajsinh-Chauhan <[email protected]>

* remove unwanted code

Signed-off-by: Brindrajsinh-Chauhan <[email protected]>

---------

Signed-off-by: Brindrajsinh-Chauhan <[email protected]>
* removes commenting on pr action, in favor of an updated pr template. also updates gradle build actions
* adds fix for nightly docker builds
* uses consolidation status to determine re-runs
* mention docker image registry in changelog
---------

Signed-off-by: Justin Florentine <[email protected]>
Signed-off-by: Sally MacFarlane <[email protected]>
Co-authored-by: Sally MacFarlane <[email protected]>
* Add Besu version to DB metadata. Check for downgrades and reject if version < version recorded in DB metadata.

Signed-off-by: Matthew Whitehead <[email protected]>

* Add --allow-downgrade CLI arg. If set it allows the downgrade and updates the Besu version in the metadata file to the downgraded version.

Signed-off-by: Matthew Whitehead <[email protected]>

* Update gradle verification XML

Signed-off-by: Matthew Whitehead <[email protected]>

* Add and update tests

Signed-off-by: Matthew Whitehead <[email protected]>

* Refactoring

Signed-off-by: Matthew Whitehead <[email protected]>

* Remove versioning from RocksDB, now in separate VERSION_DATADATA.json

Signed-off-by: Matthew Whitehead <[email protected]>

* Tidy up and tests for the new class

Signed-off-by: Matthew Whitehead <[email protected]>

* Move downgrade logic into VersionMetadata as BesuCommand is already very big

Signed-off-by: Matthew Whitehead <[email protected]>

* Add more tests

Signed-off-by: Matthew Whitehead <[email protected]>

* Refactor the naming of the option to version-compatibility-protection

Signed-off-by: Matthew Whitehead <[email protected]>

* Remove remaining references to allow-downgrade

Signed-off-by: Matthew Whitehead <[email protected]>

* Rename test

Signed-off-by: Matthew Whitehead <[email protected]>

* Update comments

Signed-off-by: Matthew Whitehead <[email protected]>

* Metadata verification update

Signed-off-by: Matthew Whitehead <[email protected]>

* gradle fix

Signed-off-by: Matthew Whitehead <[email protected]>

* Enable version downgrade protection by default for non-named networks

Signed-off-by: Matthew Whitehead <[email protected]>

* Fix default logic

Signed-off-by: Matthew Whitehead <[email protected]>

* Update ethereum/core/src/main/java/org/hyperledger/besu/ethereum/core/VersionMetadata.java

Co-authored-by: Fabio Di Fabio <[email protected]>
Signed-off-by: Matt Whitehead <[email protected]>

* Update ethereum/core/src/main/java/org/hyperledger/besu/ethereum/core/VersionMetadata.java

Co-authored-by: Fabio Di Fabio <[email protected]>
Signed-off-by: Matt Whitehead <[email protected]>

* mock-maker-inline no longer needed

Signed-off-by: Matthew Whitehead <[email protected]>

---------

Signed-off-by: Matthew Whitehead <[email protected]>
Signed-off-by: Matt Whitehead <[email protected]>
Signed-off-by: Matt Whitehead <[email protected]>
Co-authored-by: Fabio Di Fabio <[email protected]>
…ger#6557)

The BerlinGasCalculator had a side effect of warming addresses when
calculating call gas costs. By introducing a boolean we can keep the
side effects in the operation instead of in the gas calculator. This
makes gas cost estimation by downstream users of the API safer.

Signed-off-by: Danno Ferrin <[email protected]>
* append docker pull command for the release to notes

---------

Signed-off-by: Justin Florentine <[email protected]>
* invalid payload error code

* use non-zero timestamp

* use invalidPayloadAttributes error for V2 if non-null beacon root pre cancun

Signed-off-by: Sally MacFarlane <[email protected]>

---------

Signed-off-by: Sally MacFarlane <[email protected]>
@matthew1001 matthew1001 requested a review from alexwood February 27, 2024 18:00
@matthew1001 matthew1001 changed the title Kaleido noderunner dev 24.2.0 rc2 Sync kaleido-noderunner-dev with Besu 24.2.0 RC2 Feb 27, 2024
Copy link

@alexwood alexwood left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving based on testing in dev

@matthew1001 matthew1001 merged commit 6dde792 into kaleido-noderunner-dev Feb 28, 2024
32 checks passed
@matthew1001
Copy link
Author

Note: after this PR merged I realised it should have been a rebase & merge to avoid having an additional merge commit in our branch (we want our branch to be at the same commit as the Besu OSS release). I've since done a force push to bring the kaleido-noderunner-dev branch to the same commit as the PR before the merge commit was added.

I've also updated the Kaleido Besu repo settings to only allow rebase & merge PRs in the future.

@matthew1001 matthew1001 deleted the kaleido-noderunner-dev-24.2.0-rc2 branch March 12, 2024 09:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.