Skip to content

Commit

Permalink
PolyBFT consensus protocol (#899)
Browse files Browse the repository at this point in the history
* Move PolyBFT to Edge (#774)

* Move PolyBFT to edge

Co-authored-by: Igor Crevar <[email protected]>
Co-authored-by: Sasa Prsic <[email protected]>
Co-authored-by: Stefan Negovanović <[email protected]>
Co-authored-by: Victor Castell <[email protected]>

* Build block with Txs in PolyBFT (#789)


Co-authored-by: Stefan Negovanović <[email protected]>
Co-authored-by: Sasa Prsic <[email protected]>

* EVM-48: Add RootChain helper (#773)

* EVM-57 PolyBft state transactions introduction (#809)

* EVM-57 PolyBFT bridge enabled state txs

Co-authored-by: Sasa Prsic <[email protected]>

* EVM-57 PolyBFT bridge enabled state txs

Co-authored-by: Sasa Prsic <[email protected]>

* tidy.

* pr fixes

* pr fix 2

* small fix

* TransactionType should be populated in receipt

* nonce fix

* write change original tx fix

* createStateTransactionWithData change

* pr fix more

* revert ComputeHash for tx because id does not really matter

Co-authored-by: Sasa Prsic <[email protected]>

* Migrate polybft e2e framework to edge (#797)

Back-port the e2e framework + tests from v3 to edge as a separate set of tests.

* v3 smart contracts integration (#796)

* Initial changes

* Init v3-contract submodule

* Uptime changes and linter fixes

* Use local build and test workflows (temporary)

* Test workflow expand with init git submodules

* Native transfer precompile

* CommitTransaction in blockBuilder

* adding workflow call secrets input (#799)

* Add BLS aggregated signatures verification precompile

* Change required gas for introduced precompiles

* Add console precompile

* Additional functions in bls package

* Register validator command

* Linter fixes, register subcommand for register validator command

* Adapt genesis command

* Cleanup

* Change addresses of sidechain contracts so that they don't clash with precompile addresses

* Marshal from address in transaction

* Remove CommitTransaction and use WriteTx instead

* Fix panic in register validator command

* Get rid of panics in register validator

* Increase gas limit for ChildValidatorSet initialization

* Cleanup GenesisPostHookFactory

* Fix register validator command

* Change gas limit to 100M for initialize child validator set

* Change make compile v3-contracts

* Recover CI (1st attempt)

* Recover CI (2nd attempt)

* Recover CI (3rd attempt)

* Cleanup Makefile (remove compilation of SCs when unneeded). Remove cloning submodules in lint workflow

* Recover lint

* Remove init submodules from test workflow

* Address some go security issues

* small make fix

* init root contracts command

* Remove rootchain contracts deployment from rootchain server command

* Provide new url for smart contracts

* Introduce constants for register validator and contracts root folder

* Comments

* Comments (part 2)

* Reference dev-ethernal commit from core-contracts

* Fix polybft e2e workflow

* Fix genesis cmd invocation from e2e framework

* contract_initializer fix

* Fix polybft e2e tests

* Comments (part 3)

* Populate validators balance based on provided genesis configuration.
Comments (part 4)

* Fix balance unmarshal for polybft.Validator

* Initialize native transfer contract

Co-authored-by: Goran Rojovic <[email protected]>
Co-authored-by: Epikichi <[email protected]>
Co-authored-by: Igor Crevar <[email protected]>

* EVM-129 Implement bridge workflow (#828)

* Add bridge flag.

* Populate bridge config.

* Fix bridge address.

* rootchain panic fixes

* changes. deploying contracts to root still does not work in test

* fix deploying contracts and running bridge. Emit command change

* Read balance from genesis

* Remove Geth reference from Edge module

* bridge test fixes

* remove test-bootnode. add WithBootnodeCount to test-cluster. Change network_test to be more relaxed

Co-authored-by: Sasa Prsic <[email protected]>
Co-authored-by: Igor Crevar <[email protected]>

* Transfer unit tests from v3 to Edge (#825)

* test-bridge fix + temporary fix

* Read balance from genesis

* state, state transaction, event tracker

* FSM tests (part 1)

* ConsensusRuntime tests

* FSM tests (part 2)

* FSM tests (part 3)

* SystemState tests

* ValidatorAccount and ValidatorSnapshot tests

* buildLogsFromReceipts test

* FSM tests (part 4)

* FSM fix remaining tests

* t.parallel

* Introduce parallel execution in tests (part 2)

* Lint fixes

* Rename test helpers file to exclude it from build

* Add test suffix to mocks

* Install solidity compiler on test workflow

* Assign execute permission to setup-ci.sh script

* polybft tests

* all tests merged

* setting epoch snapshot added

* Lint fixes

* Use constant for IstanbulExtraVanity

* Description of polybft test

Co-authored-by: Igor Crevar <[email protected]>
Co-authored-by: Nemanja0x <[email protected]>
Co-authored-by: Goran Rojovic <[email protected]>

* [Checkpointing] `ExitEvent` storage and proof generation (#838)

* Exit event storage and query functions

* ExitProof generation

* Reorganize tests

* Optimize key composition

* Extract common function for decoding state sync and exit events

* Comments fix

* Comments fix

* [Checkpointing] JSON RPC point for generating exit proofs (#844)

* JSON RPC point for generating exit proofs

* Comments fix

Co-authored-by: Stefan Negovanović <[email protected]>

* Checkpoint data signing (#840)

* Add CheckpointData to Extra

* Don't store chainID, blockNumber and blockHash to the blockchain

* Set checkpoint data to the Header.Extra

* Fix and simplify FSM unit tests

* getSignHash static function, fix verify header implementation

* Increase block waiting time in bridge test

* Remove unnecessary condition

* Set checkpoint to each extra

* Sign checkpoint hash instead of aggregated hash

* Make nextValidatorsHash same as currentValidatorsHash for non-epoch ending blocks

* Set event root to the checkpoint object

* [Checkpoints] Parse receipt logs and insert new exit events (#852)

* Parse receipt logs and insert exit events

* Comments fix

* Comments fix

* Comments fix

* Remove TODO

* Comments fix

* Comments fix

* Comments fix

* Consolidate BLS verification ABI type (#867)

* Checkpoint submission (#861)

* Change error messages for checkpoint hash calculation

* Determine checkpoint blocks

* Checkpoint submission (draft)

* Introduce checkpoint manager

* Lint fixes

* Fix tests

* Checkpoint manager checkpoint data ABI encode unit tests

* Unit test for getCurrentCheckpointID

* Unit test for submitCheckpoint

* Remove leftover code

* Comments

* Determine epoch ending blocks by comparing epoch numbers in adjacent headers

* Micro optimization

* Skip querying first block for end of epoch

* Optimization

* Simplify checkpoints offset

* Adapt comment

* Checkpoint block detection optimization (#872)

* Simplify checkpoint block detection

* Resolve conflicts and pass tests

Co-authored-by: Boris <[email protected]>

* Simplify

* Submit pending checkpoints in unit test

* Extract parentHeader retrieval outside of the for

* Submit end of epoch blocks for pending checkpoints

* Update submitCheckpoint comment

Co-authored-by: Victor Castell <[email protected]>

* Feedback

* Separate import declarations

* Create var block for `parentExtra` and `parentHeader`

Co-authored-by: Victor Castell <[email protected]>

Co-authored-by: Goran Rojovic <[email protected]>
Co-authored-by: Boris <[email protected]>
Co-authored-by: Victor Castell <[email protected]>

* Mock Host implementation in native transfer test (#887)

* Mock Host implementation in native transfer test

* Lint fix

* Fix polybft e2e tests

* Implement the logic for quorum check (#837)

* Implement the logic for quorum check

Depending on the message type it will implement different kinds of quorum checks.

* Remove MaximumFaulty

* Integrate go-ibft consensus engine to polybft consensus protocol (#891)

* EVM 149 - syncer go ibft coordination (#843)

* consensus runtime backend metohod (not buildable)

* transport

* polybft - run cycle

* transport fix 2

* small changes

* less complex ibft consennsus wrapper.Example: https://replit.com/@crewce/Newest#main.go

* small changes

* runSequence better

* remove pbftTransportWrapper

* noone likes temp variables

* Comment fixed

* jm fixes

Co-authored-by: Igor Crevar <[email protected]>

* EVM-151 Use (or not) edge hooks support (#849)

* FSM that supports go-ibft backend interface (#865)

* consensus runtime backend metohod (not buildable)

* transport

* polybft - run cycle

* transport fix 2

* less complex ibft consennsus wrapper.Example: https://replit.com/@crewce/Newest#main.go

* Implement MessageConstructor interface.

* Tidy linter errors.

* runSequence better

* remove pbftTransportWrapper

* Add quorum size implementation.

* Remove one part of pbft.NodeID reference.

* Add some comments.

* IsValid, Build proposal on FSM

* Add block validation.

* Add stub for IsValidSender method.

* Sender validation is always true.

* fsm - Insert block

* simplify build proposal

* Tidy logs.

* IsValidSender

* pbft.Proposal removed

* Tidy errors.

* restartEpoch if epoch is nil, remove hook in fsm

* initialization fix

* Fix compare.

* quorum fixes and error propagation

* evm-151 changes + additional needed in this task

* Increase test timeout.

* bring back some part of fsm

* just execute syn test 100 times in a row

* Tidy part of errors.

* Revert e2e test.

* Remove unused code and move message creator interface implementation.

* minor fix

* Add comments.

* code organization - valdateSender + recoverAddress

* Fix compile after merge.

* small change

* restart epoch on better place. lastBuiltBlock and epoch should not be nil

* small reorg in polybft

* pr fix

* Update comments

* pr fixes no 2

* Add voting power to the validators (#880)

* Add voting power to the validator account

* Rename ValidatorAccount to ValidatorMetadata

* Set voting power to the existing test

* Comment fixes

* Rename files

* Address comments

* Remove extra space from the comment

* pr fixes

* Rename GetValidatorAccount to GetValidatorMetadata

Co-authored-by: Nemanja0x <[email protected]>
Co-authored-by: Igor Crevar <[email protected]>
Co-authored-by: Stefan Negovanović <[email protected]>
Co-authored-by: Stefan Negovanović <[email protected]>

* Accommodate go-ibft `BuildProposal` updated function (#893)

* Populate round info and change proposal hash

* Adopt currentRound from BuildProposal

* Fix logs

* Fix proposal hash calculation inside IsValidProposalHash

* Update go-ibft version

* Igor's feedback addressed

* Merge fix

* EVM-187 Handling bundles on both sync and consensus (#882)

* EVM-187 Handling bundles on both sync and consensus

* consensus_runtime tests

* UT fix

Co-authored-by: Goran Rojovic <[email protected]>

* Implement HasQuorum function in consensusRuntime

Co-authored-by: Nemanja Nedic <[email protected]>
Co-authored-by: Igor Crevar <[email protected]>
Co-authored-by: Saša Pršić <[email protected]>
Co-authored-by: Nemanja0x <[email protected]>
Co-authored-by: Goran Rojovic <[email protected]>

* Remove Polybft dummy smart contracts (#909)

* Change ValidatorMetadata methods to pointer receivers (#911)

* EVM-152 Fix Unit Tests (#910)

Co-authored-by: Goran Rojovic <[email protected]>

* adding devnet-v3 workflow for parity testing. Adding needed secrets for other workflows (#916)

* Introduce local development cluster script (#912)

* Introduce local development cluster script

* Fix assigned boot nodes port number in genesis (#930)

* Local cluster - adapt port for ibft to polybft (#931)

* Fix unit test for event tracker (#935)

* Fix linters

* Fix TestEventTracker_TrackSyncEvents

* Comment

* EVM-232 Keep polybft bls and ecdsa keys in separate secrets manager namespaces (#953)

* Track updates for validators (#939)

* Implement equality checks for PublicKey and ValidatorMetadata.
Expand ValidatorSetDelta

* Handle updates in ApplyDelta

* Additional tests

* Epikichi/edge 973 change v3 parity workflow to devnet (#957)

* reflecting changes to use ephemeral environment instead

* Epikichi/edge 964 implement multiple erc mode support (#954)

* adding support for ERC20 and ERC721 mode for Pandora's Box stress testing

* adding in default for devnet ephemeral anticipation

* fixing markdown format

* reverting for a separate PR

* adding changes for devnet ephemeral workflow to allow dispatching

* adding correct job names, removed dependencies since concurrency protection is set on Pandora' workflow env-wide

* reflecting chnages from develop

* EVM-220 TestClusterBlockSync/BLS fails in voting power branch (#926)

* Fix data race to polybft unit test (#965)

* Fix some race conditions.

* Polybft voting power (#944)

voting power feature

* Align `v3-parity` with `develop` branch (#971)

* Epikichi/edge 964 implement multiple erc mode support (#954)

* adding support for ERC20 and ERC721 mode for Pandora's Box stress testing

* adding in default for devnet ephemeral anticipation

* fixing markdown format

* reverting for a separate PR

* introducing workflow dispatch trigger for Devnet V3 workflow. (#927)

* introducing workflow dispatch trigger for Devnet V3 workflow.

* adding ephemeral environment naming

* adding pandora modes

* Add performance tests to devnet deployments (#918)

* added performance tests to devnet deployment
* removing depricated loadbot performance tests

* Docker setup: Wait for genesis file to exist before starting nodes (#949)

* Docker setup: Wait for genesis file to exist before starting nodes

* Docker setup: don't run init if genesis file already exists

* adding conditional dependency for compose to wait for successful  node

Co-authored-by: epikichi <[email protected]>

Co-authored-by: Epikichi <[email protected]>
Co-authored-by: ZeljkoBenovic <[email protected]>
Co-authored-by: mediremi-antithesis <[email protected]>
Co-authored-by: epikichi <[email protected]>

* Fix e2e tests

* Set address to the result instance on applyCreate

* Remove State Sync execution step (#964)

* Remove commitment execution related stuff
* Add the RPC endpoint to obtain StateSync Proofs
* Rootchain data to gitignore
* Return the state sync along with proof
* Better params for cluster script
* E2E test executing state syncs
* Fix tests that rely on bundle execution
* Remove commitment execution related stuff

* Replace BLS library with github.com/kilic/bn254 (#978)

* Integrate with CheckpointManager.sol (#952)

* Integrate with CheckpointManager.sol

* Hex encode ECDSA private key

* Skip failing tests for checkpoint manager

* Fix submit function signature

* Remove logs and set From field to submit checkpoint transaction

* Pass signer key to rootchain.SendTxn function

* Pull console precompile Geth docker image

* Populate voting power on checkpoint manager initialization

* Hash change

* Minor fixes (ordering of abi ancoded parameters)

* Populate voting power on genesis block

* Fix balance scaling

* Fix TestCheckpointManager_isCheckpointBlock

* Remove sorting by addresses on validators snapshot calculation

* Sort validators by addresses on genesis header population

* Sort validators by addresses on rootchain initialization

* Logging...

* Update core-contracts

* Fix checkpoint manager unit tests

* Change comment + unit tests for helper functions

* Revert usage of Geth console fork

* Include Geth console fork again

* Rename latestCheckpointBlockNumber to currentCheckpointBlockNumber

* Remove logs and revert to use official Geth image for rootchain server

* Code cleanup

* Add meaningful logs to the checkpoint manager

* not buildable

* full functional bn254

* Use proper domain when initializing CheckpointManager contract

* Fix go mod

* Revert helper bash scripts

* Fix e2e tests

* Address Igor's comment

* Provide `epochSize` to the `ChildValidatorSet` (#983)

* Provide epochSize to the ChildValidatorSet

* Rename newBls map key

* Rename parameters

* Point to our branch in core-contracts

Co-authored-by: Goran Rojovic <[email protected]>

Co-authored-by: Stefan Negovanović <[email protected]>
Co-authored-by: Nemanja0x <[email protected]>
Co-authored-by: Igor Crevar <[email protected]>
Co-authored-by: Stefan Negovanović <[email protected]>

* Change genesis premine logic (#981)

* Fix data race in fsm (#982)

* Fix data race in several places accessing fsm and fix copy validators function.

* tidy linter.

* Fix tests after changes and user RLock.

* Removing `GenesisTarget` and adding the `NodeID` field to `Validator` struct (#985)

* Refactored GenesisTarget removing it and adding the NodeID field to Validator struct.
* Refactored generatePolyBFTConfig to use the new struct and simplify the code and take into account validators passed from params
* Add NodeID as a field needed for validators passed from params
* Tune timeout for bridge test

Co-authored-by: Stefan Negovanović <[email protected]>

* Provide rootchain IP address from CLI (#989)

* Refactor RootchainInteractor abstraction

* Lint fix

* Provide JSON RPC address flags

* Remove lock from rootchain interactor.
Send transaction remove nonce parameter.

* Address comments

* Reduce amount of affected files

* Implement TxRelayer

* Address comments (part 2)

* Create txn with nonce set

* Option pattern in TxRelayer

* Use txRelayer in e2e tests

* Let TxRelayer take care of setting the nonce

* Fix tx pool e2e tests

* Added london hardfork to the chain config (#999)

* Fix pending checkpoints submission (#992)

* Fix epoch ending blocks detection in pending checkpoints

* Update TestCheckpointManager_submitCheckpoint

* Remove redundant signerAddress

* Make unit test more understandable

* Update core-contracts (#1007)

* Bundle build fix (#1014)

* Check for nil proposal (#1009)

* Remove gasLimit and gas fields from txrelayer call (#1020)

* Add default gas and gasPrice to local transactions (#1021)

* Use geth console fork as default with flag to disable (#1022)

* Checkpoint submission e2e test (#990)

* Checkpoint submission e2e test

* Utilize TxRelayer in test

* Fix init contracts command

* Fix e2e test

* git modules: shallow and fix branch (#987)

* shallow and fix branch

* feat-polybft-release

* Remove bool return value from deliverMessage signature (#1027)

* Fix/immutable trie race (#1017)

* Add lock to trie state.

* Use total stake for voting power (#1038)

* Use total stake as voting power

* Lower log level to debug log in local cluster script

* Fix unit test

* RLP Marshal/Unmarshal update field in `ValidatorSetDelta` (#1026)

* RLP Marshal/Unmarshal update field in ValidatorSetDelta

* Comment + additional unit test

* Remove unused fields (#1039)

* Introduce metrics for BoltDB for the state (#1045)

* Expose `PolyBFT` metrics to Prometheus API (#1040)

* Introduce metrics to polybft

* Add comment

* Update comment

* Remove total state syncs metrics

* Include metrics prefix and rename file

* Address comment

* Fix consensus runtime tests

* EVM-308 Add commands we are missing (#1050)

* Added new commands

* Merge params and result files

* Reorganize commands

* Comments fix

* Fix loop closure violations in unit tests (#1053)

* Fix loop closures

* Make TestExtra_CreateValidatorSetDelta_Cases deterministic

* Avoid for loop when assigning voting power

* Minimize changes

* Fix

* Integrate L2 exit workflow (#1024)

* e2e exit test

* fix lint

* add static L1 exit contract data

* add contract comilation to e2e tests

* fix ci

* save artifacts in client

* fix tests

* review fix

* review fix

* Update consensus/polybft/contractsapi/gen/main.go

Co-authored-by: Stefan Negovanović <[email protected]>

* review fix

Co-authored-by: Stefan Negovanović <[email protected]>

* Add pending checkpoints assertions to checkpoint submission e2e test (#1052)

* Decouple rootchain contracts deployment from development environment (#1005)

* Rootchain init contracts read validators from the genesis configuration.
Introduce manifest.json.

* Introduce admin-key flag to the rootchain init-contracts

* Revert polybft params changes

* Fix comment

* Populate rootchain admin address to manifest

* Update Manifest struct and adapt init-contracts workflow

* Remove sidechain smart contract addresses from genesis spec

* Update genesis command and cleanup genesis from unnecessary parameters

* Remove rootchain contracts hardcoded addresses

* Decouple emit command from contant StateSender address

* Manifest command

* Polybft manifest command

* Adapt e2e tests workflow

* Update comment

* Adapt cluster script and change flags descriptions in manifest command

* Comments

* Adapt TestE2E_CheckpointSubmission

* Remove manifest.json in cluster script

* Initial version of polybft readme

* Update readme with a precondition step

* Address comments (part 1)

* Update manifest command short help

* Comments

* Use sidechain addresses from the polybft config in fsm UTs

* Minimize changes

* Merge fix

Co-authored-by: Goran Rojovic <[email protected]>

* EVM-302  Save proposer priorities after each block (#1018)


* EVM-302 Save proposer priorities after each block

Co-authored-by: Nemanja0x <[email protected]>
Co-authored-by: Stefan Negovanović <[email protected]>
Co-authored-by: Sasa Prsic <[email protected]>

* Remove GossipMessage (#1057)

* Consolidate `ExtendByteSlice` implementations (#1058)

* Consolidate ExtendByteSlice implementations

* Add comment and fix

* Introduce trim parameter (fix unit tests)

* Update bitmap unit tests

* Address comment

* Add a unit test

* Remove readContractBytecode (#1061)

* Evm 303: Fix race conditions after a block is inserted (#1047)


Co-authored-by: Igor Crevar <[email protected]>
Co-authored-by: Stefan Negovanović <[email protected]>
Co-authored-by: Sasa Prsic <[email protected]>

* Feature/validator set changes (#1048)

* Update priorities when validator set changes.

* Remove code related to bundles for state syncs (#1013)

Related to RFC-139, we're removing bundles concept from the codebase.
We're introducing the latest Smart Contracts version in this PR

* Create a full block object during polybft (#1063)

* Create a full block object during polybft

* Remove print

* Remove now

* Fix tests

* Fix segmentation fault panic on `ValidateSender` (#1054)

* Add nil check for FSM

* Different approach: consensus runtime validates message sender.
Remove Includes function from validatorSet.

* Subscribe IBFT topic only after epoch is initialized

* Renames and comment fixes

* Revert sender validation to the fsm

* Rebase fix

* Add log

* RFC173 Checkpoint consensus issue (#1064)

* RFC 173 changes

* Comments fix

* Comments fix

* Fix uptime total blocks calculation (#1069)

* Fix uptime total blocks calculation

* Update test for uptime calculation

* Simplify

* Simplify unit test

* Typo fix

* Implement the relayer (#1044)

* Implement the relayer
* The statesyncrelayer package will reside inside polybft

* Immutable trie simplification (#1068)

* Refactor immutable trie.

* Remove state from trie.

* Tidy.

* Remove storage from trie.

* Remove voting power scaling (#1060)

* Change voting power from uint64 to big.Int

* Provide desired scaling factor to validator set

* Fix unit tests after rebase

* Fix validatorSetToABISlice comment

* Introduce relative voting power

* Change ProposerPriority to big.Int

* Fix log

* Fix computeMaxMinPriorityDiff

* Remove max total voting power hard limit

* Use truncated division for rescaling priorities

* Add ceil to quorum size calculation

* Revert unnecessary changes

* Remove safe_math

* Compress quorum calculation

Co-authored-by: Victor Castell <[email protected]>

* Extract helper function for big int with ceil division

Co-authored-by: Victor Castell <[email protected]>

* Epikichi/edge 988 add polybft consensus support to edge (#1051)

* Added support for polybft consensus docker cluster
* added customisation options
* added/edited readme

Co-authored-by: ZeljkoBenovic <[email protected]>
Co-authored-by: ZeljkoBenovic <[email protected]>

* Implement `TestE2E_Bridge_ChangeVotingPower` (#1075)

* Implement TestE2E_Bridge_ChangeVotingPower

* Increase epoch rewards

* Add log to the fsm.getCurrentValidators

* Move default epoch reward to genesis cmd

* Dump validators to log only when debug logs enabled

* Multiple validators are staking

* Remove unnecessary panics from polybft (#1077)

* Register validator remove panics and expose json rpc flag

* Remove panic from blockchainWrapper

* Remove json rpc flag from register validator cmd

* Polybft log instead of panic if block synchronization fails

* Remove unused vars

* Handle createValidatorSetDelta error

* Rename test

* Revert panic in stateProvider

* Dynamic epoch size (#897)

* Decouple uptime calculation from fixed epoch size

* Tests fix

* Rename isEndOfEpoch

* Return error on apply delta to validator set

* Comments fix

* Comments fix

* Comments fix

* Comments fix

* Comments fix

* Bug fix

* Comments fix

* Rebase fix

* Rebase fix

* Rebase fix

* Rebase fix

* Modify snapshot calculation for dynamic epoch (#997)

* Modify snapshot calculation for dynamic epoch

* Comments fix

* Comments fix

* Move tests to new file

* Comments fix

* Rebase fix

* Comments fix

* Comments fix

* Feature - State Sync manager (#1067)

* initial work

* Rework on event tracker and tests

* More stuff

* Migrate last test

* More changes

* Remove unused test

* Use correct core-contracts version

* Remove logs

* Use eventTracker from tracker package, code cleanup, bug fixes

* Code cleanup

* Code cleanup

* Comments fix

* Dummy state sync manager when bridge is not enabled

* Comments fix

* Comments fix

* Rebase fix

Co-authored-by: Goran Rojovic <[email protected]>

* Core contracts to dev-audit (#1095)

* Core contracts to dev-audit
* Fix event name

* Implement `Stringer` interface in `AccountSet` (#1103)

* Implement stringer in AccountSet

* Use strings.Join instead of defining buffer explicitly

* Add disclaimer message when running polybft consensus protocol (#1107)

* Add disclaimer message when running polybft consensus protocol

* Print disclaimer message to logger and CLI outputter

* Register validator e2e test (#1084)

* Initial approach

* Rely on system state in TestE2E_Consensus_RegisterValidator

* Add stake and balance flags to register validator cmd

* Remove debug log

* Add appropriate assertions

* Revert scientific notation changes

* Add comments

* Remove grpc flag from register validator cmd

* Delegation and undelegation e2e test (#1086)

* Initial approach

* Delegation logic

* Minor changes to unstake and withdraw commands

* Withdrawal and undelegate in test

* Change unstake cmd description

* Query for current block number before outputting delegator info

* Handled go routines in event tracker (#1080)

* Manage goroutines in EventTracker

* Stop StateSyncRelayer

* Remove closeCh from polybft and use context instead

* Create named logger in the event tracker itself

* Update consensus/polybft/polybft_test.go assertion message

Co-authored-by: Victor Castell <[email protected]>

* Remove context field from consensusRuntime

* Isolate changes (revert changes in polybft to rely on channel)

* Add function comments

* Fix test

Co-authored-by: Victor Castell <[email protected]>

* Multiple state sync commitments per epoch (#1083)

* Initial changes

* e2e test

* Code cleanup

* Comments fix

* Lock fix

* Comments fix

* Comments fix

* Comments fix

* Comments fix

* Update comment in state_sync_manager

Co-authored-by: Stefan Negovanović <[email protected]>

* Return FullBlock from syncer (#1110)

* Return FullBlock from syncer

* Comments fix

* Comments fix

* Extract common parts of WriteBlock and WriteFullBlock

* Revert "Extract common parts of WriteBlock and WriteFullBlock"

This reverts commit 62166e6.

Co-authored-by: Stefan Negovanović <[email protected]>

* EVM-356 Update checkpoint out of fsm (#1113)

* Move logic from fsm to checkpoint manager

* PostBlock and PostEpoch in checkpointManager

* Function comment fix

* Call PostEpoch on checkpointManager

* Test fix

* e2e tests fix

* Comments fix

* Comments fix

* Change test

* Comments fix

* Move BuildEventRoot to checkpointManager (#1115)

* Initialize `closeCh` in `stateSyncManager` (#1116)

* Initialize closeCh in stateSyncManager

* Rename test

* Add test for state sync relayer

* Move GenerateExitProof to checkpointManager (#1118)

* Invert logical condition in `WaitUntil` functions (#1123)

* Invert logic for bridge WaitUntil function

* Invert WaitUntil logic in TestCluster

* Change condition in TestE2E_Bridge_ChangeVotingPower

* Fix flaky register validator e2e test (#1122)

* Invert logic for bridge WaitUntil function

* Invert WaitUntil logic in TestCluster

* Change condition in TestE2E_Bridge_ChangeVotingPower

* Fix flaky test

* Query validators multiple times

* TestE2E_Bridge_L2toL1Exit to work with multiple users  (#1132)

TestE2E_Bridge_L2toL1Exit to work with multiple users

* Use PostBlock handler in ProposerCalculator (#1139)

* Add missing cluster stop invocations in e2e tests (#1138)

* Optimized docker build in docker-compose (#1135)

* Validator unstake e2e test (#1112)

* Initial approach

* Tweak test

* Invert WaitUntil condition

* Premine balance to child smart contracts

* Working withdrawal

* Remove redundant inline function

* Pre-allocate ChildValidatorSet in genesis with initial validator set total stake

* Rebase fix

* Address comments

* E2E Relayer (#1131)

E2E Tests for Relayer

We need to check if StateSyncs are being executed correctly by one of the nodes of the cluster, so we are removing the "manual" execution previously present in e2e test and relay on the Relayer feature to do it as normal operation.

* BuildProofs for missed commitments (#1141)

* BuildProofs for missed commitments

* Comments fix

* Comments fix

* EVM-360 Invalid state root error on v0.7.1-alpha2 (#1133)

Change block builder Reset to include timestamp for current block and call Reset each time before proposal is created and not just once when we are creating fsm

* Rely on contracts api package when dealing with ABI abstractions (#1144)

* Add remaining smart contract artifacts

* Use ABI functions, events and types from Artifact abstractions

* Unexport stateSyncABIType variable

* Extract StateSynced event from correct contract

* Rename parameters so that they match smart contract spec

* Fix unstake and undelegate

* Typo fix

* Fix undelegate e2e test

* Align RLP marshaling logic according to eip-2718 (#1114)

* EVM-360 e2e test (#1158)

* EVM-353 Different voting power property based test (#1149)

* Remove GetNonce function (#1165)

* Expose ValidatorInfo struct (#1162)

* Feature contractsapi gen (#1062)

* Add initial gen api

* Remove method suffix

* Add contracts to the main function

* Add struct encoding

* Small fix

* Generate contract namespace (placeholder)

* Handle leading underscore in field name

* Comments fix

* Comments fix

* Take events from artifact

* Event name fix

* Remove contracts stubs from generated code

* Remove unnecessary code

* Real type names

* Comments fix

* Comments fix

Co-authored-by: Goran Rojovic <[email protected]>

* TestCommitEpoch unit test (#1167)

TestCommitEpoch

* EVM-395 Implement RFC-175 BLS integration (#1161)

* EVM-395 Implement RFC-175
* Add hash to point unit tests
* Avoid random secret keys with zero value
* Check point at infinity and subgroup

Co-authored-by: Ferran Borreguero <[email protected]>

* Removed legacy logic from tx pool (#1171)

* Introduce test smart contracts as artifacts (#1163)

* Introduce L1DummyStateReceiver as artifact

* Extract TestL1StateReceiver smart contract to separate file and embed it

* Include TestWriteBlockMetadata artifact

* Introduce constant for test-contracts dir

* Checkpoint data validations (#1160)

* Validate checkpoint data

* Separate Validate and ValidateBasic checkpoint data validations

* Summarize logical conditions

* Vefify state transactions before writing commit epoch transaction

* Prevent double execution of commit epoch transaction by providing callback to ProcessBlock

* Simplify

* EVM-378 Use contracts api generated stubs in client code (#1166)

* Use contracts api for commitments

* Move ExitProof and StateSyncProof in another package

* Use state synced event from generated code

* Use generated structs for comit epoch (uptime)

* Rebase fix

* Use contracts api in checkpoints

* Use contracts api in bridge test

* Remove unnecessary logs

* Rebase fix

* Comments fix

* Comments fix

* Rebase fix

* Remove unused constants

* Comments fix

* Comments fix

* Rename functions and add comments

---------

Co-authored-by: Stefan Negovanović <[email protected]>

* Fix EVM-372 & EVM-373 already existing directories and files are not checked for ownership and permissions (#1164)

* Implemented helper create file and dir functions

* Replace writeFile with CreateFileSafe

* replace mkdirall with CreateDirSafe

* change config file error message

* Add extra empty path guard

* Fix not overwriting files issue

* remove shouldNotExist parameter

* remove chmod update

* fix lint

* rename to save file

* comment fix

* comment fix

* one-liner return and handle errors

* update error message

* Adapt go-ibft implementation for polybft (#1185)

* Fix hash calculation for proposal for go-ibft's update (#1065)

* Changing the signatures of the backend methods.

* Fixed all method signatures that expect a proposal.

* Verify committed seals created by ethereum block and round number

* Remove unnecessary fields before hash calculation for CS

* Fix missing field in packProposerSealIntoExtra

* Fix hash calculation for the block including round number

* Refactor hash calculation with round

* Fix build error test

* Revert "Refactor hash calculation with round"

This reverts commit b527ecb.

* Fix failed test

* Fix HasQuorum

* Rename method in IBFT Backend

* Fix method comment

* Rename IsValidBlock to IsValidProposal in IBFTBackend

* Fix parsing of RoundNumber in IBFTExtra

* Fix packCommittedSealsAndRoundNumberIntoExtra

* Revert packCommittedSealsAndRoundNumberIntoExtra

* Fix hash calculation for committed seals and revert interface of block verifier

* Update go-ibft

* Fix lint error

* Fix marshaling of IBFT Extra

* Fix minor issues

* Fix comment

---------

Co-authored-by: Lazar Travica <[email protected]>

* Integrate updated go-ibft changes

* Error handling in BuildProposal

* Fix logs

* Add view instance to BuildProposal

* Update go-ibft version

* Formatting

---------

Co-authored-by: kourin <[email protected]>
Co-authored-by: Lazar Travica <[email protected]>

* [EVM-370] Added --insecure flag to both ibft and polybft (#1182)

* Added --insecure flag to both `ibft` and `polybft`
* Provide insecure flag in `polybft` e2e framework
* Provide insecure flag to `ibft` e2e framework
---------

Co-authored-by: Stefan Negovanović <[email protected]>

* Reject contract creation transactions with large init data (#1181)

* Fix  validation of state transactions (#1186)

* Fix  validation of state transactions

* Add a comment

* Comments fix

* Add unit test

* Return error if two uptime transactions are in a block

---------

Co-authored-by: Stefan Negovanović <[email protected]>

* EVM-416 Use one struct for Exit and StateSync proofs in jsonrpc (#1170)

* Use one struct

* Fix state sync execution

* Remove abi functions from StateSyncProof

* Comments fix

* Comment

---------

Co-authored-by: Nemanja Nedic <[email protected]>
Co-authored-by: Igor Crevar <[email protected]>
Co-authored-by: Sasa Prsic <[email protected]>
Co-authored-by: Victor Castell <[email protected]>
Co-authored-by: Roman Behma <[email protected]>
Co-authored-by: Victor Castell <[email protected]>
Co-authored-by: Goran Rojovic <[email protected]>
Co-authored-by: Epikichi <[email protected]>
Co-authored-by: Nemanja0x <[email protected]>
Co-authored-by: Goran Rojovic <[email protected]>
Co-authored-by: Boris <[email protected]>
Co-authored-by: Saša Pršić <[email protected]>
Co-authored-by: ZeljkoBenovic <[email protected]>
Co-authored-by: mediremi-antithesis <[email protected]>
Co-authored-by: epikichi <[email protected]>
Co-authored-by: Ferran Borreguero <[email protected]>
Co-authored-by: Evgeny Danilenko <[email protected]>
Co-authored-by: b00ris <[email protected]>
Co-authored-by: ZeljkoBenovic <[email protected]>
Co-authored-by: Ahmet Yazıcı <[email protected]>
Co-authored-by: kourin <[email protected]>
Co-authored-by: Lazar Travica <[email protected]>
  • Loading branch information
23 people authored Feb 3, 2023
1 parent df32bb4 commit f434a9b
Show file tree
Hide file tree
Showing 253 changed files with 33,996 additions and 1,111 deletions.
7 changes: 7 additions & 0 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,9 @@ name: Build
on: # yamllint disable-line rule:truthy
workflow_dispatch:
workflow_call:
secrets:
GH_TOKEN_CLONE_PRIVATE:
required: true

jobs:
go_build:
Expand All @@ -11,6 +14,10 @@ jobs:
steps:
- name: Checkout code
uses: actions/checkout@v3
with:
submodules: recursive
token: ${{ secrets.GH_TOKEN_CLONE_PRIVATE }}

- name: Setup Go environment
uses: actions/setup-go@v3
with:
Expand Down
3 changes: 3 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,14 @@ jobs:
build:
name: Build
uses: ./.github/workflows/build.yml
secrets:
GH_TOKEN_CLONE_PRIVATE: ${{ secrets.GH_TOKEN_CLONE_PRIVATE }}

test:
name: Test
uses: ./.github/workflows/test.yml
needs: build
secrets:
GH_TOKEN_CLONE_PRIVATE: ${{ secrets.GH_TOKEN_CLONE_PRIVATE }}
SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
SONAR_HOST_URL: ${{ secrets.SONAR_HOST_URL }}
3 changes: 3 additions & 0 deletions .github/workflows/deploy.devnet.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,12 +23,15 @@ jobs:
build:
name: Build
uses: ./.github/workflows/build.yml
secrets:
GH_TOKEN_CLONE_PRIVATE: ${{ secrets.GH_TOKEN_CLONE_PRIVATE }}

test:
name: Test
uses: ./.github/workflows/test.yml
needs: build
secrets:
GH_TOKEN_CLONE_PRIVATE: ${{ secrets.GH_TOKEN_CLONE_PRIVATE }}
SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
SONAR_HOST_URL: ${{ secrets.SONAR_HOST_URL }}

Expand Down
3 changes: 3 additions & 0 deletions .github/workflows/deploy.testnet.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@ jobs:
build:
uses: ./.github/workflows/build.yml
name: Build
secrets:
GH_TOKEN_CLONE_PRIVATE: ${{ secrets.GH_TOKEN_CLONE_PRIVATE }}

test:
uses: ./.github/workflows/test.yml
Expand All @@ -30,6 +32,7 @@ jobs:
secrets:
SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
SONAR_HOST_URL: ${{ secrets.SONAR_HOST_URL }}
GH_TOKEN_CLONE_PRIVATE: ${{ secrets.GH_TOKEN_CLONE_PRIVATE }}

container_image_build:
name: Build/Push Container Image
Expand Down
60 changes: 30 additions & 30 deletions .github/workflows/deploy_edgenet.yaml
Original file line number Diff line number Diff line change
@@ -1,30 +1,30 @@
---
on: # yamllint disable-line rule:truthy
release:
types:
- published

jobs:
deploy_to_edge_net:
name: Update EdgeNet
runs-on: ubuntu-latest
steps:
- name: Get aws-commander
run: |
wget https://github.com/Trapesys/aws-commander/releases/download/v0.2.0/aws-commander_0.2.0_Linux_x86_64.tar.gz
tar -xf aws-commander_0.2.0_Linux_x86_64.tar.gz
sudo mv aws-commander /usr/local/bin
- name: Checkout code
uses: actions/checkout@v3

- name: Run deployment Ansible
env:
AWS_ACCESS_KEY_ID: ${{ secrets.EDGENET_AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.EDGENET_AWS_SECRET_ACCESS_KEY }}
run: >
/usr/local/bin/aws-commander
-instances i-039f7c0b3328a00f8,i-035b9f2d78cfb8ea9,i-00a6c7cb3a213f21f,i-03ac2f42ddcba6120
-mode ansible
-playbook .github/workflows/ansible/update_edgenet.yaml
-aws-zone us-west-2
---
on: # yamllint disable-line rule:truthy
release:
types:
- published

jobs:
deploy_to_edge_net:
name: Update EdgeNet
runs-on: ubuntu-latest
steps:
- name: Get aws-commander
run: |
wget https://github.com/Trapesys/aws-commander/releases/download/v0.2.0/aws-commander_0.2.0_Linux_x86_64.tar.gz
tar -xf aws-commander_0.2.0_Linux_x86_64.tar.gz
sudo mv aws-commander /usr/local/bin
- name: Checkout code
uses: actions/checkout@v3

- name: Run deployment Ansible
env:
AWS_ACCESS_KEY_ID: ${{ secrets.EDGENET_AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.EDGENET_AWS_SECRET_ACCESS_KEY }}
run: >
/usr/local/bin/aws-commander
-instances i-039f7c0b3328a00f8,i-035b9f2d78cfb8ea9,i-00a6c7cb3a213f21f,i-03ac2f42ddcba6120
-mode ansible
-playbook .github/workflows/ansible/update_edgenet.yaml
-aws-zone us-west-2
36 changes: 36 additions & 0 deletions .github/workflows/e2e-polybft.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
---
name: PolyBFT E2E tests
on: # yamllint disable-line rule:truthy
push:
branches:
- main
- develop
pull_request:

jobs:
build:
runs-on: ubuntu-latest
env:
E2E_TESTS: true
E2E_LOGS: true
CI_VERBOSE: true
steps:
- uses: actions/checkout@v3
with:
submodules: recursive
token: ${{ secrets.GH_TOKEN_CLONE_PRIVATE }}
- name: Install Go
uses: actions/setup-go@v3
with:
go-version: 1.18.x
- name: Compile core contracts
run: make compile-core-contracts
- name: Run tests
run: make test-e2e-polybft
- name: Archive test logs
if: always()
uses: actions/upload-artifact@v3
with:
name: e2e-logs
path: e2e-logs-*/
retention-days: 30
5 changes: 4 additions & 1 deletion .github/workflows/e2e.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,10 @@ jobs:
E2E_LOGS: true
CI_VERBOSE: true
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v3
with:
submodules: recursive
token: ${{ secrets.GH_TOKEN_CLONE_PRIVATE }}
- name: Install Go
uses: actions/setup-go@v3
with:
Expand Down
6 changes: 6 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ on: # yamllint disable-line rule:truthy
workflow_dispatch:
workflow_call:
secrets:
GH_TOKEN_CLONE_PRIVATE:
required: true
SONAR_TOKEN:
required: true
SONAR_HOST_URL:
Expand All @@ -23,6 +25,10 @@ jobs:
uses: actions/checkout@v3
with:
submodules: recursive
token: ${{ secrets.GH_TOKEN_CLONE_PRIVATE }}

- name: Install Dependencies
run: ./setup-ci.sh

- name: Run Go Test
run: go test -coverprofile coverage.out -timeout 20m `go list ./... | grep -v e2e`
Expand Down
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,12 +1,14 @@
peers.json
genesis.json
manifest.json
secretsManagerConfig.json
*-packr.go
config*.json

bin/

test-chain*
test-rootchain*
polygon-edge-chain*
.idea

Expand All @@ -21,8 +23,10 @@ main

# exclude build folder
artifacts
node_modules

# Log files
*.log

vendor/
coverage.out
8 changes: 7 additions & 1 deletion .gitmodules
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
[submodule "tests/tests"]
path = tests/tests
url = https://github.com/ethereum/tests.git
url = https://github.com/ethereum/tests.git
shallow = true
[submodule "core-contracts"]
path = core-contracts
url = https://github.com/0xPolygon/core-contracts
shallow = true
branch = feat-polybft-release
5 changes: 5 additions & 0 deletions .golangci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,11 @@ issues:
- gosec
- unparam
- lll
- path: gen_sc_data\.go
linters:
- wsl
- lll
- stylecheck
include:
- EXC0012 # Exported (.+) should have comment( \(or a comment on this block\))? or be unexported
- EXC0013 # Package comment should be of the form "(.+)...
Expand Down
28 changes: 18 additions & 10 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@

.PHONY: download-spec-tests
download-spec-tests:
.PHONY: download-submodules
download-submodules:
git submodule init
git submodule update

Expand All @@ -11,10 +11,11 @@ bindata:
.PHONY: protoc
protoc:
protoc --go_out=. --go-grpc_out=. ./server/proto/*.proto
protoc --go_out=. --go-grpc_out=. ./protocol/proto/*.proto
#protoc --go_out=. --go-grpc_out=. ./protocol/proto/*.proto
protoc --go_out=. --go-grpc_out=. ./network/proto/*.proto
protoc --go_out=. --go-grpc_out=. ./txpool/proto/*.proto
protoc --go_out=. --go-grpc_out=. ./consensus/ibft/**/*.proto
protoc --go_out=. --go-grpc_out=. ./consensus/polybft/**/*.proto

.PHONY: build
build:
Expand All @@ -39,7 +40,7 @@ generate-bsd-licenses:

.PHONY: test
test:
go test -timeout=20m `go list ./... | grep -v e2e`
go test -coverprofile coverage.out -timeout=20m `go list ./... | grep -v e2e`

.PHONY: test-e2e
test-e2e:
Expand All @@ -49,11 +50,18 @@ test-e2e:
go build -race -o artifacts/polygon-edge .
env EDGE_BINARY=${PWD}/artifacts/polygon-edge go test -v -timeout=30m ./e2e/...

.PHONY: run-local
run-local:
docker-compose -f ./docker/local/docker-compose.yml up -d --build
.PHONY: test-e2e-polybft
test-e2e-polybft:
# We can not build with race because of a bug in boltdb dependency
go build -o artifacts/polygon-edge .
env EDGE_BINARY=${PWD}/artifacts/polygon-edge E2E_TESTS=true E2E_LOGS=true go test -v -timeout=30m ./e2e-polybft/...

.PHONY: stop-local
stop-local:
docker-compose -f ./docker/local/docker-compose.yml stop
.PHONY: compile-core-contracts
compile-core-contracts:
cd core-contracts && npm install && npm run compile
$(MAKE) generate-smart-contract-bindings

.PHONY: generate-smart-contract-bindings
generate-smart-contract-bindings:
go run ./consensus/polybft/contractsapi/artifacts-gen/main.go
go run ./consensus/polybft/contractsapi/bindings-gen/main.go
4 changes: 2 additions & 2 deletions archive/restore.go
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ type blockchainInterface interface {
GetBlockByNumber(uint64, bool) (*types.Block, bool)
GetHashByNumber(uint64) types.Hash
WriteBlock(*types.Block, string) error
VerifyFinalizedBlock(*types.Block) error
VerifyFinalizedBlock(*types.Block) (*types.FullBlock, error)
}

// RestoreChain reads blocks from the archive and write to the chain
Expand Down Expand Up @@ -78,7 +78,7 @@ func importBlocks(chain blockchainInterface, blockStream *blockStream, progressi
nextBlock := firstBlock

for {
if err := chain.VerifyFinalizedBlock(nextBlock); err != nil {
if _, err := chain.VerifyFinalizedBlock(nextBlock); err != nil {
return err
}

Expand Down
4 changes: 2 additions & 2 deletions archive/restore_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -55,8 +55,8 @@ func (m *mockChain) WriteBlock(block *types.Block, _ string) error {
return nil
}

func (m *mockChain) VerifyFinalizedBlock(block *types.Block) error {
return nil
func (m *mockChain) VerifyFinalizedBlock(block *types.Block) (*types.FullBlock, error) {
return &types.FullBlock{Block: block}, nil
}

func (m *mockChain) SubscribeEvents() blockchain.Subscription {
Expand Down
Loading

0 comments on commit f434a9b

Please sign in to comment.