Add benchmark test to node block process #892
Merged
+555
−1
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #892 +/- ##
==========================================
+ Coverage 60.62% 60.63% +0.01%
==========================================
Files 215 215
Lines 23099 23105 +6
==========================================
+ Hits 14003 14009 +6
Misses 7947 7947
Partials 1149 1149 ☔ View full report in Codecov by Sentry. 🚨 Try these New Features:
|
darrenvechain
approved these changes
Nov 25, 2024
darrenvechain
added a commit
that referenced
this pull request
Dec 9, 2024
* fix(documentation): use absolute links in markdown (#889) * Add benchmark test to node block process (#892) * Add benchmark test to node block process * added file-based storage * use tempdir * update dependency go-ethereum (#895) * chore: update API metrics bucket and endpoint names (#893) * chore: update API metrics bucket and endpoint names * fix: typo & tests * fix: lint * chore: add websocket total counter * fix: txs endpoints names & ws subject * fix: unit tests * chore: standardise naming convention * chore: add websocke duration & http code * chore: add websocke duration & http code * fix: lint issues * fix: sync issues with metrics * chore: update websocket durations bucket * fix: PR comments - use sync.Once * chore: update builtin generation (#896) * chore: update builtin generation * fix: update GHA * Thor client (#818) * feat: add thorclient * refactor: remove roundTripper * refactor: change null check * clean: remove commented code * feat: add account revision and pending tx * fix: add licence headers and fix linter issue * refactor: rename package * refactor: change revision type to string * refactor: rename GetLogs and GetTransfers to FilterEvents and FilterTransfers * refactor: change FilterEvents and FilterTransactions request type to EventFilter * Adding common.EventWrapper to handle channel errors * tweak * update rawclient + update account tests * tidy up names * update tests * pr comments * adding raw tx * Tidy up method names and calls * options client * tweaks * pr comments * Update thorclient/common/common.go Co-authored-by: libotony <[email protected]> * pr comments * Adding Subscriptions * Pr comments * adjust func orders * pr comments * changing subscribe to use the channel close vs multiple channels * adding go-doc * no error after unsubscribe * pr comments * checking status code is 2xx * fix: change FilterTransfers argument --------- Co-authored-by: otherview <[email protected]> Co-authored-by: libotony <[email protected]> * Add testchain package (#844) * Refactor thor node * thorchain allows insertion of blocks * remove thorNode, added testchain * clean up + comments * adding license headers * adding templating tests for thorclient * Remove test event hacks * remove types * removed chain_builder + added logdb to testchain * pr comments * Update test/testchain/chain.go Co-authored-by: libotony <[email protected]> --------- Co-authored-by: libotony <[email protected]> * cmd/thor: update instance dir to v4 * trie: implement varint-prefix coder * deps: add github.com/qianbin/drlp * trie: implement appendHexToCompact & compactLen * trie: temporarily remove merkle proof stuff * trie: many changes * disk usage reduced by 33% (force embedding shortnode) * new encoding method for storing nodes * optimize trie hashing * versioning standalone nodes * remove extended trie * improve trie interface * simplify NodeIterator, remove unused codes * trie: optimize full-node encoding/decoding * trie: tweak shortnode encoding * muxdb: move engine pkg * trie: add Version() method for node interface * muxdb: refactor due to trie updates and: * remove leafbank stuff * simplify muxdb.Trie implementation * improve root node cache using ttl eviction * add leaf key filter * chain: a lot of changes * improve block content storage scheme * remove steady block tracking * remove tx & receipt cache * state: changes due to update of trie * lowrlp: remove this pkg * txpool: changes due to underlying pkg update * genesis: changes due to underlying pkg update * consensus: changes due to underlying pkg update * builtin: changes due to underlying pkg update * runtime: changes due to underlying pkg update * api: changes due to underlying pkg update * cmd/thor/pruner: rename pkg optimizer to pruner * cmd/thor: changes due to underlying pkg update * muxdb: abandon leaf filter * cmd/thor/pruner: use smaller period when nearly synced * muxdb: improve trie node path encoding * trie: treat short nodes as standalone nodes when skipping hash * cmd/thor: fix disablePrunerFlag not work * trie: improve refNode encoding/decoding * muxdb: improve history node key encoding * cmd/thor: adjust pruner parameters * build: fix test cases * lint: fix lint error * muxdb: fix ver encoding in node blob cache * muxdb: add test cases for cache * runtime: fix test compile error * make build and test pass after rebase * add back pruner tests * add tests for node encoding * minor typo * update named store space prefix * add more tests * fix block summary in repo * make build and test pass after rebase * add back pruner tests * remove SetBestBlockID from tests * minor improvement * pr comments * adding a comment * Metrics: Cache hit/miss (#886) * change * reverted to previous format * Add dummy cache for inmem ops (#883) * Add empty cache for inmem ops * changing name to dummyCache * Metrics: Reuse `shouldLog` so we get cache hit/miss data at the same pace (#888) * first commit * first commit * Metrics: Disk IO reads and writes (#890) * changes * removed log * sleeping any way * pr review * Pedro/maindb v4/benchmarks (#891) * Adding Benchmark tests * processing txs * Working benchmarks * lint * adding tempdir * improve cache stats log and metric * totally removed SetBestBlockID * Maindb v4 Transaction benchmark plus cache (#894) * Thor client (#818) * feat: add thorclient * refactor: remove roundTripper * refactor: change null check * clean: remove commented code * feat: add account revision and pending tx * fix: add licence headers and fix linter issue * refactor: rename package * refactor: change revision type to string * refactor: rename GetLogs and GetTransfers to FilterEvents and FilterTransfers * refactor: change FilterEvents and FilterTransactions request type to EventFilter * Adding common.EventWrapper to handle channel errors * tweak * update rawclient + update account tests * tidy up names * update tests * pr comments * adding raw tx * Tidy up method names and calls * options client * tweaks * pr comments * Update thorclient/common/common.go Co-authored-by: libotony <[email protected]> * pr comments * Adding Subscriptions * Pr comments * adjust func orders * pr comments * changing subscribe to use the channel close vs multiple channels * adding go-doc * no error after unsubscribe * pr comments * checking status code is 2xx * fix: change FilterTransfers argument --------- Co-authored-by: otherview <[email protected]> Co-authored-by: libotony <[email protected]> * Add testchain package (#844) * Refactor thor node * thorchain allows insertion of blocks * remove thorNode, added testchain * clean up + comments * adding license headers * adding templating tests for thorclient * Remove test event hacks * remove types * removed chain_builder + added logdb to testchain * pr comments * Update test/testchain/chain.go Co-authored-by: libotony <[email protected]> --------- Co-authored-by: libotony <[email protected]> * cmd/thor: update instance dir to v4 * trie: implement varint-prefix coder * deps: add github.com/qianbin/drlp * trie: implement appendHexToCompact & compactLen * trie: temporarily remove merkle proof stuff * trie: many changes * disk usage reduced by 33% (force embedding shortnode) * new encoding method for storing nodes * optimize trie hashing * versioning standalone nodes * remove extended trie * improve trie interface * simplify NodeIterator, remove unused codes * trie: optimize full-node encoding/decoding * trie: tweak shortnode encoding * muxdb: move engine pkg * trie: add Version() method for node interface * muxdb: refactor due to trie updates and: * remove leafbank stuff * simplify muxdb.Trie implementation * improve root node cache using ttl eviction * add leaf key filter * chain: a lot of changes * improve block content storage scheme * remove steady block tracking * remove tx & receipt cache * state: changes due to update of trie * lowrlp: remove this pkg * txpool: changes due to underlying pkg update * genesis: changes due to underlying pkg update * consensus: changes due to underlying pkg update * builtin: changes due to underlying pkg update * runtime: changes due to underlying pkg update * api: changes due to underlying pkg update * cmd/thor/pruner: rename pkg optimizer to pruner * cmd/thor: changes due to underlying pkg update * muxdb: abandon leaf filter * cmd/thor/pruner: use smaller period when nearly synced * muxdb: improve trie node path encoding * trie: treat short nodes as standalone nodes when skipping hash * cmd/thor: fix disablePrunerFlag not work * trie: improve refNode encoding/decoding * muxdb: improve history node key encoding * cmd/thor: adjust pruner parameters * build: fix test cases * lint: fix lint error * muxdb: fix ver encoding in node blob cache * muxdb: add test cases for cache * runtime: fix test compile error * make build and test pass after rebase * add back pruner tests * add tests for node encoding * minor typo * update named store space prefix * add more tests * fix block summary in repo * make build and test pass after rebase * add back pruner tests * remove SetBestBlockID from tests * minor improvement * pr comments * adding a comment * Metrics: Cache hit/miss (#886) * change * reverted to previous format * Add dummy cache for inmem ops (#883) * Add empty cache for inmem ops * changing name to dummyCache * Metrics: Reuse `shouldLog` so we get cache hit/miss data at the same pace (#888) * first commit * first commit * Metrics: Disk IO reads and writes (#890) * changes * removed log * sleeping any way * pr review * Pedro/maindb v4/benchmarks (#891) * Adding Benchmark tests * processing txs * Working benchmarks * lint * adding tempdir * Adding transactions benchmark + repository cache * improve cache stats log and metric * totally removed SetBestBlockID * removed unused tests * update bench tests * getreceipts metrics + lint --------- Co-authored-by: Paolo Galli <[email protected]> Co-authored-by: libotony <[email protected]> Co-authored-by: qianbin <[email protected]> Co-authored-by: Darren Kelly <[email protected]> Co-authored-by: Miguel Angel Rojo <[email protected]> * reduce clauses() allocations * bug: fix logs endpoints query (#900) * Thor client (#818) * feat: add thorclient * refactor: remove roundTripper * refactor: change null check * clean: remove commented code * feat: add account revision and pending tx * fix: add licence headers and fix linter issue * refactor: rename package * refactor: change revision type to string * refactor: rename GetLogs and GetTransfers to FilterEvents and FilterTransfers * refactor: change FilterEvents and FilterTransactions request type to EventFilter * Adding common.EventWrapper to handle channel errors * tweak * update rawclient + update account tests * tidy up names * update tests * pr comments * adding raw tx * Tidy up method names and calls * options client * tweaks * pr comments * Update thorclient/common/common.go Co-authored-by: libotony <[email protected]> * pr comments * Adding Subscriptions * Pr comments * adjust func orders * pr comments * changing subscribe to use the channel close vs multiple channels * adding go-doc * no error after unsubscribe * pr comments * checking status code is 2xx * fix: change FilterTransfers argument --------- Co-authored-by: otherview <[email protected]> Co-authored-by: libotony <[email protected]> * Add testchain package (#844) * Refactor thor node * thorchain allows insertion of blocks * remove thorNode, added testchain * clean up + comments * adding license headers * adding templating tests for thorclient * Remove test event hacks * remove types * removed chain_builder + added logdb to testchain * pr comments * Update test/testchain/chain.go Co-authored-by: libotony <[email protected]> --------- Co-authored-by: libotony <[email protected]> * cmd/thor: update instance dir to v4 * trie: implement varint-prefix coder * deps: add github.com/qianbin/drlp * trie: implement appendHexToCompact & compactLen * trie: temporarily remove merkle proof stuff * trie: many changes * disk usage reduced by 33% (force embedding shortnode) * new encoding method for storing nodes * optimize trie hashing * versioning standalone nodes * remove extended trie * improve trie interface * simplify NodeIterator, remove unused codes * trie: optimize full-node encoding/decoding * trie: tweak shortnode encoding * muxdb: move engine pkg * trie: add Version() method for node interface * muxdb: refactor due to trie updates and: * remove leafbank stuff * simplify muxdb.Trie implementation * improve root node cache using ttl eviction * add leaf key filter * chain: a lot of changes * improve block content storage scheme * remove steady block tracking * remove tx & receipt cache * state: changes due to update of trie * lowrlp: remove this pkg * txpool: changes due to underlying pkg update * genesis: changes due to underlying pkg update * consensus: changes due to underlying pkg update * builtin: changes due to underlying pkg update * runtime: changes due to underlying pkg update * api: changes due to underlying pkg update * cmd/thor/pruner: rename pkg optimizer to pruner * cmd/thor: changes due to underlying pkg update * muxdb: abandon leaf filter * cmd/thor/pruner: use smaller period when nearly synced * muxdb: improve trie node path encoding * trie: treat short nodes as standalone nodes when skipping hash * cmd/thor: fix disablePrunerFlag not work * trie: improve refNode encoding/decoding * muxdb: improve history node key encoding * cmd/thor: adjust pruner parameters * build: fix test cases * lint: fix lint error * muxdb: fix ver encoding in node blob cache * muxdb: add test cases for cache * runtime: fix test compile error * make build and test pass after rebase * add back pruner tests * add tests for node encoding * minor typo * update named store space prefix * add more tests * fix block summary in repo * make build and test pass after rebase * add back pruner tests * remove SetBestBlockID from tests * minor improvement * pr comments * adding a comment * Metrics: Cache hit/miss (#886) * change * reverted to previous format * Add dummy cache for inmem ops (#883) * Add empty cache for inmem ops * changing name to dummyCache * Metrics: Reuse `shouldLog` so we get cache hit/miss data at the same pace (#888) * first commit * first commit * Metrics: Disk IO reads and writes (#890) * changes * removed log * sleeping any way * pr review * Pedro/maindb v4/benchmarks (#891) * Adding Benchmark tests * processing txs * Working benchmarks * lint * adding tempdir * improve cache stats log and metric * totally removed SetBestBlockID * fix: logs API not returning results when to=0,from=omitted * minor update * fix: PR comments + lint * improve convert range logic * chore: remove debug log --------- Co-authored-by: Paolo Galli <[email protected]> Co-authored-by: otherview <[email protected]> Co-authored-by: libotony <[email protected]> Co-authored-by: qianbin <[email protected]> Co-authored-by: Miguel Angel Rojo <[email protected]> * chore(chain): add repo cache metrics (#910) * chore(chain): add repo cache metrics * refactor(chain): cache hit miss --------- Co-authored-by: Darren Kelly <[email protected]> Co-authored-by: Pedro Gomes <[email protected]> Co-authored-by: Paolo Galli <[email protected]> Co-authored-by: qianbin <[email protected]> Co-authored-by: Miguel Angel Rojo <[email protected]> Co-authored-by: Darren Kelly <[email protected]>
otherview
added a commit
that referenced
this pull request
Dec 9, 2024
* fix(documentation): use absolute links in markdown (#889) * Add benchmark test to node block process (#892) * Add benchmark test to node block process * added file-based storage * use tempdir * update dependency go-ethereum (#895) * chore: update API metrics bucket and endpoint names (#893) * chore: update API metrics bucket and endpoint names * fix: typo & tests * fix: lint * chore: add websocket total counter * fix: txs endpoints names & ws subject * fix: unit tests * chore: standardise naming convention * chore: add websocke duration & http code * chore: add websocke duration & http code * fix: lint issues * fix: sync issues with metrics * chore: update websocket durations bucket * fix: PR comments - use sync.Once * chore: update builtin generation (#896) * chore: update builtin generation * fix: update GHA * getreceipts metrics + lint (#902) * chore: add flag to enable/disable deprecated APIs (#897) * chore: add flag to enable/disable deprecated APIs * chore: update for PR comments * chore: update for PR comments * fix: update e2e commit sha * fix: update e2e commit sha * fix: update flag name * fix: solo start flags (#906) * chore: make thorclient configurable + fix type error (#908) * chore: make thorclient configurable * fix: subscriptions block type * fix: compile errors * fix: remove test with lint error * add 'raw' query parameter to the blocks (#899) * add 'raw' query parameter to the blocks * summary -> summary.Header Co-authored-by: libotony <[email protected]> * change variable name * make expanded and raw mutually exclusive * add unit tests * fix linting --------- Co-authored-by: libotony <[email protected]> * Adding Health endpoint (#836) * Adding Health endpoint * pr comments + 503 if not healthy * refactored admin server and api + health endpoint tests * fix health condition * fix admin routing * added comments + changed from ChainSync to ChainBootstrapStatus * Adding healthcheck for solo mode * adding solo + tests * fix log_level handler funcs * refactor health package + add p2p count * remove solo methods * moving health service to api pkg * added defaults + api health query * pr comments * pr comments * pr comments * Update cmd/thor/main.go * Darren/admin api log toggler (#877) * Adding Health endpoint * pr comments + 503 if not healthy * refactored admin server and api + health endpoint tests * fix health condition * fix admin routing * added comments + changed from ChainSync to ChainBootstrapStatus * Adding healthcheck for solo mode * adding solo + tests * fix log_level handler funcs * feat(admin): toggle api logs via admin API * feat(admin): add license headers * refactor health package + add p2p count * remove solo methods * moving health service to api pkg * added defaults + api health query * pr comments * pr comments --------- Co-authored-by: otherview <[email protected]> * Darren/chore/backport metrics (#909) * chore(muxdb): backport muxdb cache metrics * chore(muxdb): backport muxdb cache metrics * chore(metrics): backport disk IO * chore(metrics): fix lint * chore(chain): add repo cache metrics * fix(chain): fix cache return value * refactor(chain): cache hit miss * chore(thor): update version (#912) * chore(thor): update version * chore(openapi): version * feat(api/debug): support debug trace without blockId (#905) * api/debug: support debug with txhash Signed-off-by: jsvisa <[email protected]> api/debug: blockId should use tx's instead Signed-off-by: jsvisa <[email protected]> fix tests Signed-off-by: jsvisa <[email protected]> * debug: add test Signed-off-by: jsvisa <[email protected]> * improve parseTarget Signed-off-by: jsvisa <[email protected]> * update doc Signed-off-by: jsvisa <[email protected]> * fix tests Signed-off-by: jsvisa <[email protected]> --------- Signed-off-by: jsvisa <[email protected]> Co-authored-by: tony <[email protected]> * version --------- Signed-off-by: jsvisa <[email protected]> Co-authored-by: Darren Kelly <[email protected]> Co-authored-by: libotony <[email protected]> Co-authored-by: YeahNotSewerSide <[email protected]> Co-authored-by: Delweng <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Benchmark tests for Node Block Process.
Fixes # (issue)
Type of change
Please delete options that are not relevant.
Checklist: