Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Thor client #818

Merged
merged 40 commits into from
Oct 29, 2024
Merged

Thor client #818

merged 40 commits into from
Oct 29, 2024

Conversation

paologalligit
Copy link
Member

Description

This PR introduces a thor client. This refers to this gh issue.
The structure was taken from the Draupnir's client.
The client supports the following operations:

  • GetTransactionReceipt
  • InspectClauses
  • InspectTxClauses
  • SendTransaction
  • SendEncodedTransaction
  • GetLogEvents
  • GetLogTransfer
  • GetAccount
  • GetAccountForRevision
  • GetAccountCode
  • GetAccountCodeForRevision
  • GetStorage
  • GetExpandedBlock
  • GetBlock
  • GetBestBlock
  • GetTransaction
  • GetPendingTransaction
  • GetPeers
  • ChainTag
  • SubscribeBlocks
  • SubscribeEvents
  • SubscribeTransfers
  • SubscribeTxPool
  • SubscribeBeats
  • SubscribeBeats2

Type of change

  • New feature (non-breaking change which adds functionality)
  • This change requires a documentation update

How Has This Been Tested?

  • Unit tests

Checklist:

  • My code follows the style guidelines of this project
  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • New and existing E2E tests pass locally with my changes
  • Any dependent changes have been merged and published in downstream modules
  • I have not added any vulnerable dependencies to my code

@paologalligit paologalligit requested a review from a team as a code owner August 13, 2024 14:46
@codecov-commenter
Copy link

codecov-commenter commented Aug 13, 2024

Codecov Report

Attention: Patch coverage is 58.75486% with 212 lines in your changes missing coverage. Please review.

Project coverage is 60.53%. Comparing base (175050c) to head (a1e92c3).

Files with missing lines Patch % Lines
thorclient/thorclient.go 34.09% 86 Missing and 1 partial ⚠️
thorclient/httpclient/client.go 62.50% 47 Missing and 25 partials ⚠️
thorclient/wsclient/client.go 74.45% 32 Missing and 3 partials ⚠️
thorclient/httpclient/http_request.go 59.09% 11 Missing and 7 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #818      +/-   ##
==========================================
- Coverage   62.60%   60.53%   -2.08%     
==========================================
  Files         209      213       +4     
  Lines       18946    22931    +3985     
==========================================
+ Hits        11862    13881    +2019     
- Misses       5983     7914    +1931     
- Partials     1101     1136      +35     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@libotony
Copy link
Member

Can we abandon the API we have already marked as deprecated? Such as SubscribeBeats .

thorclient/thorclient.go Outdated Show resolved Hide resolved
thorclient/thorclient.go Outdated Show resolved Hide resolved
@libotony
Copy link
Member

@otherview Appended a commit a2569e0

thorclient/thorclient.go Outdated Show resolved Hide resolved
thorclient/thorclient.go Outdated Show resolved Hide resolved
}

// FilterTransfers filters transfers based on the provided filter request.
func (c *Client) FilterTransfers(req *events.EventFilter) ([]*transfers.FilteredTransfer, error) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be TransferFilter

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We already have filter - transfer style in the code, like here, also the returned data structure is called FilteredTransfer. I would keep the naming consistent, wdyt?

Copy link
Member

@libotony libotony Oct 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the confusion, I mean the parameter in the function. Should be github.com/vechain/thor/api/transfers.TransferFilter

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ohh, you're right, good catch!

@paologalligit paologalligit merged commit b6380d5 into master Oct 29, 2024
15 checks passed
@paologalligit paologalligit deleted the paolo/feat/client branch October 29, 2024 10:21
paologalligit added a commit that referenced this pull request Nov 8, 2024
* Thor client (#818)

* feat: add thorclient

* refactor: remove roundTripper

* refactor: change null check

* clean: remove commented code

* feat: add account revision and pending tx

* fix: add licence headers and fix linter issue

* refactor: rename package

* refactor: change revision type to string

* refactor: rename GetLogs and GetTransfers to FilterEvents and FilterTransfers

* refactor: change FilterEvents and FilterTransactions request type to EventFilter

* Adding common.EventWrapper to handle channel errors

* tweak

* update rawclient + update account tests

* tidy up names

* update tests

* pr comments

* adding raw tx

* Tidy up method names and calls

* options client

* tweaks

* pr comments

* Update thorclient/common/common.go

Co-authored-by: libotony <[email protected]>

* pr comments

* Adding Subscriptions

* Pr comments

* adjust func orders

* pr comments

* changing subscribe to use the channel close vs multiple channels

* adding go-doc

* no error after unsubscribe

* pr comments

* checking status code is 2xx

* fix: change FilterTransfers argument

---------

Co-authored-by: otherview <[email protected]>
Co-authored-by: libotony <[email protected]>

* Show all issues on lint (#869)

* Show all issues on lint

* fix lint

* fix(docker): using AWS docker repo for trivy (#872)

* fix(docker): using AWS docker repo for trivy

* fix(docker): using AWS docker repo for trivy

* Darren/feat/add subscription cache (#866)

* ehancement: create a cache for block based subscriptions

* minor: change function names for subscriptions

* test: add unit test for message cache

* chore: add license headers

* refactor: fix up error handling

* fix: remove bad test

* fix: PR comments

* fix: PR comments - remove block cache

* refactor(subscriptions): store structs in cache, not bytes

* fix(license): add license header

* chore(subscriptions): revert unit test changes

* enhancement: resolve pr comments to use simplelru

* enhancement: resolve pr comments - use id as key

* Add additional block tests (#863)

* enhancement(logging): leverage trace level (#873)

* Add testchain package (#844)

* Refactor thor node

* thorchain allows insertion of blocks

* remove thorNode, added testchain

* clean up + comments

* adding license headers

* adding templating tests for thorclient

* Remove test event hacks

* remove types

* removed chain_builder + added logdb to testchain

* pr comments

* Update test/testchain/chain.go

Co-authored-by: libotony <[email protected]>

---------

Co-authored-by: libotony <[email protected]>

* chore(docs): update spec for validator nodes (#875)

* chore(docs): update spec for validator nodes

* chore(docs): update cores

* chore(docs): remove public node stuff

* Darren/logdb remove leading zeros (#865)

* feat: add new txIndex column to event meta response

* test: add convert event test

* feat: make txLog and txIndex as optional return params

* chore: update swagger with new event optional data

* feat: save logIndex in sequence

* feat: tweaked bits in sequence

* refactor: rename optional log meta field

* refactor: comments, yaml and txIndex counts

* rebase to master

* fix: remove stale struct

* add txIndex to returned logdb query

* reset to 0 eventCount and transferCount each receipt and write blockId only once

* fix lint

* rephrase logIndex description in yaml file

* refactor: use filter.Option instead of eventFilter.Option

* move includeIndexes to api

---------

Co-authored-by: otherview <[email protected]>
Co-authored-by: libotony <[email protected]>
Co-authored-by: Darren Kelly <[email protected]>
Co-authored-by: Makis Christou <[email protected]>
libotony added a commit that referenced this pull request Nov 12, 2024
* Thor client (#818)

* feat: add thorclient

* refactor: remove roundTripper

* refactor: change null check

* clean: remove commented code

* feat: add account revision and pending tx

* fix: add licence headers and fix linter issue

* refactor: rename package

* refactor: change revision type to string

* refactor: rename GetLogs and GetTransfers to FilterEvents and FilterTransfers

* refactor: change FilterEvents and FilterTransactions request type to EventFilter

* Adding common.EventWrapper to handle channel errors

* tweak

* update rawclient + update account tests

* tidy up names

* update tests

* pr comments

* adding raw tx

* Tidy up method names and calls

* options client

* tweaks

* pr comments

* Update thorclient/common/common.go

Co-authored-by: libotony <[email protected]>

* pr comments

* Adding Subscriptions

* Pr comments

* adjust func orders

* pr comments

* changing subscribe to use the channel close vs multiple channels

* adding go-doc

* no error after unsubscribe

* pr comments

* checking status code is 2xx

* fix: change FilterTransfers argument

---------

Co-authored-by: otherview <[email protected]>
Co-authored-by: libotony <[email protected]>

* Show all issues on lint (#869)

* Show all issues on lint

* fix lint

* fix(docker): using AWS docker repo for trivy (#872)

* fix(docker): using AWS docker repo for trivy

* fix(docker): using AWS docker repo for trivy

* Darren/feat/add subscription cache (#866)

* ehancement: create a cache for block based subscriptions

* minor: change function names for subscriptions

* test: add unit test for message cache

* chore: add license headers

* refactor: fix up error handling

* fix: remove bad test

* fix: PR comments

* fix: PR comments - remove block cache

* refactor(subscriptions): store structs in cache, not bytes

* fix(license): add license header

* chore(subscriptions): revert unit test changes

* enhancement: resolve pr comments to use simplelru

* enhancement: resolve pr comments - use id as key

* Add additional block tests (#863)

* enhancement(logging): leverage trace level (#873)

* Add testchain package (#844)

* Refactor thor node

* thorchain allows insertion of blocks

* remove thorNode, added testchain

* clean up + comments

* adding license headers

* adding templating tests for thorclient

* Remove test event hacks

* remove types

* removed chain_builder + added logdb to testchain

* pr comments

* Update test/testchain/chain.go

Co-authored-by: libotony <[email protected]>

---------

Co-authored-by: libotony <[email protected]>

* chore(docs): update spec for validator nodes (#875)

* chore(docs): update spec for validator nodes

* chore(docs): update cores

* chore(docs): remove public node stuff

* Darren/logdb remove leading zeros (#865)

* feat: add new txIndex column to event meta response

* test: add convert event test

* feat: make txLog and txIndex as optional return params

* chore: update swagger with new event optional data

* feat: save logIndex in sequence

* feat: tweaked bits in sequence

* refactor: rename optional log meta field

* refactor: comments, yaml and txIndex counts

* rebase to master

* fix: remove stale struct

* add txIndex to returned logdb query

* reset to 0 eventCount and transferCount each receipt and write blockId only once

* fix lint

* rephrase logIndex description in yaml file

* refactor: use filter.Option instead of eventFilter.Option

* move includeIndexes to api

---------

Co-authored-by: otherview <[email protected]>
Co-authored-by: libotony <[email protected]>
Co-authored-by: Darren Kelly <[email protected]>
Co-authored-by: Makis Christou <[email protected]>
libotony added a commit that referenced this pull request Nov 12, 2024
* feat: add thorclient

* refactor: remove roundTripper

* refactor: change null check

* clean: remove commented code

* feat: add account revision and pending tx

* fix: add licence headers and fix linter issue

* refactor: rename package

* refactor: change revision type to string

* refactor: rename GetLogs and GetTransfers to FilterEvents and FilterTransfers

* refactor: change FilterEvents and FilterTransactions request type to EventFilter

* Adding common.EventWrapper to handle channel errors

* tweak

* update rawclient + update account tests

* tidy up names

* update tests

* pr comments

* adding raw tx

* Tidy up method names and calls

* options client

* tweaks

* pr comments

* Update thorclient/common/common.go

Co-authored-by: libotony <[email protected]>

* pr comments

* Adding Subscriptions

* Pr comments

* adjust func orders

* pr comments

* changing subscribe to use the channel close vs multiple channels

* adding go-doc

* no error after unsubscribe

* pr comments

* checking status code is 2xx

* fix: change FilterTransfers argument

---------

Co-authored-by: otherview <[email protected]>
Co-authored-by: libotony <[email protected]>
libotony added a commit that referenced this pull request Nov 29, 2024
* feat: add thorclient

* refactor: remove roundTripper

* refactor: change null check

* clean: remove commented code

* feat: add account revision and pending tx

* fix: add licence headers and fix linter issue

* refactor: rename package

* refactor: change revision type to string

* refactor: rename GetLogs and GetTransfers to FilterEvents and FilterTransfers

* refactor: change FilterEvents and FilterTransactions request type to EventFilter

* Adding common.EventWrapper to handle channel errors

* tweak

* update rawclient + update account tests

* tidy up names

* update tests

* pr comments

* adding raw tx

* Tidy up method names and calls

* options client

* tweaks

* pr comments

* Update thorclient/common/common.go

Co-authored-by: libotony <[email protected]>

* pr comments

* Adding Subscriptions

* Pr comments

* adjust func orders

* pr comments

* changing subscribe to use the channel close vs multiple channels

* adding go-doc

* no error after unsubscribe

* pr comments

* checking status code is 2xx

* fix: change FilterTransfers argument

---------

Co-authored-by: otherview <[email protected]>
Co-authored-by: libotony <[email protected]>
otherview added a commit that referenced this pull request Nov 29, 2024
* Thor client (#818)

* feat: add thorclient

* refactor: remove roundTripper

* refactor: change null check

* clean: remove commented code

* feat: add account revision and pending tx

* fix: add licence headers and fix linter issue

* refactor: rename package

* refactor: change revision type to string

* refactor: rename GetLogs and GetTransfers to FilterEvents and FilterTransfers

* refactor: change FilterEvents and FilterTransactions request type to EventFilter

* Adding common.EventWrapper to handle channel errors

* tweak

* update rawclient + update account tests

* tidy up names

* update tests

* pr comments

* adding raw tx

* Tidy up method names and calls

* options client

* tweaks

* pr comments

* Update thorclient/common/common.go

Co-authored-by: libotony <[email protected]>

* pr comments

* Adding Subscriptions

* Pr comments

* adjust func orders

* pr comments

* changing subscribe to use the channel close vs multiple channels

* adding go-doc

* no error after unsubscribe

* pr comments

* checking status code is 2xx

* fix: change FilterTransfers argument

---------

Co-authored-by: otherview <[email protected]>
Co-authored-by: libotony <[email protected]>

* Add testchain package (#844)

* Refactor thor node

* thorchain allows insertion of blocks

* remove thorNode, added testchain

* clean up + comments

* adding license headers

* adding templating tests for thorclient

* Remove test event hacks

* remove types

* removed chain_builder + added logdb to testchain

* pr comments

* Update test/testchain/chain.go

Co-authored-by: libotony <[email protected]>

---------

Co-authored-by: libotony <[email protected]>

* cmd/thor: update instance dir to v4

* trie: implement varint-prefix coder

* deps: add github.com/qianbin/drlp

* trie: implement appendHexToCompact & compactLen

* trie: temporarily remove merkle proof stuff

* trie: many changes

* disk usage reduced by 33% (force embedding shortnode)
* new encoding method for storing nodes
* optimize trie hashing
* versioning standalone nodes
* remove extended trie
* improve trie interface
* simplify NodeIterator, remove unused codes

* trie: optimize full-node encoding/decoding

* trie: tweak shortnode encoding

* muxdb: move engine pkg

* trie: add Version() method for node interface

* muxdb: refactor due to trie updates and:

* remove leafbank stuff
* simplify muxdb.Trie implementation
* improve root node cache using ttl eviction
* add leaf key filter

* chain: a lot of changes

* improve block content storage scheme
* remove steady block tracking
* remove tx & receipt cache

* state: changes due to update of trie

* lowrlp: remove this pkg

* txpool: changes due to underlying pkg update

* genesis: changes due to underlying pkg update

* consensus: changes due to underlying pkg update

* builtin: changes due to underlying pkg update

* runtime: changes due to underlying pkg update

* api: changes due to underlying pkg update

* cmd/thor/pruner: rename pkg optimizer to pruner

* cmd/thor: changes due to underlying pkg update

* muxdb: abandon leaf filter

* cmd/thor/pruner: use smaller period when nearly synced

* muxdb: improve trie node path encoding

* trie: treat short nodes as standalone nodes when skipping hash

* cmd/thor: fix disablePrunerFlag not work

* trie: improve refNode encoding/decoding

* muxdb: improve history node key encoding

* cmd/thor: adjust pruner parameters

* build: fix test cases

* lint: fix lint error

* muxdb: fix ver encoding in node blob cache

* muxdb: add test cases for cache

* runtime: fix test compile error

* make build and test pass after rebase

* add back pruner tests

* add tests for node encoding

* minor typo

* update named store space prefix

* add more tests

* fix block summary in repo

* make build and test pass after rebase

* add back pruner tests

* remove SetBestBlockID from tests

* minor improvement

* pr comments

* adding a comment

* Metrics: Cache hit/miss (#886)

* change

* reverted to previous format

* Add dummy cache for inmem ops (#883)

* Add empty cache for inmem ops

* changing name to dummyCache

* Metrics: Reuse `shouldLog` so we get cache hit/miss data at the same pace (#888)

* first commit

* first commit

* Metrics: Disk IO reads and writes (#890)

* changes

* removed log

* sleeping any way

* pr review

* Pedro/maindb v4/benchmarks (#891)

* Adding Benchmark tests

* processing txs

* Working benchmarks

* lint

* adding tempdir

* Adding transactions benchmark + repository cache

* improve cache stats log and metric

* totally removed SetBestBlockID

* removed unused tests

* update bench tests

* getreceipts metrics + lint

---------

Co-authored-by: Paolo Galli <[email protected]>
Co-authored-by: libotony <[email protected]>
Co-authored-by: qianbin <[email protected]>
Co-authored-by: Darren Kelly <[email protected]>
Co-authored-by: Miguel Angel Rojo <[email protected]>
darrenvechain added a commit that referenced this pull request Dec 3, 2024
* Thor client (#818)

* feat: add thorclient

* refactor: remove roundTripper

* refactor: change null check

* clean: remove commented code

* feat: add account revision and pending tx

* fix: add licence headers and fix linter issue

* refactor: rename package

* refactor: change revision type to string

* refactor: rename GetLogs and GetTransfers to FilterEvents and FilterTransfers

* refactor: change FilterEvents and FilterTransactions request type to EventFilter

* Adding common.EventWrapper to handle channel errors

* tweak

* update rawclient + update account tests

* tidy up names

* update tests

* pr comments

* adding raw tx

* Tidy up method names and calls

* options client

* tweaks

* pr comments

* Update thorclient/common/common.go

Co-authored-by: libotony <[email protected]>

* pr comments

* Adding Subscriptions

* Pr comments

* adjust func orders

* pr comments

* changing subscribe to use the channel close vs multiple channels

* adding go-doc

* no error after unsubscribe

* pr comments

* checking status code is 2xx

* fix: change FilterTransfers argument

---------

Co-authored-by: otherview <[email protected]>
Co-authored-by: libotony <[email protected]>

* Add testchain package (#844)

* Refactor thor node

* thorchain allows insertion of blocks

* remove thorNode, added testchain

* clean up + comments

* adding license headers

* adding templating tests for thorclient

* Remove test event hacks

* remove types

* removed chain_builder + added logdb to testchain

* pr comments

* Update test/testchain/chain.go

Co-authored-by: libotony <[email protected]>

---------

Co-authored-by: libotony <[email protected]>

* cmd/thor: update instance dir to v4

* trie: implement varint-prefix coder

* deps: add github.com/qianbin/drlp

* trie: implement appendHexToCompact & compactLen

* trie: temporarily remove merkle proof stuff

* trie: many changes

* disk usage reduced by 33% (force embedding shortnode)
* new encoding method for storing nodes
* optimize trie hashing
* versioning standalone nodes
* remove extended trie
* improve trie interface
* simplify NodeIterator, remove unused codes

* trie: optimize full-node encoding/decoding

* trie: tweak shortnode encoding

* muxdb: move engine pkg

* trie: add Version() method for node interface

* muxdb: refactor due to trie updates and:

* remove leafbank stuff
* simplify muxdb.Trie implementation
* improve root node cache using ttl eviction
* add leaf key filter

* chain: a lot of changes

* improve block content storage scheme
* remove steady block tracking
* remove tx & receipt cache

* state: changes due to update of trie

* lowrlp: remove this pkg

* txpool: changes due to underlying pkg update

* genesis: changes due to underlying pkg update

* consensus: changes due to underlying pkg update

* builtin: changes due to underlying pkg update

* runtime: changes due to underlying pkg update

* api: changes due to underlying pkg update

* cmd/thor/pruner: rename pkg optimizer to pruner

* cmd/thor: changes due to underlying pkg update

* muxdb: abandon leaf filter

* cmd/thor/pruner: use smaller period when nearly synced

* muxdb: improve trie node path encoding

* trie: treat short nodes as standalone nodes when skipping hash

* cmd/thor: fix disablePrunerFlag not work

* trie: improve refNode encoding/decoding

* muxdb: improve history node key encoding

* cmd/thor: adjust pruner parameters

* build: fix test cases

* lint: fix lint error

* muxdb: fix ver encoding in node blob cache

* muxdb: add test cases for cache

* runtime: fix test compile error

* make build and test pass after rebase

* add back pruner tests

* add tests for node encoding

* minor typo

* update named store space prefix

* add more tests

* fix block summary in repo

* make build and test pass after rebase

* add back pruner tests

* remove SetBestBlockID from tests

* minor improvement

* pr comments

* adding a comment

* Metrics: Cache hit/miss (#886)

* change

* reverted to previous format

* Add dummy cache for inmem ops (#883)

* Add empty cache for inmem ops

* changing name to dummyCache

* Metrics: Reuse `shouldLog` so we get cache hit/miss data at the same pace (#888)

* first commit

* first commit

* Metrics: Disk IO reads and writes (#890)

* changes

* removed log

* sleeping any way

* pr review

* Pedro/maindb v4/benchmarks (#891)

* Adding Benchmark tests

* processing txs

* Working benchmarks

* lint

* adding tempdir

* improve cache stats log and metric

* totally removed SetBestBlockID

* fix: logs API not returning results when to=0,from=omitted

* minor update

* fix: PR comments + lint

* improve convert range logic

* chore: remove debug log

---------

Co-authored-by: Paolo Galli <[email protected]>
Co-authored-by: otherview <[email protected]>
Co-authored-by: libotony <[email protected]>
Co-authored-by: qianbin <[email protected]>
Co-authored-by: Miguel Angel Rojo <[email protected]>
darrenvechain added a commit that referenced this pull request Dec 9, 2024
* fix(documentation): use absolute links in markdown (#889)

* Add benchmark test to node block process (#892)

* Add benchmark test to node block process

* added file-based storage

* use tempdir

* update dependency go-ethereum (#895)

* chore: update API metrics bucket and endpoint names (#893)

* chore: update API metrics bucket and endpoint names

* fix: typo & tests

* fix: lint

* chore: add websocket total counter

* fix: txs endpoints names & ws subject

* fix: unit tests

* chore: standardise naming convention

* chore: add websocke duration & http code

* chore: add websocke duration & http code

* fix: lint issues

* fix: sync issues with metrics

* chore: update websocket durations bucket

* fix: PR comments - use sync.Once

* chore: update builtin generation (#896)

* chore: update builtin generation

* fix: update GHA

* Thor client (#818)

* feat: add thorclient

* refactor: remove roundTripper

* refactor: change null check

* clean: remove commented code

* feat: add account revision and pending tx

* fix: add licence headers and fix linter issue

* refactor: rename package

* refactor: change revision type to string

* refactor: rename GetLogs and GetTransfers to FilterEvents and FilterTransfers

* refactor: change FilterEvents and FilterTransactions request type to EventFilter

* Adding common.EventWrapper to handle channel errors

* tweak

* update rawclient + update account tests

* tidy up names

* update tests

* pr comments

* adding raw tx

* Tidy up method names and calls

* options client

* tweaks

* pr comments

* Update thorclient/common/common.go

Co-authored-by: libotony <[email protected]>

* pr comments

* Adding Subscriptions

* Pr comments

* adjust func orders

* pr comments

* changing subscribe to use the channel close vs multiple channels

* adding go-doc

* no error after unsubscribe

* pr comments

* checking status code is 2xx

* fix: change FilterTransfers argument

---------

Co-authored-by: otherview <[email protected]>
Co-authored-by: libotony <[email protected]>

* Add testchain package (#844)

* Refactor thor node

* thorchain allows insertion of blocks

* remove thorNode, added testchain

* clean up + comments

* adding license headers

* adding templating tests for thorclient

* Remove test event hacks

* remove types

* removed chain_builder + added logdb to testchain

* pr comments

* Update test/testchain/chain.go

Co-authored-by: libotony <[email protected]>

---------

Co-authored-by: libotony <[email protected]>

* cmd/thor: update instance dir to v4

* trie: implement varint-prefix coder

* deps: add github.com/qianbin/drlp

* trie: implement appendHexToCompact & compactLen

* trie: temporarily remove merkle proof stuff

* trie: many changes

* disk usage reduced by 33% (force embedding shortnode)
* new encoding method for storing nodes
* optimize trie hashing
* versioning standalone nodes
* remove extended trie
* improve trie interface
* simplify NodeIterator, remove unused codes

* trie: optimize full-node encoding/decoding

* trie: tweak shortnode encoding

* muxdb: move engine pkg

* trie: add Version() method for node interface

* muxdb: refactor due to trie updates and:

* remove leafbank stuff
* simplify muxdb.Trie implementation
* improve root node cache using ttl eviction
* add leaf key filter

* chain: a lot of changes

* improve block content storage scheme
* remove steady block tracking
* remove tx & receipt cache

* state: changes due to update of trie

* lowrlp: remove this pkg

* txpool: changes due to underlying pkg update

* genesis: changes due to underlying pkg update

* consensus: changes due to underlying pkg update

* builtin: changes due to underlying pkg update

* runtime: changes due to underlying pkg update

* api: changes due to underlying pkg update

* cmd/thor/pruner: rename pkg optimizer to pruner

* cmd/thor: changes due to underlying pkg update

* muxdb: abandon leaf filter

* cmd/thor/pruner: use smaller period when nearly synced

* muxdb: improve trie node path encoding

* trie: treat short nodes as standalone nodes when skipping hash

* cmd/thor: fix disablePrunerFlag not work

* trie: improve refNode encoding/decoding

* muxdb: improve history node key encoding

* cmd/thor: adjust pruner parameters

* build: fix test cases

* lint: fix lint error

* muxdb: fix ver encoding in node blob cache

* muxdb: add test cases for cache

* runtime: fix test compile error

* make build and test pass after rebase

* add back pruner tests

* add tests for node encoding

* minor typo

* update named store space prefix

* add more tests

* fix block summary in repo

* make build and test pass after rebase

* add back pruner tests

* remove SetBestBlockID from tests

* minor improvement

* pr comments

* adding a comment

* Metrics: Cache hit/miss (#886)

* change

* reverted to previous format

* Add dummy cache for inmem ops (#883)

* Add empty cache for inmem ops

* changing name to dummyCache

* Metrics: Reuse `shouldLog` so we get cache hit/miss data at the same pace (#888)

* first commit

* first commit

* Metrics: Disk IO reads and writes (#890)

* changes

* removed log

* sleeping any way

* pr review

* Pedro/maindb v4/benchmarks (#891)

* Adding Benchmark tests

* processing txs

* Working benchmarks

* lint

* adding tempdir

* improve cache stats log and metric

* totally removed SetBestBlockID

* Maindb v4 Transaction benchmark plus cache (#894)

* Thor client (#818)

* feat: add thorclient

* refactor: remove roundTripper

* refactor: change null check

* clean: remove commented code

* feat: add account revision and pending tx

* fix: add licence headers and fix linter issue

* refactor: rename package

* refactor: change revision type to string

* refactor: rename GetLogs and GetTransfers to FilterEvents and FilterTransfers

* refactor: change FilterEvents and FilterTransactions request type to EventFilter

* Adding common.EventWrapper to handle channel errors

* tweak

* update rawclient + update account tests

* tidy up names

* update tests

* pr comments

* adding raw tx

* Tidy up method names and calls

* options client

* tweaks

* pr comments

* Update thorclient/common/common.go

Co-authored-by: libotony <[email protected]>

* pr comments

* Adding Subscriptions

* Pr comments

* adjust func orders

* pr comments

* changing subscribe to use the channel close vs multiple channels

* adding go-doc

* no error after unsubscribe

* pr comments

* checking status code is 2xx

* fix: change FilterTransfers argument

---------

Co-authored-by: otherview <[email protected]>
Co-authored-by: libotony <[email protected]>

* Add testchain package (#844)

* Refactor thor node

* thorchain allows insertion of blocks

* remove thorNode, added testchain

* clean up + comments

* adding license headers

* adding templating tests for thorclient

* Remove test event hacks

* remove types

* removed chain_builder + added logdb to testchain

* pr comments

* Update test/testchain/chain.go

Co-authored-by: libotony <[email protected]>

---------

Co-authored-by: libotony <[email protected]>

* cmd/thor: update instance dir to v4

* trie: implement varint-prefix coder

* deps: add github.com/qianbin/drlp

* trie: implement appendHexToCompact & compactLen

* trie: temporarily remove merkle proof stuff

* trie: many changes

* disk usage reduced by 33% (force embedding shortnode)
* new encoding method for storing nodes
* optimize trie hashing
* versioning standalone nodes
* remove extended trie
* improve trie interface
* simplify NodeIterator, remove unused codes

* trie: optimize full-node encoding/decoding

* trie: tweak shortnode encoding

* muxdb: move engine pkg

* trie: add Version() method for node interface

* muxdb: refactor due to trie updates and:

* remove leafbank stuff
* simplify muxdb.Trie implementation
* improve root node cache using ttl eviction
* add leaf key filter

* chain: a lot of changes

* improve block content storage scheme
* remove steady block tracking
* remove tx & receipt cache

* state: changes due to update of trie

* lowrlp: remove this pkg

* txpool: changes due to underlying pkg update

* genesis: changes due to underlying pkg update

* consensus: changes due to underlying pkg update

* builtin: changes due to underlying pkg update

* runtime: changes due to underlying pkg update

* api: changes due to underlying pkg update

* cmd/thor/pruner: rename pkg optimizer to pruner

* cmd/thor: changes due to underlying pkg update

* muxdb: abandon leaf filter

* cmd/thor/pruner: use smaller period when nearly synced

* muxdb: improve trie node path encoding

* trie: treat short nodes as standalone nodes when skipping hash

* cmd/thor: fix disablePrunerFlag not work

* trie: improve refNode encoding/decoding

* muxdb: improve history node key encoding

* cmd/thor: adjust pruner parameters

* build: fix test cases

* lint: fix lint error

* muxdb: fix ver encoding in node blob cache

* muxdb: add test cases for cache

* runtime: fix test compile error

* make build and test pass after rebase

* add back pruner tests

* add tests for node encoding

* minor typo

* update named store space prefix

* add more tests

* fix block summary in repo

* make build and test pass after rebase

* add back pruner tests

* remove SetBestBlockID from tests

* minor improvement

* pr comments

* adding a comment

* Metrics: Cache hit/miss (#886)

* change

* reverted to previous format

* Add dummy cache for inmem ops (#883)

* Add empty cache for inmem ops

* changing name to dummyCache

* Metrics: Reuse `shouldLog` so we get cache hit/miss data at the same pace (#888)

* first commit

* first commit

* Metrics: Disk IO reads and writes (#890)

* changes

* removed log

* sleeping any way

* pr review

* Pedro/maindb v4/benchmarks (#891)

* Adding Benchmark tests

* processing txs

* Working benchmarks

* lint

* adding tempdir

* Adding transactions benchmark + repository cache

* improve cache stats log and metric

* totally removed SetBestBlockID

* removed unused tests

* update bench tests

* getreceipts metrics + lint

---------

Co-authored-by: Paolo Galli <[email protected]>
Co-authored-by: libotony <[email protected]>
Co-authored-by: qianbin <[email protected]>
Co-authored-by: Darren Kelly <[email protected]>
Co-authored-by: Miguel Angel Rojo <[email protected]>

* reduce clauses() allocations

* bug: fix logs endpoints query (#900)

* Thor client (#818)

* feat: add thorclient

* refactor: remove roundTripper

* refactor: change null check

* clean: remove commented code

* feat: add account revision and pending tx

* fix: add licence headers and fix linter issue

* refactor: rename package

* refactor: change revision type to string

* refactor: rename GetLogs and GetTransfers to FilterEvents and FilterTransfers

* refactor: change FilterEvents and FilterTransactions request type to EventFilter

* Adding common.EventWrapper to handle channel errors

* tweak

* update rawclient + update account tests

* tidy up names

* update tests

* pr comments

* adding raw tx

* Tidy up method names and calls

* options client

* tweaks

* pr comments

* Update thorclient/common/common.go

Co-authored-by: libotony <[email protected]>

* pr comments

* Adding Subscriptions

* Pr comments

* adjust func orders

* pr comments

* changing subscribe to use the channel close vs multiple channels

* adding go-doc

* no error after unsubscribe

* pr comments

* checking status code is 2xx

* fix: change FilterTransfers argument

---------

Co-authored-by: otherview <[email protected]>
Co-authored-by: libotony <[email protected]>

* Add testchain package (#844)

* Refactor thor node

* thorchain allows insertion of blocks

* remove thorNode, added testchain

* clean up + comments

* adding license headers

* adding templating tests for thorclient

* Remove test event hacks

* remove types

* removed chain_builder + added logdb to testchain

* pr comments

* Update test/testchain/chain.go

Co-authored-by: libotony <[email protected]>

---------

Co-authored-by: libotony <[email protected]>

* cmd/thor: update instance dir to v4

* trie: implement varint-prefix coder

* deps: add github.com/qianbin/drlp

* trie: implement appendHexToCompact & compactLen

* trie: temporarily remove merkle proof stuff

* trie: many changes

* disk usage reduced by 33% (force embedding shortnode)
* new encoding method for storing nodes
* optimize trie hashing
* versioning standalone nodes
* remove extended trie
* improve trie interface
* simplify NodeIterator, remove unused codes

* trie: optimize full-node encoding/decoding

* trie: tweak shortnode encoding

* muxdb: move engine pkg

* trie: add Version() method for node interface

* muxdb: refactor due to trie updates and:

* remove leafbank stuff
* simplify muxdb.Trie implementation
* improve root node cache using ttl eviction
* add leaf key filter

* chain: a lot of changes

* improve block content storage scheme
* remove steady block tracking
* remove tx & receipt cache

* state: changes due to update of trie

* lowrlp: remove this pkg

* txpool: changes due to underlying pkg update

* genesis: changes due to underlying pkg update

* consensus: changes due to underlying pkg update

* builtin: changes due to underlying pkg update

* runtime: changes due to underlying pkg update

* api: changes due to underlying pkg update

* cmd/thor/pruner: rename pkg optimizer to pruner

* cmd/thor: changes due to underlying pkg update

* muxdb: abandon leaf filter

* cmd/thor/pruner: use smaller period when nearly synced

* muxdb: improve trie node path encoding

* trie: treat short nodes as standalone nodes when skipping hash

* cmd/thor: fix disablePrunerFlag not work

* trie: improve refNode encoding/decoding

* muxdb: improve history node key encoding

* cmd/thor: adjust pruner parameters

* build: fix test cases

* lint: fix lint error

* muxdb: fix ver encoding in node blob cache

* muxdb: add test cases for cache

* runtime: fix test compile error

* make build and test pass after rebase

* add back pruner tests

* add tests for node encoding

* minor typo

* update named store space prefix

* add more tests

* fix block summary in repo

* make build and test pass after rebase

* add back pruner tests

* remove SetBestBlockID from tests

* minor improvement

* pr comments

* adding a comment

* Metrics: Cache hit/miss (#886)

* change

* reverted to previous format

* Add dummy cache for inmem ops (#883)

* Add empty cache for inmem ops

* changing name to dummyCache

* Metrics: Reuse `shouldLog` so we get cache hit/miss data at the same pace (#888)

* first commit

* first commit

* Metrics: Disk IO reads and writes (#890)

* changes

* removed log

* sleeping any way

* pr review

* Pedro/maindb v4/benchmarks (#891)

* Adding Benchmark tests

* processing txs

* Working benchmarks

* lint

* adding tempdir

* improve cache stats log and metric

* totally removed SetBestBlockID

* fix: logs API not returning results when to=0,from=omitted

* minor update

* fix: PR comments + lint

* improve convert range logic

* chore: remove debug log

---------

Co-authored-by: Paolo Galli <[email protected]>
Co-authored-by: otherview <[email protected]>
Co-authored-by: libotony <[email protected]>
Co-authored-by: qianbin <[email protected]>
Co-authored-by: Miguel Angel Rojo <[email protected]>

* chore(chain): add repo cache metrics (#910)

* chore(chain): add repo cache metrics

* refactor(chain): cache hit miss

---------

Co-authored-by: Darren Kelly <[email protected]>
Co-authored-by: Pedro Gomes <[email protected]>
Co-authored-by: Paolo Galli <[email protected]>
Co-authored-by: qianbin <[email protected]>
Co-authored-by: Miguel Angel Rojo <[email protected]>
Co-authored-by: Darren Kelly <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants