No json rpc streaming #2779
Merged
No json rpc streaming #2779
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Collaborator
AskAlexSharov
commented
Oct 5, 2021
AskAlexSharov
commented
- Drop feature of streaming json to network because JSON-RPC 2.0 is not ready for this. It's just impossible to return error which happened in the middle of stream and don't break JSON format.
- But keep jsoniter.Stream itself - because it reducing allocations, allow use streams buffer pool, etc...
mandrigin
added a commit
that referenced
this pull request
Oct 7, 2021
* begin 2021.10.2 release cycle * Revert "rpcdaemon: (#2752)" (#2762) This reverts commit 2afd028. * Pool v2: --txpool.pricelimit support (#2763) * --txpoo.pricelimit support * Pool v2: --txpool.accountslots flag support (#2765) * Update signal_windows.go (#2767) Trap os.interrupt instead of SIGINT and SIGTERM * Update stage_finish.go : notifications to rpc daemon (#2755) * Dockerfile: switch to go1.17 and alpine3.14 (#2766) * add logs in recoverFromDb func (#2769) * eip 1559 in miner (#2773) * Inner errors (#2774) * Clean up DEBUG category logs (#2776) - move many DEBUG logs into TRACE category * Decoding incarnation implemented (#2764) * WIP decoding incarnation specifically * Changed decodeIncarnation to be an external function * added tests to for decoding incarnations * ran gofmt -w -s * changed test name, and changed incarnations to 4 * Created a test which tests if it returns an error when there is one * ran gofmt * Capitalized all tests and made breaking test * added an error check * changed decodingForStorage for decodingIncarnationFromStorage * ran gofmt -w -s * No senders is fine (#2775) * IntermediateHash stage - switch from incremental to re-generate mode - if jump > 100K blocks (#2781) * Enable "State stream" by default (#2780) * No json rpc streaming (#2779) * reduce_bach_concurrency_default * RPCDaemon: reduce --rpc.batch.concurrency default from 50 to 2 (#2784) * Integration to print right stage in logs (#2785) * remove debug prints * RemoteDB: don't spend time to close cursors on end of tx - server will cleanup everything well (#2786) * Fermion genesis block (#2787) * updated Fermion genesis block * Updated Fermion genesis block: added precompiles * Rpcdaemon: add --tevm flag to enable experiment (#2788) * Reworkings of state compression experiments (#2790) * Changes * Progress * Another way * More * More * Produce encoding * Add uncoded characters * cleanup * Add sortdict * Fixes * Use patricia from erigon-lib * Cleanup * Switch to dynamic programming, optimise allocations in FindMatches * Optimise allocations * Reduce allocations * Switch to main branch of erigon-lib, reduce allocations further Co-authored-by: Alexey Sharp <[email protected]> Co-authored-by: Alex Sharp <[email protected]> * Ignore MaxPeers param for staticpeers (#2789) Co-authored-by: Aleksandr Borodulin <[email protected]> * less warnings * Update skip_analysis.go (#2792) * Extend preverified hashes for mainnet and ropsten (#2793) Co-authored-by: Alexey Sharp <[email protected]> Co-authored-by: Alex Sharov <[email protected]> Co-authored-by: Andrea Lanfranchi <[email protected]> Co-authored-by: Enrique Jose Avila Asapche <[email protected]> Co-authored-by: e-danko <[email protected]> Co-authored-by: ledgerwatch <[email protected]> Co-authored-by: Alexey Sharp <[email protected]> Co-authored-by: Alex Sharp <[email protected]> Co-authored-by: Alexandr Borodulin <[email protected]> Co-authored-by: Aleksandr Borodulin <[email protected]>
|
We decided to enable back streaming feature by default: #4647 Erigon has enalbed json streamin for some heavy endpoints (like trace_*). It's treadoff: greatly reduce amount of RAM (in some cases from 30GB to 30mb), but it produce invalid json format if error happened in the middle of streaming (because json is not streaming-friendly format) We decided that value from this streaming is higher than handling "error happen in the middle" rare corner case. But added flag: --rpc.streaming.disable if users wish to pay for correctnesses or compatibility. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.