forked from jl777/SuperNET
-
Notifications
You must be signed in to change notification settings - Fork 94
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(utxo): prioritize electrum connections #1966
Merged
Merged
Changes from 204 commits
Commits
Show all changes
206 commits
Select commit
Hold shift + click to select a range
bd00a99
stash
rozhkovdmitrii a31179a
stash
da4e5f9
save state of being able to be compiled
38e3cc4
save state
50c8052
introduce conn_mng guarded state
4ba4a1a
stash
9466e53
get rid of warnings
ae5ed65
pushing secondary connection is implemented
c1bffff
move connect_to
03af8ce
Conn manager works as it was supposed
b31a9ff
refactor: get `get_conn` and `get_conn_by_address` apart
f325ebf
refactor: electrum_request_multi rolled back
39a7475
feat: introduce conn_mng_policy and start moving parts into the files…
0e612bb
feat: started to implement multiple_conn_mng based on dev branch
d5fa172
feat: stash
e7a10f0
feat: multiple conn mng has been simplified
cfbeea0
doc: get rid of typo
99408ce
refactor: Bring Priority enum into mm2_rpc::rpc_data
de14a3f
feat: Provide setting conn_mng_policy during config initialization us…
8363353
refactor: Prefer &str in some places
04eae35
feat: implement rotate_servers for conn_mng_multiple
48d507c
get rid of redundant trace
a193f5d
revert doc comment
7b98067
revert commenting
3663716
process creating subsystem errors
7f10b19
Merge branch 'dev' into 1520-prioritize-nodes
4c1a577
commit working state
da4d58d
make wasm version working
5686619
Get rid of some warnings
3534ddb
Renaming
d98521f
refactor: implement `duplicate_suspend_timeout` and `reset_supend_tim…
9031c9f
fix: getting of connecting state
fa25b6f
refactor: spit and polish
6500dbd
refactor: get_conn_ctx and get_conn_ctx_mut for the selective con_mng
41c1872
refactor: bring ConnMngTrait into separate file
22116e1
refactor: introduce ConnMngError
0d09323
refactor: rollback some changes
9d045ef
refactor: rollback some changes
4fce08b
Revert "refactor: rollback some changes"
95bf9fa
refactor: rollback some changes
2373174
refactor: rollback some changes
9608966
refactor: spit and polish
d05347d
refactor: spit and polish
291d901
refactor: spit and polish
5e51798
refactor: spit and polish
5b23e37
refactor: spit and polish
eb82125
refactor: spit and polish
3e98a42
refactor: spit and polish
5663668
refactor: spit and polish
79e6850
refactor: spit and polish
0e10bcb
refactor: spit and polish
6f96d6e
Merge branch 'dev' into 1520-prioritize-nodes
borngraced c630ee0
fix merge conflicts
borngraced 71d9664
fix wasm clippy warnings
borngraced b2434da
rename conn_mng -> connection_manager
borngraced 935bee5
merge ConnectionManagerSelectiveQueue with ConnectionManagerSelective…
borngraced 2097e53
manage sending multiple sub requests for same scripthash and other fi…
borngraced 0ed6c3a
Merge remote-tracking branch 'origin/dev' into 1520-prioritize-nodes
borngraced e6585ee
fix merge conflicts
borngraced 9cc0378
fix review notes and improvements
borngraced 0e59572
rename guarded to state_guard and move connect impl to impl trait for…
borngraced e6f2ba3
move get_connection_ctx_mut and get_connection_ctx to ConnectionManag…
borngraced 5eede71
move ElectrumConnCtx to rpc_clients::connection_manager_common and co…
borngraced 350a992
merge get_suspend_timeout and duplicate_suspend_timeout and other ver…
borngraced 2d24da3
connection is_connected is properly checked for selective
borngraced 7b7110d
move state related/only method to it's impl
borngraced 10ff5c8
revert is_connected impl for multiple
borngraced ef6ff41
handle error in ConnectionManagerMultipleImpl
borngraced f87278d
Refactor ConnectionManagerMultiple & ConnectionManagerSelective
mariocynicys 61af931
fix merge conflicts
borngraced a1d0a9a
set the state as running when we start executing the future
mariocynicys ebb5394
this type annotation isn't needed
mariocynicys 9bed9d1
remove deadcode
mariocynicys 5666936
stashing
mariocynicys 426037c
stashing
mariocynicys 94a90aa
stashing
mariocynicys 20c70bf
pushing working tree for starter review - still messy tho
mariocynicys 0471735
stashing
mariocynicys bf3781f
stashing
mariocynicys 2129a8f
compiles for linux targets only
mariocynicys dafff72
atomically start connections
mariocynicys a9a56fe
fix wasm build
mariocynicys 7cffd67
some lint fixes and formatting
mariocynicys 1b49971
add back scripthash notification sender since it's used to send Subsc…
mariocynicys af03de3
multiple connection manager
mariocynicys b296728
record a fixme
mariocynicys 5a06371
fix lint issues
mariocynicys 85c3609
abort all jobs in the queue but keep it intact
mariocynicys c2f4488
add get_all_server_addresses method to connection manager trait
mariocynicys dd7cb96
fix: reuse type def - RpcTransportEventHandler
mariocynicys 6bb5e4f
fix: get rid of useless async closure
mariocynicys 2d72df4
add fixme: reminder to fire a ping loop to keep connections alive
mariocynicys d31f648
fix: send ping requests to all the active servers
mariocynicys d8c11b9
fix: add a pinging loop and manage it from the connection manager
mariocynicys 5f91ca5
store next suspend time in terms of seconds and not milliseconds
mariocynicys 518819b
bound establish_connecion_loop method with a timeout
mariocynicys 2be78df
refactor constant names
mariocynicys dfc4bc7
discuss -> fixme, to better find problems in the PR
mariocynicys 769304b
complete the selective connection manager, fixmes left
mariocynicys dc47f67
fix some bugs and fixmes
mariocynicys cce2ec4
temp fix: leak the abortable system to keep its children systems alive
mariocynicys 4c76491
remove panic
mariocynicys ed6801d
wait till one of the electrum server connects
mariocynicys 3d9680a
don't query for the version again
mariocynicys 229a809
don't create an extra child subsystem, use the parent directly
mariocynicys 1e0e162
wait till one of the electrum server connects, applied while initiali…
mariocynicys 8d9ac1c
set the test timeout back to 120s
mariocynicys 4c8d834
get rid of debuging printlns
mariocynicys 732e00e
fix wasm issues
mariocynicys 3185860
merge with origin/dev
mariocynicys ffa086d
fix adex-cli compilation issue
mariocynicys d908ec6
fix wrong electrum script hash calculation
mariocynicys eddb9ee
adapt test_unavailable_electrum_proto_version to the new design
mariocynicys c4592fc
fmt
mariocynicys f7297b5
implement common trait methods for duplicated impls
mariocynicys 84fc19d
also generalize add_subscriptions method
mariocynicys 76b2f68
fix wasm tests: use wasm's `now_ms` func & don't mutate the original …
mariocynicys ef4f458
add doc comments for the configuration types
mariocynicys a447eff
use hashset instead of vec for the subs list
mariocynicys 0103f0b
package `disconnected_at` and `next_suspend_time` in one mutex
mariocynicys 6dab399
don't collect connections list to avoid unnecessary allocations
mariocynicys d81bd8b
move priority next to electrum stuff :/
mariocynicys 8ae8d02
play safe: call abort on the previous system only after the new system
mariocynicys 220d70b
use mem::take instead of mem::replace + default
mariocynicys 54acd8e
remove fixmes
mariocynicys d9549ab
connection context: use atomics for u64 feilds
mariocynicys 260192e
adex cli: use closer versions in cargo toml
mariocynicys 5b92281
remove irrelevant comment
mariocynicys 0eedd97
don't convert to json back and forth in wasm
mariocynicys 0395e2c
merge with origin/dev
mariocynicys 8513b37
decide the connection manager policy per coin in the electrum request
mariocynicys 62993a0
merge with origin/dev
mariocynicys 33075e9
retry connections in parallel in selective mode
mariocynicys 064a12d
make sure we don't stuck in the case where all connections are suspended
mariocynicys ae04b68
add important note
mariocynicys bf01ba0
add more robustness to `get_active_connections`
mariocynicys 21b0799
endlessly try `electrum_request_multi`
mariocynicys 22fb7ff
only try endlessly on non-test env
mariocynicys 484143a
cfg! is better structured than #[cfg]
mariocynicys 6d71546
trigger disconnecting a connection if it errored
mariocynicys 6c48485
fix: resolve a couple of fixmes optimisitcally
mariocynicys 4b2464c
refactor: replace multiple & selective manager with one more dynamic …
mariocynicys 1a0058c
some macros to simplify if-let & let-else patterns
mariocynicys 7a458e3
only aquire write lock when needed
mariocynicys ced724b
try connections sequentially on the background task
mariocynicys 21fda09
remove trigger_disconnect
mariocynicys 09f34ca
resolve fixme: version negotiation logic is correct
mariocynicys 7e5e8dd
introduce a connection establishment synchronizer
mariocynicys 4809812
simplify timeouting with a new macro
mariocynicys f836f04
revert to trying connections in parallel in background task (to some …
mariocynicys 678601f
clean up unused methods, structs, fields
mariocynicys a6e8d69
merge with origin/dev
mariocynicys 6945ad1
fix the qrc20 issue on CI
mariocynicys 48f5ddd
fix adex-cli compilation (removing traces of conn_manager_policy)
mariocynicys 62d26c7
refactor expirable map
mariocynicys 4a16d78
let the responses map in eletrum connections be expriable
mariocynicys ea0cbe3
make wait_till_connected test-only
mariocynicys c37cc9b
split the connection manager into multiple impl blocks
mariocynicys 8fd92df
let connections map inside the manager be dynaimc
mariocynicys 122cf7d
not important; organization
mariocynicys bf3608d
resolve the last fixmes
mariocynicys 3fb958b
simplfying establish_connection_loop method as much as possible
mariocynicys 69db230
don't retry forever, and also try all connections when the active one…
mariocynicys eead6a8
properly respect the max_connected threshold
mariocynicys ffd2052
suggestion from omar - 1
mariocynicys f9866b1
suggestion from omar - 2 - (un)maintain methods
mariocynicys d98fd21
testing if this would fix tests
mariocynicys ff8833a
Revert "testing if this would fix tests"
mariocynicys df2fdcc
bug: don't disconnect the connection we want to connect while version…
mariocynicys 0971d9c
don't disconnect immediately
mariocynicys fe90391
abandon the wait_till_connected test-only method
mariocynicys 69a6810
fix test_electrum_rpc_client_error
mariocynicys 9bda46b
fix: flaky test_unavailable_electrum_proto_version
mariocynicys 2c61c61
yield in both test and non-test environment
mariocynicys 25b174a
suggestions from omar: don't send to all
mariocynicys b546bbd
merge with origin/dev
mariocynicys 49564b3
add a todo regarding abortablequeue abort vs finish
mariocynicys 4c2a5e0
patch on_future_finished to not mark id as finished if it was aborted
mariocynicys 2f9c568
add a todo
mariocynicys 417d054
fix test_one_unavailable_electrum_proto_version
mariocynicys b791204
review(omar): let version mismatch be an irrecoverable error
mariocynicys 989ec9e
review(omar): use simpiler method args
mariocynicys e0e7708
simplyfy timeouting - handle timeout in establish_connection_loop only
mariocynicys db93005
simplify erroring: handle disconnection in establish_connection_loop …
mariocynicys d2906ff
fix: electrum server labels as irrecoverable when DNS is down
mariocynicys 1e92dfd
review(omar): don't provide a defualt impl for event handler trait
mariocynicys fc3f172
actually validate min & max connected
mariocynicys dc8a84e
merge with origin/dev
mariocynicys c93bcb9
add clear() back to expirable map
mariocynicys be219ee
reduce the btree key of json bach request to its hash
mariocynicys 556b65f
use a deterministically-selected rpc_id from the batch
mariocynicys 72b3ab9
don't overwrite connection data that's already set
mariocynicys ee7f6a4
omit unnecessary json conversion
mariocynicys e99fd25
label version mismatch only when the electrum server replys
mariocynicys a2e4348
add coin name in error logs
mariocynicys decacb8
to revert: try to deser the electrum response as json and log it
mariocynicys 89bad4f
disable the coin if the balance query fails
mariocynicys 5e38086
run rpc futures in a tokio thread till completion
mariocynicys 643d1f3
reduce the default connection establishment timeout to 20s
mariocynicys 2060704
cap the electrum request to 5s
mariocynicys d5b2ef3
mitigate the noisy logs from the endless retries
mariocynicys 00b71c3
Revert "to revert: try to deser the electrum response as json and log…
mariocynicys cae9842
convert async mutexs to sync ones
mariocynicys 596cba0
parallelize the request when all connections fail
mariocynicys 2457b96
review(sami): let max & min connected be usize
mariocynicys 24a64e3
review(sami): split up the background_task of the connection manager
mariocynicys File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We added over 20 dependencies (some of them duplicates) simply by adding
mm2_core = { path = "../mm2_core" }
toadex_cli
. To avoid this issue, you can specify the exact versions of those packages in the mm2 packages.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Used the versions used by other mm2 crates here: 260192e
We still unfortunately have some duplicates tho :(