-
Notifications
You must be signed in to change notification settings - Fork 11
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
kad: Refactor FindNode query, keep K best results and add tests #114
Conversation
Signed-off-by: Alexandru Vasile <[email protected]>
Signed-off-by: Alexandru Vasile <[email protected]>
Signed-off-by: Alexandru Vasile <[email protected]>
Signed-off-by: Alexandru Vasile <[email protected]>
Signed-off-by: Alexandru Vasile <[email protected]>
Signed-off-by: Alexandru Vasile <[email protected]>
Signed-off-by: Alexandru Vasile <[email protected]>
Signed-off-by: Alexandru Vasile <[email protected]>
candidates Signed-off-by: Alexandru Vasile <[email protected]>
Signed-off-by: Alexandru Vasile <[email protected]>
Signed-off-by: Alexandru Vasile <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice PR! And good to see we have FIND_NODE
tested now.
|
||
// always mark the peer as queried to prevent it getting queried again | ||
self.queried.insert(peer.peer); | ||
|
||
// TODO: could this be written in another way? | ||
// TODO: only insert nodes from whom a response was received |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this TODO still relevant? It looks like nothing has changed in its respect.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I dont think this is addressed indeed, will briefly look over libp2p or create an issue if this may be too involved / not covered by our testing atm, thanks!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
While looking at this I realized a few things:
- we were registering the first 20 (repl factor) responses
- whenever we got a new response (better) we tried to update just the peerID_ of the last entry
This has a few implications:
- the first 19 entries are never updated with a better (closer) peer
- the distance of the last entry was never updated. And because of this, we might end up in a situation where the record is updated with a furthest distance
// Presume:
last_entry: distance = 10, peer = A
// Registering a new response
new_response: distance = 2, peer = B
-> last entry: distance = 10, peer = B
// Registering a new response
new_response: distance = 7, peer = C
-> last entry: distance = 10, peer = C
// we should've kept B as best
In this case, we should have updated other records and only keep the window of the best 20 peers.
To mitigate this, if we receive a response better than the last entry:
- insert the response with distance and peer to our btreemap, if and only if we didn't track that distance before
- pop the last entry to keep only K entries
Have also added a test for this keep_k_best_results
Signed-off-by: Alexandru Vasile <[email protected]>
last entry Signed-off-by: Alexandru Vasile <[email protected]>
|
||
// The response received from the peer is closer than the furthest response. | ||
if distance < furthest_distance { | ||
// Update the entries only if the distance is not already present. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What is the drawback of overwriting the entry with this distance? I.e., there should be exactly one peer with such distance as it uniquely identifies the hash, so it should be safe to overwrite without checking.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think I followed the libp2p here. However, I believe we can overwrite the entries without checking here, and we'll have a "fresher" report in the end. Will do that, thanks!
Signed-off-by: Alexandru Vasile <[email protected]>
## [0.5.0] - 2023-05-24 This is a small patch release that makes the `FindNode` command a bit more robst: - The `FindNode` command now retains the K (replication factor) best results. - The `FindNode` command has been updated to handle errors and unexpected states without panicking. ### Changed - kad: Refactor FindNode query, keep K best results and add tests ([#114](#114)) ## [0.4.0] - 2023-05-23 This release introduces breaking changes to the litep2p crate, primarily affecting the `kad` module. Key updates include: - The `GetRecord` command now exposes all peer records, not just the latest one. - A new `RecordType` has been introduced to clearly distinguish between locally stored records and those discovered from the network. Significant refactoring has been done to enhance the efficiency and accuracy of the `kad` module. The updates are as follows: - The `GetRecord` command now exposes all peer records. - The `GetRecord` command has been updated to handle errors and unexpected states without panicking. Additionally, we've improved code coverage in the `kad` module by adding more tests. ### Added - Add release checklist ([#115](#115)) - Re-export `multihash` & `multiaddr` types ([#79](#79)) - kad: Expose all peer records of `GET_VALUE` query ([#96](#96)) ### Changed - multistream_select: Remove unneeded changelog.md ([#116](#116)) - kad: Refactor `GetRecord` query and add tests ([#97](#97)) - kad/store: Set memory-store on an incoming record for PutRecordTo ([#88](#88)) - multistream: Dialer deny multiple /multistream/1.0.0 headers ([#61](#61)) - kad: Limit MemoryStore entries ([#78](#78)) - Refactor WebRTC code ([#51](#51)) - Revert "Bring `rustfmt.toml` in sync with polkadot-sdk (#71)" ([#74](#74)) - cargo: Update str0m from 0.4.1 to 0.5.1 ([#95](#95)) ### Fixed - Fix clippy ([#83](#83)) - crypto: Don't panic on unsupported key types ([#84](#84)) --------- Signed-off-by: Alexandru Vasile <[email protected]>
## [0.5.0] - 2023-05-24 This is a small patch release that makes the `FindNode` command a bit more robst: - The `FindNode` command now retains the K (replication factor) best results. - The `FindNode` command has been updated to handle errors and unexpected states without panicking. ### Changed - kad: Refactor FindNode query, keep K best results and add tests ([#114](paritytech/litep2p#114)) --------- Signed-off-by: Alexandru Vasile <[email protected]>
## [0.5.0] - 2023-05-24 This is a small patch release that makes the `FindNode` command a bit more robst: - The `FindNode` command now retains the K (replication factor) best results. - The `FindNode` command has been updated to handle errors and unexpected states without panicking. ### Changed - kad: Refactor FindNode query, keep K best results and add tests ([#114](paritytech/litep2p#114)) --------- Signed-off-by: Alexandru Vasile <[email protected]>
This is a small patch release that makes the `FindNode` command a bit more robst: - The `FindNode` command now retains the K (replication factor) best results. - The `FindNode` command has been updated to handle errors and unexpected states without panicking. - kad: Refactor FindNode query, keep K best results and add tests ([#114](paritytech/litep2p#114)) --------- Signed-off-by: Alexandru Vasile <[email protected]>
This is a small patch release that makes the `FindNode` command a bit more robst: - The `FindNode` command now retains the K (replication factor) best results. - The `FindNode` command has been updated to handle errors and unexpected states without panicking. - kad: Refactor FindNode query, keep K best results and add tests ([#114](paritytech/litep2p#114)) --------- Signed-off-by: Alexandru Vasile <[email protected]>
## [0.5.0] - 2023-05-24 This is a small patch release that makes the `FindNode` command a bit more robst: - The `FindNode` command now retains the K (replication factor) best results. - The `FindNode` command has been updated to handle errors and unexpected states without panicking. ### Changed - kad: Refactor FindNode query, keep K best results and add tests ([paritytech#114](paritytech/litep2p#114)) --------- Signed-off-by: Alexandru Vasile <[email protected]>
## [0.5.0] - 2023-05-24 This is a small patch release that makes the `FindNode` command a bit more robst: - The `FindNode` command now retains the K (replication factor) best results. - The `FindNode` command has been updated to handle errors and unexpected states without panicking. ### Changed - kad: Refactor FindNode query, keep K best results and add tests ([paritytech#114](paritytech/litep2p#114)) --------- Signed-off-by: Alexandru Vasile <[email protected]>
This PR refactors the FindNode query to make it more robust.
It also fixes a bug where only the last peer ID of our results was updated.
This had a few implications:
Testing Done
Closes #100
cc @paritytech/networking