multi: Use an APBF for per peer known addrs. #2583
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dnldd
reviewed
Feb 1, 2021
davecgh
force-pushed
the
multi_known_addrs_abpf
branch
from
4bd9924
to
d7586d7
Compare
matheusd
approved these changes
Feb 2, 2021
rstaudt2
approved these changes
Feb 3, 2021
dnldd
approved these changes
Feb 3, 2021
davecgh
force-pushed
the
multi_known_addrs_abpf
branch
from
d7586d7
to
e6cf306
Compare
JoeGruffins
approved these changes
Feb 8, 2021
This modifies the server to track the per peer known addresses using a much more efficient APBF instead of an LRU cache which has significant overhead in addition to having to store the entirety of all items added to it. False positives are acceptable as the goal is to avoid sending duplicate addresses which is an ideal case for APBFs. More concretely, the current LRU cache that stores the known addresses can grow to around 1.69 MiB *per peer* when full while the new APBF only takes around 40 KiB, a reduction of around 97.7%, while exhibiting roughly the same computational performance. Since there is a maximum of 125 peers by default, that means the current LRU cache can potentially grow to around ~211 MiB if fully populated for max peers. The new APBF, on the other hand, will only consume ~4.88 MiB under the same scenario.
davecgh
force-pushed
the
multi_known_addrs_abpf
branch
from
e6cf306
to
1512058
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This requires #2579 and is rebased on #2580.
This modifies the server to track the per peer known addresses using a much more efficient APBF instead of an LRU cache which has significant overhead in addition to having to store the entirety of all items added to it.
False positives are acceptable as the goal is to avoid sending duplicate addresses which is an ideal case for APBFs.
More concretely, the current LRU cache that stores the known addresses can grow to around 1.69 MiB per peer when full while the new APBF only takes around 40 KiB, a reduction of around 97.7%, while exhibiting roughly the same computational performance.
Since there is a maximum of 125 peers by default, that means the current LRU cache can potentially grow to around ~211 MiB if fully populated for max peers. The new APBF, on the other hand, will only consume ~4.88 MiB under the same scenario.