Disable bloom filtering by default. #16152
Conversation
|
Concept ACK |
|
Also needs release notes and a post to the mailing list |
TheBlueMatt
force-pushed
the
2019-06-fix-dos
branch
from
e5d9566 to
e7cf3f7
Compare
|
concept and code-review ACK, only point of discussion is what release to set as milestone here |
TheBlueMatt
force-pushed
the
2019-06-fix-dos
branch
from
e7cf3f7 to
d5449d8
Compare
|
I'd presume next-major. No need to rush it into a backport, let alone to ensure there is always gonna be reasonable coverage of NODE_BLOOM nodes available. |
TheBlueMatt
force-pushed
the
2019-06-fix-dos
branch
from
d5449d8 to
f7b3e18
Compare
BIP 37 bloom filters have been well-known to be a significant DoS target for some time. However, in order to provide continuity for SPV clients relying on it, the NODE_BLOOM service flag was added, and left as a default, to ensure sufficient nodes exist with such a flag. NODE_BLOOM is, at this point, well-established and, as long as there exist 0.18 nodes with default config (which I'd anticipate will be true for many years), will be available from some peers. By that time, the continued slowdown of BIP 37-based filtering will likely have rendered it useless (though this is already largely the case). Further, BIP 37 was deliberately never updated to support witness-based filtering as newer wallets are expected to migrate to some yet-to-be-network-exposed filters.
TheBlueMatt
force-pushed
the
2019-06-fix-dos
branch
from
f7b3e18 to
f27309f
Compare
|
utACK f27309f |
This is not a protocol change, only a default argument change. No need to do a mailing list post. I added a release notes entry. |
|
utACK, without affirmation of future plans mentioned in PR description. |
|
ACK 048e0dc I've read the code and it seems reasonable to me. It's surprising that bloom filters aren't tested by the functional tests. This line could be removed: bitcoin/test/functional/p2p_mempool.py Line 20 in 8a503a6 -peerbloomfilters now defaults to false, but no harm in leaving it.
|
|
Even BIP157 come short for this usecase as it needlessly waste bandwidth. (and I think BIP157, as implemented now does not work in pruned mode -might be wrong here-) |
|
@NicolasDorier That seems like an orthogonal issue. There is currently no code which enables it by default for whitelisted peers, so feel free to open a separate PR that does that. |
|
I don't believe it is completely orthogonal. Changing this default will break existing software which rely on the old default, at least we can limit damage to non-whitelisted peers only. But yeh, anyway Concept ACK, not worth wasting ink on this, would just prefer if it was done here as well. |
|
0.19 is a ways off, no big deal either way :p |
| @@ -44,7 +44,7 @@ | |||
|
|
|||
| NODE_NETWORK = (1 << 0) | |||
| # NODE_GETUTXO = (1 << 1) | |||
| NODE_BLOOM = (1 << 2) | |||
There was a problem hiding this comment.
Do not disable this, I will make a test later for allowing NODE_BLOOM services to whitelisted peers.
There was a problem hiding this comment.
No need to invalidate reviews. Just uncomment it in your PR.
There was a problem hiding this comment.
(also linting tools like vulture may flag an error if this constant is defined but not used)
There was a problem hiding this comment.
That sounds like a bug in the linting tool IMO.
|
Exactly, if you don't comment that line the linters decide that you've added a Clear Bug and must be shot.
… On Jun 9, 2019, at 14:55, John Newbery ***@***.***> wrote:
@jnewbery commented on this pull request.
In test/functional/test_framework/messages.py:
> @@ -44,7 +44,7 @@
NODE_NETWORK = (1 << 0)
# NODE_GETUTXO = (1 << 1)
-NODE_BLOOM = (1 << 2)
(also linting tools like vulture may flag an error if this constant is defined but not used)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.
|
|
Concept ACK |
| @@ -0,0 +1,3 @@ | |||
| Miscellaneous CLI Changes | |||
There was a problem hiding this comment.
This is not a CLI change, but a change of default p2p behavior. (You remove the NODE_BLOOM service bit by default)
There was a problem hiding this comment.
I dont know what the list of categories are, but I think a default parameter change is a CLI change. Happy to change to whatever.
|
Concept ACK |
… after which they might as well switch to a protocol that is efficient for that use such as electrum (which doesn't really have any worse privacy expectations), and everyone, including users of that wallet software will be better off 😊 |
|
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers. ConflictsReviewers, this pull request conflicts with the following ones:
If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first. |
|
Release notes fixed here: https://github.com/jnewbery/bitcoin/tree/pr16152_release_notes @TheBlueMatt - I think if you reset to that branch, this should be mergeable, since the code changes have already been ACKed by @ajtowns , @jnewbery , @MarcoFalke, @luke-jr, @jtimon and @laanwj, and the release notes will be re-reviewed at release time. |
|
As mentioned earlier (#16152 (comment)), this probably requires a very brief post to the mailing list, that explains the change in p2p network behavior and mentions how long wallet developers have until their software breaks. |
|
It won't break. People will continue to use old versions for years into the future. Some might turn NODE_BLOOM back on, too. |
|
Running versions of Bitcoin Core that are EOL is not supported. So if there was a DOS vector against them, it wouldn't be fixed and the nodes could be shut down remotely. After those are shut down, the ones that have it turned on explicitly might also die off because they are flooded with filters at that point. So effectively after 0.18 is EOL, this p2p feature should be considered dead. Implying that it will be available forever is not helpful. I'd rather put a clear date on its expiry to give 3rd party developers a timescale to fix things. Otherwise they will rely on it never dying off and keep shipping software that might break at any point in time. |
I agree. Much better to overcommunicate to the ecosystem through the mailing list than undercommunicate. |
|
Sure, no harm notifying people. My point was simply that 0.19 isn't suddenly a cut off. It will disappear gradually at most. And this only disables it by default - it's still there if someone wants to enable it (eg, for their own usage). It would even make sense to enable it for trusted peers by default, until a better solution exists. So it's not really dead at all yet. |
|
ACK 2518c06 ; just changes the release notes which now look good to me. Unsurprisingly, code still looks good, still compiles and tests pass for me. |
TheBlueMatt
force-pushed
the
2019-06-fix-dos
branch
from
048e0dc to
bead32e
Compare
|
Tweaked the release notes to use the "P2P" header (thanks @jnewbery for answering my question of what the proper header is, happy to credit you in the commit if you want!), and rewrote the text to indicate that we don't anticipate it being a fast process (if anything, I'm happy to run a few of these, they aren't that bad if you have a super beefy machine). Happy to send a mail to the ML with the release notes text if this gets merged. |
|
ACK |
bead32e Add release notes for DEFAULT_BLOOM change (Matt Corallo) f27309f Move DEFAULT_PEERBLOOMFILTERS from validation.h to net_processing.h (Matt Corallo) 5efcb77 Disable bloom filtering by default. (Matt Corallo) Pull request description: BIP 37 bloom filters have been well-known to be a significant DoS target for some time. However, in order to provide continuity for SPV clients relying on it, the NODE_BLOOM service flag was added, and left as a default, to ensure sufficient nodes exist with such a flag. NODE_BLOOM is, at this point, well-established and, as long as there exist 0.18 nodes with default config (which I'd anticipate will be true for many years), will be available from some peers. By that time, the continued slowdown of BIP 37-based filtering will likely have rendered it useless (though this is already largely the case). Further, BIP 37 was deliberately never updated to support witness-based filtering as newer wallets are expected to migrate to some yet-to-be-network-exposed filters. ACKs for top commit: jnewbery: ACK bead32e kallewoof: ACK bead32e Tree-SHA512: ecd901898e8efe1a7c82b471af0acc2373c2282ac633eb58d9aae7c35deda1999d0f79fb0485e6cecbda7246aeda00206cd82c7fa36866e2ac64705ba93f9390
Yes, please |
| This resolves well-known DoS vectors in Bitcoin Core, especially for nodes with spinning disks. It is not anticipated that | ||
| this will result in a significant lack of availability of NODE_BLOOM-enabled nodes in the coming years, however, clients | ||
| which rely on the availability of NODE_BLOOM-supporting nodes on the P2P network should consider the process of migrating | ||
| to a more modern (and less trustful and privacy-violating) alternative over the coming years. |
|
ACK |
|
I fully understand the purpose of having bloom filters disabled by default. |
|
Bloom filters are the way that lightweight wallets operate without centralized servers. There is no evidence of a significant DoS issue. This is a clear effort to sabotage Bitcoin. |
|
What's the matter? Liquid network is not the smash hit Blockstream hoped it would be and you need to do this to cripple SPV wallets? Or what? |
BIP 37 bloom filters have been well-known to be a significant DoS
target for some time. However, in order to provide continuity for
SPV clients relying on it, the NODE_BLOOM service flag was added,
and left as a default, to ensure sufficient nodes exist with such a
flag.
NODE_BLOOM is, at this point, well-established and, as long as
there exist 0.18 nodes with default config (which I'd anticipate
will be true for many years), will be available from some peers. By
that time, the continued slowdown of BIP 37-based filtering will
likely have rendered it useless (though this is already largely the
case). Further, BIP 37 was deliberately never updated to support
witness-based filtering as newer wallets are expected to migrate to
some yet-to-be-network-exposed filters.