Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Peer connections eclipse attack #829

Closed
chainum opened this issue Dec 17, 2019 · 2 comments
Closed

Peer connections eclipse attack #829

chainum opened this issue Dec 17, 2019 · 2 comments
Assignees
Labels
type:bug Something isn't working

Comments

@chainum
Copy link

chainum commented Dec 17, 2019

Describe the bug
It's possible to eclipse attack nodes because of them having a fixed set of 300 maximum peer connections. It also seems that the node isn't validating that the connected peers are actually real nodes and not just simple peers (i.e. bogus/fake seed nodes).

To Reproduce
I wrote a seed node DDOS:er as well as a eclipse tool that spawns hundreds/thousands of seednodes.

This results in there being a lot of bogus peer connections for nodes to connect to - and these peer connections don't do anything, they just take up valuable space in the limited 300 peer connection pool.

In order to start off the eclipse attack nodes had to be forcefully restarted in order for them to get back to peer discovery mode. I used my previous spam attack tool (#799) to initiate this phase (phase 1). After the nodes were OOM-reaped/restarted they started connecting to the fake seed peer servers I previously had set up (phase 2).

The end result was that plenty of nodes ended up having a running process - but were essentially stuck at 0 peer connections because all peer connections were made to the fake seed peer nodes. AFAIK, there definitely was quite a few people in the Elrond Validators chat that also reported being stuck on 0 peers.

As a result of this all shards (except meta) eventually stalled / wasn't producing blocks (at least according to the explorer) while the attack was operational.

Steps to reproduce the behavior:

  1. Clone the repo elrond-peer-attacker
  2. cd cmd/seednodeddos && go build && ./seednodeddos --count 250 to start 250 workers DDoSing the original seed node provided by Elrond.
  3. cd cmd/eclipse && go build && ./eclipse --count 500 to start 500 seed nodes that will interfere with effective peer discovery.
  4. Repeat accordingly for every server you want to turn into a fake seed peer host (for my final attack I used about 10 different VPS:es running between 500 and 1000 separate seed nodes per server).
  5. Launch the tx spammer (see https://github.com/SebastianJ/elrond-tx-sender/ for details)

Expected behavior
There should be some kind of filtering to make sure that nodes only connect to valid peers and that fake/bogus peers are disregarded.

Screencast
I recorded when one of my own nodes got eclipsed by fake peers and got stuck on 0 peers until I stopped all attack tools and restarted the node.

First 8 minutes is just the node running with 0 peers to prove that it couldn't connect to anything for a long time.

After that I shut down the attack tools and restart the node - and the node then proceeds to find real peers again.

@iulianpascalau
Copy link
Contributor

Nice attack. Will provide a fix for this, not allowing a peer to connect to more than a fixed low value number of peers originating from the same ip address.

@iulianpascalau iulianpascalau self-assigned this Dec 23, 2019
@iulianpascalau iulianpascalau added type:bug Something isn't working P0 labels Dec 23, 2019
@iulianpascalau
Copy link
Contributor

Done, the network sharding subsystem will prevent this. The nodes must be all validators to try to eclipse a new node and have a more favorable hash. Extremely improbable that this can occur in the wild. Reference PR #1201

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type:bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants