p2p: group peers by AS (bitcoin's asmap)

[Malinero]

Relevant links (and credits):

• https://blog.bitmex.com/call-to-action-testing-and-improving-asmap/
• [RFC] bitcoin-asmap utility bitcoin/bitcoin#18573

[maogo]

Nice, another link #7090

[vtnerd]

When I looked into this, my issue was that we could not verify/recreate the provided database ...

[Malinero]

Thanks @vtnerd

Just a couple question before closing this PR:

we could not verify/recreate the provided database ...

not even someone with a public AS number ?

Does GeoIP databases suffers from the same problem ?

[selsta]

I wouldn't close it yet.

[Malinero]

This PR doesn't deal with the generation of the mapping file and is not specific to Autonomous Systems, it "only" requires to associate CIDRs to an uint32. There are multiple ways to do it, but none perfect (due to how BGP/internet works). For example, this little python script shows 2 ways:

• GeoIP database (maxmind in this case, used by I2P)
• cloud providers self-declared public IP addresses (IMO good enough against script kiddies)

Unfortunately, Sipa's encoding, while quite efficient, doesn't seem to support partial mappings. For example with this mapping file, the lookup of 4.0.0.1 returns AS9:

1.0.0.0/8 AS1
2.0.0.0/8 AS2
3.0.0.0/8 AS2
5.0.0.0/8 AS9

[Malinero]

As stated here, Sipa published new python scripts to encode the mapping file. The new version allows for partial mappings.

Here's a sample usage to consider AWS as a single group (it falls back to /16 grouping for other IP addresses) :

git clone https://github.com/sipa/asmap -b nextgen && cd asmap
curl https://ip-ranges.amazonaws.com/ip-ranges.json \
    | jq -r ".prefixes| .[].ip_prefix" \
    | sed "s/\$/ AS1/" \
    | python3 asmap-tool.py encode > mapping.bin
monerod --asmap /apath/to/mapping.bin ...

[plowsof]

For visibility, comment reg licensing of this PR from ypdvim (to be confirmed/food for thought)

The core code of asmap PR is copied from bitcoin, I wonder if the PR is worth review/merge by reason for license or other something?

[0xFFFC0000]

Thanks for your PR. I understand this part of the code comes from asmap file in Bitcoin source. But they have documentation for these in their .py file.

We don't have any other file related to asmap. Can we have a comment explaining each Instruction in asmap format? I saw this and it looks very beautiful to me.

[maogo]

give more love to asmap !