DHT doesn't work under proxy #4078

proninyaroslav · 2019-11-10T10:42:47Z

libtorrent version (or branch): libtorrent4j-1.2.1.0 (libtorrent 1.2.1)

platform/architecture: Android

Hello. In my torrent client for Android I use libtorrent. The only problem I encountered: DHT does not work if I setup a proxy. I checked this in similar apps that use libtorrent - DHT under proxy works fine. Here is the full session and DHT log right after launch: session.log
Basic moments:

...
I: [SESSION_LOG]not starting DHT announce timer: m_dht == nullptr
...
I: [SESSION_LOG]UDP error: 0.0.0.0:0 (125) Operation Canceled
I: [SESSION_LOG]about to stop DHT, running: false
I: [SESSION_LOG]starting DHT, running: false, router lookups: 0
I: [DHT_LOG][TRACKER]starting IPv4 DHT tracker with node id: 8cd612cddfc0423fb4993f2892c473d0f914b95d
...
I: [DHT_LOG][TRAVERSAL][0] bootstrap done, pinging remaining nodes
I: [DHT_LOG][TRAVERSAL][0] bootstrap DONE
I: [DHT_LOG][TRAVERSAL][0] not alive: 67.215.246.10:6881
I: [DHT_LOG][TRAVERSAL][0] not alive: 87.98.162.88:6881
I: [DHT_LOG][TRAVERSAL][0] not alive: 192.3.31.116:25401
I: [DHT_LOG][TRAVERSAL][0] not alive: 212.129.33.59:6881
I: [DHT_LOG][TRAVERSAL][0] COMPLETED distance: 160 type: bootstrap
...

I also used the option proxy_peer_connections = true, but this didn't work.

The text was updated successfully, but these errors were encountered:

arvidn · 2019-11-13T15:34:27Z

yes, the log shows that the DHT packets aren't going through. what kind of proxy are you using? does it support UDP?

proninyaroslav · 2019-11-13T15:46:10Z

I think yes, because, for example, in qBittorrent or tTorrent DHT and UDP trackers works when using this proxy.

arvidn · 2019-11-13T17:32:35Z

qbittorrent uses libtorrent, and there are options to bypass the proxy for UDP, if the proxy doesn't support it. What kind of proxy are you using? i.e. what are you setting the type field to in the proxy_settings?

proninyaroslav · 2019-11-13T17:35:47Z

http_pw (if the proxy with password), http and socks5. None of them work with UDP.

arvidn · 2019-11-13T19:06:26Z

HTTP proxies can only proxy TCP streams. the DHT runs over UDP, so it doesn't work over an HTTP proxy

proninyaroslav · 2019-11-13T19:14:24Z

So, qbittorrent just doesn't connect DHT over HTTP proxy? But how?
Also I tried Tor SOCKS proxy (localhost:9050), but UDP still didn't work.

arvidn · 2019-11-13T19:21:14Z

I don't know what the default qbt settings are.
I would not be surprised if the Tor SOCKS5 proxy does not support UDP. Do you have reasons to believe it does?
you may want to try a real socks5 proxy, like Dante for instance

proninyaroslav · 2019-11-13T19:23:21Z

Thank you, I will check SOCKS and write about the results.

proninyaroslav · 2019-11-14T11:26:52Z

I configured Dante on my server and allowed it UDP:

pass {
        from: 0.0.0.0/0 to: 0.0.0.0/0
        protocol: tcp udp
}

I tested the same tracker in qbittorrent (PC network), my application (mobile network) and ttorrent (mobile network) - udp://tracker.leechers-paradise.org:6969
In qbittorrent and ttorrent, this works fine, DHT and UDP trackers connect seamlessly. But in my application they don't work.
Logs:
start_session.log
add_torrent.log

Main moments from start_session.log:

...
I: [SESSION_LOG]UDP error: 0.0.0.0:0 (125) Operation Canceled
I: [SESSION_LOG]UDP error: 0.0.0.0:0 (125) Operation Canceled
I: [SESSION_LOG]applying settings pack, reopen_listen_port=false
I: [SESSION_LOG]update listen interfaces: 0.0.0.0:44333,[::]:44333
I: [SESSION_LOG]about to stop DHT, running: false
I: [SESSION_LOG]starting DHT, running: false, router lookups: 0
I: [DHT_LOG][TRACKER]starting IPv4 DHT tracker with node id: 71f3733277349838c0162b471ca2ea69feeaf499
I: [DHT_LOG][NODE]adding router node: 192.3.31.116:25401
I: [DHT_LOG][NODE]adding router node: 67.215.246.10:6881
I: [DHT_LOG][NODE]adding router node: 87.98.162.88:6881
I: [DHT_LOG][NODE]adding router node: 212.129.33.59:6881
I: [DHT_LOG][NODE]adding router node: 192.3.31.116:25401
I: [DHT_LOG][NODE]adding router node: 67.215.246.10:6881
I: [DHT_LOG][NODE]adding router node: 87.98.162.88:6881
I: [DHT_LOG][NODE]adding router node: 212.129.33.59:6881
...
I: [DHT_LOG][RPC_MANAGER][0] invoking get_peers -> 212.129.33.59:6881
I: [DHT_LOG][TRAVERSAL][0] bootstrap done, pinging remaining nodes
I: [DHT_LOG][TRAVERSAL][0] bootstrap DONE
I: [DHT_LOG][TRAVERSAL][0] not alive: 67.215.246.10:6881
I: [DHT_LOG][TRAVERSAL][0] not alive: 87.98.162.88:6881
I: [DHT_LOG][TRAVERSAL][0] not alive: 192.3.31.116:25401
I: [DHT_LOG][TRAVERSAL][0] not alive: 212.129.33.59:6881
I: [DHT_LOG][TRAVERSAL][0] COMPLETED distance: 160 type: bootstrap

And add_torrent.log (DHT/tracker time out and network is unreachable):

...
I: [TORRENT_LOG]==> TRACKER REQUEST "udp://tracker.leechers-paradise.org:6969" event: started abort: 0 ssl: 0x0 port: 44333 ssl-port: 0 fails: 0 upd: 0
I: [TORRENT_LOG]*** QUEUE_TRACKER_REQUEST [ listen_port: 44333 ]
I: [TORRENT_LOG]==> UDP_TRACKER_CONNECT [ failed: Network is unreachable ]
...
I: [TORRENT_LOG]*** tracker: "udp://tracker.leechers-paradise.org:6969" [ tiers: 0 trackers: 0 found: 0 i->tier: 0 tier: 2147483647 working: 1 fails: 0 limit: 0 upd: 1 ]
I: [TORRENT_LOG]*** update tracker timer: next_announce < now 0 m_waiting_tracker: 0 next_announce_in: -2147475308
I: [SESSION_LOG]==> LSD: ih: 26e8d03aff486d164e262d0a09cadbe8fc4f71a9 port: 44333
I: [SESSION_LOG]aborting DHT announce timer (125): Operation Canceled
I: [TORRENT_LOG]*** tracker error: (101) Network is unreachable
I: [TORRENT_LOG]*** increment tracker fail count [1]
...
I: [DHT_LOG][TRAVERSAL][2] 1ST_TIMEOUT id: bc92a93c5ae4df75037b7b1f7fa750f530198282 distance: 159 addr: 192.3.31.116 branch-factor: 6 invoke-count: 4 type: get_peers
I: [DHT_LOG][TRAVERSAL][2] 1ST_TIMEOUT id: 34ce2d4fad89000b149f7e50234694d4d1e4b277 distance: 156 addr: 212.129.33.59 branch-factor: 7 invoke-count: 4 type: get_peers
I: [DHT_LOG][TRAVERSAL][2] 1ST_TIMEOUT id: 2c408072a7cc858a5cfa78ad8cb11a7122b075a3 distance: 155 addr: 67.215.246.10 branch-factor: 8 invoke-count: 4 type: get_peers
I: [DHT_LOG][TRAVERSAL][2] 1ST_TIMEOUT id: 2691a20ec371c6ecc128ff9b2a2a995efedbaf06 distance: 150 addr: 87.98.162.88 branch-factor: 9 invoke-count: 4 type: get_peers
I: [SESSION_LOG]==> LSD: ih: 26e8d03aff486d164e262d0a09cadbe8fc4f71a9 port: 44333
I: [SESSION_LOG]<== LSD: ignoring packet (cookie matched our own): 2356c308
I: [SESSION_LOG]<== LSD: ignoring packet (cookie matched our own): 2356c308
I: [TORRENT_LOG]*** UDP_TRACKER [ timed out url: udp://tracker.leechers-paradise.org:6969 ]
I: [TORRENT_LOG]*** tracker error: (36) timed out
I: [TORRENT_LOG]*** increment tracker fail count [1]
I: [TORRENT_LOG]*** tracker: "udp://tracker.leechers-paradise.org:6969" [ tiers: 0 trackers: 0 i->tier: 0 tier: 2147483647 working: 0 limit: 0 can: 0 sent: 0 ]
...

proninyaroslav · 2019-12-01T08:09:14Z

Could this be due to the fact that I didn't enabled an option that is responsible for this? I studied the documentation but didn’t find anything like it (except proxy_peer_connections).

arvidn · 2019-12-02T11:53:23Z

The proper way to debug this is to get a wireshark capture of your communication with the SOCKS5 proxy. Both the TCP and the UDP stream.

I would be happy to look at a capture to see what's wrong.

There is a unit test (that's passing) exercising the UDP over SOCKS5, here. As a sanity check, you could run this test to make sure it passes for you too.

proninyaroslav · 2019-12-02T18:16:16Z

I made wireshark dump. Before dumping, I discovered another moment:
Connection to the proxy via Wi-Fi (in the same subnet as the proxy server): DHT works
Connection to the proxy via Mobile LTE: DHT doesn't work

So, the initial parameters before wireshark dumping:

Client
- Device: Android phone
- Connection type: Mobile LTE
- Global IP: 176.59.39.10
Proxy server
- Proxy software: Dante
- Allowed protocols: TCP, UDP
- Connection type: Ethernet (via Wi-Fi router)
- Local IP: 192.168.1.101
- Global IP: 37.235.201.30
- Proxy port: 1080 (opened)
Proxy params (libtorrent)
- Type: SOCKS5 with authorization (socks5_pw)
- Hostname: proninyaroslav.ru
- IP: 37.235.201.30
- Port: 1080
- proxy_peer_connections == true
- proxy_tracker_connections == true
- proxy_hostnames == true
Test torrent: 26e8d03aff486d164e262d0a09cadbe8fc4f71a9.zip
Test tracker: udp://tracker.leechers-paradise.org:6969

Dump 1:

I made wireshark dump only on a proxy server machine:

session_log_1.txt
server_dump_1.zip

Dump 2:

I made wireshark dump both on the proxy server and on the client (Android). Android allows packets sniffing only through VpnService, but it doesn't support UDP, so the results are most likely incorrect:

session_log_2.txt
client_and_server_dump.zip

proninyaroslav · 2019-12-03T07:53:49Z

Dump 3:

I found a workaround: I launched the Android emulator on my PC and turned on Wireshark. As a network, I used a USB LTE modem. Now I was able to sniff UDP on the client side.

Client
- Device: Android emulator on PC
- Network type: LTE modem
- Local IP: 192.168.42.200
- Global IP: 176.59.35.132

session_log_3.txt
android_emulator_client_dump_3.zip

arvidn · 2019-12-04T08:28:41Z

is 37.235.201.30 the SOCKS5 proxy?

I see multiple successful UDP ASSOCIATE commands sent. However, each one responds with a different bind address.

The first one is:
c0 a8 01 65 9a 07 = 192.168.1.101:39431

followed by:
c0 a8 01 65 91 7e = 192.168.1.101:37246
c0 a8 01 65 e2 8a = 192.168.1.101:57994
c0 a8 01 65 ec dd = 192.168.1.101:60637
etc.

I see all UDP traffic going to 192.168.1.101:39431, with the appropriate SOCKS5 UDP header. Which seems correct. Perhaps the proxy only allows one UDP associate per source IP, but that would be weird as you could have multiple processes creating pinholes independently.

I'm not entirely sure why libtorrent creates so many UDP associate calls, perhaps it does one per local listen interface. You could try to bind the local listen socket to a single IP/interface, then there will probably only be one socket created.

So, this could be improved in libtorrent by moving the UDP SOCKS5 logic to a higher level, that replaces all listen sockets. Or perhaps more accurately, allow annotating which sockets are running over SOCKS5 in the listen_interfaces settings.

Either way, as far as I can see, all the interaction with the proxy is correct from libtorrent's point of view.

proninyaroslav · 2019-12-04T18:29:39Z

is 37.235.201.30 the SOCKS5 proxy?

Yes

Or perhaps more accurately, allow annotating which sockets are running over SOCKS5 in the listen_interfaces settings.

I use 0.0.0.0:port,[::]:port where port is randomly selected in the range [37000, 57000] and max_retry_port_bind to select the nearest port if it's busy. I tried to remove this and return the libtorrent settings to default (as far as I know this is 0.0.0.0:6881,[::]:6881), but this didn't help.

arvidn · 2019-12-04T19:06:16Z

what I meant is that if you listen on two sockets, there will be two sockets that are created and mapped on the SOCKS5 proxy. If the proxy only supports one at a time, that might be problematic.
That's why I suggested listening to just one socket. For example, you could set listen_interfaces to just your specific IP address + port.

proninyaroslav · 2019-12-05T10:47:42Z

That is, use local_netcard_ip:random_port? I tried this, but it also doesn't work.

arvidn · 2019-12-05T11:02:18Z

how many listen_succeeded_alert do you get?
do you still get multiple SOCKS5 UDP ASSOCIATE commands sent to the proxy?

proninyaroslav · 2019-12-05T15:36:01Z

listen_interfaces = 192.168.200.2:55972

how many listen_succeeded_alert do you get?

Once: ListenSucceededAlert: address=192.168.200.2, port=55972. 192.168.200.2 is the emulator inet.

do you still get multiple SOCKS5 UDP ASSOCIATE commands sent to the proxy?

Seems to be yes. android_emulator_client_dump_4.zip .

PC inet: 192.168.42.200
Emulator inet (running on PC): 192.168.200.2
Server inet: 192.168.1.101

proninyaroslav · 2019-12-05T15:53:20Z

For comparison, I connected the emulator to the same network as the proxy server and set listen_interfaces = 0.0.0.0:50973,[::]:50973 (my default configuration). DHT immediately started working after adding torrent and the download started (but UDP tracker still didn't work). I don't understand why this doesn't work through LTE network or another non-local network.
android_emulator_client_dump_5.zip

ListenSucceededAlert: address=0.0.0.0, port=50973
ListenSucceededAlert: address=fec0::a1e3:a78b:5b1b:4688, port=50973
ListenSucceededAlert: address=fec0::a1e3:a78b:5b1b:4688, port=50973
ListenSucceededAlert: address=fec0::e07f:7fff:fe10:882e, port=50973
ListenSucceededAlert: address=fec0::e07f:7fff:fe10:882e, port=50973
ListenSucceededAlert: address=fe80::e07f:7fff:fe10:882e%radio0, port=50973
ListenSucceededAlert: address=fe80::e07f:7fff:fe10:882e%radio0, port=50973

stale · 2020-03-04T16:06:02Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

proninyaroslav · 2020-03-04T16:07:33Z

@arvidn Hi. Please see the results that I managed to get earlier.

arvidn · 2020-03-04T16:46:22Z

I believe current RC_1_2 will fix this. At least the issue of multiple SOCKS messages, that's obvious from the dump

stale · 2020-06-02T17:27:42Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

master255 · 2021-03-11T00:49:04Z

@arvidn DHT does not work on any type of proxy.

glassez · 2024-07-22T12:33:10Z

qbittorrent uses libtorrent, and there are options to bypass the proxy for UDP, if the proxy doesn't support it

@arvidn
What kind of option did you mean? I couldn't find one.

arvidn · 2024-07-23T00:04:55Z

qbittorrent uses libtorrent, and there are options to bypass the proxy for UDP, if the proxy doesn't support it

@arvidn What kind of option did you mean? I couldn't find one.

I must have mis-typed. Confusing the settings to control whether peers and trackers use the proxy.

glassez · 2024-07-23T03:25:39Z

qbittorrent uses libtorrent, and there are options to bypass the proxy for UDP, if the proxy doesn't support it

@arvidn What kind of option did you mean? I couldn't find one.

I must have mis-typed. Confusing the settings to control whether peers and trackers use the proxy.

Wouldn't it make sense to add one? Or maybe even separately for DHT...

stalkerok · 2024-08-10T10:28:11Z

@arvidn Please pay attention to this. There is no point in using a proxy for DHT, because DHT stops working. You need to ask the user if they want to proxy DHT traffic.

arvidn · 2024-08-11T06:22:31Z

please stop declaring that things "don't work". please provide a wireshark dump demonstrating it not working. (Or some other useful record of it)

not proxying DHT traffic would make other peers start contacting you directly, not via the proxy. That doesn't seem desirable.

stalkerok · 2024-08-11T08:16:03Z

@arvidn qbittorrent/qBittorrent#14868 (comment) qbittorrent/qBittorrent#14868 (comment)

proninyaroslav mentioned this issue Nov 11, 2019

Trackers error "Method Not Allowed" if HTTP proxy enabled #4079

Closed

stale bot added the stale label Mar 4, 2020

stale bot removed the stale label Mar 4, 2020

stale bot added the stale label Jun 2, 2020

stale bot closed this as completed Jun 22, 2020

master255 mentioned this issue Mar 11, 2021

Bug: DHT doesn't work under any proxy #6053

Closed

thalieht mentioned this issue Apr 28, 2021

DHT doesn't work when HTTP proxy is enabled qbittorrent/qBittorrent#14868

Open

This comment was marked as off-topic.

Sign in to view

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DHT doesn't work under proxy #4078

DHT doesn't work under proxy #4078

proninyaroslav commented Nov 10, 2019

arvidn commented Nov 13, 2019

proninyaroslav commented Nov 13, 2019

arvidn commented Nov 13, 2019

proninyaroslav commented Nov 13, 2019

arvidn commented Nov 13, 2019

proninyaroslav commented Nov 13, 2019 •

edited

Loading

arvidn commented Nov 13, 2019

proninyaroslav commented Nov 13, 2019

proninyaroslav commented Nov 14, 2019 •

edited

Loading

proninyaroslav commented Dec 1, 2019

arvidn commented Dec 2, 2019

proninyaroslav commented Dec 2, 2019 •

edited

Loading

proninyaroslav commented Dec 3, 2019 •

edited

Loading

arvidn commented Dec 4, 2019

proninyaroslav commented Dec 4, 2019 •

edited

Loading

arvidn commented Dec 4, 2019

proninyaroslav commented Dec 5, 2019

arvidn commented Dec 5, 2019

proninyaroslav commented Dec 5, 2019 •

edited

Loading

proninyaroslav commented Dec 5, 2019 •

edited

Loading

stale bot commented Mar 4, 2020

proninyaroslav commented Mar 4, 2020

arvidn commented Mar 4, 2020

stale bot commented Jun 2, 2020

master255 commented Mar 11, 2021

glassez commented Jul 22, 2024

arvidn commented Jul 23, 2024

glassez commented Jul 23, 2024

stalkerok commented Aug 10, 2024

This comment was marked as off-topic.

arvidn commented Aug 11, 2024

stalkerok commented Aug 11, 2024

DHT doesn't work under proxy #4078

DHT doesn't work under proxy #4078

Comments

proninyaroslav commented Nov 10, 2019

arvidn commented Nov 13, 2019

proninyaroslav commented Nov 13, 2019

arvidn commented Nov 13, 2019

proninyaroslav commented Nov 13, 2019

arvidn commented Nov 13, 2019

proninyaroslav commented Nov 13, 2019 • edited Loading

arvidn commented Nov 13, 2019

proninyaroslav commented Nov 13, 2019

proninyaroslav commented Nov 14, 2019 • edited Loading

proninyaroslav commented Dec 1, 2019

arvidn commented Dec 2, 2019

proninyaroslav commented Dec 2, 2019 • edited Loading

proninyaroslav commented Dec 3, 2019 • edited Loading

arvidn commented Dec 4, 2019

proninyaroslav commented Dec 4, 2019 • edited Loading

arvidn commented Dec 4, 2019

proninyaroslav commented Dec 5, 2019

arvidn commented Dec 5, 2019

proninyaroslav commented Dec 5, 2019 • edited Loading

proninyaroslav commented Dec 5, 2019 • edited Loading

stale bot commented Mar 4, 2020

proninyaroslav commented Mar 4, 2020

arvidn commented Mar 4, 2020

stale bot commented Jun 2, 2020

master255 commented Mar 11, 2021

glassez commented Jul 22, 2024

arvidn commented Jul 23, 2024

glassez commented Jul 23, 2024

stalkerok commented Aug 10, 2024

This comment was marked as off-topic.

arvidn commented Aug 11, 2024

stalkerok commented Aug 11, 2024

proninyaroslav commented Nov 13, 2019 •

edited

Loading

proninyaroslav commented Nov 14, 2019 •

edited

Loading

proninyaroslav commented Dec 2, 2019 •

edited

Loading

proninyaroslav commented Dec 3, 2019 •

edited

Loading

proninyaroslav commented Dec 4, 2019 •

edited

Loading

proninyaroslav commented Dec 5, 2019 •

edited

Loading

proninyaroslav commented Dec 5, 2019 •

edited

Loading