Fix triggering loss for uTP packets that were never sent #7107
Conversation
|
This could possibly be related to #3542 Looking at the log attached to this comment, the cwnd is cut 21 times due to loss, contributing to the sawtooth shape in the graph. Of these 21 cuts, 20 were due to loss of packets that appear to never have been sent in the first place due to 'stalling'. Here they are with the loss event and the following cwnd halving tied to the preceding 'stall' where the packet wasn't sent. Packet |
|
I believe your understanding of |
|
I've added this to |
This fix may partially help heavy-load high-contention usage. But my test scenarios are almost the exact opposite of that...with 1 seed (uT) to 1 peer (qBT) via uTP transfer. I've tested with and without speed limits on both the seed and peer, from 10 KB/sec to 1000 KB/sec. Almost all of my testing is relying on uTorrent's graphs...specifically uTP "retransmission upload" under "Network Overhead". qBitTorrent may be corrupting uTorrent's data packets and/or failing to reply back (missing ACKs) in a timely manner. 5-25% retransmission rate seems a high price to pay for uTorrent seeds uploading to qBitTorrent peers using uTP ...when uTorrent uTP uploading to non-libtorrent peers shows effectively no retransmissions. |
What happens when you send from libtorrent to libtorrent using uTP? Is it better than uTP in uTorrent? |
It's...complicated. Test conditions typically consisted of local loopback (127.0.0.1) and running multiple BT clients at once on a single computer, with the torrents' contents stored on a ramdrive, using a single seed to single peer at a time. Other people's test results: |
That should be false. On optical links and links over oceans uTP is always faster, just as QUIC aka http3, compressed with brotli. Where is BTW brotli or others for uTP? All tests should be redone AFTER many recent fixes into libtorrent, uTP patches should be merged into Wireshark. Also, I do not understand what stops you from using uTorrent utp lib code directly? It is open source. Finally, there is at least one place where utorrent is wrong: #7071, that should be fixed upstream. |
|
Someone else can do the tests, I've spent years on it already. |
Did you ever test much faster Tixati? And anyway, these pulls should be tested. Also there is a key allegation from yourself:
Was that fixed!? Why even that is not obvious. This delays the connection for 15 seconds, which makes all other tests useless.
Which is BTW rediculous and MUST be a bug in uTorrent. Also the main issue is closed by a stale bot, so not really closed is very bad. |
I have tested Tixati (v3.19 even) and I find it cripplingly slow with very high CPU usage depending on what window section I have open.
Please provide a link to where you saw that quote as I don't recall all the context and situation. Situations change, and in my later tests I see something that may look like the opposite: |
|
References: Okay, so Tixati is useless. |
In my recent tests... [2023-07-01 04:41:29] Incoming connection from 127.0.0.1:65505 And...then I realized qBT was still set in advanced options to allow multiple connections to the same ip. Once that was disabled, this still happened: [2023-07-01 04:43:36] Incoming connection from 127.0.0.1:65505 So...I closed and restarted qBT and then restarted both torrents: [2023-07-01 04:50:32] Incoming connection from 127.0.0.1:65505 ...Which worked correctly, except for the fact that the combined speeds was only 1.2-1.5 MB/sec and resends (packet loss) were from 800 to just over 1000 KB/sec! While I was doing connection/handshake testing again, I spotted something a little like that: qBitTorrent would initially make 3 uTP connections (1 second interval between each), then wait a minute and retry uTP once, repeating that every minute.
I have used Tixati in particular in testing because its core code as far as I know isn't based on uTorrent/BitTorrent or qBitTorrent/libtorrent code. I need to try Transmission's uTP performance as well, but there doesn't seem to be a recent official Windows version for it. |
https://build.transmissionbt.com/job/trunk-linux/ BTW, why not use directly uTorrent open source library? I do not get it. |
|
Because I am not much of a programmer and in poor health. This isn't my project -- I was a uTorrent beta-tester for a long time, so that's where I draw my experience from. Even Arvidn seems stumped by this problem, so my lack of precise network training (I don't know what to expect) means I really can't help much more. |
|
I gave you a link for windows nightly builds for Transmission. Here is even more specific https://build.transmissionbt.com/job/trunk-win32/ |
|
I tried to test uTP on Transmission...and I didn't get Transmission to use uTP at all! I didn't get Transmission to use uTP in the past either: Transmission uses a fork of BitTorrent's uTP library, which hasn't been updated in 5 or 6 years.
Theory is irrelevant when other problems block putting that theory into practice. If there is a perceived advantage from not following the rules, some BT clients will do so.
And that's why everything that interfaces with buggy software has to work around their bugs...even if the workarounds aren't great. The whole point of using dissimilar BT client software is to determine where the underlying problems are -- in a particular BT client, in the OS used, or elsewhere. |
Did you enable uTP? It is an option in Transmission. |
|
Yes... plus it's enabled by default. It's found under Transmission Preferences, Network, Options, 1st Enable checkbox there. If you happen to have Transmission installed you can test its uTP as well with qBitTorrent. I wouldn't mind knowing if the problem is specific to Transmission on Windows or not. |
Transmission uses libutp from uTorrent. So it is a bug then. Can you open a bug in Transmission? |
|
You need to verify the bug before I'll bother. Transmission seems to get incoming TCP connections on its listening port just fine...but possibly a uTP bug in how qBitTorrent/libtorrent contacts it? |
libtorrent/include/libtorrent/aux_/utp_stream.hpp
Lines 964 to 969 in 8786d17
A uTP packet that fails to send due to a 'stall' isn't marked as
need_resend. The send is only retried once the packet has timed out or is picked up by a SACK, causing it to trigger loss and thecwndto be halved despite never having been sent yet. This also means packets with a higherseq_nrmay be sent out first if the socket becomes writable before the unsent packet has timed out/been picked up by SACK.From this utp log note that 6542-6544 stall while 6545-6594 were sent as normal as the socket became writable again. 6542-6544 were only sent once they are noticed as missing from a SACK, and are then treated as lost packets:
This suggested change marks such packets as
need_resendwhen the stall happens, meaning they will be sent the next time send_pkt() is called once the socket is writable again. This should prevent packets that weren't sent from being 'lost' and needlessly halving thecwnd, and also sending packets out of order.I've marked this as a draft for the time being as I'm not yet sure that I'm not missing/misunderstanding something.