Add support for packet loss

[kura]

Some Ookla speedtest servers support a packet loss value. I'm not sure why it's only some, rather than all...

For example these server IDs should all support packet loss:

• 30690
• 40788
• 32950
• 34948
• 11445
• 48354
• 40788

It gets return by the Ookla CLI in both normal text output and also JSON output (note I've removed a bunch of stuff from this output to make it easier to read):

$ speedtest --accept-license --accept-gdpr -f json-pretty -s 32950
{
    "type": "result",
    "timestamp": "2023-12-27T17:33:06Z",
    "ping": {
    },
    "download": {
    },
    "upload": {
    },
    "packetLoss": 0,
    "isp": "My ISP",
    "interface": {
    },
    "server": {
        "id": 32950,
        "host": "speedtest.boxbroadband.co.uk",
        "port": 8080,
        "name": "Box Broadband Ltd",
        "location": "London",
        "country": "United Kingdom",
        "ip": "45.10.101.252"
    },
    "result": {
    }
}

Would it be possible to add packet loss support to speedtest-go? I like the fact the Ookla CLI can handle the packet loss data, but recently I switched away because it stopped being able to get anywhere close to my max connection speed but the lack of packet loss data is bugginng me.

[r3inbowari]

Hi @kura, I don't know the protocol defined by speedtest.net, so we can only implement the uplink packet loss. Is this what you need?

[r3inbowari]

And we may only be able to implement this feature on specific platforms.

[kura]

@r3inbowari I'm not sure about the speedtest.net protocol either tbh.

Any kind of packet loss measurement would be good, otherwise I guess the only option for me would be to have something running iperf and dumping measurements.

What exactly would the uplink packet loss measurement be?

I only care about it working on Linux so if you do find you can only implement it on specific platforms then I'd guess Linux is probably the most likely to be supported?

[r3inbowari]

What exactly would the uplink packet loss measurement be?

Maybe like this? I am not sure.

retransmission rate = tcpOutboundRetransBytes / (tcpOutboundRetransBytes + actuallyOutboundSize) * 100%

I don't know the protocol defined by speedtest.net, so we can only implement the uplink packet loss.

This is just the retransmission rate, which is quite different from packet loss.

speedtest-go_windows_amd64.zip

[kura]

Hmmm, yeah retrans isn't really the same as packet loss.

I assume what the official Ookla client is doing is sending UDP packets to the server and measuring packet loss (similar to the iperf approach), which might explain why the speedtest server listens on udp/5060 and/or udp/8080.

[kura]

Ok, the official Ookla client is definitely doing some UDP traffic.

# Before running Ookla speedtest
$> cat /proc/net/udp
   sl  local_address rem_address   st tx_queue rx_queue tr tm->when retrnsmt   uid  timeout inode ref pointer drops

 2590: 00000000:14E9 00000000:0000 07 00000000:00000000 00:00000000 00000000   119        0 17860 2 0000000000000000 0

 3121: 00000000:96FC 00000000:0000 07 00000000:00000000 00:00000000 00000000   119        0 17862 2 0000000000000000 0

 5482: 3500007F:0035 00000000:0000 07 00000000:00000000 00:00000000 00000000   101        0 19625 2 0000000000000000 0

 5540: 00000000:006F 00000000:0000 07 00000000:00000000 00:00000000 00000000     0        0 17782 2 0000000000000000 0

 5752: 0100007F:0143 00000000:0000 07 00000000:00000000 00:00000000 00000000     0        0 20484 2 0000000000000000 0

# During a run of Ookla speedtest
$> cat /proc/net/udp
   sl  local_address rem_address   st tx_queue rx_queue tr tm->when retrnsmt   uid  timeout inode ref pointer drops

 2590: 00000000:14E9 00000000:0000 07 00000000:00000000 00:00000000 00000000   119        0 17860 2 0000000000000000 0

 3121: 00000000:96FC 00000000:0000 07 00000000:00000000 00:00000000 00000000   119        0 17862 2 0000000000000000 0

 5482: 3500007F:0035 00000000:0000 07 00000000:00000000 00:00000000 00000000   101        0 19625 2 0000000000000000 0

 5540: 00000000:006F 00000000:0000 07 00000000:00000000 00:00000000 00000000     0        0 17782 2 0000000000000000 0

 5752: 0100007F:0143 00000000:0000 07 00000000:00000000 00:00000000 00000000     0        0 20484 2 0000000000000000 0

 6180: 00000000:E2EF 00000000:0000 07 00000000:00000000 00:00000000 00000000  1000        0 292130 2 0000000000000000 0

 7644: 00000000:E8A7 00000000:0000 07 00000000:00000000 00:00000000 00000000  1000        0 292132 2 0000000000000000 0

During the Ookla run 2 new lines appear, 1000 is the UID of my user.

 6180: 00000000:E2EF 00000000:0000 07 00000000:00000000 00:00000000 00000000  1000        0 292130 2 0000000000000000 0

 7644: 00000000:E8A7 00000000:0000 07 00000000:00000000 00:00000000 00000000  1000        0 292132 2 0000000000000000 0

I decided to capture UDP packets with tshark from a speedtest.net run and it's definitely sending UDP packets in the background. I haven't figured out exactly what each part means yet but here is a single capture:

0000   00 15 5d 71 7d e5 00 15 5d fa aa b5 08 00 45 00   ..]q}...].....E.
0010   00 5f a0 4a 40 00 40 11 76 c6 ac 15 e4 61 2d 0a   ._.J@[email protected].
0020   65 fc 94 e6 1f 90 00 4b 23 da 4c 4f 53 53 20 36   e......K#.LOSS 6
0030   39 39 37 37 34 34 30 20 31 34 20 44 33 42 41 46   9977440 14 D3BAF
0040   42 34 45 2d 33 35 41 43 2d 34 39 44 37 2d 38 37   B4E-35AC-49D7-87
0050   37 39 2d 38 42 45 33 33 46 38 35 32 41 43 33 20   79-8BE33F852AC3 
0060   31 34 30 32 30 30 38 33 30 37 33 37 0a            140200830737.

• 69977440 value doesn't change in a single run, maybe some kind of ID?
• 14 this looks to be incrementing counter. It's definitely going from 1 to n
• D3BAFB4E-35AC-49D7-8779-8BE33F852AC3 not sure what this is. Some kind of UUID? It doesn't change in a single run
• 140200830737 timestamp

[r3inbowari]

This packet looks like it reports a loss counter. and 140200830737 does not look like a timestamp?
I don't know. any more?

[kura]

You are right. That is not a timestamp, not sure why I thought it was.

I'll do another complete run and packet capture tomorrow and post a larger set of packets and see if I can figure out more of the data that is being sent.

[r3inbowari]

I think the udp protocol of speedtest may be similar to tcp.
https://gist.github.com/sdstrowes/411fca9d900a846a704f68547941eb97

140200830737 more likes share id.
eg https://www.speedtest.net/result/1568942211

[r3inbowari]

• D3BAFB4E-35AC-49D7-8779-8BE33F852AC3 not sure what this is. Some kind of UUID? It doesn't change in a single run

This UUID widely used in a single test process and is randomly generated by the user client. Its purpose is to mark that the test is associated with the upload\download resource. likes this:
https://speedtest.boxbroadband.co.uk:8080/download?size=25000000&guid=6b7df3db-584b-44b9-9b04-94cbd5d14268

[kura]

Ok, so I did a PCAP capture of an entire test again. It's attached here as test.pcap.gz

The last 54-70 bytes are the payload, and they tend to alternate in size as the payload content alternates. The increase in size looks like it's the counters going in to double and then triple digits.

# 1 packet 68 bytes
LOSS 200231533 6 0 D41D9238-2B08-491B-BE8A-5AC2 A0FCA3D5 1925215 24835

# 2 packet 54 bytes
LOSS 672475500 0 70A65ECE-9199-47CA-926F-C9D84 9BB405B

# 3 packet 68 bytes
LOSS 200231533 6 1 D41D9238-2B08-491B-BE8A-5AC2 A0FCA3D5 1925215 77000

# 4 packet 54 bytes
LOSS 672475500 1 70A65ECE-9199-47CA-926F-C9D84 9BB405B

# 5 packet 68 bytes
LOSS 200231533 6 2 D41D9238-2B08-491B-BE8A-5AC2 A0FCA3D5 1925216 33921

# 6 packet 54 bytes
LOSS 672475500 2 70A65ECE-9199-47CA-926F-C9D84 9BB405B

I think you are right about it being like the TCP protocol.

I'm not sure why there are 2 UUIDs, 70A65ECE-9199-47CA-926F-C9D84 is the UUID of the test itself but I don't know what D41D9238-2B08-491B-BE8A-5AC2 references.

It also sends these packets on 2 different UDP connections, the odd numbered packets are being sent from src: 40913, dst: 8080 and the even numbered packets are src: 33527, dst: 8080