cloud: add optional connection timeout retries

[stevendanna]

This optionally adds connection timeout errors to the set of errors to retry in the resuming reader. Additionally, it adds the file name to error messages returned by the resuming reader.

We test for ETIMEDOUT. Linux returns ETIMEDOUT in cases where the remote end of a connection fails to acknowledge a configured number of TCP Keep-Alive messages.

This shouldn't catch timeouts that are the result of IO deadlines or context cancellations. But, we don't enable this retrying by default, to avoid changing any current behaviour.

Epic: none

Release note: None

[blathers-crl]

It looks like your PR touches production code but doesn't add or edit any test code. Did you consider adding tests to your PR?

_{🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.}

[cockroach-teamcity]

This change is 

[stevendanna]

For the permanent record, I tested this by:

0. Setting sudo sysctl -w net.ipv4.tcp_keepalive_probes=2 to shorten how long the following test takes. This is how many missed keepalives result in the connection being torn down. The default is 9 which takes a few minutes. Keep-Alives are set every 30s on our current version of Go.
1. Creating some calling code that started a read and then slept for a while.
2. While that code was stopped, using ss -ontap to find the local ephemeral port
3. Using iptables to drop inbound traffic on that port: sudo iptables -A INPUT -p tcp --dport 33256 -j DROP
4. Wait for the connection to fail with a timeout.

With the setting set to false, the error is returned to the user. With the setting set to true, we get a retry:

E230328 22:13:28.744289 3804 cloud/cloud_io.go:262 ⋮ [n1,client=127.0.0.1:56942,user=‹demo›] 754  read ‹cockroach-v20.1.11.darwin-10.9-amd64›: read tcp ‹10.142.0.102:33256› -> ‹52.219.110.2:443›: read: connection timed out
W230328 22:13:28.744408 3804 cloud/cloud_io.go:169 ⋮ [n1,client=127.0.0.1:56942,user=‹demo›] 755  retrying connection timed out because ‹cloudstorage.connection_timed_out_retries.enabled› = true
E230328 22:13:28.744421 3804 cloud/cloud_io.go:270 ⋮ [n1,client=127.0.0.1:56942,user=‹demo›] 756  Retry IO error: read ‹cockroach-v20.1.11.darwin-10.9-amd64›: read tcp ‹10.142.0.102:33256› -> ‹52.219.110.2:443›: read: connection timed out

[dt]

does this need a forward port to master and 23.1?