download_sysext: HTTP client timeout and retry

[dongsupark]

Set the default timeout to 20 seconds, so HTTP client can abort after that timeout.

If HTTP client fails to download, retry to download once in 1000 msec, up to 20 times.

Fixes #15

[pothos]

It doesn't retry in this case: During download I deactivated my wifi and the connection was stuck because we don't have the read timeout (see issue).
When activating my wifi again the download seems to continue but directly failed without retry:

read 13926158/28371306 ( 49%)Error: unable to download data(url Url { scheme: "https", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("update.release.flatcar-linux.net")), port: None, path: "/amd64-usr/3815.0.0/oem-azure.gz", query: None, fragment: None })

Caused by:
    0: failed to get response chunk
    1: request or response body error: error reading a body from connection: end of file before message length reached
    2: error reading a body from connection: end of file before message length reached
    3: end of file before message length reached

[pothos]

It doesn't retry in this case: During download I deactivated my wifi and the connection was stuck because we don't have the read timeout (see issue). When activating my wifi again the download seems to continue but directly failed without retry:

read 13926158/28371306 ( 49%)Error: unable to download data(url Url { scheme: "https", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("update.release.flatcar-linux.net")), port: None, path: "/amd64-usr/3815.0.0/oem-azure.gz", query: None, fragment: None })

Caused by:
    0: failed to get response chunk
    1: request or response body error: error reading a body from connection: end of file before message length reached
    2: error reading a body from connection: end of file before message length reached
    3: end of file before message length reached

I think I see the problem: We only retry getting a response (the header) but we don't retry the full download. We need to wrap the whole function and can't split it into two parts.

[dongsupark]

As discussed, this PR is re-implemented without tryhard crate.

[pothos]

This continuously appends, we need to truncate the file first to a length of 0 bytes before writing again.

[pothos]

This does not retry on "Connection refused"

[pothos]

If we want to keep ? for error handling we should retry the whole function - eg. with a helper function download_and_hash_retry that loops around it.

[pothos]

If we want to keep ? for error handling we should retry the whole function - eg. with a helper function download_and_hash_retry that loops around it.

With #44 this will be easier

[dongsupark]

Brought back to a simple local wrapper version without tryhard, without passing File, without tokio, based on #44.

[pothos]

Thanks, works well now