perf: copy entire file before unpacking

[nichmor]

Description

This PR introduce a small performance improvement for streaming tar.bz2 files.

Based on the benchmarks, downloading file into SpooledTempFile and then un-compressing is faster than download & extract at the same time.

Benchmarks for python tar.bz2:

Without changes

Running benches/extract.rs (/Users/graf/projects/oss/rattler-1/target/release/deps/extract-f947b197b5ad3a13)
Gnuplot not found, using plotters backend
extract tar             time:   [1.8692 s 1.8893 s 1.9125 s]
                        change: [-1.4149% +0.5238% +2.4181%] (p = 0.61 > 0.05)
                        No change in performance detected.

with spooled file 5 mb

     Running benches/extract.rs (/Users/graf/projects/oss/rattler-1/target/release/deps/extract-f947b197b5ad3a13)
Gnuplot not found, using plotters backend
extract tar             time:   [1.8519 s 1.8676 s 1.8811 s]
                        change: [-2.5692% -1.5904% -0.5665%] (p = 0.01 < 0.05)
                        Change within noise threshold.

We can see a small 1% improvement. Similar improvement can be observed also for bat.tar.bz2

I've also added a small webserver that will serve files from the memory, so it can be used in benchmarks.

[baszalmstra]

This basically means we are no longer downloading and extracting in parallel. I wonder if that makes a difference with a slower connection.

[baszalmstra]

Do you think it would be possible to implement the server in rust instead? That would make the whole thing more standalone.

[baszalmstra]

Look for StaticFileServer (from top of mind) we already have something similar.

[wolfv]

I think my idea would be that we ideally keep parallel downloading and extraction, but if downloading is much faster than extraction, that we store the remaining data in a (spooled) temporary file, so that download can progress at maximum speed.

[wolfv]

I think what could be also interesting is to try out async_compression (uv uses this). uv also controls the BufReader size more, and uses a 128Kb buffer (124 * 1024). I think we end up using a default BufReader which has a 8Kb buffer which might lead to less optimal download speeds.