Skip to content

Commit

Permalink
Add a test demonstrating underread of underlying buffers
Browse files Browse the repository at this point in the history
When reading from a decompressor that was constructed using `Decoder::with_buffer`, the decoder may not consume all of the input by the time it returns all of the decompressed output.  This means when you call `.finish()` on the decoder to get the underlying stream back, it is not pointing after the end of the compressed data.

This commit adds a test that demonstrates the issue.
  • Loading branch information
markbt committed Oct 19, 2023
1 parent 92f87dc commit 3ae7ad6
Show file tree
Hide file tree
Showing 2 changed files with 38 additions and 0 deletions.
Binary file added assets/zeros64.zst
Binary file not shown.
38 changes: 38 additions & 0 deletions tests/issue_n.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
use std::io::Read;

#[test]
fn test_issue_n() {
// This is 64 compressed zero bytes.
let compressed_data = include_bytes!("../assets/zeros64.zst");
let decompressed_size = 64;

// Construct a decompressor using `with_buffer`. This should be ok as
// `Cursor` is `BufRead`.
let reader = std::io::Cursor::new(compressed_data);
let mut decomp = zstd::Decoder::with_buffer(reader).unwrap();

// Count how many bytes we decompress.
let mut total = 0;

// Decompress four bytes at a time (this is necessary to trigger underrun
// behaviour).
for _ in 0..(decompressed_size / 4) {
let mut buf = [0u8; 4];
let count = decomp.read(&mut buf).unwrap();
total += count;
}

// Finish reading and get the buffer back.
let reader = decomp.finish();

// The cursor should now be at the end of the compressed data.
println!("We read {total}/{decompressed_size} decompressed bytes");
println!(
"The underlying cursor is at position {} of {} compressed bytes",
reader.position(),
compressed_data.len()
);

assert_eq!(total, decompressed_size);
assert_eq!(reader.position() as usize, compressed_data.len());
}

0 comments on commit 3ae7ad6

Please sign in to comment.