Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

flush data still available in the decoder when no input (#123) #247

Merged
merged 2 commits into from
Jan 17, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0),

## Unreleased

- Flush available data in decoder even when there's no incoming input.

## 0.4.5

- Add `{Lzma, Xz}Decoder::with_mem_limit()` methods.
Expand Down
12 changes: 10 additions & 2 deletions src/codec/gzip/decoder.rs
Original file line number Diff line number Diff line change
Expand Up @@ -80,8 +80,16 @@ impl GzipDecoder {

State::Decoding => {
let prior = output.written().len();
let done = inner(self, input, output)?;
self.crc.update(&output.written()[prior..]);

let res = inner(self, input, output);

if (output.written().len() > prior) {
// update CRC even if there was an error
self.crc.update(&output.written()[prior..]);
}

let done = res?;

if done {
self.state = State::Footer(vec![0; 8].into())
}
Expand Down
29 changes: 24 additions & 5 deletions src/futures/bufread/generic/decoder.rs
Original file line number Diff line number Diff line change
Expand Up @@ -65,18 +65,37 @@ impl<R: AsyncBufRead, D: Decode> Decoder<R, D> {
) -> Poll<Result<()>> {
let mut this = self.project();

let mut first = true;

loop {
*this.state = match this.state {
State::Decoding => {
let input = ready!(this.reader.as_mut().poll_fill_buf(cx))?;
if input.is_empty() {
// Avoid attempting to reinitialise the decoder if the reader
// has returned EOF.
let input = if first {
&[][..]
} else {
ready!(this.reader.as_mut().poll_fill_buf(cx))?
};

if input.is_empty() && !first {
// Avoid attempting to reinitialise the decoder if the
// reader has returned EOF.
*this.multiple_members = false;

State::Flushing
} else {
let mut input = PartialBuffer::new(input);
let done = this.decoder.decode(&mut input, output)?;
let done = this.decoder.decode(&mut input, output).or_else(|err| {
// ignore the first error, occurs when input is empty
// but we need to run decode to flush
if first {
Ok(false)
} else {
Err(err)
}
})?;

first = false;

let len = input.written().len();
this.reader.as_mut().consume(len);
if done {
Expand Down