Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable multiple_members for GzipDecoder #354

Merged
merged 2 commits into from
Apr 13, 2023

Conversation

cmackenzie1
Copy link
Contributor

This commit enables support for multiple_members which allows the GzipDecoder to support gzip streams consisting of multiple, concatenated gzip's - which in turn is a valid gzip.

Before this commit, when using the decompression middleware, the input stream would be closed after reading the first gzip chunk, ignoring the rest of the gzip - meaning the resulting decompressed body could be missing data but still considered "OK".

An example of a concatenated gzip:

echo "hello" | gzip > hello.txt.gz
echo "world" | gzip > world.txt.gz

cat hello.txt.gz world.txt.gz > hello-world.txt.gz

gzip -d < hello-world.txt.gz
hello
world

Fixes: #353

Motivation

#353

Solution

Set multiple_members(true) on the GzipDecoder.

This commit enables support for `multiple_members` which allows the
`GzipDecoder` to support gzip streams consisting of multiple,
concatenated gzip's - which in turn is a valid gzip.

Before this commit, when using the decompression middleware, the input
stream would be closed after reading the first gzip chunk, ignoring the
rest of the gzip - meaning the resulting decompressed body could be
missing data but still considered "OK".

An example of a concatenated gzip:

```bash
echo "hello" | gzip > hello.txt.gz
echo "world" | gzip > world.txt.gz

cat hello.txt.gz world.txt.gz > hello-world.txt.gz

gzip -d < hello-world.txt.gz
hello
world
```

Fixes: tower-rs#353
@Nehliin Nehliin requested a review from davidpdrsn April 13, 2023 11:56
Copy link
Member

@davidpdrsn davidpdrsn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! I think this looks good. Great that that you also added tests.

Do you wanna update the changelog as well? Then this is good to merge.

@cmackenzie1
Copy link
Contributor Author

Sure, just pushed the changes for the CHANGELOG.md!

Copy link
Member

@davidpdrsn davidpdrsn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@davidpdrsn davidpdrsn enabled auto-merge (squash) April 13, 2023 22:11
@davidpdrsn davidpdrsn merged commit 92d1954 into tower-rs:master Apr 13, 2023
@cmackenzie1 cmackenzie1 deleted the cole/issue-353 branch April 13, 2023 22:30
gezihuzi pushed a commit to gezihuzi/tower-http that referenced this pull request Jul 19, 2023
* Enable `multiple_members` for `GzipDecoder`

This commit enables support for `multiple_members` which allows the
`GzipDecoder` to support gzip streams consisting of multiple,
concatenated gzip's - which in turn is a valid gzip.

Before this commit, when using the decompression middleware, the input
stream would be closed after reading the first gzip chunk, ignoring the
rest of the gzip - meaning the resulting decompressed body could be
missing data but still considered "OK".

An example of a concatenated gzip:

```bash
echo "hello" | gzip > hello.txt.gz
echo "world" | gzip > world.txt.gz

cat hello.txt.gz world.txt.gz > hello-world.txt.gz

gzip -d < hello-world.txt.gz
hello
world
```

Fixes: tower-rs#353

* update changelog for tower-rs#354
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support concatenated GZIP streams
3 participants