Stream-based zip encoder and decoder #880
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Part of #851.
Depends on #875.
Implements stream-based ZIP encodes and decoder using the CompressionStream and DecompressionStream class.
Here's what we get:
ZipArchive
To that last point:
ZIP as a remote, virtual filesystem
This change enables fast previewing of even 10GB-large zipped exports via partial downloads.
Imagine previewing a large site export with many photos and videos. The
decodeRemoteZip
function knows how to request just the list of files first, filter out the large ones, and then issue multiple fetch() requests to download the rest.Effectively, we would only download ~5MB - 10MB of data for the initial preview, and then only download these larger assets once they're needed.
Technical details
Here's a few interesting functions shipped by this PR. Note the links point to a specific commit and may get outdated:
Remaining work
There's a few more things to do here, but I still wanted to get some reviews in before spending the time on these just in case the API would substantially change:
API changes
Breaking changes
This PR isn't a breaking change yet. One of the follow-up PRs will very likely propose some breaking changes, but this one only extends the available API.
Without this PR
Without this PR, unzipping a file requires writing it to Playground, calling PHP's
unzip
, and removing the temporaryzip
file:With this PR
With this PR, unzipping can be done as follows:
More examples
Here's what else the streaming API unlocks. Not all of these functions are shipped here, but they are quite easy to implement:
cc @dmsnell