Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Compression method Deflate is unsupported #131

Closed
RReverser opened this issue May 8, 2021 · 8 comments · Fixed by #132
Closed

Compression method Deflate is unsupported #131

RReverser opened this issue May 8, 2021 · 8 comments · Fixed by #132

Comments

@RReverser
Copy link
Contributor

I'm trying to use image crate to read TIFF images produced (compressed) by ImageMagick with -compress zip. My understanding is that image crate relies on this one to do actual reading, so raising issue here.

This is what Exiftool reports on those images:

> exiftool -compression pleiades4.tif
Compression                     : Adobe Deflate

README suggests that decoding Deflate TIFFs should be supported:

image

However, when trying to read them using image::open, I'm getting:

The decoder for Tiff does not support the format features Compression method Deflate is unsupported

Am I holding it wrong? Or is there some specific to this "Adobe Deflate" that ImageMagick uses that makes it different from Deflate supported by this crate?
@RReverser
Copy link
Contributor Author

Or is there some specific to this "Adobe Deflate" that ImageMagick uses that makes it different from Deflate supported by this crate?

Looking at the code a bit deeper, it does seem there are Deflate and OldDeflate compression methods defined, but only OldDeflate supported for decoding. So is this a documentation issue / README support table being misleading?

@RReverser
Copy link
Contributor Author

I played around a bit, it appears that "Adobe Deflate" has the same Zlib-compatible header, and if I change match pattern to OldDeflate | Deflate going to the same decoder, it seems to work.

I'll play around a bit more and make a PR - now running into another issue with the compressed.len() > max_uncompressed_length check.

@RReverser
Copy link
Contributor Author

RReverser commented May 8, 2021 •
edited
Loading

Hm, the max_uncompressed_length seems to be a separate issue. It looks like something not configurable by the end user, but I don't get how it's set? I've tried to println it, and it shows it's calculated as 578304 which seems to be very low for most images - my TIFFs are easily ~130MB uncompressed for 24MP image x 16-bit RGB.

@RReverser
Copy link
Contributor Author

Looks like it's calculated as multiplication of these two numbers, which are the same for uncompressed or compressed variant of the same image:

(buffer.len(), buffer.byte_len()) = (
    289152,
    2,
)

@RReverser
Copy link
Contributor Author

I think I see what's happening... looks like that length is meant to be for a tile not for the whole file, so the error is misleading (I guess that's what

image-tiff/src/decoder/stream.rs

Lines 142 to 143 in a874b9c

// TODO: Implement streaming compression, and remove this (temporary) and somewhat
// misleading workaround.
tries to warn about).

@RReverser
Copy link
Contributor Author

UPD: fixed that too, now battling with satisfying the crate's MSRV... 1.34.2 is fairly restrictive by now, are there any plans to lift that?

@RReverser
Copy link
Contributor Author

Ah, it only needs to build, not to test, so I can ignore criterion's errors. That's easier.

@RReverser RReverser mentioned this issue May 8, 2021
Merged
RReverser added a commit to RReverser/image-tiff that referenced this issue May 8, 2021
@RReverser
Improve deflate support
a6fb855 
 - Switch to streaming decompression via `flate2`. Aside from performance improvements and lower RAM consumption, this fixes a bug where `max_uncompressed_length` was precalculated for a single tile but then used as a hard limit on the whole data, failing to decompress any tiled images.
 - Deprecate `InflateError` - this was used for wrapping `miniz_oxide` raw status, but `flate2` already reports them as `io::Error` instead. This makes errors from both Lzw and Deflate modes more consistent - they're all now just `io::Error`.
 - Add support for new `Deflate` tag in addition to `OldDeflate` - format-wise it's same Zlib, and on practice decodes successfully with the same underlying code.
 - Slightly simplify partial reads from the `Reader` by using the `std::io::copy` helper instead of doing it manually.

Fixes image-rs#131.
@RReverser
Copy link
Contributor Author

Fix & other improvements in #132.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Improve deflate support RReverser/image-tiff
1 participant
@RReverser