skopeo copy → oci-archive: cannot disable compression

[dilyanpalauzov]

As discussed at containers/image#1313, the zstd compression, offered by skopeo copy v1.4.1, is worse that the Zstandard compression offered by the zstd command line tool. I want to use scopeo copy to oci-archive: destination. I want that the blobs are not compressed.

It turns out, that there is no option to offer uncompressed blobs.

That is, the image has "layers":[{"mediaType":"application/vnd.oci.image.layer.v1.tar+gzip","digest":", I want it to have the final +gzip stripped.

Rationale: it is more safe efficient, if the image is compressed with the zstd command and sent as .tar.zst file.

• Extend the parameter skopeo copy --dest-decompress to function also for the non-dir transports.

[mtrmac]

Thanks for your report. Is this still relevant, now that both containers/image#1313 and #1406 has been fixed? It should now be possible to request compression using the appropriate Zstd level.

Or do you truly need decompression as a goal, not just an intermediate step for compressing the image?

[dilyanpalauzov]

#1406 will land in skopeo > 1.4.1, which is not released yet. so I do not know.

klauspost/compress#410 provides in total four compression levels: SpeedFastest, SpeedDefault, SpeedBetterCompression, and (for level >= 10) SpeedBestCompression. The comment above the values is // Many input values will provide the same compression level. zstd has compression levels between 1 and 19. I am sceptical about zstd-level19 being equivalent to SpeedBestCompression.

As you stated in containers/image#1313 (comment) “we are not going to dig deep down into alternative compression algorithm implementations to figure out how they differ and to work on changing them to squeeze extra efficiency”. Since squeezing extra efficiency is out of scope, there shall be possible to have uncompressed output.

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[mtrmac]

Skopeo 1.5.0 has been released since. @dilyanpalauzov can you re-test whether that produces acceptable compression levels, please?

[dilyanpalauzov]

$ skopeo --version
skopeo version 1.5.0

$ skopeo copy --dest-compress=false containers-storage:f714eca81fe4 oci-archive:uncompressed.tar
Getting image source signatures
Copying blob 82dd7034db04 done  
Copying config f714eca81f done  
Writing manifest to image destination
Storing signatures

$ skopeo copy --dest-compress-format=zstd --dest-compress-level=20 containers-storage:f714eca81fe4 oci-archive:zstd20-compressed.tar
Getting image source signatures
Copying blob 82dd7034db04 done  
Copying config f714eca81f done  
Writing manifest to image destination
Storing signatures

The blobs/sha256-file is gzip compressed, --dest-compress=false does not work. https://github.com/containers/skopeo/blob/main/docs/skopeo-copy.1.md says --dest-compress only works for the dir transport.

If I copy to oci:dir, uncompress the gzip-blob, then tar the whole oci:directory, and zstd -19 it, the result is 2121816bytes, while --dest-compress-format=zstd --dest-compress-level=20 produces zstd20-compressed.tar=2385408 bytes.

[mtrmac]

Oh, there’s actually been a --dest-oci-accept-uncompressed-layers for a long time. It won’t decompress a pre-existing compressed blob, but for the case of copying from containers-storage, that’s not a concern.

Is that was was necessary?

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[mtrmac]

@dilyanpalauzov please reopen if --dest-oci-accept-uncompressed-layers doesn’t work as required.