Move the tar-split digest into the TOC #1902
Conversation
pkg/chunked/internal/compression.go
Outdated
| @@ -85,7 +86,7 @@ func GetType(t byte) (string, error) { | |||
| const ( | |||
| ManifestChecksumKey = "io.github.containers.zstd-chunked.manifest-checksum" | |||
| ManifestInfoKey = "io.github.containers.zstd-chunked.manifest-position" | |||
| TarSplitChecksumKey = "io.github.containers.zstd-chunked.tarsplit-checksum" | |||
| TarSplitChecksumKey = "io.github.containers.zstd-chunked.tarsplit-checksum" // Deprecated: Use the TOC.TarSplitDigest field instead, this one is not authenticated by ManifestChecksumKey. | |||
There was a problem hiding this comment.
I think it is fine if we just break now, and not have to worry about it
There was a problem hiding this comment.
The alternative would be to ignore the tar-split. That would make the current images consumable, but they would IIRC break on re-pushing the layers.
mtrmac
force-pushed
the
tar-split-digest
branch
from
a2ea4cc to
710533e
Compare
mtrmac
force-pushed
the
tar-split-digest
branch
8 times, most recently
from
ff1e003 to
791869f
Compare
|
#1888 must have a solution. This is one possible solution; it also breaks all currently-existing zstd:chunked images, which might be fine or very bad, I don’t know which one it is. |
It is fine to break these images now. The tarsplit data must not be mandatory. If the tarsplit data is missing, we should just skip fetching it (and deal with errors to recreate the original image later) |
This is a microptimization, we call strings.ToLower only once, but more importantly it will make it easier to add more fields. Signed-off-by: Miloslav Trmač <[email protected]>
Other TOC formats don't fill the data in. For now, this only increases memory usage, but we will need the data soon. Signed-off-by: Miloslav Trmač <[email protected]>
... so that we can uniquely identify partially-pulled layers by the TOC digest. Signed-off-by: Miloslav Trmač <[email protected]>
Include more details in the returned error text. Don't continue in tests when we fail to obtain a TOC. Signed-off-by: Miloslav Trmač <[email protected]>
|
With this PR:
Alternatively, in the last case, we could accept the layer and ignore the tar-split data. If we do that, the images will continue to pull fine; but previously they also pushed fine and that would now break. Marking ready for review now. |
could we just drop it from the image generation and always ignore it? So we don't have to worry about it in future |
I’m not sure which part you mean should be dropped. We need:
We can’t drop the tar-split offset. Do you mean to stop adding the |
yes, to stop doing that and use only the information in the TOC |
We are already not reading it, so simplify the code. Signed-off-by: Miloslav Trmač <[email protected]>
|
Done. |
mtrmac
changed the title
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: giuseppe, mtrmac The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
/lgtm |
A minimal prototype for a fix of #1888 .
Note that this would immediately break pulling all currently-existing layers.
Also this probably increases memory requirements because the TOC exists as individual Go values longer.
Cc: @giuseppe