-
Notifications
You must be signed in to change notification settings - Fork 181
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
oras cp
fails if org.opencontainers.image.ref.name
contains a full reference
#1505
Comments
That’s a fair requirement as per image spec. Coincidentally this point came up during the OCI calls. Thanks for being willing to contribute the fix @mauriciovasquezbernal |
Opened #1507 with (hopefully) a right fix for it. |
@sajayantony I have a concern about this. According to the annotations.md ref ::= component ("/" component)*
component ::= alphanum (separator alphanum)*
alphanum ::= [A-Za-z0-9]+
separator ::= [-._:@+] | "--" Reference names like @mauriciovasquezbernal If you look at the image-layout.md, there is a note:
Therefore, I'm not sure if storing the full reference in |
@mauriciovasquezbernal FYI, {
"schemaVersion": 2,
"manifests": [
{
"mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
"digest": "sha256:4edbd2beb5f78b1014028f4fbb99f3237d9561100b6881aabbf5acce2c4f9454",
"size": 1638,
"annotations": {
"io.containerd.image.name": "docker.io/library/alpine:latest",
"org.opencontainers.image.ref.name": "latest"
}
}
]
} References: |
Nevertheless, |
Thank @mauriciovasquezbernal for the fix (#1507). However, fixing this issue is not trivial as we need to consider paths for various OSs and edge cases. For instances,
Again, taking In other words, we need to find a special separator, which is not in the oras cp --from-oci-layout ./mygadget.tar.folder#ghcr.io/inspektor-gadget/gadget/mygadget:latest localhost:5000/foo:latest Or, we need a new flag such as oras cp --from-oci-layout-path ./mygadget.tar.folder ghcr.io/inspektor-gadget/gadget/mygadget:latest localhost:5000/foo:latest Note: This flag should be mutual exclusive with I think proposal 2 is clearer, less ambiguous, and backward compatible. |
@shizhMSFT If the source image in OCI layout is
|
@FeynmanZhou Here's the difference. Original: oras cp --from-oci-layout <path>:<reference> <destination> Proposal 2: There is no separator oras cp --from-oci-layout-path <path> <reference> <destination> |
Is it really worth it to introduce a new flag? What about continue supporting the In any case, I'll let you as a maintainers make the decision and I will implement it. |
The For example,
Both parsing strategies are valid. |
For proposal 2, can we move the
|
The original design of |
IMO, the proposal #2 is much flexible than the proposal #1. However, the only ambiguity of proposal #2 is that users might be confused about two parameters oras cp --from-oci-layout-path <path> <reference> <destination> For example, I thought oras cp --from-oci-layout-path ./mygadget.tar.folder ghcr.io/inspektor-gadget/gadget/mygadget:latest localhost:5000/foo:latest How about we make the new flag |
cc more ORAS maintainers @TerryHowe @oras-project/oras-cli for inputs |
It could be hard to parse the command line in proposal 2 |
@TerryHowe were you going for? oras cp --from-oci-layout-ref '<reference> <path>' |
Sorry a little bit lost, will there still be 2 positional arguments for proposal |
Hi @TerryHowe , just to double confirm, does |
@FeynmanZhou no this is not what I mean. I am confirming whether |
Confirming since @shizhMSFT and I are suggesting a flag that take string value as parameter, but what @TerryHowe suggested is a switch flag? |
What I should have said is the usage would not change, but the way it is used would. Current usage:
would parse this just fine:
|
I'd like to elaborate more on the original design of The original command is oras cp <source> <destination> which means "use oras to copy from source to destination by references". With the bool flag oras cp <source> <destination> --from-oci-layout --to-oci-layout which means "use oras to copy from source to destination by references where the source reference is in OCI image layout form and the destination reference is in OCI image layout form". oras cp --from-oci-layout <source> --to-oci-layout <destination> Similarly, with the string value flag oras cp <source> <destination> --from-oci-layout-path <source_path> --to-oci-layout-path <destination_path> which means "use oras to copy from source to destination by references where the source reference can be found in the given OCI image layout path and the destination is the reference in the given OCI image layout path". oras cp --from-oci-layout-path <source_path> <source> --to-oci-layout-path <destination_path> <destination> If only source or destination is OCI image layout, it can be oras cp --from-oci-layout-path <source_path> <source> <destination> or oras cp --to-oci-layout-path <destination_path> <destination> @TerryHowe @qweeah @FeynmanZhou @mauriciovasquezbernal @sajayantony I recall that the bool flag oras/cmd/oras/internal/option/target.go Lines 76 to 88 in 1d60e22
Therefore, the following commands should be equivalent by design. oras cp --from-oci-layout <source> --to-oci-layout <destination>
oras cp --from-target type=oci-layout <source> --to-target type=oci-layout <destination> It is also natural to have oras cp --from-oci-layout-path <source_path> <source> --to-oci-layout-path <destination_path> <destination>
oras cp --from-target type=oci-layout,src=<source_path> <source> --to-target type=oci-layout,dest=<destination_path> <destination>
|
The proposal Actually, I'm not a fan of using space as a separator inside an argument since many scripts forget to add quotes. For example, the following script seems correct #!/bin/bash
function test {
echo "$# parameters for test"
ref=$1
oras cp --from-oci-layout-ref $ref localhost:5000/test:latest
}
function oras {
echo "$# parameters for oras"
echo "ref is $3"
echo "destination is $4"
}
test "ghcr.io/oras-project/oras:v1.2.0 oras.tar" but it is not as it prints
The trick is that quotes are required for - oras cp --from-oci-layout-ref $ref localhost:5000/test:latest
+ oras cp --from-oci-layout-ref "$ref" localhost:5000/test:latest it prints
|
IMHO we shouldn't rely on users to correctly set the quotes. Also, paths can have empty spaces, making this command a bit more difficult to understand: With the IMO |
I think it would be more consisten to have a
from and to oci-layout-path should take an argument rather than be booleans. |
@TerryHowe Thanks for pointing out. It was a typo in #1505 (comment) and I've fixed that. |
What happened in your environment?
We use OCI images to package eBPF programs in Inspektor Gadget. We have an
ig image export
command that creates an oci layout file with the image. I wanted to useoras cp
to copy the image to a remote registry, however it fails with:The issue seems to be related to the fact that the
org.opencontainers.image.ref.name
annotation is set toghcr.io/inspektor-gadget/gadget/mygadget:latest
in index.json:mygadget.tar.folder/index.json:
According to https://github.com/opencontainers/image-spec/blob/v1.1.0-rc5/annotations.md,
ghcr.io/inspektor-gadget/gadget/mygadget:latest
is a valid value for the annotation.Possible Fix
The following diff fixes the issue:
I'm willing to open a fix once we agree on a solution.
What did you expect to happen?
oras cp
should have copied the image to the remote repositoryHow can we reproduce it?
Set the
org.opencontainers.image.ref.name
annotation to something likefoo:bar
in index.json and try to runoras cp --from-oci-layout ./myfolder:foo:bar localhost:5000/foo:latest
What is the version of your ORAS CLI?
Manually compiled from 961e9f8
What is your OS environment?
Ubuntu 22.04 (it really doesn't matter for this bug)
Are you willing to submit PRs to fix it?
The text was updated successfully, but these errors were encountered: