feat(oci-layout): support in `oras cp` #748

qweeah · 2023-01-13T03:40:25Z

Related to #378

Signed-off-by: Billy Zha <[email protected]>

codecov-commenter · 2023-01-13T03:54:04Z

Codecov Report

Attention: Patch coverage is 23.33333% with 69 lines in your changes missing coverage. Please review.

Project coverage is 63.96%. Comparing base (9013115) to head (440731e).
Report is 445 commits behind head on main.

Files with missing lines	Patch %	Lines
cmd/oras/internal/option/target.go	19.76%	69 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #748      +/-   ##
==========================================
- Coverage   70.12%   63.96%   -6.17%     
==========================================
  Files          18       19       +1     
  Lines         616      702      +86     
==========================================
+ Hits          432      449      +17     
- Misses        153      222      +69     
  Partials       31       31

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Signed-off-by: Billy Zha <[email protected]>

cmd/oras/internal/option/target.go

cmd/oras/internal/option/target_test.go

cmd/oras/internal/display/print.go

cmd/oras/cp.go

cmd/oras/internal/option/target.go

Signed-off-by: Billy Zha <[email protected]>

qweeah · 2023-01-13T09:19:00Z

cmd/oras/cp.go

-
-Example - Copy the artifact tagged with 'v1' from repository 'localhost:5000/net-monitor' to 'localhost:5000/net-monitor-copy' with multiple tags and concurrency level tuned
-  oras cp --concurrency 6 localhost:5000/net-monitor:v1 localhost:5000/net-monitor-copy:v1,tag2,tag3
+Example - Copy the artifacts:


@FeynmanZhou I have grouped examples by features, can you help review, thanks!

Signed-off-by: Billy Zha <[email protected]>

shizhMSFT · 2023-01-13T16:02:50Z

cmd/oras/cp.go

+Example - Copy the artifact tagged 'v1':
+  oras cp localhost:5000/net-monitor:v1 localhost:5000/net-monitor-copy:v1  # copy between repositories
+  oras cp localhost:5000/net-monitor:v1 localhost:5000/net-monitor-copy     # copy without tagging in the destination
+  oras cp --to-oci localhost:5000/net-monitor:v1 test:v1                    # download into an OCI layout folder 'test'
+  oras cp --from-oci test:v1 localhost:5000/net-monitor:v1                  # upload from an OCI layout folder 'test'
+  oras cp --from-oci test.tar:v1 localhost:5000/net-monitor:v1              # upload from an OCI layout tar archive 'test.tar'
+
+Example - Copy the artifact tagged 'v1' and its referrers:
+  oras cp -r localhost:5000/net-monitor:v1 localhost:5000/net-monitor-copy:v1  # copy between repositories
+  oras cp -r --to-oci localhost:5000/net-monitor:v1 test:v1                    # download into an OCI image layout folder 'test'
+  oras cp -r --from-oci test:v1 localhost:5000/net-monitor:v1                  # upload from an OCI image layout folder 'test'
+
+Example - Copy certain platform of the artifact 'v1':
+  oras cp --platform linux/arm/v5 localhost:5000/net-monitor:v1 localhost:5000/net-monitor-copy:v1  # copy between repositories
+  oras cp --platform linux/arm/v5 --to-oci localhost:5000/net-monitor:v1 test:v1                    # download into an OCI layout folder 'test'
+  oras cp --platform linux/arm/v5 --from-oci test:v1 localhost:5000/net-monitor:v1                  # upload from an OCI layout folder 'test'
+
+Example - Copy the artifact 'v1' with multiple tags:
+  oras cp localhost:5000/net-monitor:v1 localhost:5000/net-monitor-copy:tag1,tag2,tag3                   # copy between repositories
+  oras cp --concurrency 10 localhost:5000/net-monitor:v1 localhost:5000/net-monitor-copy:tag1,tag2,tag3  # copy between repositories with concurrency level tuned
+  oras cp localhost:5000/net-monitor:v1 test:tag1,tag2,tag3 --to-oci                                     # download into an OCI layout folder 'test'
+  oras cp test:v1 localhost:5000/net-monitor-copy:tag1,tag2,tag3 --from-oci                              # upload from an OCI layout folder 'test'


Those examples will be wrapped by terminals, rendering hard to read for users.

@FeynmanZhou we might need to have a better way to present the examples.

cmd/oras/internal/option/target.go

cmd/oras/cp.go

Signed-off-by: Billy Zha <[email protected]>

cmd/oras/internal/option/target.go

TerryHowe · 2023-01-14T16:25:58Z

cmd/oras/cp.go

-  oras cp --concurrency 6 localhost:5000/net-monitor:v1 localhost:5000/net-monitor-copy:v1,tag2,tag3
+Example - Copy the artifact tagged 'v1':
+  oras cp localhost:5000/net-monitor:v1 localhost:5000/net-monitor-copy:v1  # copy between repositories
+  oras cp localhost:5000/net-monitor:v1 localhost:5000/net-monitor-copy     # copy without tagging in the destination


I think from a customers perspective --to-oci would be to an oci registry. I feel like it would be more intuitive to say --to-file or a little worse to me --to-directory

Thinking about it more, I like the --to=format and --from=format better because that would leave the possibility of other formats other than registry and directory. Having a TGZ format would be very handy for example. Having an input of stdin or stndout might be convenient as well.

The oci image layout is not a format but a layout, see spec

So for short flags, how about --from-layout, --to-layout for oras cp and --layout to other commands?

--layout is equivalent to --layout type=oci. To support more layouts in the future, we can add --layout type=docker (just taking docker image layout as an example, doesn't mean we need to support it)

Can you elaborate on the scenarios of using an input of stdin? To me the UX sounds looks like below:

some-cmd | oras cp - localhost/owner/name:v1

Also for uploading .tgz image layout, I might take a closer look tomorrow to see if it's already supported or what we need to do to support it.

Having an input of stdin or stndout might be convenient as well.

Currently, oras / oras-go requires random access for read and write for performance considerations, and thus stdin and stdout are not supported. Maybe we can consider that after oras v1.0.0, trading convenience off performance.

I don't want to get too side tracked by all the possible formats and destinations, but also don't want to design into a corner. It definitely might be convenient to support stdin/stdout for example, but maybe that should be done in another command. The tgz thing also might make more sense for another command since tgz input/output make sense for multiple repositories.

It definitely might be convenient to support stdin/stdout for example, but maybe that should be done in another command. The tgz thing also might make more sense for another command since tgz input/output make sense for multiple repositories.

@TerryHowe Can you help create enhancement issues with detailed scenario for those? We can continue discussion there.

I think from a customers perspective --to-oci would be to an oci registry. I feel like it would be more intuitive to say --to-file or a little worse to me --to-directory.

@TerryHowe To avoid ambiguity, how about using --to-oci-layout and --from-oci-layout in oras copy respectively?

cmd/oras/internal/option/target.go

Signed-off-by: Billy Zha <[email protected]>

cmd/oras/cp.go

Signed-off-by: Billy Zha <[email protected]>

cmd/oras/internal/fileref/unix.go

cmd/oras/internal/fileref/windows_test.go

cmd/oras/internal/fileref/windows.go

cmd/oras/internal/option/target.go

Signed-off-by: Billy Zha <[email protected]>

cmd/oras/internal/option/target.go

Signed-off-by: Billy Zha <[email protected]>

shizhMSFT

LGTM

Related to oras-project#378 Signed-off-by: Billy Zha <[email protected]>

qweeah force-pushed the oci-copy branch from 43e07b8 to 5735307 Compare January 13, 2023 03:43

feat: support oci image layout in oras cp

83c4af1

Signed-off-by: Billy Zha <[email protected]>

qweeah force-pushed the oci-copy branch from 5735307 to 83c4af1 Compare January 13, 2023 03:44

qweeah added 3 commits January 13, 2023 11:46

add example

f8d4a27

Signed-off-by: Billy Zha <[email protected]>

add example

2026d9d

Signed-off-by: Billy Zha <[email protected]>

add go mod

071c792

Signed-off-by: Billy Zha <[email protected]>

qweeah added 2 commits January 13, 2023 12:08

rename

c244cd5

Signed-off-by: Billy Zha <[email protected]>

doc clean

a5fd875

Signed-off-by: Billy Zha <[email protected]>

shizhMSFT requested review from shizhMSFT and FeynmanZhou and removed request for shizhMSFT January 13, 2023 06:05

shizhMSFT reviewed Jan 13, 2023

View reviewed changes

qweeah added 8 commits January 13, 2023 14:39

update license header

d4f96d3

Signed-off-by: Billy Zha <[email protected]>

add interface for printer

b0e8e28

Signed-off-by: Billy Zha <[email protected]>

doc clean

3df7915

Signed-off-by: Billy Zha <[email protected]>

rename

ec2f556

Signed-off-by: Billy Zha <[email protected]>

Merge remote-tracking branch 'origin_src/main' into oci-copy

07dba15

update examples

f5bb892

Signed-off-by: Billy Zha <[email protected]>

update doc

6dbd0a6

Signed-off-by: Billy Zha <[email protected]>

doc clean

d71860c

Signed-off-by: Billy Zha <[email protected]>

qweeah commented Jan 13, 2023

View reviewed changes

qweeah added 4 commits January 13, 2023 17:25

resolve comments

6d11cef

Signed-off-by: Billy Zha <[email protected]>

add wrapper type for tag status printer

01c5674

Signed-off-by: Billy Zha <[email protected]>

remove long oci image layout flag

c20e948

Signed-off-by: Billy Zha <[email protected]>

doc clean

3962a55

Signed-off-by: Billy Zha <[email protected]>

shizhMSFT reviewed Jan 13, 2023

View reviewed changes

qweeah added 2 commits January 14, 2023 10:19

resolve comments

33f6561

Signed-off-by: Billy Zha <[email protected]>

Merge remote-tracking branch 'origin_src/main' into oci-copy

fca9db1

fix unit test

2a70519

Signed-off-by: Billy Zha <[email protected]>

TerryHowe reviewed Jan 14, 2023

View reviewed changes

qweeah added 2 commits January 16, 2023 08:37

rename target type

6036d0d

Signed-off-by: Billy Zha <[email protected]>

simplify examples

1e463d4

Signed-off-by: Billy Zha <[email protected]>

shizhMSFT reviewed Jan 16, 2023

View reviewed changes

cmd/oras/cp.go Outdated Show resolved Hide resolved

cmd/oras/cp.go Outdated Show resolved Hide resolved

qweeah added 5 commits January 16, 2023 17:17

fix bug for windows path parsing and add tests

c83b54b

Signed-off-by: Billy Zha <[email protected]>

fix unit test

b4b2a6c

Signed-off-by: Billy Zha <[email protected]>

resolve comments

1585f5a

Signed-off-by: Billy Zha <[email protected]>

doc clean

12fc8ea

Signed-off-by: Billy Zha <[email protected]>

doc clean

87771b6

Signed-off-by: Billy Zha <[email protected]>

shizhMSFT reviewed Jan 16, 2023

View reviewed changes

qweeah added 3 commits January 16, 2023 19:14

rename parsing arguments

6af8f50

Signed-off-by: Billy Zha <[email protected]>

doc clean

56e29f9

Signed-off-by: Billy Zha <[email protected]>

resolve comments

b9f5df3

Signed-off-by: Billy Zha <[email protected]>

shizhMSFT reviewed Jan 16, 2023

View reviewed changes

fix doc

440731e

Signed-off-by: Billy Zha <[email protected]>

qweeah changed the title ~~feat: support OCI image layout in oras cp~~ feat(oci-layout): support in oras cp Jan 17, 2023

shizhMSFT approved these changes Jan 17, 2023

View reviewed changes

shizhMSFT merged commit 6447e8e into oras-project:main Jan 17, 2023

TerryHowe pushed a commit to TerryHowe/oras that referenced this pull request Feb 2, 2023

feat(oci-layout): support in oras cp (oras-project#748)

71091f4

Related to oras-project#378 Signed-off-by: Billy Zha <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(oci-layout): support in `oras cp` #748

feat(oci-layout): support in `oras cp` #748

qweeah commented Jan 13, 2023 •

edited

Loading

codecov-commenter commented Jan 13, 2023 •

edited by codecov bot

Loading

qweeah Jan 13, 2023

shizhMSFT Jan 13, 2023 •

edited

Loading

TerryHowe Jan 14, 2023

TerryHowe Jan 14, 2023

qweeah Jan 15, 2023 •

edited

Loading

qweeah Jan 15, 2023 •

edited

Loading

qweeah Jan 15, 2023

shizhMSFT Jan 15, 2023

TerryHowe Jan 15, 2023

qweeah Jan 16, 2023

FeynmanZhou Jan 16, 2023 •

edited

Loading

TerryHowe Jan 16, 2023

shizhMSFT left a comment

feat(oci-layout): support in oras cp #748

feat(oci-layout): support in oras cp #748

Conversation

qweeah commented Jan 13, 2023 • edited Loading

codecov-commenter commented Jan 13, 2023 • edited by codecov bot Loading

Codecov Report

Choose a reason for hiding this comment

shizhMSFT Jan 13, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

qweeah Jan 15, 2023 • edited Loading

Choose a reason for hiding this comment

qweeah Jan 15, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

FeynmanZhou Jan 16, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shizhMSFT left a comment

Choose a reason for hiding this comment

feat(oci-layout): support in `oras cp` #748

feat(oci-layout): support in `oras cp` #748

qweeah commented Jan 13, 2023 •

edited

Loading

codecov-commenter commented Jan 13, 2023 •

edited by codecov bot

Loading

shizhMSFT Jan 13, 2023 •

edited

Loading

qweeah Jan 15, 2023 •

edited

Loading

qweeah Jan 15, 2023 •

edited

Loading

FeynmanZhou Jan 16, 2023 •

edited

Loading