Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: enhance manifest fetch #541

Merged
merged 16 commits into from
Sep 16, 2022
Merged

Conversation

yuehaoliang
Copy link
Contributor

Resolve #531

Signed-off-by: Haoliang Yue [email protected]

Signed-off-by: Haoliang Yue <[email protected]>
@codecov-commenter
Copy link

codecov-commenter commented Sep 7, 2022

Codecov Report

Merging #541 (5287bf1) into main (b1415c0) will decrease coverage by 1.24%.
The diff coverage is 66.66%.

@@            Coverage Diff             @@
##             main     #541      +/-   ##
==========================================
- Coverage   73.68%   72.44%   -1.25%     
==========================================
  Files          13       13              
  Lines         532      508      -24     
==========================================
- Hits          392      368      -24     
  Misses        112      112              
  Partials       28       28              
Impacted Files Coverage Δ
cmd/oras/internal/option/cache.go 66.66% <66.66%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

cmd/oras/manifest/fetch.go Outdated Show resolved Hide resolved
@shizhMSFT shizhMSFT added this to the v0.15.0 milestone Sep 8, 2022
Comment on lines 110 to 116
if opts.cacheRoot != "" {
ociStore, err := oci.New(opts.cacheRoot)
if err != nil {
return err
}
buf.WriteByte('\n')
content = buf.Bytes()
src = cache.New(src, ociStore)
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we refactor it into the option package?

/cc @qweeah

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we refactor it into the option package?

/cc @qweeah

Created a option Cache.

cmd/oras/manifest/fetch.go Outdated Show resolved Hide resolved
cmd/oras/manifest/fetch.go Outdated Show resolved Hide resolved
Comment on lines 162 to 171
if opts.OutputDescriptor {
descBytes, err := json.Marshal(desc)
if err != nil {
return err
}
err = opts.Output(os.Stdout, descBytes)
if err != nil {
return err
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This code block appears everywhere if --descriptor and --pretty and both needed, any idea to refactor it? It doesn't necessarily need to be done in this PR /cc @lizMSFT

Copy link
Contributor Author

@yuehaoliang yuehaoliang Sep 15, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're discussing about this change offline.

if err != nil {
t.Fatal("error calling oci.New(), error =", err)
}
want := cache.New(mockTarget, ociStore)
Copy link
Contributor Author

@yuehaoliang yuehaoliang Sep 15, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was trying to write a better test here, but I didn't figure out a good way unless exporting the cache.target struct.

want := struct {
	oras.ReadOnlyTarget
	cache content.Storage
}{
	mockTarget,
	ociStore,
}

Building a struct like above doesn't work as want and got are not equal.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not doable even if cache.target struct is exported because target.cache is not exported. I think we can keep it this way.


var desc ocispec.Descriptor
var content []byte
if opts.OutputDescriptor && opts.outputPath == "" {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This if block can be merged with the if block at the end:

  1. If !opts.OutputDescriptor || opt.outputPath != "" => fetch manifest content
  2. If opts.OutputDescriptor => resolve and output to stdout

Copy link
Contributor Author

@yuehaoliang yuehaoliang Sep 16, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I couldn't really follow your idea here.

In the case of fetching manifest content, there still might be a need to output descriptor, eg. --output manifest.json --descriptor. Since oras.FetchBytes returns both the content and the descriptor, so if content has already been fetched, we don't need to call oras.Resolve again.

Current logic:

if opts.OutputDescriptor && opts.outputPath == "" {
	// oras.Resolve   fetch manifest descriptor only
} else {
	// oras.FetchBytes   fetch manifest descriptor content

	if opts.outputPath == "" || opts.outputPath == "-" {
		// output manifest content

		return
	}

	// save manifest content into the local file
}

if opts.OutputDescriptor {
	// output manifest descriptor
}

In the logic below, for --output manifest.json --descriptor, there's a redundant call of oras.Resolve.

if !opts.OutputDescriptor || opt.outputPath != "" {
	// oras.FetchBytes   fetch manifest descriptor content

	if opts.outputPath == "" || opts.outputPath == "-" {
		// output manifest content

		return
	}

	// save manifest content into the local file
} 

if opts.OutputDescriptor {
	// oras.Resolve   fetch manifest descriptor only

	// output manifest descriptor
}

Copy link
Contributor

@qweeah qweeah Sep 19, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since oras.FetchBytes returns both the content and the descriptor, so if content has already been fetched, we don't need to call oras.Resolve again

This can be avoided by checking if the desc is an empty struct.

cmd/oras/manifest/fetch.go Outdated Show resolved Hide resolved
cmd/oras/internal/option/cache.go Outdated Show resolved Hide resolved
cmd/oras/internal/option/cache_test.go Show resolved Hide resolved
cmd/oras/manifest/fetch.go Outdated Show resolved Hide resolved
if err != nil {
t.Fatal("error calling oci.New(), error =", err)
}
want := cache.New(mockTarget, ociStore)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not doable even if cache.target struct is exported because target.cache is not exported. I think we can keep it this way.

Copy link
Contributor

@shizhMSFT shizhMSFT left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@shizhMSFT shizhMSFT merged commit 6f08e11 into oras-project:main Sep 16, 2022
TerryHowe pushed a commit to TerryHowe/oras that referenced this pull request Feb 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
No open projects
Status: Done
Development

Successfully merging this pull request may close these issues.

Enhance manifet fetch
4 participants