Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for Project Descriptor 0.2 #1241

Merged

Conversation

YousefHaggyHeroku
Copy link
Contributor

@YousefHaggyHeroku YousefHaggyHeroku commented Jul 28, 2021

Summary

Implementing this without breaking the old descriptor parsing. I'm not sure how to approach documentation with a change like this

Output

Before

After

Documentation

  • Should this change be documented?
    • Yes, see #___
    • No

Related

Resolves #1127

@github-actions github-actions bot added this to the 0.20.0 milestone Jul 28, 2021
@github-actions github-actions bot added the type/enhancement Issue that requests a new feature or improvement. label Jul 28, 2021
@YousefHaggyHeroku YousefHaggyHeroku changed the title Draft: Project Descriptor 0.2, versioning project descriptor Draft: Project Descriptor 0.2, supporting different project descriptor schemas Jul 28, 2021
@YousefHaggyHeroku YousefHaggyHeroku force-pushed the yh/reverse-domain-implementation branch 2 times, most recently from 8a751c6 to c8526d3 Compare July 29, 2021 16:29
Update references, t

Signed-off-by: Yousef Haggy <[email protected]>
@@ -37,7 +37,7 @@ import (
"github.com/buildpacks/pack/internal/style"
"github.com/buildpacks/pack/logging"
"github.com/buildpacks/pack/pkg/archive"
"github.com/buildpacks/pack/project"
projectCommon "github.com/buildpacks/pack/project/common"
Copy link
Contributor Author

@YousefHaggyHeroku YousefHaggyHeroku Jul 29, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if there's a better name for this 🤷 , maybe just common? thoughts?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No strong feeling over here!

@YousefHaggyHeroku YousefHaggyHeroku changed the title Draft: Project Descriptor 0.2, supporting different project descriptor schemas Project Descriptor 0.2, supporting different project descriptor schemas Jul 29, 2021
@YousefHaggyHeroku YousefHaggyHeroku marked this pull request as ready for review July 29, 2021 16:43
@YousefHaggyHeroku YousefHaggyHeroku requested a review from a team as a code owner July 29, 2021 16:43
Copy link
Contributor

@aemengo aemengo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great! You're pretty good at this 🙂

--

Also wanted to the highlight that the following windows CI errors aren't related to this PR.

# ...
Setting up auth
  "C:\Program Files\Git\cmd\git.exe" config --local --name-only --get-regexp core\.sshCommand
  "C:\Program Files\Git\cmd\git.exe" submodule foreach --recursive "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :"
  Error: fatal: No url found for submodule path 'internal/registry/registry139553254' in .gitmodules
  Error: The process 'C:\Program Files\Git\cmd\git.exe' failed with exit code 128

@@ -37,7 +37,7 @@ import (
"github.com/buildpacks/pack/internal/style"
"github.com/buildpacks/pack/logging"
"github.com/buildpacks/pack/pkg/archive"
"github.com/buildpacks/pack/project"
projectCommon "github.com/buildpacks/pack/project/common"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No strong feeling over here!

)

type Script struct {
API string `toml:"api"`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are internal only representations right? Meaning these won't be Marshalled(), Unmarshalled() to the filesystem?

If so, then these toml:"..." annotations won't be used.

Copy link
Contributor Author

@YousefHaggyHeroku YousefHaggyHeroku Aug 2, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are meant to be primarily internal representations. However, currently v01/package.go and v02/package.go reference these structs for Unmarshalling some of the unchanged attributes (ex: v02 uses common.Buildpack).

I'm not sure whether we should just copy over the structs and remove struct tags from common.go or just leave as is, I'm comfortable with either.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm comfortable straddling both sides and treating common primarily as the internal representations with the (un)marshalling until there is a deviation between different versions.

}

func ReadProjectDescriptor(pathToFile string) (Descriptor, error) {
func ReadProjectDescriptor(pathToFile string) (common.Descriptor, error) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm personally finding the self-updating structs a little awkward.

Suggested change
func ReadProjectDescriptor(pathToFile string) (common.Descriptor, error) {
func ReadProjectDescriptor(pathToFile string) (common.Descriptor, error) {
pathToFile = filepath.Clean(pathToFile)
var versionDescriptor struct {
Project struct {
Version string `toml:"schema-version"`
} `toml:"_"`
}
_, err := toml.DecodeFile(pathToFile, &versionDescriptor)
if err != nil {
return common.Descriptor{}, errors.Wrapf(err, "parsing schema version")
}
switch versionDescriptor.Project.Version {
// _.schema-version does not exist in 0.1
case "0.1", "":
descriptor, err := v01.NewDescriptor(pathToFile)
if err != nil {
return common.Descriptor{}, err
}
return descriptor, validate(descriptor)
case "0.2":
descriptor, err := v02.NewDescriptor(pathToFile)
if err != nil {
return common.Descriptor{}, err
}
return descriptor, validate(descriptor)
default:
return common.Descriptor{}, fmt.Errorf("unknown project descriptor schema version %s", versionDescriptor.Project.Version)
}
}

Copy link
Contributor Author

@YousefHaggyHeroku YousefHaggyHeroku Aug 2, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this suggestion a lot! I think it looks a lot cleaner and less awkward. The only possible downside I could see is if we have lots of versions in the future, the switch statement could get a little large and redundant. What are your thoughts on the following:

var NewDescriptor = map[string]common.NewDescriptorType{
	"0.1": v01.NewDescriptor,
	"0.2": v02.NewDescriptor,
}

func ReadProjectDescriptor(pathToFile string) (common.Descriptor, error) {
	projectTomlContents, err := ioutil.ReadFile(filepath.Clean(pathToFile))
	if err != nil {
		return common.Descriptor{}, err
	}

	var versionDescriptor struct {
		Project struct {
			Version string `toml:"schema-version"`
		} `toml:"_"`
	}

	_, err = toml.Decode(string(projectTomlContents), &versionDescriptor)
	if err != nil {
		return common.Descriptor{}, errors.Wrapf(err, "parsing schema version")
	}

	version := versionDescriptor.Project.Version
	if version == "" {
		version = "0.1"
	}

	var descriptor common.Descriptor

	if _, ok := NewDescriptor[version]; !ok {
		return common.Descriptor{}, fmt.Errorf("unknown project descriptor schema version %s", version)
	}

	descriptor, err = NewDescriptor[version](projectTomlContents)
	if err != nil {
		return common.Descriptor{}, err
	}

	return descriptor, validate(descriptor)

}

I'm also totally happy to land on your suggestion; I do think it's the cleanest solution for the present. No strong feelings here, just wanted to put out one more thought 😄

Copy link
Contributor

@aemengo aemengo Aug 18, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know why I didn't get a notification. I'm so sorry for missing this! 🙇🏾

Ultimately, I think it's a style preference which is why I have no problem with what you've presented (both of your suggestions). Your code is plenty clear as it is. In my personal projects, large and redundant switch statements are very common; But I understand that it's not for everyone!

Copy link
Member

@jromero jromero Aug 18, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we slacked a bit here @YousefHaggyHeroku I'm going through the process of creating a polish PR for this so we can get this in ASAP. I liked and therefore am taking your latest suggestion into the follow up PR. I'll also take care of the merge conflicts for you.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good! Thanks for making that PR

@jromero jromero modified the milestones: 0.20.0, 0.21.0 Aug 9, 2021
@jromero jromero mentioned this pull request Aug 18, 2021
@jromero jromero changed the title Project Descriptor 0.2, supporting different project descriptor schemas Add support for Project Descriptor 0.2 Aug 18, 2021
jromero added a commit that referenced this pull request Aug 18, 2021
@jromero jromero merged commit 715bfa2 into buildpacks:main Aug 18, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/enhancement Issue that requests a new feature or improvement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

RFC 140: Project Descriptor Domains
4 participants