adding support for public and private git providers

[mike-hoang]

Signed-off-by: Michael Hoang [email protected]

What does this PR do?:

Adds ability to parse Github, GitLab, and Bitbucket urls along with updating util.CloneGitRepo and util.DownloadInMemory to support private repos with personal auth tokens.

Which issue(s) this PR fixes:

fixes devfile/api#984

PR acceptance criteria:

Testing and documentation do not need to be complete in order for this PR to be approved. We just need to ensure tracking issues are opened.

• Unit/Functional tests

• QE Integration test

• Documentation

• Client Impact

• Gosec scans

How to test changes / Special notes to the reviewer:

To set up manual testing:

func main() {
    os.Setenv("GITHUB_TOKEN", <github pat>)
    os.Setenv("GITLAB_TOKEN", <gitlab pat>)
    os.Setenv("BITBUCKET_TOKEN", <bitbucket pat>)

    // private GitHub case
    args := parser.ParserArgs{
        Url: "https://github.com/<private repo>/devfile.yaml",
        Token: os.Getenv("GITHUB_TOKEN"),
    }

    devfile, _, err := devfilepkg.ParseDevfileAndValidate(args)
}

The private repo contains:

<private repo>
├── docker
│   └──  Dockerfile
├── kubernetes 
│   └──  deploy.yaml
└── devfile.yaml

The urls can be both a raw file path or a link to the devfile:

Github
https://github.com/mike-hoang/private-repo-example/blob/main/devfile.yaml
https://raw.githubusercontent.com/mike-hoang/private-repo-example/main/devfile.yaml

Gitlab
https://gitlab.com/mike-hoang/private-testing-repo/-/blob/main/devfile.yaml
https://gitlab.com/mike-hoang/private-testing-repo/-/raw/main/devfile.yaml

Bitbucket
https://bitbucket.org/mike-hoang/private-repo-example/src/main/devfile.yaml
https://bitbucket.org/mike-hoang/private-repo-example/raw/main/devfile.yaml

Running ParseDevfileAndValidate against private GitHub, Gitlab, and Bitbucket repos containing the files structure above should result in the docker/Dockerfile and kubernetes/deploy.yaml downloaded.

[codecov]

Codecov Report

Patch coverage: 47.83% and project coverage change: -2.03 ⚠️

Comparison is base (c1b23d2) 61.28% compared to head (7acc246) 59.25%.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #160      +/-   ##
==========================================
- Coverage   61.28%   59.25%   -2.03%     
==========================================
  Files          37       40       +3     
  Lines        4373     4926     +553     
==========================================
+ Hits         2680     2919     +239     
- Misses       1531     1832     +301     
- Partials      162      175      +13     

Impacted Files	Coverage Δ
pkg/devfile/parser/context/content.go	79.59% <0.00%> (-5.20%)	⬇️
pkg/git/mock.go	0.00% <0.00%> (ø)
pkg/devfile/parser/parse.go	60.62% <9.61%> (-3.39%)	⬇️
pkg/git/util.go	32.29% <32.29%> (ø)
pkg/util/util.go	35.87% <58.06%> (-2.83%)	⬇️
pkg/git/git.go	80.40% <80.40%> (ø)
pkg/devfile/parser/context/context.go	32.46% <100.00%> (+9.17%)	⬆️

... and 1 file with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

[kim-tsao]

Can you update the license headers on all modified files to the current year?

[kim-tsao]

/lgtm

@mike-hoang can you also add the manual test case coverage in the PR description under the How to test changes section? That way, we have it on record what additional coverage and validation was done outside of the unit tests

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: kim-tsao, mike-hoang

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [kim-tsao]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment