Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support git repo caching on package create #785

Conversation

Racer159
Copy link
Contributor

@Racer159 Racer159 commented Sep 26, 2022

Description

Add the ability to locally cache large git repositories when pulling them with Zarf

Related Issue

Fixes #750

Type of change

  • New feature (non-breaking change which adds functionality)

Checklist before merging

  • Tests have been added/updated as necessary (add the needs-tests label)
  • Documentation has been updated as necessary (add the needs-docs label)
  • An ADR has been written as necessary (add the needs-adr label) [ 1 2 3 ]
  • (Optional) Changes have been linted locally with golangci-lint. (NOTE: We haven't turned on lint checks in the pipeline yet so linting may be hard if it shows a lot of lint errors in places that weren't touched by changes. Thus, linting is optional right now.)

Racer159 and others added 30 commits September 22, 2022 10:23
If the 'external' git server still exists within the k8s cluster, create a tunnel when pushing repos to it
general cleanup and oranization of functions in hooks package for zarf agent
Makes sure the svc url comes at the end of the provided URL. This captures edge cases where someone has a subdomain that looks just like a service url would. eg) https://svc.cluster.local.example.com
update GitHub workflows to not explicitly create init-package if it's not needed (if it already exists)
The get-admin-password was a bad name because it wasn't clear what 'admin' meant. We are now making the name more explicit
update the flag git-username to git-account and add a note in the usage text to describe what to do if the repository belongs under an org
verify that the git-user and git-password has been provided if using an external git server
give users the ability to set some of the flags for the GitServerInfo without needing to host an extneral repository
ready check now verifies that the zarf-state has been loaded by verifying the distro field has been populated
instead of essentially hardcoding the container URL to be localhost:31999 we are now loading the containger registry information from the zarf state to get the address of the registry
instead of having a single 'secret' value and expanding that into other secrets for internal resources, we now store all the values necessary within the ZarfState
src/internal/git/utils.go Outdated Show resolved Hide resolved
src/cmd/package.go Outdated Show resolved Hide resolved
src/cmd/package.go Outdated Show resolved Hide resolved
src/cmd/package.go Outdated Show resolved Hide resolved
@Racer159
Copy link
Contributor Author

Racer159 commented Oct 4, 2022

Looks like mac has issues with the given test location chosen

Screen Shot 2022-10-01 at 1 54 57 PM

Switched to using os.TempDir instead of /tmp if that helps?

src/internal/git/pull.go Outdated Show resolved Hide resolved
Copy link
Contributor

@jeff-mccoy jeff-mccoy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🥳 LGTM

@jeff-mccoy jeff-mccoy merged commit c15f371 into master Oct 4, 2022
UncleGedd pushed a commit that referenced this pull request Oct 6, 2022
## Description

Add the ability to locally cache large git repositories when pulling
them with Zarf

## Related Issue

Fixes #750

## Type of change

- [X] New feature (non-breaking change which adds functionality)

## Checklist before merging

- [x] Tests have been added/updated as necessary (add the `needs-tests`
label)
- [x] Documentation has been updated as necessary (add the `needs-docs`
label)
- [x] An ADR has been written as necessary (add the `needs-adr` label) [
[1](https://github.com/joelparkerhenderson/architecture-decision-record)
[2](https://cognitect.com/blog/2011/11/15/documenting-architecture-decisions)
[3](https://adr.github.io/) ]
- [x] (Optional) Changes have been linted locally with
[golangci-lint](https://github.com/golangci/golangci-lint). (NOTE: We
haven't turned on lint checks in the pipeline yet so linting may be hard
if it shows a lot of lint errors in places that weren't touched by
changes. Thus, linting is optional right now.)

Co-authored-by: Jon Perry <[email protected]>
Co-authored-by: Megamind <[email protected]>
@bdfinst bdfinst deleted the 750-support-a-git-cache-directory-similar-to-the-image-cache-directory branch January 12, 2023 22:33
Noxsios pushed a commit that referenced this pull request Mar 8, 2023
## Description

Add the ability to locally cache large git repositories when pulling
them with Zarf

## Related Issue

Fixes #750

## Type of change

- [X] New feature (non-breaking change which adds functionality)

## Checklist before merging

- [x] Tests have been added/updated as necessary (add the `needs-tests`
label)
- [x] Documentation has been updated as necessary (add the `needs-docs`
label)
- [x] An ADR has been written as necessary (add the `needs-adr` label) [
[1](https://github.com/joelparkerhenderson/architecture-decision-record)
[2](https://cognitect.com/blog/2011/11/15/documenting-architecture-decisions)
[3](https://adr.github.io/) ]
- [x] (Optional) Changes have been linted locally with
[golangci-lint](https://github.com/golangci/golangci-lint). (NOTE: We
haven't turned on lint checks in the pipeline yet so linting may be hard
if it shows a lot of lint errors in places that weren't touched by
changes. Thus, linting is optional right now.)

Co-authored-by: Jon Perry <[email protected]>
Co-authored-by: Megamind <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support a git cache directory, similar to the image cache directory
3 participants