Make Zarf a Library (#934)

This PR is a major refactor of the zarf code base primarily designed around better structure to support the current Zarf API as well as expose some key portions of Zarf as an SDK for other tools to consume. This major refactor came out of growing API needs as well as needs in #892 and the [zarf-controller](https://github.com/defenseunicorns/zarf-controller) prototype. Major changes: - Migrate all global configs used by the packager into [receiver methods](https://medium.com/globant/go-method-receiver-pointer-vs-value-ffc5ab7acdb) to avoid concurrency issues - Migrate from heavy `fatal` use to conventional error bubbling / wrapping - Expose core packager code as `github.com/defenseunicorns/zarf/src/pkg/packager` - Split the k8s package into `github.com/defenseunicorns/zarf/src/pkg/k8s` (external) and `src/internal/cluster` (internal) - Expose `message`, `pki`, and `utils` as `github.com/defenseunicorns/zarf/src/pkg/*` - Move the remaining `src/internal/*` packages (except for `agent` & `api`) into `src/internal/packager/*` - Change K8s connections to a reusable connection, except for `cluster.Tunnel` - Switch to Revive for linting - Introduce language file and establish basic patterns, related to #631 - Add SPDX headers to all files - Begin work to standardize on the [Uber Go Style Guide](https://github.com/uber-go/guide/blob/master/style.md) - Address a lot of lint issues - Return confirm yaml display to only show populated values Co-authored-by: Jon Perry <[email protected]> Co-authored-by: Wayne Starr <[email protected]>
zarf-dev · Nov 30, 2022 · a3d8c1f · a3d8c1f
1 parent 9bec658
commit a3d8c1f
Show file tree

Hide file tree

Showing 230 changed files with 5,524 additions and 4,409 deletions.
diff --git a/.github/codeql.yaml b/.github/codeql.yaml
@@ -0,0 +1,6 @@
+paths-ignore:
+  - src/pkg/packager/network.go
+
+query-filters:
+  - exclude:
+      id: go/path-injection
diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md
@@ -25,4 +25,3 @@ Fixes # (issue)
 - [ ] Tests have been added/updated as necessary (add the `needs-tests` label)
 - [ ] Documentation has been updated as necessary (add the `needs-docs` label)
 - [ ] An ADR has been written as necessary (add the `needs-adr` label) [ [1](https://github.com/joelparkerhenderson/architecture-decision-record) [2](https://cognitect.com/blog/2011/11/15/documenting-architecture-decisions) [3](https://adr.github.io/) ]
-- [ ] (Optional) Changes have been linted locally with [golangci-lint](https://github.com/golangci/golangci-lint). (NOTE: We haven't turned on lint checks in the pipeline yet so linting may be hard if it shows a lot of lint errors in places that weren't touched by changes. Thus, linting is optional right now.)
diff --git a/.github/workflows/build-rust-injector.yml b/.github/workflows/build-rust-injector.yml
@@ -8,8 +8,7 @@ on:
         required: true
       branchName:
         description: "Branch to build the injector from"
-        required: false
-        default: "master"
+        required: true
 
 jobs:
   build-injector:

diff --git a/.github/workflows/scan-codeql.yml b/.github/workflows/scan-codeql.yml
@@ -2,7 +2,7 @@ name: Analyze CodeQL
 
 on:
   push:
-    branches: ["master"]
+    branches: ["main"]
   pull_request:
     paths-ignore:
       - "**.md"
@@ -40,22 +40,20 @@ jobs:
       - name: Setup golang
         uses: ./.github/actions/golang
 
-      - name: Build
-        run: make build-cli-linux-amd
-
       # Initializes the CodeQL tools for scanning.
       - name: Initialize CodeQL
         uses: github/codeql-action/init@v2
+        env:
+          CODEQL_EXTRACTOR_GO_BUILD_TRACING: on
         with:
           languages: ${{ matrix.language }}
-          # If you wish to specify custom queries, you can do so here or in a config file.
-          # By default, queries listed here will override any specified in a config file.
-          # Prefix the list here with "+" to use these queries and those in the config file.
+          config-file: ./.github/codeql.yaml
 
-          # Details on CodeQL's query packs refer to : https://docs.github.com/en/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/configuring-code-scanning#using-queries-in-ql-packs
-          # queries: security-extended,security-and-quality
+      - name: Build
+        run: make build-cli-linux-amd
 
       - name: Perform CodeQL Analysis
         uses: github/codeql-action/analyze@v2
         with:
           category: "/language:${{matrix.language}}"
+
diff --git a/.github/workflows/scan-lint.yml b/.github/workflows/scan-lint.yml
@@ -0,0 +1,18 @@
+name: Validate Lint
+on: pull_request
+
+jobs:
+  validate:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v3
+
+      - name: Run Revive Action by pulling pre-built image
+        uses: docker://morphy/revive-action:v2
+        with:
+          config: revive.toml
+          # Exclude patterns, separated by semicolons (optional)
+          exclude: "src/cmd/viper.go"
+          # Path pattern (default: ./...)
+          path: "./src/..."
diff --git a/.golangci.yml b/.golangci.yml
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
@@ -28,7 +28,6 @@ repos:
     rev: v0.4.0
     hooks:
       - id: go-fmt
-      - id: golangci-lint
   - repo: local
     hooks:
       - id: generate-zarf-schema

diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -49,7 +49,6 @@ In this repo we use [pre-commit](https://pre-commit.com/) hooks for automated va
 
 1. Install [pre-commit](https://pre-commit.com/)
 1. Install [go](https://golang.org/)
-1. Install [golangci-lint](https://github.com/golangci/golangci-lint)
 1. Run `pre-commit install` in the repo to install the pre-commit hooks. This will make the hooks run automatically each time you `git commit`. If you want to skip the hooks for any reason you can run `git commit --no-verify` to skip them.
 
 > ℹ️ **HINT:** *Consider [automatically enabling the hooks in every Git repository](https://pre-commit.com/#automatically-enabling-pre-commit-on-repositories)*

diff --git a/Makefile b/Makefile
@@ -1,3 +1,6 @@
+# SPDX-License-Identifier: Apache-2.0
+# SPDX-FileCopyrightText: 2021-Present The Zarf Authors
+
 # Provide a default value for the operating system architecture used in tests, e.g. " APPLIANCE_MODE=true|false make test-e2e ARCH=arm64"
 ARCH ?= amd64
 # The image tag used for the zarf agent, defaults to a dev image tag
@@ -173,3 +176,6 @@ test-cves: ensure-ui-build-dir
 
 cve-report: ensure-ui-build-dir
 	go run main.go tools sbom packages . -o json | grype -o template -t .hooks/grype.tmpl > build/zarf-known-cves.csv
+
+lint-go:
+	revive -config revive.toml -exclude src/cmd/viper.go -formatter stylish ./src/...
diff --git a/docs/4-user-guide/1-the-zarf-cli/100-cli-commands/zarf.md b/docs/4-user-guide/1-the-zarf-cli/100-cli-commands/zarf.md
@@ -1,6 +1,11 @@
 ## zarf
 
-DevSecOps Airgap Toolkit
+DevSecOps for Airgap
+
+### Synopsis
+
+Zarf eliminates the complexity of air gap software delivery for Kubernetes clusters and cloud native workloads
+using a declarative packaging strategy to support DevSecOps in offline and semi-connected environments.
 
 ```
 zarf [COMMAND] [flags]
@@ -27,5 +32,5 @@ zarf [COMMAND] [flags]
 * [zarf package](zarf_package.md)	 - Zarf package commands for creating, deploying, and inspecting packages
 * [zarf prepare](zarf_prepare.md)	 - Tools to help prepare assets for packaging
 * [zarf tools](zarf_tools.md)	 - Collection of additional tools to make airgap easier
-* [zarf version](zarf_version.md)	 - Displays the version of the Zarf binary
+* [zarf version](zarf_version.md)	 - SBOM tools provided by Anchore Syft
 
diff --git a/docs/4-user-guide/1-the-zarf-cli/100-cli-commands/zarf_completion.md b/docs/4-user-guide/1-the-zarf-cli/100-cli-commands/zarf_completion.md
@@ -27,7 +27,7 @@ See each sub-command's help for details on how to use the generated script.
 
 ### SEE ALSO
 
-* [zarf](zarf.md)	 - DevSecOps Airgap Toolkit
+* [zarf](zarf.md)	 - DevSecOps for Airgap
 * [zarf completion bash](zarf_completion_bash.md)	 - Generate the autocompletion script for bash
 * [zarf completion fish](zarf_completion_fish.md)	 - Generate the autocompletion script for fish
 * [zarf completion powershell](zarf_completion_powershell.md)	 - Generate the autocompletion script for powershell

diff --git a/docs/4-user-guide/1-the-zarf-cli/100-cli-commands/zarf_connect.md b/docs/4-user-guide/1-the-zarf-cli/100-cli-commands/zarf_connect.md
@@ -41,6 +41,6 @@ zarf connect {REGISTRY|LOGGING|GIT|connect-name} [flags]
 
 ### SEE ALSO
 
-* [zarf](zarf.md)	 - DevSecOps Airgap Toolkit
+* [zarf](zarf.md)	 - DevSecOps for Airgap
 * [zarf connect list](zarf_connect_list.md)	 - List all available connection shortcuts.
 
diff --git a/docs/4-user-guide/1-the-zarf-cli/100-cli-commands/zarf_destroy.md b/docs/4-user-guide/1-the-zarf-cli/100-cli-commands/zarf_destroy.md
@@ -37,5 +37,5 @@ zarf destroy [flags]
 
 ### SEE ALSO
 
-* [zarf](zarf.md)	 - DevSecOps Airgap Toolkit
+* [zarf](zarf.md)	 - DevSecOps for Airgap
 
diff --git a/docs/4-user-guide/1-the-zarf-cli/100-cli-commands/zarf_init.md b/docs/4-user-guide/1-the-zarf-cli/100-cli-commands/zarf_init.md
@@ -39,7 +39,7 @@ zarf init [flags]
 ### Options
 
 ```
-      --components string               Comma-separated list of components to install.
+      --components string               Specify which optional components to install.  E.g. --components=git-server,logging
       --confirm                         Confirm the install without prompting
       --git-pull-password string        Password for the pull-only user to access the git server
       --git-pull-username string        Username for pull-only access to the git server
@@ -54,7 +54,7 @@ zarf init [flags]
       --registry-push-username string   Username to access to the registry Zarf is configured to use (default "zarf-push")
       --registry-secret string          Registry secret value
       --registry-url string             External registry url address to use for this Zarf cluster
-      --storage-class string            Describe the StorageClass to be used
+      --storage-class string            Specify the storage class to use for the registry.  E.g. --storage-class=standard
 ```
 
 ### Options inherited from parent commands
@@ -70,5 +70,5 @@ zarf init [flags]
 
 ### SEE ALSO
 
-* [zarf](zarf.md)	 - DevSecOps Airgap Toolkit
+* [zarf](zarf.md)	 - DevSecOps for Airgap
 
diff --git a/docs/4-user-guide/1-the-zarf-cli/100-cli-commands/zarf_package.md b/docs/4-user-guide/1-the-zarf-cli/100-cli-commands/zarf_package.md
@@ -21,7 +21,7 @@ Zarf package commands for creating, deploying, and inspecting packages
 
 ### SEE ALSO
 
-* [zarf](zarf.md)	 - DevSecOps Airgap Toolkit
+* [zarf](zarf.md)	 - DevSecOps for Airgap
 * [zarf package create](zarf_package_create.md)	 - Use to create a Zarf package from a given directory or the current directory
 * [zarf package deploy](zarf_package_deploy.md)	 - Use to deploy a Zarf package from a local file or URL (runs offline)
 * [zarf package inspect](zarf_package_inspect.md)	 - Lists the payload of a Zarf package (runs offline)

diff --git a/docs/4-user-guide/1-the-zarf-cli/100-cli-commands/zarf_prepare.md b/docs/4-user-guide/1-the-zarf-cli/100-cli-commands/zarf_prepare.md
@@ -21,7 +21,7 @@ Tools to help prepare assets for packaging
 
 ### SEE ALSO
 
-* [zarf](zarf.md)	 - DevSecOps Airgap Toolkit
+* [zarf](zarf.md)	 - DevSecOps for Airgap
 * [zarf prepare find-images](zarf_prepare_find-images.md)	 - Evaluates components in a zarf file to identify images specified in their helm charts and manifests
 * [zarf prepare generate-config](zarf_prepare_generate-config.md)	 - Generates a config file for Zarf
 * [zarf prepare patch-git](zarf_prepare_patch-git.md)	 - Converts all .git URLs to the specified Zarf HOST and with the Zarf URL pattern in a given FILE.  NOTE: 

diff --git a/docs/4-user-guide/1-the-zarf-cli/100-cli-commands/zarf_tools.md b/docs/4-user-guide/1-the-zarf-cli/100-cli-commands/zarf_tools.md
@@ -21,12 +21,12 @@ Collection of additional tools to make airgap easier
 
 ### SEE ALSO
 
-* [zarf](zarf.md)	 - DevSecOps Airgap Toolkit
-* [zarf tools archiver](zarf_tools_archiver.md)	 - Compress/Decompress tools for Zarf packages
-* [zarf tools clear-cache](zarf_tools_clear-cache.md)	 - Clears the configured git and image cache directory
+* [zarf](zarf.md)	 - DevSecOps for Airgap
+* [zarf tools archiver](zarf_tools_archiver.md)	 - Compress/Decompress generic archives, including Zarf packages.
+* [zarf tools clear-cache](zarf_tools_clear-cache.md)	 - Clears the configured git and image cache directory.
 * [zarf tools gen-pki](zarf_tools_gen-pki.md)	 - Generates a Certificate Authority and PKI chain of trust for the given host
 * [zarf tools get-git-password](zarf_tools_get-git-password.md)	 - Returns the push user's password for the Git server
-* [zarf tools monitor](zarf_tools_monitor.md)	 - Launch K9s tool for managing K8s clusters
-* [zarf tools registry](zarf_tools_registry.md)	 - Collection of registry commands provided by Crane
-* [zarf tools sbom](zarf_tools_sbom.md)	 - SBOM tools provided by Anchore Syft
+* [zarf tools monitor](zarf_tools_monitor.md)	 - Launch a terminal UI to monitor the connected cluster using K9s.
+* [zarf tools registry](zarf_tools_registry.md)	 - Tools for working with container registries using go-containertools.
+* [zarf tools sbom](zarf_tools_sbom.md)	 - Generates a Software Bill of Materials (SBOM) for the given package
 
diff --git a/docs/4-user-guide/1-the-zarf-cli/100-cli-commands/zarf_tools_archiver.md b/docs/4-user-guide/1-the-zarf-cli/100-cli-commands/zarf_tools_archiver.md
@@ -1,6 +1,6 @@
 ## zarf tools archiver
 
-Compress/Decompress tools for Zarf packages
+Compress/Decompress generic archives, including Zarf packages.
 
 ### Options
 
@@ -22,6 +22,6 @@ Compress/Decompress tools for Zarf packages
 ### SEE ALSO
 
 * [zarf tools](zarf_tools.md)	 - Collection of additional tools to make airgap easier
-* [zarf tools archiver compress](zarf_tools_archiver_compress.md)	 - Compress a collection of sources based off of the destination file extension
-* [zarf tools archiver decompress](zarf_tools_archiver_decompress.md)	 - Decompress an archive (package) to a specified location
+* [zarf tools archiver compress](zarf_tools_archiver_compress.md)	 - Compress a collection of sources based off of the destination file extension.
+* [zarf tools archiver decompress](zarf_tools_archiver_decompress.md)	 - Decompress an archive or Zarf package based off of the source file extension.
 
diff --git a/docs/4-user-guide/1-the-zarf-cli/100-cli-commands/zarf_tools_archiver_compress.md b/docs/4-user-guide/1-the-zarf-cli/100-cli-commands/zarf_tools_archiver_compress.md
@@ -1,6 +1,6 @@
 ## zarf tools archiver compress
 
-Compress a collection of sources based off of the destination file extension
+Compress a collection of sources based off of the destination file extension.
 
 ```
 zarf tools archiver compress {SOURCES} {ARCHIVE} [flags]
@@ -25,5 +25,5 @@ zarf tools archiver compress {SOURCES} {ARCHIVE} [flags]
 
 ### SEE ALSO
 
-* [zarf tools archiver](zarf_tools_archiver.md)	 - Compress/Decompress tools for Zarf packages
+* [zarf tools archiver](zarf_tools_archiver.md)	 - Compress/Decompress generic archives, including Zarf packages.
 
diff --git a/.../4-user-guide/1-the-zarf-cli/100-cli-commands/zarf_tools_archiver_decompress.md b/.../4-user-guide/1-the-zarf-cli/100-cli-commands/zarf_tools_archiver_decompress.md
@@ -1,6 +1,6 @@
 ## zarf tools archiver decompress
 
-Decompress an archive (package) to a specified location
+Decompress an archive or Zarf package based off of the source file extension.
 
 ```
 zarf tools archiver decompress {ARCHIVE} {DESTINATION} [flags]
@@ -25,5 +25,5 @@ zarf tools archiver decompress {ARCHIVE} {DESTINATION} [flags]
 
 ### SEE ALSO
 
-* [zarf tools archiver](zarf_tools_archiver.md)	 - Compress/Decompress tools for Zarf packages
+* [zarf tools archiver](zarf_tools_archiver.md)	 - Compress/Decompress generic archives, including Zarf packages.
 
diff --git a/docs/4-user-guide/1-the-zarf-cli/100-cli-commands/zarf_tools_clear-cache.md b/docs/4-user-guide/1-the-zarf-cli/100-cli-commands/zarf_tools_clear-cache.md
@@ -1,6 +1,6 @@
 ## zarf tools clear-cache
 
-Clears the configured git and image cache directory
+Clears the configured git and image cache directory.
 
 ```
 zarf tools clear-cache [flags]

diff --git a/docs/4-user-guide/1-the-zarf-cli/100-cli-commands/zarf_tools_monitor.md b/docs/4-user-guide/1-the-zarf-cli/100-cli-commands/zarf_tools_monitor.md
@@ -1,6 +1,6 @@
 ## zarf tools monitor
 
-Launch K9s tool for managing K8s clusters
+Launch a terminal UI to monitor the connected cluster using K9s.
 
 ```
 zarf tools monitor [flags]

diff --git a/docs/4-user-guide/1-the-zarf-cli/100-cli-commands/zarf_tools_registry.md b/docs/4-user-guide/1-the-zarf-cli/100-cli-commands/zarf_tools_registry.md
@@ -1,6 +1,6 @@
 ## zarf tools registry
 
-Collection of registry commands provided by Crane
+Tools for working with container registries using go-containertools.
 
 ### Options
 

diff --git a/docs/4-user-guide/1-the-zarf-cli/100-cli-commands/zarf_tools_registry_catalog.md b/docs/4-user-guide/1-the-zarf-cli/100-cli-commands/zarf_tools_registry_catalog.md
@@ -3,7 +3,7 @@
 List the repos in a registry
 
 ```
-zarf tools registry catalog [flags]
+zarf tools registry catalog [REGISTRY] [flags]
 ```
 
 ### Options
@@ -25,5 +25,5 @@ zarf tools registry catalog [flags]
 
 ### SEE ALSO
 
-* [zarf tools registry](zarf_tools_registry.md)	 - Collection of registry commands provided by Crane
+* [zarf tools registry](zarf_tools_registry.md)	 - Tools for working with container registries using go-containertools.
 
diff --git a/docs/4-user-guide/1-the-zarf-cli/100-cli-commands/zarf_tools_registry_copy.md b/docs/4-user-guide/1-the-zarf-cli/100-cli-commands/zarf_tools_registry_copy.md
@@ -25,5 +25,5 @@ zarf tools registry copy SRC DST [flags]
 
 ### SEE ALSO
 
-* [zarf tools registry](zarf_tools_registry.md)	 - Collection of registry commands provided by Crane
+* [zarf tools registry](zarf_tools_registry.md)	 - Tools for working with container registries using go-containertools.
 
diff --git a/docs/4-user-guide/1-the-zarf-cli/100-cli-commands/zarf_tools_registry_login.md b/docs/4-user-guide/1-the-zarf-cli/100-cli-commands/zarf_tools_registry_login.md
@@ -28,5 +28,5 @@ zarf tools registry login [OPTIONS] [SERVER] [flags]
 
 ### SEE ALSO
 
-* [zarf tools registry](zarf_tools_registry.md)	 - Collection of registry commands provided by Crane
+* [zarf tools registry](zarf_tools_registry.md)	 - Tools for working with container registries using go-containertools.
 
diff --git a/docs/4-user-guide/1-the-zarf-cli/100-cli-commands/zarf_tools_registry_pull.md b/docs/4-user-guide/1-the-zarf-cli/100-cli-commands/zarf_tools_registry_pull.md
@@ -28,5 +28,5 @@ zarf tools registry pull IMAGE TARBALL [flags]
 
 ### SEE ALSO
 
-* [zarf tools registry](zarf_tools_registry.md)	 - Collection of registry commands provided by Crane
+* [zarf tools registry](zarf_tools_registry.md)	 - Tools for working with container registries using go-containertools.
 
diff --git a/docs/4-user-guide/1-the-zarf-cli/100-cli-commands/zarf_tools_registry_push.md b/docs/4-user-guide/1-the-zarf-cli/100-cli-commands/zarf_tools_registry_push.md
@@ -31,5 +31,5 @@ zarf tools registry push PATH IMAGE [flags]
 
 ### SEE ALSO
 
-* [zarf tools registry](zarf_tools_registry.md)	 - Collection of registry commands provided by Crane
+* [zarf tools registry](zarf_tools_registry.md)	 - Tools for working with container registries using go-containertools.