Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update readme content #672

Merged
merged 32 commits into from
Aug 22, 2022
Merged

Update readme content #672

Changes from 9 commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
5eade9e
Initial README structure
Racer159 Aug 14, 2022
86c85fb
Refine README content
Racer159 Aug 14, 2022
7ae40df
Copy updates
Racer159 Aug 14, 2022
a963718
Drop systems
Racer159 Aug 14, 2022
f803efb
Refine NPS wording
Racer159 Aug 14, 2022
d6876ae
Refine badges
Racer159 Aug 15, 2022
8a205f8
Fix badges
Racer159 Aug 15, 2022
b84229b
Fix go version
Racer159 Aug 15, 2022
6f35fb5
Copy type updates
Racer159 Aug 15, 2022
e840ca6
Add changes per feedback
Racer159 Aug 15, 2022
ae20bda
Update readme text and add build badge per feedback
Racer159 Aug 16, 2022
e620f25
Add flair to the subnote
Racer159 Aug 16, 2022
f27f7b6
Get rid of the fold
Racer159 Aug 16, 2022
29a90e6
Get rid of another fold
Racer159 Aug 16, 2022
d05ba11
Quick test
Racer159 Aug 16, 2022
cd4c731
Revert test
Racer159 Aug 16, 2022
1026405
Dark mode support for the diagram
Racer159 Aug 17, 2022
bb4eee4
Fix some diagram spacing
Racer159 Aug 17, 2022
3763b6d
Perfect the spacing
Racer159 Aug 17, 2022
bcce682
Perfect the arrow
Racer159 Aug 17, 2022
f68e672
Actually perfect the arrow
Racer159 Aug 17, 2022
1702963
Fix error rendering at certain sizes
Racer159 Aug 17, 2022
a272038
Fix contrast
Racer159 Aug 17, 2022
3f4244d
Render text better
Racer159 Aug 17, 2022
cf64265
Fix line spacing
Racer159 Aug 17, 2022
27469e1
Merge branch 'master' into update-readme-content
jeff-mccoy Aug 20, 2022
395f000
readme tweaks & video update
jeff-mccoy Aug 22, 2022
5add684
better video
jeff-mccoy Aug 22, 2022
12b1a00
Merge branch 'master' into update-readme-content
jeff-mccoy Aug 22, 2022
45a7838
Merge branch 'master' into update-readme-content
jeff-mccoy Aug 22, 2022
e78b344
Merge branch 'master' into update-readme-content
jeff-mccoy Aug 22, 2022
cd46e7e
Fix small gramatical error
Racer159 Aug 22, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
312 changes: 26 additions & 286 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,306 +1,46 @@
# Zarf - DevSecOps for Air Gap Systems
# Zarf - DevSecOps for Air Gap

[![Latest Release](https://img.shields.io/github/v/release/defenseunicorns/zarf)](https://github.com/defenseunicorns/zarf/releases)
[![Zarf Slack Channel](https://img.shields.io/badge/k8s%20slack-zarf-6d87c3)](https://kubernetes.slack.com/archives/C03B6BJAUJ3)
[![Zarf Documentation](https://img.shields.io/badge/web-zarf.dev-775ba1)](https://zarf.dev/)
YrrepNoj marked this conversation as resolved.
Show resolved Hide resolved
[![Go version](https://img.shields.io/github/go-mod/go-version/defenseunicorns/zarf?filename=go.mod)](https://go.dev/)

<img align="right" alt="zarf logo" src=".images/zarf-logo.png" height="256" />

Zarf simplifies the setup & administration of Kubernetes clusters, cyber systems & workloads that support DevSecOps "across the [air gap](https://en.wikipedia.org/wiki/Air_gap_(networking))."

It provides a static go binary (can run anywhere) CLI that can pull, package, and install all the things your clusters need to run and any necessary resources to standup infrastructure such as Terraform. Zarf also caches downloads (for speed), hashes packages (for security), and can _even install the Kubernetes cluster itself_ if you needed.

Zarf runs on [a bunch of operating systems](./docs/supported-oses.md). It aims to support configurations ranging from "I want to run one, simple app" to "I need to support & dependency control a _bunch_ of internet-disconnected clusters."
It provides a static go binary (can run anywhere) CLI that can pull, package, and install everything a cluster needs to run alongside any necessary infrastructure resources like Terraform. Zarf also caches downloads (for speed), hashes packages (for security), and can _even install the Kubernetes cluster itself_ if you needed.

Zarf was theorized and initially demonstrated in Naval Postgraduate School research:
Zarf runs on [a bunch of operating systems](./docs/supported-oses.md), and aims to support configurations ranging from "I want to run one, simple app" to "I need to support & dependency control a _bunch_ of internet-disconnected clusters."

[A DEVSECOPS APPROACH FOR DEVELOPING AND DEPLOYING CONTAINERIZED CLOUD-BASED SOFTWARE ON SUBMARINES](https://calhoun.nps.edu/handle/10945/68688).
Zarf was initially theorized and demonstrated in research from Naval Postgraduate School which you can read [here](https://calhoun.nps.edu/handle/10945/68688).

## Demo

[![asciicast](https://asciinema.org/a/475530.svg)](https://asciinema.org/a/475530)

## Why is Zarf Needed?
Most of the software ecosystem assumes your systems have access to the internet. The world (for good reasons) has become more and more dependent upon Software as a Service (SaaS), which assumes a robust connection to the internet and a willingness to inherently trust 3rd party providers. Although this makes sense for most of the world, certain secure systems must operate either fully disconnected, semi-disconnected, or might need the ability to disconnect in case of emergencies (like while under an active cyber attack). Although only a small percentage of systems, these secure systems make up some of the most vital systems globally, such as Aerospace and Defense, Finance, Healthcare, Energy, Water, Sewage, and many Federal, Local, and State Government systems.

## Explain Zarf Like I'm Ten(ish)
jeff-mccoy marked this conversation as resolved.
Show resolved Hide resolved

Zarf allows you to bundle portions of "the internet" into a single package to be installed later following specific instructions. A Zarf package is just a single file that includes everything you need to manage a system or capability while fully disconnected. Think of a disconnected system as a system that always is or sometimes is on airplane mode.

You bring this single file (we call it a package) to the system you want to install or update new software on. The package includes instructions on how to assemble all the pieces of software (components) once on the other side. These instructions are fully "declarative," which means that everything is represented by code and automated instead of manual. The hardest part is assembling the declarative package on the connected side. But once everything is packaged, Zarf makes even very complex systems easy to install, update, and maintain within disconnected systems.

Such packages also become highly distributable, as they can now run on edge, embedded systems, secure cloud, data centers, or even in a local environment. This is incredibly helpful for organizations that need to integrate and deploy software from multiple, secure development environments from disparate development teams into disconnected IT operational environments. Zarf helps ensure that development teams can integrate with the production environment they are deploying to, even if they will never actually touch that environment.

Zarf makes DevSecOps for air gap possible.

<!--
##########
# This block is about LEARNING TO USE Zarf
##########
-->
## If you're *just getting into Zarf*, you should...

<table>
<tbody>

<!-- row start: cuz markdown hates html indention -->
<tr valign="top">
<td width="150">

**Get Started**

_Using Zarf_

</td>
<td>

Experience just how easy it is to go from _**zero** to **chainsaw-wielding hero** (of the Kubernetes cluster)_ using Zarf!

</td>
<td>

[Read](./examples/game/)

</td>
</tr>
<!-- row end -->

<!-- row start -->
<tr valign="top">
<td>

**Add Logging**

_Zarf components_

</td>
<td>

Sometimes running your app in-cluster is enough&mdash; usually, it's not. Find out how to inject commonly-required, uber-useful functionality through Zarf components.

</td>
<td>

[Read](./examples/game/add-logging.md)

</td>
</tr>
<!-- row end -->

<!-- row start -->
<tr valign="top">
<td>

**Join Slack Channels**

</td>
<td>

Join our project channels on K8s slack. <a href="https://kubernetes.slack.com/archives/C03B6BJAUJ3">#zarf</a> <a href="https://kubernetes.slack.com/archives/C03BP9Z3CMA">#zarf-dev</a>


</td>
<td>

https://slack.k8s.io/

</td>
</tr>
<!-- row end -->
</tbody>
</table>

<!--
##########
# This block is about DEVELOPING Zarf
##########
-->
## If you'd rather *help develop Zarf*, you should read...

<table>
<tbody>

<!-- row start: cuz markdown hates html indention -->
<tr valign="top">
<td width="150">

**Workstation Setup**

</td>
<td>

Are you thinking about hacking on the Zarf binary itself? Or, perhaps you want to run the examples in an isolated environment (the same way we do)? Then, get your machine setup _just right_ using these instructions!

</td>
<td>

[Read](./docs/workstation.md)

</td>
</tr>
<!-- row end -->

<!-- row start -->
<tr valign="top">
<td>

**Build Your First Zarf**

</td>
<td>

You've got a development workstation setup, so... now what? Why not _build your own Zarf_? Step-by-step instructions here.
## Docs

</td>
<td>
To learn more about Zarf and its use cases visit [docs.zarf.dev](https://docs.zarf.dev/docs/zarf-overview).

[Read](./docs/first-time-build.md)
From there you can learn about [installation](https://docs.zarf.dev/docs/operator-manual/set-up-and-install), [using the CLI](https://docs.zarf.dev/docs/user-guide/the-zarf-cli/), [making packages](https://docs.zarf.dev/docs/user-guide/zarf-packages/), and the [Zarf package schema](https://docs.zarf.dev/docs/user-guide/zarf-schema).

</td>
</tr>
<!-- row end -->
## Developing

<!-- row start -->
<tr valign="top">
<td>
To contribute, please see our [Contributor Guide](https://docs.zarf.dev/docs/developer-guide/contributor-guide). Below is an architectural diagram showing the basics of how Zarf functions which you can read more about [here](https://docs.zarf.dev/docs/developer-guide/nerd-notes).

**Contribution Guide**

</td>
<td>

As with most collaborative efforts, there are guidelines for contributing to the project. Find out what they are & how to make them work here.

</td>
<td>

[Read](./CONTRIBUTING.md)

</td>
</tr>
<!-- row end -->

</tbody>
</table>

<!--
##########
# This block is about the MINUTIA & UNDERSTANDING WHY Zarf is the way it is
##########
-->
## Or, for *details & design decisions*, check out...

<table>
<tbody>

<!-- row start: cuz markdown hates html indention -->
<tr valign="top">
<td width="150">

**Supported OSes**

</td>
<td>

Zarf is intended to run on a variety of Operating Systems&mdash; you can find out which _and_ discover how to take Zarf for a test drive (in a VM of your favorite flavor) by clicking the link!

</td>
<td>

[Read](./docs/supported-oses.md)

</td>
</tr>
<!-- row end -->

<!-- row start -->
<tr valign="top">
<td>

**Zarf Components**

</td>
<td>

Need to understand what's happening in your cluster? Zarf can give you visibility by injecting a _Logging_ component.

Find out all the other stuff Zarf offers here.

</td>
<td>

[Read](./docs/components.md)

</td>
</tr>
<!-- row end -->

<!-- row start -->
<tr valign="top">
<td>

**Usage Examples**

</td>
<td>

There are a bunch of interesting ways to use Zarf beyond "space-marine-ing" your way through _the_ pixel demon invasion. Browse our examples directory to find out what other neat things are available.

</td>
<td>

[Read](./examples)

</td>
</tr>
<!-- row end -->

<!-- row start -->
<tr valign="top">
<td>

**Iron Bank** <br/>

_Hardened image registry_

</td>
<td>

Zarf can build deployment packages using images pulled from pretty much anywhere...but for gov't-approved & "[hardened](https://en.wikipedia.org/wiki/Hardening_(computing))" container images, check out Platform One's [Iron Bank](https://p1.dso.mil/#/products/iron-bank/).

</td>
<td>

[Read](./docs/ironbank.md)

</td>
</tr>
<!-- row end -->

</tbody>
</table>


&nbsp;
![Architecture Diagram](./docs/architecture.drawio.svg)

[Source DrawIO](docs/architecture.drawio.svg)

## Zarf Nerd Notes
## Special Thanks

Zarf is written entirely in [go](https://go.dev/), except for a single 400Kb binary for the injector system written in [rust](https://www.rust-lang.org/), so we can fit it in a [configmap](https://kubernetes.io/docs/concepts/configuration/configmap/). All assets are bundled together into a single [zstd](https://facebook.github.io/zstd/) tarball on each `zarf package create` operation. On the air gap / offline side, `zarf package deploy` extracts the various assets and places them on the filesystem or installs them in the cluster, depending on what the [zarf package](Zarf.yaml) says to do. Some important ideas behind Zarf:
We would also like to thank the following awesome libraries and projects without which Zarf would not be possible!

- All workloads are installed in the cluster via the [Helm SDK](https://helm.sh/docs/topics/advanced/#go-sdk)
- The OCI Registries used are both from [Docker](https://github.com/distribution/distribution)
- Currently, the Registry and Git servers _are not HA_, see [#375](https://github.com/defenseunicorns/zarf/issues/376) and [#376](https://github.com/defenseunicorns/zarf/issues/376) for discussion on this
- To avoid TLS issues, Zarf binds to `127.0.0.1:31999` on each node as a [NodePort](https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport) to allow all nodes to access the pod(s) in the cluster
- A [mutating webhook](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/) we call the Zarf Agent handles updating pod's [ImagePullSecrets](https://kubernetes.io/docs/concepts/containers/images/#specifying-imagepullsecrets-on-a-pod) so the deployed resources use the NodePort binding
- Note: Existing namespaces in the cluster prior to `zarf init` are marked as ignored by the Zarf Agent so you should not use them with Zarf-deployed resources. Most common would be the `default` or `kube-system` namespaces
- Zarf uses a custom injector system to bootstrap a new cluster. See the PR [#329](https://github.com/defenseunicorns/zarf/pull/329) and [ADR](docs/adr/0003-image-injection-into-remote-clusters-without-native-support.md) for more details on how we came to this solution. The general steps are listed below:
- Get a list of images in the cluster
- Attempt to create an ephemeral pod using an image from the list
- A small rust binary that is compiled using [musl](https://www.musl-libc.org/) to keep the size the max binary size of ~ 672 KBs is injected into the pod
- The mini zarf registry binary and `docker:2` images are put in a tar archive and split into 512 KB chunks; larger sizes tended to cause latency issues on low-resource control planes
- An init container runs the rust binary to re-assemble and extract the zarf binary and registry image
- The container then starts and runs the zarf binary to host the registry image in an embedded docker registry
- After this, the main docker registry chart is deployed, pulls the image from the ephemeral pod, and finally destroys the created configmaps, pod, and service
- We have published a [Homebrew Tap](https://github.com/defenseunicorns/homebrew-tap) for this project that you can install with `brew tap defenseunicorns/tap && brew install zarf`
- This recipe is updated with each new version release via [GoReleaser](https://goreleaser.com/customization/homebrew/) as part of the release pipeline that runs on GitHub Actions.
&nbsp;
### Zarf Architecture
![Architecture Diagram](./docs/architecture.drawio.svg)

[Source DrawIO](docs/architecture.drawio.svg)
[![pterm/pterm](https://img.shields.io/badge/pterm%2Fpterm-007d9c?logo=go&logoColor=white)](https://github.com/pterm/pterm)
[![mholt/archiver](https://img.shields.io/badge/mholt%2Farchiver-007d9c?logo=go&logoColor=white)](https://github.com/mholt/archiver)
[![spf13/cobra](https://img.shields.io/badge/spf13%2Fcobra-007d9c?logo=go&logoColor=white)](https://github.com/spf13/cobra)
[![go-git/go-git](https://img.shields.io/badge/go--git%2Fgo--git-007d9c?logo=go&logoColor=white)](https://github.com/go-git/go-git)
[![sigstore/cosign](https://img.shields.io/badge/sigstore%2Fcosign-2a1e71?logo=linuxfoundation&logoColor=white)](https://github.com/sigstore/cosign)
[![helm.sh/helm](https://img.shields.io/badge/helm.sh%2Fhelm-0f1689?logo=helm&logoColor=white)](https://github.com/helm/helm)
[![kubernetes](https://img.shields.io/badge/kubernetes-316ce6?logo=kubernetes&logoColor=white)](https://github.com/kubernetes)