Prepare v0.16.3 release (#983)

Signed-off-by: Bridget McErlean <[email protected]>

pkg/buildinfo/version.go

@@ -20,7 +20,7 @@ limitations under the License.
 package buildinfo
 
 // Version is the current version of Sonobuoy, set by the go linker's -X flag at build time
-var Version = "v0.16.2"
+var Version = "v0.16.3"
 
 // GitSHA is the actual commit that is being built, set by the go linker's -X flag at build time.
 var GitSHA string

site/_config.yml

@@ -52,6 +52,12 @@ defaults:
       version: master
       gh: https://github.com/vmware-tanzu/sonobuoy/tree/master
       layout: "docs"
+  - scope:
+      path: docs/v0.16.3
+    values:
+      version: v0.16.3
+      gh: https://github.com/vmware-tanzu/sonobuoy/tree/v0.16.3
+      layout: "docs"
   - scope:
       path: docs/v0.16.2
     values:
@@ -142,9 +148,10 @@ collections:
   - casestudies
 
 versioning: true
-latest: v0.16.2
+latest: v0.16.3
 versions:
 - master
+- v0.16.3
 - v0.16.2
 - v0.16.1
 - v0.16.0

site/_data/toc-mapping.yml

@@ -5,6 +5,7 @@
 # Below is a commented out example of what this may look like:
 
 master: master-toc
+v0.16.3: v0-16-3toc
 v0.16.2: v0-16-2toc
 v0.16.1: v0-16-1toc
 v0.16.0: v0-16-0toc

site/_data/v0-16-3toc.yml

@@ -0,0 +1,28 @@
+toc:
+  - title: Basics
+    subfolderitems:
+      - page: Overview
+        url: /index.html
+      - page: Checking Results
+        url: /results
+  - title: Plugins
+    subfolderitems:
+      - page: Overview
+        url: /plugins
+      - page: E2E & Conformance
+        url: /e2eplugin
+      - page: Examples
+        url: /examples
+        github: true
+  - title: Advanced
+    subfolderitems:
+      - page: Detailed result contents
+        url: /snapshot
+      - page: Configuration Options
+        url: /sonobuoy-config
+      - page: Custom Registries & Airgap Testing
+        url: /airgap
+      - page: Using Private Images
+        url: /pullsecrets
+      - page: Advanced Customization
+        url: /gen

site/docs/v0.16.3/README.md

@@ -0,0 +1,180 @@
+# <img src="img/sonobuoy-logo.png" width="400px" > [![CircleCI](https://circleci.com/gh/vmware-tanzu/sonobuoy.svg?style=svg)](https://circleci.com/gh/vmware-tanzu/sonobuoy)
+
+## [Overview][oview]
+
+Sonobuoy is a diagnostic tool that makes it easier to understand the
+state of a Kubernetes cluster by running a set of plugins (including [Kubernetes][k8s] conformance
+tests) in an accessible and non-destructive manner. It is a customizable,
+extendable, and cluster-agnostic way to generate clear, informative reports
+about your cluster.
+
+Its selective data dumps of Kubernetes resource objects and cluster nodes allow
+for the following use cases:
+
+* Integrated end-to-end (e2e) [conformance-testing][e2ePlugin]
+* Workload debugging
+* Custom data collection via extensible plugins
+
+Sonobuoy supports 3 Kubernetes minor versions: the current release and 2 minor versions before. Sonobuoy is currently versioned to track the Kubernetes minor version to clarify the support matrix. For example, Sonobuoy v0.14.x would support Kubernetes 1.14.x, 1.13.x, and 1.12.x.
+
+> Note: You can skip this version enforcement by running Sonobuoy with the `--skip-preflight` flag.
+
+## Prerequisites
+
+* Access to an up-and-running Kubernetes cluster. If you do not have a cluster,
+  we recommend following the [AWS Quickstart for Kubernetes][quickstart] instructions.
+
+* An admin `kubeconfig` file, and the KUBECONFIG environment variable set.
+
+* For some advanced workflows it may be required to have `kubectl` installed. See [installing via Homebrew (MacOS)][brew] or [building
+  the binary (Linux)][linux].
+
+* The `sonobuoy images` subcommand requires [Docker](https://www.docker.com) to be installed. See [installing Docker](docker).
+
+## Installing
+
+We recommend installing Sonobuoy via downloading one of the releases directly from [here][releases].
+
+You can use the web UI to download a release or from the terminal:
+
+```
+$ VERSION=0.16.1 OS=darwin && \
+    curl -L "https://github.com/vmware-tanzu/sonobuoy/releases/download/v${VERSION}/sonobuoy_${VERSION}_${OS}_amd64.tar.gz" --output $HOME/bin/sonobuoy.tar.gz && \
+    tar -xzf $HOME/bin/sonobuoy.tar.gz -C $HOME/bin && \
+    chmod +x $HOME/bin/sonobuoy && \
+    rm $HOME/bin/sonobuoy.tar.gz
+```
+
+> Note: Be sure to update the OS to your local value. Supported values are: "linux", "darwin", and "windows".
+
+If building locally, you should clone the repository and run `make`. To build locally, Docker is required.
+
+## Getting Started
+
+To launch conformance tests (ensuring [CNCF][cncf] conformance) and wait until they are finished run:
+
+```bash
+sonobuoy run --wait
+```
+
+> Note: Using `--mode quick` will significantly shorten the runtime of Sonobuoy. It runs just a single test, helping to quickly validate your Sonobuoy and Kubernetes configuration.
+
+Get the results from the plugins (e.g. e2e test results):
+
+```bash
+results=$(sonobuoy retrieve)
+```
+
+Inspect results for test failures.  This will list the number of tests failed and their names:
+
+```bash
+sonobuoy results $results
+```
+
+> Note: The `results` command has lots of useful options for various situations. See the [results page][results] for more details.
+
+You can also extract the entire contents of the file to get much more [detailed data][snapshot] about your cluster.
+
+Sonobuoy creates a few resources in order to run and expects to run within its
+own namespace.
+
+Deleting Sonobuoy entails removing its namespace as well as a few cluster
+scoped resources.
+
+```bash
+sonobuoy delete --wait
+```
+
+> Note: The --wait option ensures the Kubernetes namespace is deleted, avoiding conflicts if another Sonobuoy run is started quickly.
+
+### Other Tests
+
+By default, `sonobuoy run` runs the Kubernetes conformance tests but this can easily be configured. The same plugin that has the conformance tests has all the Kubernetes end-to-end tests which include other tests such as:
+
+* tests for specific storage features
+* performance tests
+* scaling tests
+* provider specific tests
+* and many more
+
+To modify which tests you want to run, checkout our page on the [e2e plugin][e2ePlugin].
+
+If you want to run other tests or tools which are not a part of the Kubernetes end-to-end suite, refer to our documentation on [custom plugins][customPlugins].
+
+### Monitoring Sonobuoy during a run
+
+You can check on the status of each of the plugins running with:
+
+```bash
+sonobuoy status
+```
+
+You can also inspect the logs of all Sonobuoy containers:
+
+```bash
+sonobuoy logs
+```
+
+## Troubleshooting
+
+If you encounter any problems that the documentation does not address, [file an
+issue][issue].
+
+## Known Issues
+
+### Leaked End-to-end namespaces
+
+There are some Kubernetes e2e tests that may leak resources. Sonobuoy can
+help clean those up as well by deleting all namespaces prefixed with `e2e`:
+
+```bash
+sonobuoy delete --all
+```
+
+### Run on Google Cloud Platform (GCP)
+
+Sonobuoy requires admin permissions which won't be automatic if you are running via Google Kubernetes Engine (GKE) cluster. You must first create an admin role for the user under which you run Sonobuoy:
+
+```bash
+kubectl create clusterrolebinding <your-user-cluster-admin-binding> --clusterrole=cluster-admin --user=<[email protected]>
+```
+
+## Contributing
+
+Thanks for taking the time to join our community and start contributing! We
+welcome pull requests. Feel free to dig through the [issues][issue] and jump in.
+
+### Before you start
+
+* Please familiarize yourself with the [Code of Conduct][coc] before
+  contributing.
+* See [CONTRIBUTING.md][contrib] for instructions on the developer certificate
+  of origin that we require.
+* There is a [Slack channel][slack] if you want to
+  interact with other members of the community
+
+## Changelog
+
+See [the list of releases][releases] to find out about feature changes.
+
+[airgap]: airgap
+[brew]: https://kubernetes.io/docs/tasks/tools/install-kubectl/#install-with-homebrew-on-macos
+[cncf]: https://github.com/cncf/k8s-conformance#certified-kubernetes
+[coc]: https://github.com/vmware-tanzu/sonobuoy/blob/master/CODE_OF_CONDUCT.md
+[contrib]: https://github.com/vmware-tanzu/sonobuoy/blob/master/CONTRIBUTING.md
+[docker]: https://docs.docker.com/install
+[docs]: https://sonobuoy.io/docs/v0.16.3
+[e2ePlugin]: e2eplugin
+[customPlugins]: plugins
+[gen]: gen
+[issue]: https://github.com/vmware-tanzu/sonobuoy/issues
+[k8s]: https://github.com/kubernetes/kubernetes
+[linux]: https://kubernetes.io/docs/tasks/tools/install-kubectl/#tabset-1
+[oview]: https://youtu.be/k-P4hXdruRs?t=9m27s
+[plugins]: plugins
+[quickstart]: https://aws.amazon.com/quickstart/architecture/vmware-kubernetes/
+[releases]: https://github.com/vmware-tanzu/sonobuoy/releases
+[results]: results.md
+[slack]: https://kubernetes.slack.com/messages/sonobuoy
+[snapshot]:snapshot
+[sonobuoyconfig]: sonobuoy-config

site/docs/v0.16.3/airgap.md

@@ -0,0 +1,60 @@
+# Custom registries and air-gapped testing
+
+In air-gapped deployments where there is no access to the public Docker registries
+Sonobuoy supports running end-to-end tests with custom registries. This enables
+you to test your air-gapped deployment once you've loaded the necessary images
+into a registry that is reachable by your cluster.
+
+## Test images
+
+Just provide the `--e2e-repo-config` parameter and pass it the path to a local
+YAML file pointing to the registries you'd like to use. This will instruct the
+Kubernetes end-to-end suite to use your registries instead of the default ones.
+
+```
+sonobuoy run --e2e-repo-config custom-repos.yaml
+```
+
+The registry list is a YAML document specifying a few different registry
+categories and their values:
+
+```
+dockerLibraryRegistry: docker.io/library
+e2eRegistry: gcr.io/kubernetes-e2e-test-images
+gcRegistry: k8s.gcr.io
+etcdRegistry: quay.io/coreos
+privateRegistry: gcr.io/k8s-authenticated-test
+sampleRegistry: gcr.io/google-samples
+```
+
+The keys in that file are specified in the Kubernetes test framework itself. You
+may provide a subset of those and the defaults will be used for the others.
+
+## Other required images
+
+The list of custom registries is consumed by the Kubernetes end-to-end tests, but outside of that there are 2 images you will need to be able to access:
+ - the sonobuoy image
+ - the conformance image
+
+To get those images into your cluster you need to pull/tag/push those images yourself:
+
+```
+PRIVATE_REG=<private registry>
+SONO_VERSION=<version of Sonobuoy you are targeting; e.g. v0.14.0>
+CLUSTER_VERSION=<version of k8s you are targeting; e.g. v1.14.0>
+
+docker pull gcr.io/google-containers/conformance:$CLUSTER_VERSION
+docker pull gcr.io/heptio-images/sonobuoy:$SONO_VERSION
+
+docker tag gcr.io/google-containers/conformance:$CLUSTER_VERSION $PRIVATE_REG/conformance:$CLUSTER_VERSION
+docker tag gcr.io/heptio-images/sonobuoy:$SONO_VERSION $PRIVATE_REG/sonobuoy:$SONO_VERSION
+
+docker push $PRIVATE_REG/conformance:$CLUSTER_VERSION
+docker push $PRIVATE_REG/sonobuoy:$SONO_VERSION
+```
+
+If you want to run the systemd_logs plugin you'll need to pull/tag/push it as well. In addition, you'll have to manually specify the image you want to use via `sonobuoy gen` -> `kubectl apply` since that image is not overridable on the CLI. The default value is: `gcr.io/heptio-images/sonobuoy-plugin-systemd-logs:latest`
+
+If you do not wish to run it in your air-gapped cluster, just remove it from the list of [plugins][plugins] to be run (again, using `sonobuoy gen` -> `kubectl apply`).
+
+[plugins]: plugins.md#choosing-which-plugins-to-run

site/docs/v0.16.3/e2eplugin.md

@@ -0,0 +1,74 @@
+# The Kubernetes End-To-End Testing Plugin
+
+The Kubernetes end-to-end testing plugin (the e2e plugin) is used to run tests which are maintained by the upstream Kubernetes community in the [kubernetes/kubernetes][kubernetesRepo] repo.
+
+There are numerous ways to run this plugin in order to meet your testing needs.
+
+## Choosing Which Tests To Run
+
+The most common point of customization is changing the set of tests to run. This is controlled by two environment variables the test image recognizes:
+
+* E2E_FOCUS
+* E2E_SKIP
+
+Each of these is a regular expression describing which tests tests to run or skip. The "E2E_FOCUS" value is applied first and the "E2E_SKIP" value then further restricts that list. These can be set using Sonobuoy flags:
+
+```
+sonobuoy run \
+  --e2e-focus=<run tests matching this regexp> \
+  --e2e-skip=<skip tests matching this regexp>
+```
+
+> Note: These flags are just special cases of the more general flag `--plugin-env`. For instance, you could set the env vars by using the flag `--plugin-env e2e.E2E_SKIP=<value>`
+
+# Built-In Configurations
+
+There are a few commonly run configurations which Sonobuoy hard-codes for convenience:
+
+* non-disruptive-conformance
+
+This is the default mode and will run all the tests in the `e2e` plugin which are marked `Conformance` which are known to not be disruptive to other workloads in your cluster. This mode is ideal for checking that an existing cluster continues to behave is conformant manner.
+
+> NOTE: The length of time it takes to run conformance can vary based on the size of your cluster---the timeout can be adjusted in the Server.timeoutseconds field of the Sonobuoy `config.json` or on the CLI via the `--timeout` flag.
+
+* quick
+
+This mode will run a single test from the `e2e` test suite which is known to be simple and fast. Use this mode as a quick check that the cluster is responding and reachable.
+
+* certified-conformance
+
+This mode runs all of the `Conformance` tests and is the mode used when applying for the [Certified Kubernetes Conformance Program](https://www.cncf.io/certification/software-conformance). Some of these tests may be disruptive to other workloads so it is not recommended that you run this mode on production clusters. In those situations, use the default "non-disruptive-conformance" mode.
+
+> NOTE: The length of time it takes to run conformance can vary based on the size of your cluster---the timeout can be adjusted in the Server.timeoutseconds field of the Sonobuoy `config.json` or on the CLI via the `--timeout` flag.
+
+## Dry Run
+
+When specifying your own focus/skip values, it may be useful to set the run to operate in dry run mode:
+
+```
+sonobuoy run \
+  --plugin-env e2e.E2E_FOCUS=pods \
+  --plugin-env e2e.E2E_DRYRUN=true
+```
+
+By setting `E2E_DRYRUN`, the run will execute and produce results like normal except that the actual test code won't execute, just the test selection. Each test that _would have been run_ will be reported as passing. This can help you fine-tune your focus/skip values to target just the tests you want without wasting hours on test runs which target unnecessary tests.
+
+## Why Conformance Matters
+
+With such a [wide array][configs] of Kubernetes distributions available, *conformance tests* help ensure that a Kubernetes cluster meets the minimal set of features. They are a subset of end-to-end (e2e) tests that should pass on any Kubernetes cluster.
+
+A conformance-passing cluster provides the following guarantees:
+
+* **Best practices**: Your Kubernetes is properly configured. This is useful to know whether you are running a distribution out of the box or handling your own custom setup.
+
+* **Predictability**: All your cluster behavior is well-documented. Available features in the official Kubernetes documentation can be taken as a given. Unexpected bugs should be rare, because distribution-specific issues are weeded out during the conformance tests.
+
+* **Interoperability**: Workloads from other conforming clusters can be ported into your cluster, or vice versa. This standardization of Kubernetes is a key advantage of open source software, and allows you to avoid vendor lock-in.
+
+Individual Kubernetes distributions may offer additional features beyond conformance testing, but if you change distributions, these features can't be expected to be provided.
+
+See the [official documentation][conformanceDocs] for Kubernetes's existing conformance tests.
+
+[configs]: https://docs.google.com/spreadsheets/d/1LxSqBzjOxfGx3cmtZ4EbB_BGCxT_wlxW_xgHVVa23es/edit#gid=0
+[conformanceDocs]: https://github.com/kubernetes/community/blob/master/contributors/devel/sig-testing/e2e-tests.md#conformance-tests
+[kubernetesRepo]: https://github.com/kubernetes/kubernetes/tree/master/cluster/images/conformance