Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NETOBSERV-346 revamp upstream doc / readme #125

Merged
merged 7 commits into from
Jun 23, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions .mk/sample.mk
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,17 @@
.PHONY: deploy-sample-cr
deploy-sample-cr:
@echo -e "\n==> Deploy sample CR"
sed -e 's~:main~:$(VERSION)~' ./config/samples/flows_v1alpha1_flowcollector.yaml | kubectl apply -f - || true
ifeq (main,$(VERSION))
kubectl apply -f ./config/samples/flows_v1alpha1_flowcollector.yaml || true
else
kubectl apply -f ./config/samples/flows_v1alpha1_flowcollector_versioned.yaml || true
endif
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove "|| true".  If this fails, do you really want the next potential target to continue?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is what @eranra wrote, on purpose I think .. I'm not against discussing that, but it doesn't really relate to that PR

To bring some context: I did did small change in the makefile because, while writing the doc, I figured out that deploying a "versionned" sample CR wouldn't work as expected, especially as the underlying component images are not tied anymore with the operator version. So now, I recommend switching to the version tag in git before deploying / applying CR

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, that change is fine.  We can ignore my comment.


# Undeploy the sample FlowCollector CR
.PHONY: undeploy-sample-cr
undeploy-sample-cr:
@echo -e "\n==> Undeploy sample CR"
sed -e 's~:main~:$(VERSION)~' ./config/samples/flows_v1alpha1_flowcollector.yaml | kubectl --ignore-not-found=true delete -f - || true
kubectl --ignore-not-found=true delete flowcollector cluster || true

# Deploy sample workload
.PHONY: deploy-sample-workload
Expand Down
112 changes: 112 additions & 0 deletions DEVELOPMENT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@
The NetObserv Operator is meant to run in a Kubernetes cluster like OpenShift or [Kind](https://kind.sigs.k8s.io/). These are the two options most used by the development team.

stleerh marked this conversation as resolved.
Show resolved Hide resolved
## Build / format / lint the code, run unit tests

```bash
make build test
```

## Build and deploy a Docker image

A way to test code changes is to build a Docker image from local sources, push it to your own Docker repository, and deploy it to an existing cluster. Do the following, but replace IMG value with your own registry and account:

stleerh marked this conversation as resolved.
Show resolved Hide resolved
```bash
IMG="quay.io/youraccount/network-observability-operator:test" make image-build image-push deploy
```

After the operator is deployed, set up Loki, which is used to store flows, install a `FlowCollector` custom resource to collect network flows, and optionally install Grafana to provide a user interface and dashboards.

stleerh marked this conversation as resolved.
Show resolved Hide resolved
This provides a quick and easy way to deploy Loki, Grafana and a `FlowCollector` with default values. Note this Loki setup is not for production use.

```bash
make deploy-loki deploy-grafana deploy-sample-cr
```

It will run Loki and Grafana locally, and set up a local port-forward to them. To avoid this, add `PORT_FWD=false` to the command above.

Creating a `FlowCollector` triggers the operator deploying the monitoring pipeline:

- Configures IPFIX exports
- Deploys the flow collector pods, `flowlogs-pipeline`
- Deploys the `network-observability-plugin` if using OpenShift Console

You should be able to see flows in OpenShift Console and Grafana. If not, wait up to 10 minutes. See the [FAQ on troubleshooting](./README.md#faq--troubleshooting) for more information.

### Test another one's pull request

To test a pull request opened by someone else, you just need to pull it locally. Using [GitHub CLI](https://cli.github.com/) is an easy way to do it. Then repeat the steps mentioned above to build, push an image, then deploy the operator and its custom resource.

## Deploy a specific image

Images are built and pushed through CI to [quay.io](https://quay.io/repository/netobserv/network-observability-operator?tab=tags).

You can refer to existing commits using their short-SHA as the image tag, or refer to existing releases. E.g:

```bash
# By commit SHA
VERSION="960766c" make deploy
# By release
VERSION="0.1.2" make deploy
```

It is recommended to switch to the corresponding release Git tag before deploying an old version to make sure the underlying components refer to the correct versions.

## Installing Kafka

Kafka can be used to separate flow ingestion from flow transformation. The operator does not manage kafka deployment and topic creation. We provide a quick setup for Kafka using the [strimzi operator](https://strimzi.io/).

```bash
make deploy-kafka
```

Kafka can then be enabled in the `FlowCollector` resource by setting `spec.kafka.enable` flag to `true`. If you use your own Kafka setup, make sure to configure `spec.kafka.address` and `spec.kafka.topic` accordingly.

## Deploy as bundle

For more details, refer to the [Operator Lifecycle Manager (OLM) bundle quickstart documentation](https://sdk.operatorframework.io/docs/olm-integration/quickstart-bundle/).

This task should be automatically done by the CI/CD pipeline. However, if you want to deploy as
bundle for local testing, you should execute the following commands:

```bash
export USER=<container-registry-username>
export VERSION=0.0.1
export IMG=quay.io/$USER/network-observability-operator:v$VERSION
export BUNDLE_IMG=quay.io/$USER/network-observability-operator-bundle:v$VERSION
make image-build image-push
make bundle bundle-build bundle-push
```

Optionally, you might validate the bundle:

```bash
operator-sdk bundle validate $BUNDLE_IMG
```

> Note: the base64 logo can be generated with: `base64 -w 0 <image file>`, then manually pasted in the [CSV manifest file](./config/manifests/bases/netobserv-operator.clusterserviceversion.yaml) under `spec.icon`.

### Deploy as bundle from command line

This mode is recommended to quickly test the operator during its development:

```bash
operator-sdk run bundle $BUNDLE_IMG
```

### Deploy as bundle from the Console's OperatorHub page

This mode is recommended when you want to test the customer experience of navigating through the
operators' catalog and installing/configuring it manually through the UI.

First, create and push a catalog image:

```bash
export CATALOG_IMG=quay.io/$USER/network-observability-operator-catalog:v$VERSION
make catalog-build catalog-push catalog-deploy
```

The NetObserv Operator is available in OperatorHub: https://operatorhub.io/operator/netobserv-operator

## Publish on central OperatorHub

See [RELEASE.md](./RELEASE.md#publishing-on-operatorhub).
Loading