From 778928e2c917d37ad5109fb15864b8600395b83b Mon Sep 17 00:00:00 2001 From: bakshi41c Date: Fri, 26 Apr 2024 13:23:08 +0100 Subject: [PATCH] readme-fix --- CHANGELOG.md | 8 +-- CODE_OF_CONDUCT.md | 4 +- CONTRIBUTING.md | 26 +++---- README.md | 88 ++++++++++++----------- SECURITY.md | 2 +- agent/README.md | 17 +++-- agent/plugins/syntests/curl/README.md | 5 +- agent/plugins/syntests/dns/README.md | 6 +- agent/plugins/syntests/httpPing/README.md | 8 ++- agent/plugins/syntests/netDial/README.md | 3 + agent/plugins/syntests/ping/README.md | 5 +- common/README.md | 7 +- controller/README.md | 9 ++- restapi/README.md | 2 +- 14 files changed, 109 insertions(+), 81 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index dc3c93f..c9d3839 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,11 +9,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Added - - Configurable AGENT_ID through environment variable - - Configurable prometheus metric labels +- Configurable AGENT_ID through environment variable +- Configurable prometheus metric labels ### Changes - + - Changes to plugin init and finish calls ## [v1.1.0] - 2024-04-26 @@ -23,4 +23,4 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Initial release [unreleased]: https://github.com/cisco-open/synthetic-heart/compare/v1.1.0...HEAD -[v1.1.0]: https://github.com/cisco-open/synthetic-heart/compare/v1.1.0...HEAD \ No newline at end of file +[v1.1.0]: https://github.com/cisco-open/synthetic-heart/compare/v1.1.0...HEAD diff --git a/CODE_OF_CONDUCT.md b/CODE_OF_CONDUCT.md index 169e7d9..8caad38 100644 --- a/CODE_OF_CONDUCT.md +++ b/CODE_OF_CONDUCT.md @@ -71,9 +71,9 @@ members of the project's leadership. ## Attribution This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, -available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html +available at [homepage]: https://www.contributor-covenant.org For answers to common questions about this code of conduct, see -https://www.contributor-covenant.org/faq \ No newline at end of file + diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 0663f00..db10f2a 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -16,27 +16,27 @@ any real-time space e.g., Slack, Discord, etc. ### Setting up a build environment Prerequisites: - - A MacOS or Linux dev machine. - - [Podman](https://podman-desktop.io/) - - A Kubernetes cluster (Tested with v1.25+, older versions may work) - - You can use [Kind](https://kind.sigs.k8s.io/) to create a local cluster. Follow the [Podman doc page](https://podman-desktop.io/docs/kind/creating-a-kind-cluster) to see how. - - [Go 1.22](https://go.dev/doc/install) - - [Helm v3](https://helm.sh/docs/intro/install/#helm) - - [Make](https://www.gnu.org/software/make/) + +- A MacOS or Linux dev machine. +- [Podman](https://podman-desktop.io/) +- A Kubernetes cluster (Tested with v1.25+, older versions may work) + - You can use [Kind](https://kind.sigs.k8s.io/) to create a local cluster. Follow the [Podman doc page](https://podman-desktop.io/docs/kind/creating-a-kind-cluster) to see how. +- [Go 1.22](https://go.dev/doc/install) +- [Helm v3](https://helm.sh/docs/intro/install/#helm) +- [Make](https://www.gnu.org/software/make/) Once you have a cluster to develop on, install the Helm chart to that cluster, as described in the [README](./README.md). Following that, the workflow would look something like this: - - Make code changes. - - Run `make docker-all`. - - This compiles all the code and creates container images for agent, controller and restapi with tag `dev_latest`. - - Push the latest image to the Kind cluster. Can be done through the Podman desktop UI. [Link to instructions](https://podman-desktop.io/docs/kubernetes/kind/pushing-an-image-to-kind). - - Restart the relevant pods, so the pod starts with the new image. +- Make code changes. +- Run `make docker-all`. + - This compiles all the code and creates container images for agent, controller and restapi with tag `dev_latest`. +- Push the latest image to the Kind cluster. Can be done through the Podman desktop UI. [Link to instructions](https://podman-desktop.io/docs/kubernetes/kind/pushing-an-image-to-kind). +- Restart the relevant pods, so the pod starts with the new image. Note: The Controller has special commands to make changes to CRD files etc. Please refer to the [controller README](./controller/README.md). Note 2: The Agent also has special commands to make changes to proto files. Please refer to the [agent README](./agent/README.md). - ### Writing new synthetic test plugins for the agent Synthetic-heart welcomes the community to contribute new plugins for the agents. Please refer to the [agent README](./agent/README.md) to see how to write new plugins. diff --git a/README.md b/README.md index ac9e819..cef5398 100644 --- a/README.md +++ b/README.md @@ -1,10 +1,11 @@ -# Synthetic Heart - Kubernetes synthetic testing and monitoring framework +# Synthetic Heart - Kubernetes synthetic testing and monitoring framework Synthetic Heart is a synthetic testing and monitoring framework for Kubernetes clusters. It allows operators to run synthetic tests, then check and monitor the results through an API or through metrics (via Prometheus). ## Architecture Synthetic Heart is comprised of four components: + - [Agent](./agent): Runs the actual tests, watches redis/kv-store for test configs - [Rest api](./restapi): To query test results, logs and agent details - [Controller](./controller): K8s controller (watches for SyntheticTest CRD), and stores them in redis for agents to pick up/query @@ -56,70 +57,73 @@ The plugin manager is responsible for handling the lifecycle of these plugins. I Synthetic Heart also comes with a Rest API which lets external services query test results etc. It is essentially a shim for the Redis storage. - ## Installation ### With Helm Chart + Easiest way to run synthetic-heart is to install the helm chart in the `charts/` directory. Prerequisites: - - - Kubernetes cluster (Tested with v1.25+, older versions may work) - - [Helm v3](https://helm.sh/docs/intro/install/#helm) + +- Kubernetes cluster (Tested with v1.25+, older versions may work) +- [Helm v3](https://helm.sh/docs/intro/install/#helm) Steps to follow: - - Install the [synthetic-heart](./chart/synthetic-heart) Helm chart - - Run: `helm upgrade -i synthetic-heart -n synthetic-heart .` in `charts/` directory. - - This installs the RestAPI, Agents (as DaemonSet), Controller, and Redis. - - Install the [synthetic-tests](./chart/synthetic-tests) Helm chart - - Run `helm upgrade -i synthetic-tests -n synthetic-heart .` in `charts/` directory. - - This installs two example synthetic tests. - - You may then port forward the rest api to query results. - - Run `kubectl port-forward svc/synheart-api-svc 8080:8080 -n synthetic-heart` - - and then query the tests `curl http://localhost:8080/api/v1/tests` + +- Install the [synthetic-heart](./chart/synthetic-heart) Helm chart + - Run: `helm upgrade -i synthetic-heart -n synthetic-heart .` in `charts/` directory. + - This installs the RestAPI, Agents (as DaemonSet), Controller, and Redis. +- Install the [synthetic-tests](./chart/synthetic-tests) Helm chart + - Run `helm upgrade -i synthetic-tests -n synthetic-heart .` in `charts/` directory. + - This installs two example synthetic tests. +- You may then port forward the rest api to query results. + - Run `kubectl port-forward svc/synheart-api-svc 8080:8080 -n synthetic-heart` + - and then query the tests `curl http://localhost:8080/api/v1/tests` ### Without Helm Chart It might be a bit more complex to install the components manually. Please explore the [Helm chart](./chart/synthetic-heart) to check the example configurations for different Kubernetes resources. The dependencies, and major components are described below. A few Kubenretes resources need to be installed: - - CustomResourceDefinition - The `SyntheticTest` CRD needs to be installed. [Link to CRD.](./controller/config/crd/bases/synheart.infra.webex.com_synthetictests.yaml) - - Redis - Redis v7 needs to be installed so the test configs and results can be stored. - - A `Service` is also needed, so the redis endpoint can be accessed by agents. - - Controller - Needs to be deployed as a `Deployment`. - - The controller needs `ServiceAccount`, `ClusteRole`, `ClusterRoleBinding`, allowing it to query and modify `SyntheticTests`. - - Agents - Can be deployed as a `DaemonSet`. - - The configuration of the agents is passed in as a file. The file should be mounted from a `ConfigMap` - - RestApi - Needs to be deployed as a `Deployment` - - The configuration of the agents is passed in as a file. The file should be mounted from a `ConfigMap` - - A `Service` for the Restapi is needed. - - Optionally an `Ingress` can be added to the Restapi to make the API accessible from outside the cluster. + +- CustomResourceDefinition - The `SyntheticTest` CRD needs to be installed. [Link to CRD.](./controller/config/crd/bases/synheart.infra.webex.com_synthetictests.yaml) +- Redis - Redis v7 needs to be installed so the test configs and results can be stored. + - A `Service` is also needed, so the redis endpoint can be accessed by agents. +- Controller - Needs to be deployed as a `Deployment`. + - The controller needs `ServiceAccount`, `ClusteRole`, `ClusterRoleBinding`, allowing it to query and modify `SyntheticTests`. +- Agents - Can be deployed as a `DaemonSet`. + - The configuration of the agents is passed in as a file. The file should be mounted from a `ConfigMap` +- RestApi - Needs to be deployed as a `Deployment` + - The configuration of the agents is passed in as a file. The file should be mounted from a `ConfigMap` + - A `Service` for the Restapi is needed. + - Optionally an `Ingress` can be added to the Restapi to make the API accessible from outside the cluster. ## Credits and Acknowledgements Thank you to the following folks for contributing to synthetic-heart project: - - Allen Guo: tiaguo@cisco.com - - Ben Zhao: benzhao@cisco.com - - Cristian Maugeri: cmaugeri@cisco.com - - David Hazra: dahazra@cisco.com - - Jeff Wang: jiangfwa@cisco.com - - Jerry Zhang: xiaojiez@cisco.com - - Luke Ren: lukren@cisco.com - - Lun Zhou: lunzhou@cisco.com - - Meibao Wang: meibwang@cisco.com - - Mercion Wilathgamuwage: mwilathg@cisco.com - - Shaz Balakumar: shbalaku@cisco.com + +- Allen Guo: +- Ben Zhao: +- Cristian Maugeri: +- David Hazra: +- Jeff Wang: +- Jerry Zhang: +- Luke Ren: +- Lun Zhou: +- Meibao Wang: +- Mercion Wilathgamuwage: +- Shaz Balakumar: ## Roadmap There's a lot of features that are still being worked on. [Get involved!](./CONTRIBUTING.md) - - UI (currently being worked on) - - Re-design agent deployment model (allow agents per node, per namespace etc.) - - Allow agents to be deployed in different namespace, mount secrets etc. - - Plugin versions - - Dynamically add or remove plugins at build time (to reduce binary size) - - More plugins +- UI (currently being worked on) +- Re-design agent deployment model (allow agents per node, per namespace etc.) + - Allow agents to be deployed in different namespace, mount secrets etc. +- Plugin versions +- Dynamically add or remove plugins at build time (to reduce binary size) +- More plugins ## Contributing diff --git a/SECURITY.md b/SECURITY.md index c239e27..94b6dee 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -36,4 +36,4 @@ involving the following steps: ## Comments on this Policy If you have suggestions on how this process could be improved please submit a -pull request. \ No newline at end of file +pull request. diff --git a/agent/README.md b/agent/README.md index 90e9366..a22511a 100644 --- a/agent/README.md +++ b/agent/README.md @@ -4,8 +4,10 @@ The agent is the component responsible for running the synthetic tests. The agen [Hashicorp's go-plugin](https://github.com/hashicorp/go-plugin). ## Configuration + This section is for the configuration of the agent. A sample config file agent is provided below: + ```yaml gracePeriod: 3s # When the agent is exiting, how long to wait to process/export any pending test results syncFrequency: 30s # How often to poll external storage for new syntest configs @@ -26,19 +28,22 @@ etc: ``` ## Metrics + By default the agent export the test runtimes and the test marks. If a test wants to export custom metrics, it needs to add the following to `TestResult.Details` map: + - `key`: `_prometheus` - `value`: YAML representation of `PrometheusMetrics` struct in [models.go](https://github.com/cisco-open/synthetic-heart/blob/master/common/models.go#L32) Note: At the moment only Prometheus Gauges are supported - ## Building proto files + Run `make proto` ## Testing + Run the tests, do: `make test` ## Development @@ -49,10 +54,10 @@ All Synthetic-Heart tests are [hashicorp go-plugins](https://github.com/hashicor Some comments on plugin development: - - Please try to keep plugins as generic as possible. - - Make the plugins as configurable as possible. - - Use worker pools to allow multiple instances to be run by one plugin. For example with http ping test, it's expensive to run 5 instances of the same plugins to test 5 domains, compared to 1 instance testing all 5 domains. - - Try exporting plugin specific metrics. +- Please try to keep plugins as generic as possible. +- Make the plugins as configurable as possible. +- Use worker pools to allow multiple instances to be run by one plugin. For example with http ping test, it's expensive to run 5 instances of the same plugins to test 5 domains, compared to 1 instance testing all 5 domains. +- Try exporting plugin specific metrics. To add a new synthetic test, follow: @@ -60,6 +65,7 @@ To add a new synthetic test, follow: NOTE: Please use camel cased name like (`vaultAws` or `dns`)
2. In `common/constants.go` file, Add your plugin name as a constant (with suffix `TestName`) e.g. + ```go MyTestTestName = "myTest" // <- Add this DockerPullTestName = "dockerPull" @@ -71,4 +77,3 @@ To add a new synthetic test, follow: 3. In `pluginmanager/pluginRegistration.go` file, register your plugin by adding: - `RegisterSynTestPlugin(YourTestTestName, []string{BinPath + SyntestPrefix + MyTestTestName})` - diff --git a/agent/plugins/syntests/curl/README.md b/agent/plugins/syntests/curl/README.md index d3c2000..8edbe0c 100644 --- a/agent/plugins/syntests/curl/README.md +++ b/agent/plugins/syntests/curl/README.md @@ -1,13 +1,14 @@ # Curl Test + Does a curl on an url, then prints out the result, can also export the metrics to prometheus. The `outputOptions` correspond to the `--write-out` options in curl command. [More info](https://ec.haxx.se/usingcurl/usingcurl-verbose/usingcurl-writeout) - ## Test Details map + Populates the prometheus metrics that prometheus plugin supports - ## Example Configuration + ```yaml config: | url: https://google.com diff --git a/agent/plugins/syntests/dns/README.md b/agent/plugins/syntests/dns/README.md index ae03832..dbced5a 100644 --- a/agent/plugins/syntests/dns/README.md +++ b/agent/plugins/syntests/dns/README.md @@ -1,14 +1,16 @@ # DNS Test + Resolves given domains to IPs ## Test Details map + 1. `key`: `_log` - `value`: ip addresses for each domain tested - ## Example Configuration + ```yaml config: | domains: ["google.com"] repeats: 3 -``` \ No newline at end of file +``` diff --git a/agent/plugins/syntests/httpPing/README.md b/agent/plugins/syntests/httpPing/README.md index cebbc54..3b3782a 100644 --- a/agent/plugins/syntests/httpPing/README.md +++ b/agent/plugins/syntests/httpPing/README.md @@ -1,12 +1,14 @@ # HTTP Ping + Pings a HTTP endpoint and checks if its the expected response. ## Test Details map + 1. `key`: `_log` - `value`: details of a request - ## Example Configuration + ```yaml config : | address: "localhost:9200" @@ -14,10 +16,12 @@ Pings a HTTP endpoint and checks if its the expected response. retries: 3 timeoutRetries: 1 ``` + `retries` is max retry times when http request failed except exceeded timeout `timeoutRetries` is optional, default value is 0. Max retry times only when http request exceeded timeout. Recommended value is small number, like 1. For multiple endpoints (performs the tests in parallel): + ```yaml config : | - address: "localhost:9200" @@ -27,4 +31,4 @@ For multiple endpoints (performs the tests in parallel): - address: "localhost:6400" expectedCodeRegex : ^(202|472){1} retries: 3 -``` \ No newline at end of file +``` diff --git a/agent/plugins/syntests/netDial/README.md b/agent/plugins/syntests/netDial/README.md index 5530286..95e31ae 100644 --- a/agent/plugins/syntests/netDial/README.md +++ b/agent/plugins/syntests/netDial/README.md @@ -1,10 +1,13 @@ # Net Dial Test + Dials on a port and address to check if its open ## Test Details map + No extra info ## Example Configuration + ```yaml config : | addresses: diff --git a/agent/plugins/syntests/ping/README.md b/agent/plugins/syntests/ping/README.md index 24bf451..7e8e41c 100644 --- a/agent/plugins/syntests/ping/README.md +++ b/agent/plugins/syntests/ping/README.md @@ -1,13 +1,16 @@ # Ping Test + Sends an ICMP ping ## Test Details map + No extra info ## Example Configuration + ```yaml config: | domain: 8.8.8.8 # Where to ping pings: 5 # How many pings to send privileged: true # Whether to run as root -``` \ No newline at end of file +``` diff --git a/common/README.md b/common/README.md index 2d7c400..b0429f6 100644 --- a/common/README.md +++ b/common/README.md @@ -3,6 +3,7 @@ This package contains code that is shared across all components ## Folders - - `proto/`: contains the proto files needed for communication with plugins - - `storage/`: contains the code and interface for external storage (e.g. redis) - - `utils/`: contains any common utility code that can be shared \ No newline at end of file + +- `proto/`: contains the proto files needed for communication with plugins +- `storage/`: contains the code and interface for external storage (e.g. redis) +- `utils/`: contains any common utility code that can be shared diff --git a/controller/README.md b/controller/README.md index b748aa6..ba17c16 100644 --- a/controller/README.md +++ b/controller/README.md @@ -4,6 +4,7 @@ This is a controller to handler SyntheticTest CRDs. It mirrors the CRD configs t is watched by the synthetic-heart agents. It also: + - any agents that are no longer correspond to a k8s node - any tests that do not exist - reschedules tests that are on non-active/non-existent nodes @@ -11,12 +12,14 @@ It also: The controller was built using [Kubebuilder v3.14.0](https://github.com/kubernetes-sigs/kubebuilder) kubebuilder commands that were run to generate the project: + ```sh kubebuilder init --domain infra.webex.com --license none --repo "github.com/cisco-open/synthetic-heart/controller" kubebuilder create api --group synheart.infra.webex.com --version v1 --kind SyntheticTest ``` # Synthetic Test CRD Example + ```yaml apiVersion: synheart.infra.webex.com/v1 kind: SyntheticTest @@ -32,6 +35,7 @@ spec: ``` ## Config + ``` # Needs two environment variables SYNHEART_STORE_ADDR="localhost:6379" # the address of redis @@ -41,20 +45,21 @@ AGENT_STATUS_DEADLINE="30s" # deadline for an agent before its considered not al ## Development ### Prerequisites + - go version v1.21.0+ - docker version 17.03+. - kubectl version v1.11.3+. - Access to a Kubernetes v1.11.3+ cluster. - In order to run: + - Run a local [redis server](https://redis.io/download) - `export SYNHEART_STORE_ADDR=localhost:6379` - `make install` - `make run` Useful make commands: + - `make manifests`: Builds api definitions - `make install` : Installs the CRD to the cluster using local kubeconfig - `make run` : Runs the controller using the local kubeconfig - diff --git a/restapi/README.md b/restapi/README.md index ca2d98e..3db6ad5 100644 --- a/restapi/README.md +++ b/restapi/README.md @@ -8,4 +8,4 @@ Simple REST endpoint to query redis for synthetic test results, agent details et address: "0.0.0.0:51230" # Address at which the rest api would run storageAddress: "redis:6379" # Address at which the storage is running uiAddress: "http://localhost:51230?server=http://localhost:51230" # Address to redirect to when user requests /ui -``` \ No newline at end of file +```