-
Notifications
You must be signed in to change notification settings - Fork 23
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* move existing examples to their own directory * Add k8s events example * fix example image path * fix example image file extension * Update title to include Google Logging. Specify required IAM role for service account * Add Prerequisites, use CHANGE_ME in config map. * use correct screenshot for k8s events
- Loading branch information
Joseph Sirianni
authored
Mar 18, 2021
1 parent
2a08fe1
commit 447c10d
Showing
8 changed files
with
180 additions
and
0 deletions.
There are no files selected for viewing
File renamed without changes.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,51 @@ | ||
# Kubernetes Events w/ Google Cloud Logging | ||
|
||
Stanza can be deployed as a Kubernetes Events collector by leveraging the [k8s_event_input](https://github.com/observIQ/stanza/blob/master/docs/operators/k8s_event_input.md) operator. [Minikube](https://minikube.sigs.k8s.io/docs/start/) | ||
can be used for this example. | ||
|
||
## Architecture | ||
|
||
1. Service account with permission to the Kubernetes API server | ||
2. Config map: Contains the stanza configuration file | ||
3. Credentials secret: Contains Google Cloud [service account credentials JSON file](https://cloud.google.com/docs/authentication/getting-started) | ||
4. Persistent volume: Allows the stanza database to persist between restarts and pod evictions | ||
5. Deployment: A single replica deployment for the agent | ||
|
||
## Prerequisites | ||
|
||
1. Google Cloud account with Cloud Logging API enabled | ||
2. Google service account with [roles/logging.logWriter](https://cloud.google.com/logging/docs/access-control) | ||
3. Kubernetes Cluster with a storageclass capable of providing persistent volumes | ||
4. Edit `config.yaml` to include: | ||
- Your cluster name (this is added as a label) | ||
- Your project_id | ||
|
||
## Deployment Steps | ||
|
||
Create the credentials secret. The file provided in this example should be replaced | ||
with your service account's credentials. | ||
``` | ||
kubectl create secret generic stanza-agent-credentials \ | ||
--from-file=log_credentials.json | ||
``` | ||
|
||
Create the Kubernetes Service Account | ||
``` | ||
kubectl apply -f service_account.yaml | ||
``` | ||
|
||
Create the config map | ||
``` | ||
kubectl apply -f config.yaml | ||
``` | ||
|
||
Deploy the agent | ||
``` | ||
kubectl apply -f deployment.yaml | ||
``` | ||
|
||
## Validate | ||
|
||
Log into Google Cloud Logging | ||
|
||
![Events](./assets/events.png) |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,14 @@ | ||
--- | ||
apiVersion: v1 | ||
kind: ConfigMap | ||
metadata: | ||
name: stanza-agent | ||
namespace: default | ||
data: | ||
config.yaml: | | ||
pipeline: | ||
- type: kubernetes_events | ||
cluster_name: CHANGE_ME | ||
- credentials_file: /stanza_home/log_destinations/google_cloud/log_credentials.json | ||
project_id: CHANGE_ME | ||
type: google_cloud_output |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,70 @@ | ||
--- | ||
kind: PersistentVolumeClaim | ||
apiVersion: v1 | ||
metadata: | ||
name: stanza-agent-events-persistent-volume | ||
namespace: default | ||
spec: | ||
accessModes: | ||
- ReadWriteOnce | ||
resources: | ||
requests: | ||
storage: 1Gi | ||
--- | ||
apiVersion: apps/v1 | ||
kind: Deployment | ||
metadata: | ||
name: stanza-agent-cluster-events | ||
namespace: default | ||
spec: | ||
replicas: 1 | ||
selector: | ||
matchLabels: | ||
name: stanza-agent-cluster-events | ||
template: | ||
metadata: | ||
labels: | ||
name: stanza-agent-cluster-events | ||
spec: | ||
serviceAccountName: stanza-agent | ||
containers: | ||
- image: observiq/stanza:0.13.16 | ||
imagePullPolicy: Always | ||
name: stanza-agent | ||
command: | ||
- /stanza_home/stanza | ||
args: | ||
- --config | ||
- /stanza_home/config.yaml | ||
- --database | ||
- /stanza_home/database/stanza.db | ||
- --plugin_dir | ||
- /stanza_home/plugins | ||
resources: | ||
limits: | ||
memory: "250Mi" | ||
cpu: 250m | ||
requests: | ||
memory: "250Mi" | ||
cpu: 100m | ||
volumeMounts: | ||
- mountPath: /stanza_home/config.yaml | ||
subPath: config.yaml | ||
name: stanza-agent | ||
- mountPath: /stanza_home/log_destinations/google_cloud/log_credentials.json | ||
subPath: log_credentials.json | ||
name: stanza-agent-credentials | ||
- mountPath: /stanza_home/database | ||
name: stanza-agent-events-persistent-volume | ||
restartPolicy: Always | ||
terminationGracePeriodSeconds: 30 | ||
volumes: | ||
- name: stanza-agent | ||
configMap: | ||
name: stanza-agent | ||
- name: stanza-agent-credentials | ||
secret: | ||
secretName: stanza-agent-credentials | ||
- name: stanza-agent-events-persistent-volume | ||
persistentVolumeClaim: | ||
claimName: stanza-agent-events-persistent-volume |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,12 @@ | ||
{ | ||
"type": "service_account", | ||
"project_id": "", | ||
"private_key_id": "", | ||
"private_key": "", | ||
"client_email": "", | ||
"client_id": "", | ||
"auth_uri": "https://accounts.google.com/o/oauth2/auth", | ||
"token_uri": "https://oauth2.googleapis.com/token", | ||
"auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs", | ||
"client_x509_cert_url": "" | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,33 @@ | ||
--- | ||
kind: ServiceAccount | ||
apiVersion: v1 | ||
metadata: | ||
name: stanza-agent | ||
namespace: default | ||
--- | ||
kind: ClusterRole | ||
apiVersion: rbac.authorization.k8s.io/v1 | ||
metadata: | ||
name: stanza-agent | ||
rules: | ||
- apiGroups: ["", "apps", "batch"] | ||
resources: | ||
- pods | ||
- namespaces | ||
- replicasets | ||
- jobs | ||
- events | ||
verbs: ["get", "list", "watch"] | ||
--- | ||
kind: ClusterRoleBinding | ||
apiVersion: rbac.authorization.k8s.io/v1 | ||
metadata: | ||
name: stanza-agent | ||
roleRef: | ||
apiGroup: rbac.authorization.k8s.io | ||
kind: ClusterRole | ||
name: stanza-agent | ||
subjects: | ||
- kind: ServiceAccount | ||
name: stanza-agent | ||
namespace: default |
File renamed without changes.