-
Notifications
You must be signed in to change notification settings - Fork 98
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Cluster Events and Node Log Updates (#1016)
* Additional Cluster Events Processing - Extracted default values from Cluster Events using logfmt - Added static label source=kubernetes-events - Added labelsToKeep support - Added structuredMetadata support * Normalized extraLogProcessingStages Changed extraProcessingStages to extraLogProcessingStages for consistency across charts * Additional Cluster Events Processing - Extracted all systemd unit service discovery labels - Added static label source=journal - Added labelsToKeep support - Added structuredMetadata support - Determine log level * Fixed Cluster Events Test and Rebuilt * Add the ability to use a caFile in tls sections (#1013) Signed-off-by: Pete Wall <[email protected]> * Improved validators for OpenCost prometheus URLs and basic auth (#1012) * Improved validators for OpenCost prometheus URLs and basic auth Signed-off-by: Pete Wall <[email protected]> * Sync the helpers template Signed-off-by: Pete Wall <[email protected]> --------- Signed-off-by: Pete Wall <[email protected]> * Additional Cluster Events Processing - Extracted default values from Cluster Events using logfmt - Added static label source=kubernetes-events - Added labelsToKeep support - Added structuredMetadata support * Sync'd and Rebuilt * Scyn'd and Rebuilt * Resolved Conflicts * Regenerated Cluster Events Documentation * Fixed Documentation and Unit Test --------- Signed-off-by: Pete Wall <[email protected]> Co-authored-by: Pete Wall <[email protected]>
- Loading branch information
Showing
51 changed files
with
2,822 additions
and
85 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,92 @@ | ||
| # yamllint disable rule:document-start rule:line-length rule:trailing-spaces | ||
| suite: Test namespaces | ||
| templates: | ||
| - configmap.yaml | ||
| tests: | ||
| - it: should create a ConfigMap that sets custom labels to keep | ||
| set: | ||
| deployAsConfigMap: true | ||
| labelsToKeep: | ||
| - job | ||
| - namespace | ||
| - level | ||
| - node | ||
| - name | ||
| - source | ||
| asserts: | ||
| - isKind: | ||
| of: ConfigMap | ||
| - equal: | ||
| path: data["module.alloy"] | ||
| value: |- | ||
| declare "cluster_events" { | ||
| argument "logs_destinations" { | ||
| comment = "Must be a list of log destinations where collected logs should be forwarded to" | ||
| } | ||
| loki.source.kubernetes_events "cluster_events" { | ||
| job_name = "integrations/kubernetes/eventhandler" | ||
| log_format = "logfmt" | ||
| forward_to = [loki.process.cluster_events.receiver] | ||
| } | ||
| loki.process "cluster_events" { | ||
| // add a static source label to the logs so they can be differentiated / restricted if necessary | ||
| stage.static_labels { | ||
| values = { | ||
| "source" = "kubernetes-events", | ||
| } | ||
| } | ||
| // extract some of the fields from the log line, these could be used as labels, structured metadata, etc. | ||
| stage.logfmt { | ||
| mapping = { | ||
| "component" = "sourcecomponent", // map the sourcecomponent field to component | ||
| "kind" = "", | ||
| "level" = "type", // most events don't have a level but they do have a "type" i.e. Normal, Warning, Error, etc. | ||
| "name" = "", | ||
| "node" = "sourcehost", // map the sourcehost field to node | ||
| } | ||
| } | ||
| // set these values as labels, they may or may not be used as index labels in Loki as they can be dropped | ||
| // prior to being written to Loki, but this makes them available | ||
| stage.labels { | ||
| values = { | ||
| "component" = "", | ||
| "kind" = "", | ||
| "level" = "", | ||
| "name" = "", | ||
| "node" = "", | ||
| } | ||
| } | ||
| // if kind=Node, set the node label by copying the instance label | ||
| stage.match { | ||
| selector = "{kind=\"Node\"}" | ||
| stage.labels { | ||
| values = { | ||
| "node" = "name", | ||
| } | ||
| } | ||
| } | ||
| // set the level extracted key value as a normalized log level | ||
| stage.match { | ||
| selector = "{level=\"Normal\"}" | ||
| stage.static_labels { | ||
| values = { | ||
| level = "Info", | ||
| } | ||
| } | ||
| } | ||
| // Only keep the labels that are defined in the `keepLabels` list. | ||
| stage.label_keep { | ||
| values = ["job","namespace","level","node","name","source"] | ||
| } | ||
| forward_to = argument.logs_destinations.value | ||
| } | ||
| } |
Oops, something went wrong.