Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add support for tailing custom host files #300

Merged
merged 3 commits into from
Nov 18, 2021
Merged

add support for tailing custom host files #300

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
5de08b9
add support for tailing custom host files
rockb1017 Nov 18, 2021
df19d18
add resource processors
rockb1017 Nov 18, 2021
bc74e2c
Merge branch 'main' into customFiles
dmitryax Nov 18, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions helm-charts/splunk-otel-collector/templates/config/_otel-agent.tpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -234,6 +234,10 @@ receivers:
- move:
from: log
to: $$

{{- if .Values.logsCollection.extraFileLogs }}
{{- toYaml .Values.logsCollection.extraFileLogs | nindent 2 }}
{{- end }}
{{- end }}

# By default k8sattributes and batch processors enabled.
Expand Down Expand Up @@ -456,6 +460,28 @@ service:
- splunk_hec/platform_logs
{{- end }}
{{- end }}

{{- if .Values.logsCollection.extraFileLogs }}
logs/extraFiles:
receivers:
{{- range $key, $exporterData := .Values.logsCollection.extraFileLogs }}
- {{ $key }}
{{ end }}
processors:
- memory_limiter
- batch
rockb1017 marked this conversation as resolved.
Show resolved Hide resolved
- resource
{{- if .Values.environment }}
- resource/add_environment
{{- end }}
exporters:
{{- if eq (include "splunk-otel-collector.platformLogsEnabled" .) "true" }}
- splunk_hec/platform
{{- end }}
{{- if eq (include "splunk-otel-collector.o11yLogsEnabled" .) "true" }}
- splunk_hec/o11y
{{- end }}
{{- end }}
{{- end }}

{{- if (eq (include "splunk-otel-collector.tracesEnabled" .) "true") }}
Expand Down
3 changes: 3 additions & 0 deletions helm-charts/splunk-otel-collector/values.schema.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -517,6 +517,9 @@
},
"checkpointPath": {
"type": "string"
},
"extraFileLogs": {
"type": "object"
}
}
},
Expand Down
15 changes: 15 additions & 0 deletions helm-charts/splunk-otel-collector/values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -395,6 +395,21 @@ logsCollection:

checkpointPath: "/var/addon/splunk/otel_pos"

# Files on k8s nodes to tail.
# Make sure to configure volume mounts properly at `otelAgent.extraVolumes` and `otelAgent.extraVolumeMounts`.
extraFileLogs: {}
Copy link
Contributor

@chaitanyaphalak chaitanyaphalak Nov 18, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An example of how to use volumes/emptydir would be nice to have!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should go to examples https://github.com/signalfx/splunk-otel-collector-chart/tree/main/examples, and this values.yaml comment can get a link to it. Just to avoid mixing comment for different config parts here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd recommend submit another PR for that. This one LGTM. Let me know if we shold merge it

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, its cool to merge it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, added issue for this #307

# Sample configuration to collect Audit logs. Please note hostPath can vary depending on the audit-policy.yaml configuration.
# extraFileLogs:
# filelog/audit-log:
# include: [/var/log/kubernetes/apiserver/audit.log]
# start_at: beginning
# include_file_path: true
# include_file_name: false
# resource:
# com.splunk.source: /var/log/kubernetes/apiserver/audit.log
# host.name: 'EXPR(env("K8S_NODE_NAME"))'
# com.splunk.sourcetype: kube:apiserver-audit

################################################################################
# Fluentd sidecar configuration for logs collection.
# As of now, this is the recommended way to collect k8s logs,
Expand Down