Skip to content

Commit

Permalink
Remove the quadruple dollar hack (#357)
Browse files Browse the repository at this point in the history
Double expansion issue was fixed with the recent splunk-otel-collector upgrade. Now OTel collector configs are expanded only once and hack with `$$$$` is not needed anymore.

+ temporary set splunk-otel-collector docker image to the latest dev version to keep the main in a working state.
  • Loading branch information
dmitryax authored Jan 15, 2022
1 parent e2b1175 commit 5439a80
Show file tree
Hide file tree
Showing 15 changed files with 39 additions and 26 deletions.
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,11 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).

## Unreleased

### Fixed

- Double expansion issue splunk-otel-collector (#357). See [upgrade
guideline](https://github.com/signalfx/splunk-otel-collector-chart/blob/main/UPGRADING.md#0410-to-0420)

### Removed

- Temporary helper initContainer for OTel checkpointing log path move (#358)
Expand Down
8 changes: 8 additions & 0 deletions UPGRADING.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,13 @@
# Upgrade guidelines

## 0.41.0 to 0.42.0

[#357 Double expansion issue in splunk-otel-collector is
fixed](https://github.com/signalfx/splunk-otel-collector-chart/pull/357)

If you use OTel native logs collection with any custom log processing operators
in `filelog` receiver, please replace any occurrences of `$$$$` with `$$`.

## 0.38.0 to 0.39.0

[#325 Logs collection is now disabled by default for Splunk Observability
Expand Down
6 changes: 3 additions & 3 deletions helm-charts/splunk-otel-collector/templates/_helpers.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -218,15 +218,15 @@ Create a filter expression for multiline logs configuration.
{{- $expr := "" }}
{{- if .namespaceName }}
{{- $useRegexp := eq (toString .namespaceName.useRegexp | default "false") "true" }}
{{- $expr = cat "($$$$resource[\"k8s.namespace.name\"])" (ternary "matches" "==" $useRegexp) (quote .namespaceName.value) "&&" }}
{{- $expr = cat "($$resource[\"k8s.namespace.name\"])" (ternary "matches" "==" $useRegexp) (quote .namespaceName.value) "&&" }}
{{- end }}
{{- if .podName }}
{{- $useRegexp := eq (toString .podName.useRegexp | default "false") "true" }}
{{- $expr = cat $expr "($$$$resource[\"k8s.pod.name\"])" (ternary "matches" "==" $useRegexp) (quote .podName.value) "&&" }}
{{- $expr = cat $expr "($$resource[\"k8s.pod.name\"])" (ternary "matches" "==" $useRegexp) (quote .podName.value) "&&" }}
{{- end }}
{{- if .containerName }}
{{- $useRegexp := eq (toString .containerName.useRegexp | default "false") "true" }}
{{- $expr = cat $expr "($$$$resource[\"k8s.container.name\"])" (ternary "matches" "==" $useRegexp) (quote .containerName.value) "&&" }}
{{- $expr = cat $expr "($$resource[\"k8s.container.name\"])" (ternary "matches" "==" $useRegexp) (quote .containerName.value) "&&" }}
{{- end }}
{{- $expr | trimSuffix "&&" | trim }}
{{- end -}}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -139,11 +139,11 @@ receivers:
id: get-format
routes:
- output: parser-docker
expr: '$$$$body matches "^\\{"'
expr: '$$body matches "^\\{"'
- output: parser-crio
expr: '$$$$body matches "^[^ Z]+ "'
expr: '$$body matches "^[^ Z]+ "'
- output: parser-containerd
expr: '$$$$body matches "^[^ Z]+Z"'
expr: '$$body matches "^[^ Z]+Z"'
{{- end }}
{{- if or (not .Values.logsCollection.containers.containerRuntime) (eq .Values.logsCollection.containers.containerRuntime "cri-o") }}
# Parse CRI-O format
Expand Down Expand Up @@ -199,12 +199,12 @@ receivers:
- type: metadata
id: filename
resource:
com.splunk.source: EXPR($$$$attributes["file.path"])
com.splunk.source: EXPR($$attributes["file.path"])
# Extract metadata from file path
- type: regex_parser
id: extract_metadata_from_filepath
regex: '^\/var\/log\/pods\/(?P<namespace>[^_]+)_(?P<pod_name>[^_]+)_(?P<uid>[^\/]+)\/(?P<container_name>[^\._]+)\/(?P<restart_count>\d+)\.log$'
parse_from: $$$$attributes["file.path"]
parse_from: $$attributes["file.path"]
# Move out attributes to Attributes
- type: metadata
resource:
Expand Down
4 changes: 2 additions & 2 deletions helm-charts/splunk-otel-collector/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -712,9 +712,9 @@ image:

otelcol:
# The registry and name of the opentelemetry collector image to pull
repository: quay.io/signalfx/splunk-otel-collector
repository: quay.io/signalfx/splunk-otel-collector-dev
# The tag of the Splunk OTel Collector image, default value is the chart appVersion
tag: ""
tag: 91f554caad6cc5fb52b4bcbdccc94256f0562610
# The policy that specifies when the user wants the opentelemetry collector images to be pulled
pullPolicy: IfNotPresent

Expand Down
2 changes: 1 addition & 1 deletion rendered/manifests/agent-only/daemonset.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ spec:
containerPort: 9411
hostPort: 9411
protocol: TCP
image: quay.io/signalfx/splunk-otel-collector:0.41.0
image: quay.io/signalfx/splunk-otel-collector-dev:91f554caad6cc5fb52b4bcbdccc94256f0562610
imagePullPolicy: IfNotPresent
env:
- name: SPLUNK_MEMORY_TOTAL_MIB
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ spec:
- /otelcol
- --config=/conf/relay.yaml
- --metrics-addr=0.0.0.0:8889
image: quay.io/signalfx/splunk-otel-collector:0.41.0
image: quay.io/signalfx/splunk-otel-collector-dev:91f554caad6cc5fb52b4bcbdccc94256f0562610
imagePullPolicy: IfNotPresent
env:
- name: SPLUNK_MEMORY_TOTAL_MIB
Expand Down
2 changes: 1 addition & 1 deletion rendered/manifests/gateway-only/deployment-gateway.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ spec:
- /otelcol
- --config=/conf/relay.yaml
- --metrics-addr=0.0.0.0:8889
image: quay.io/signalfx/splunk-otel-collector:0.41.0
image: quay.io/signalfx/splunk-otel-collector-dev:91f554caad6cc5fb52b4bcbdccc94256f0562610
imagePullPolicy: IfNotPresent
env:
- name: SPLUNK_MEMORY_TOTAL_MIB
Expand Down
2 changes: 1 addition & 1 deletion rendered/manifests/logs-only/daemonset.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -129,7 +129,7 @@ spec:
- name: otlp-http-old
containerPort: 55681
protocol: TCP
image: quay.io/signalfx/splunk-otel-collector:0.41.0
image: quay.io/signalfx/splunk-otel-collector-dev:91f554caad6cc5fb52b4bcbdccc94256f0562610
imagePullPolicy: IfNotPresent
env:
- name: SPLUNK_MEMORY_TOTAL_MIB
Expand Down
2 changes: 1 addition & 1 deletion rendered/manifests/metrics-only/daemonset.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ spec:
containerPort: 9943
hostPort: 9943
protocol: TCP
image: quay.io/signalfx/splunk-otel-collector:0.41.0
image: quay.io/signalfx/splunk-otel-collector-dev:91f554caad6cc5fb52b4bcbdccc94256f0562610
imagePullPolicy: IfNotPresent
env:
- name: SPLUNK_MEMORY_TOTAL_MIB
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ spec:
- /otelcol
- --config=/conf/relay.yaml
- --metrics-addr=0.0.0.0:8889
image: quay.io/signalfx/splunk-otel-collector:0.41.0
image: quay.io/signalfx/splunk-otel-collector-dev:91f554caad6cc5fb52b4bcbdccc94256f0562610
imagePullPolicy: IfNotPresent
env:
- name: SPLUNK_MEMORY_TOTAL_MIB
Expand Down
10 changes: 5 additions & 5 deletions rendered/manifests/otel-logs/configmap-agent.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -139,11 +139,11 @@ data:
operators:
- id: get-format
routes:
- expr: $$$$body matches "^\\{"
- expr: $$body matches "^\\{"
output: parser-docker
- expr: $$$$body matches "^[^ Z]+ "
- expr: $$body matches "^[^ Z]+ "
output: parser-crio
- expr: $$$$body matches "^[^ Z]+Z"
- expr: $$body matches "^[^ Z]+Z"
output: parser-containerd
type: router
- id: parser-crio
Expand Down Expand Up @@ -190,10 +190,10 @@ data:
type: json_parser
- id: filename
resource:
com.splunk.source: EXPR($$$$attributes["file.path"])
com.splunk.source: EXPR($$attributes["file.path"])
type: metadata
- id: extract_metadata_from_filepath
parse_from: $$$$attributes["file.path"]
parse_from: $$attributes["file.path"]
regex: ^\/var\/log\/pods\/(?P<namespace>[^_]+)_(?P<pod_name>[^_]+)_(?P<uid>[^\/]+)\/(?P<container_name>[^\._]+)\/(?P<restart_count>\d+)\.log$
type: regex_parser
- attributes:
Expand Down
6 changes: 3 additions & 3 deletions rendered/manifests/otel-logs/daemonset.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ spec:
app: splunk-otel-collector
release: default
annotations:
checksum/config: cc641bdc22a677c320c67f9335e7100b21b7cb654613f730df21b106da9e425d
checksum/config: ea0bb7a5bcd707268a902feec74217c929c1bff378507cf3c20cc10dab6394b1
kubectl.kubernetes.io/default-container: otel-collector
spec:
hostNetwork: true
Expand All @@ -41,7 +41,7 @@ spec:
key: node-role.kubernetes.io/master
initContainers:
- name: migrate-checkpoint
image: quay.io/signalfx/splunk-otel-collector:0.41.0
image: quay.io/signalfx/splunk-otel-collector-dev:91f554caad6cc5fb52b4bcbdccc94256f0562610
imagePullPolicy: IfNotPresent
command: ["/migratecheckpoint"]
securityContext:
Expand Down Expand Up @@ -111,7 +111,7 @@ spec:
containerPort: 9411
hostPort: 9411
protocol: TCP
image: quay.io/signalfx/splunk-otel-collector:0.41.0
image: quay.io/signalfx/splunk-otel-collector-dev:91f554caad6cc5fb52b4bcbdccc94256f0562610
imagePullPolicy: IfNotPresent
securityContext:
runAsUser: 0
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ spec:
- /otelcol
- --config=/conf/relay.yaml
- --metrics-addr=0.0.0.0:8889
image: quay.io/signalfx/splunk-otel-collector:0.41.0
image: quay.io/signalfx/splunk-otel-collector-dev:91f554caad6cc5fb52b4bcbdccc94256f0562610
imagePullPolicy: IfNotPresent
env:
- name: SPLUNK_MEMORY_TOTAL_MIB
Expand Down
2 changes: 1 addition & 1 deletion rendered/manifests/traces-only/daemonset.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ spec:
containerPort: 9411
hostPort: 9411
protocol: TCP
image: quay.io/signalfx/splunk-otel-collector:0.41.0
image: quay.io/signalfx/splunk-otel-collector-dev:91f554caad6cc5fb52b4bcbdccc94256f0562610
imagePullPolicy: IfNotPresent
env:
- name: SPLUNK_MEMORY_TOTAL_MIB
Expand Down

0 comments on commit 5439a80

Please sign in to comment.