-
Notifications
You must be signed in to change notification settings - Fork 2.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[filestorage] receiver name sanitization #20731
Comments
I can confirm this is an issue, here's a sample config: exporters:
logging:
extensions:
file_storage:
directory: .
receivers:
filelog/logs/json:
include: ./logs-json.log
start_at: beginning
storage: file_storage
service:
extensions:
- file_storage
pipelines:
logs:
exporters:
- logging
receivers:
- filelog/logs/json and the output for the current latest v0.75.0 $ ./otelcol-contrib-0.75.0-darwin_amd64 --config ./config.yaml
2023-04-06T17:03:18.713+0200 info service/telemetry.go:90 Setting up own telemetry...
2023-04-06T17:03:18.713+0200 info service/telemetry.go:116 Serving Prometheus metrics {"address": ":8888", "level": "Basic"}
2023-04-06T17:03:18.713+0200 info [email protected]/exporter.go:286 Development component. May change in the future. {"kind": "exporter", "data_type": "logs", "name": "logging"}
2023-04-06T17:03:18.715+0200 info service/service.go:129 Starting otelcol-contrib... {"Version": "0.75.0", "NumCPU": 16}
2023-04-06T17:03:18.715+0200 info extensions/extensions.go:41 Starting extensions...
2023-04-06T17:03:18.715+0200 info extensions/extensions.go:44 Extension is starting... {"kind": "extension", "name": "file_storage"}
2023-04-06T17:03:18.715+0200 info extensions/extensions.go:48 Extension started. {"kind": "extension", "name": "file_storage"}
2023-04-06T17:03:18.715+0200 info adapter/receiver.go:56 Starting stanza receiver {"kind": "receiver", "name": "filelog/logs/json", "data_type": "logs"}
2023-04-06T17:03:18.715+0200 info service/service.go:155 Starting shutdown...
2023-04-06T17:03:18.715+0200 info adapter/receiver.go:150 Stopping stanza receiver {"kind": "receiver", "name": "filelog/logs/json", "data_type": "logs"}
2023-04-06T17:03:18.715+0200 info extensions/extensions.go:55 Stopping extensions...
2023-04-06T17:03:18.715+0200 info service/service.go:169 Shutdown complete.
Error: cannot start pipelines: storage client: open receiver_filelog_logs/json: no such file or directory
2023/04/06 17:03:18 collector server run finished with error: cannot start pipelines: storage client: open receiver_filelog_logs/json: no such file or directory I agree this should be fixed by sanitizing the receiver's name by replacing the slashes with another character. How about the underscore |
Pinging code owners for extension/storage/filestorage: @djaglowski. See Adding Labels via Comments if you do not have permissions to add labels yourself. |
Thanks for looking into this. I don't have any concern to use underscore. |
I think we should use a character that is less likely to be one chosen by a user, because for example we would still have a collision between |
Related issue: #3148 |
@djaglowski would you be OK with a PR that just replaces slashes |
@astencel-sumo, that would be an improvement on what we have now, so seems fine. I think we should leave the issue open though, until we're confident that the solution is robust. |
Here's the PR: I propose to close this issue with this PR as it is a (sub)duplicate of #3148 that's only about slashes, but keep #3148 open. What do you think @djaglowski? |
This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping Pinging code owners:
See Adding Labels via Comments if you do not have permissions to add labels yourself. |
I have reworked the PR mentioned above to fix the more general issue #3148. |
This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping Pinging code owners:
See Adding Labels via Comments if you do not have permissions to add labels yourself. |
…ames (open-telemetry#20896) Fixes open-telemetry#3148 This change was initially created to only fix open-telemetry#20731 by replacing the slash `/` characters in component names with a tilde `~`. After that, we decided that it is a breaking change, and so it's better to fix other characters as well in a single breaking change.
Component(s)
No response
Is your feature request related to a problem? Please describe.
/
might be used in receiver name, for example,filelog/pod/container/stdout
, when storage is enabled(for persisting file fingerprints, offsets, etc), the path will bereceiver_filelog_pod/container/stdout
, and dirreceiver_filelog_pod/container
will need to be created first, I think it makes sense to avoid such issue via the sanitization instead of changing receiver name.opentelemetry-collector-contrib/extension/storage/filestorage/extension.go
Lines 58 to 64 in 2e2623c
Describe the solution you'd like
replace
/
with~
in receiver name, looks like such convention is used in some k8s components, for examplepods/volumes/kubernetes.io~csi
pods/volumes/kubernetes.io~empty~dir
pods/volumes/kubernetes.io~secrets
Describe alternatives you've considered
No response
Additional context
No response
The text was updated successfully, but these errors were encountered: