Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[receiver/awsfirehosereceiver] Support receiving multiple record type using single endpoint #35988

Open
kaiyan-sheng opened this issue Oct 24, 2024 · 1 comment
Labels
enhancement New feature or request needs triage New item requiring triage receiver/awsfirehose

Comments

@kaiyan-sheng
Copy link
Contributor

Component(s)

receiver/awsfirehose

Is your feature request related to a problem? Please describe.

When trying to ingest both logs and metrics from Firehose, I tried with two receivers configured with the same endpoint:

receivers:
      awsfirehose/cwmetrics:
        endpoint: {{ (.Values.receiver).hostname | default "localhost"}}:4433
        include_metadata: true
        record_type: cwmetrics
      awsfirehose/cwlogs:
        endpoint: {{ (.Values.receiver).hostname | default "localhost"}}:4433
        include_metadata: true
        record_type: cwlogs

This gave me an error:

2024-10-23T20:01:59.169Z	error	graph/graph.go:426	Failed to start component	{"error": "listen tcp 127.0.0.1:4433: bind: address already in use", "type": "Receiver", "id": "awsfirehose/cwmetrics"}

In order to workaround this issue, I have to use two different endpoints, one for metrics and one for logs.

I would like to have a single endpoint to ingest both metrics and logs.

Describe the solution you'd like

We should be able to check what does the request from Firehose looks like and figure out if it's logs or metrics, if it's JSON format metrics or OTEL1.0 format metrics and then apply the right marshaling.

Describe alternatives you've considered

No response

Additional context

No response

@kaiyan-sheng kaiyan-sheng added enhancement New feature or request needs triage New item requiring triage labels Oct 24, 2024
Copy link
Contributor

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request needs triage New item requiring triage receiver/awsfirehose
Projects
None yet
Development

No branches or pull requests

1 participant