Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add message size information into s3-sqs input metrics #40306

Closed
kaiyan-sheng opened this issue Jul 22, 2024 · 1 comment · Fixed by #40775
Closed

Add message size information into s3-sqs input metrics #40306

kaiyan-sheng opened this issue Jul 22, 2024 · 1 comment · Fixed by #40775
Assignees
Labels
Team:Cloud-Monitoring Label for the Cloud Monitoring team

Comments

@kaiyan-sheng
Copy link
Contributor

It will be very beneficial to have the information about S3 object size and number of events included in each object when debugging AWS s3-sqs input performance. These metrics are missing from the existing input metrics right now.

For input metrics, please see #25711 for more details.

@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Jul 22, 2024
@bturquet bturquet added the Team:Cloud-Monitoring Label for the Cloud Monitoring team label Jul 23, 2024
@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Jul 23, 2024
@bturquet bturquet assigned bturquet and kaiyan-sheng and unassigned bturquet Jul 23, 2024
@Kavindu-Dodan
Copy link
Contributor

Discussed with @kaiyan-sheng on what metrics to include. We decided to include a histogram to measure,

  1. Processed individual S3 object size
  2. Number of logs included in each object

These values should help to understand the distribution of processed data at beats allowing us to help users better.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Team:Cloud-Monitoring Label for the Cloud Monitoring team
Projects
None yet
3 participants