Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better metrics on OpenSearch document errors #4344

Closed
dlvenable opened this issue Mar 27, 2024 · 0 comments · Fixed by #4348
Closed

Better metrics on OpenSearch document errors #4344

dlvenable opened this issue Mar 27, 2024 · 0 comments · Fixed by #4348
Assignees
Labels
enhancement New feature or request
Milestone

Comments

@dlvenable
Copy link
Member

Is your feature request related to a problem? Please describe.

Data Prepper reports a documentErrors metric which counts all document errors. However, it may be difficult to know the exact cause. Data Prepper also has metrics related to specific known errors that happen at the _bulk response.

Neither of these lets us analyze all the document status codes.

Describe the solution you'd like

Provide a dynamic metric which includes the status from the response item.

I'd also like to do this with tags/dimensions (see #3051 (comment) for a broader proposal on this)

<pipeline-name>.opensearch.documentStatus | status=<status>

Describe alternatives you've considered (Optional)

We could choose specific codes and have discrete metrics for those. But, this may miss some errors.

Additional context

See #4343 for another metric proposal to help with similar problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Development

Successfully merging a pull request may close this issue.

1 participant