Allow elasticsearch sink response error logs to be made less verbose

[tronboto]

A note for the community

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Use Cases

Having the entire bulk api response printed as a log message by default can be extremely verbose in situations where you have a large bulk size and high throughput, resulting in the logs being extremely difficult to analyse at a time when you likely want to analyse them.

Attempted Solutions

No response

Proposal

Make it possible to disable these verbose response error logs so that the sink just prints the generalised error message as it apparently did in v0.24.

References

• fix(elasticsearch sink): Readd error log for elasticsearch sink #19846
• Logs changed from version 0.24 to 0.25 for elasticsearch sink #15886

Version

vector 0.41.1 (x86_64-unknown-linux-gnu 745babd 2024-09-11 14:55:36.802851761)

[pront]

Hi @tronboto, thank you for this request. To immediately alleviate this situation we can make this emit opt-out. We can also think about generic solutions to this in the future.