Support maximum request length configurations in the HTTP and OTel sources #3931
Labels
enhancement
New feature or request
plugin - source
A plugin to receive data from a service or location.
Milestone
Is your feature request related to a problem? Please describe.
Some configurations will not work if a request size is too large. For example, an
http
source with akafka
buffer will fail to write to Kafka if the request size is greater than 1MB. Data Prepper does not check the request size up-front leading to a confusing error.Describe the solution you'd like
Provide new configurations on the HTTP/gRPC sources (this includes all three OTel sources) to configure a maximum request length.
e.g.
Describe alternatives you've considered (Optional)
Have different exceptions thrown from the buffer to indicate the size is too large.
Additional context
This is somewhat related to the PR to increase the Kafka message size in the
kafka
buffer:#3916
The text was updated successfully, but these errors were encountered: