Bad request body from the Honeycomb sink after refactoring to the new style

[hgiasac]

A note for the community

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Problem

The Honeycomb sink responds HTTP 400 errors when we upgraded Vector to the latest version. The sink works well with Vector <= v0.32. After searching around I doubt that the breaking change happened after the HTTP client refactoring

The request body that I can capture is an object. According to Honeycomb's API docs the /1/batch/{datasetSlug}
API accepts an array of objects.

{
  "time": "2024-11-25T08:48:46.864151918Z",
  "data": {
    "User_Agent": null,
    "X_real_ip": null,
    "X_request_id": null,
    "environment": "staging"
  }
}

Expected:

[
  { "time": "2024-11-25T08:48:46.864151918Z", "data": { ... } },
  { "time": "2024-11-25T08:48:47.864151918Z", "data": { ... } }
]

Configuration

[sources.docker_logs]
type = "docker_logs"
include_labels = ["vector.enable=true"]

[sinks.honeycomb]
inputs = ["docker_logs"]
type = "honeycomb"
api_key = "xxx"
dataset = "xxx"
batch.max_bytes = 1000000
batch.max_events = 1000
batch.timeout_secs = 5

Version

= v0.33

Debug Output

http: hyper::client::pool: pooling idle connection for ("https", api.honeycomb.io)
2024-11-25T09:13:12.119020Z DEBUG http: vector::internal_events::http_client: HTTP response. status=400 Bad Request version=HTTP/1.1 headers={"date": "Mon, 25 Nov 2024 09:13:11 GMT", "content-type": "application/json", "content-length": "45", "connection": "keep-alive", "access-control-allow-origin": "*", "request-id": "ccfdbc82d74cc06cb24188248fb83352", "vary": "Accept-Encoding"} body=[45 bytes]
2024-11-25T09:13:12.119056Z  INFO vector::topology::builder: Healthcheck passed.
2024-11-25T09:13:15.955440Z DEBUG sink{component_kind="sink" component_id=honeycomb component_type=honeycomb}: vector::utilization: utilization=0.07850103649827872
2024-11-25T09:13:16.313445Z DEBUG sink{component_kind="sink" component_id=honeycomb component_type=honeycomb}:request{request_id=1}:http: vector::internal_events::http_client: Sending HTTP request.yper::client::pool: pooling idle connection for ("https", api.honeycomb.io)
2024-11-25T09:13:12.119020Z DEBUG http: vector::internal_events::http_client: HTTP response. status=400 Bad Request version=HTTP/1.1 headers={"date": "Mon, 25 Nov 2024 09:13:11 GMT", "content-type": "application/json", "content-length": "45", "connection": "keep-alive", "access-control-allow-origin": "*", "request-id": "ccfdbc82d74cc06cb24188248fb83352", "vary": "Accept-Encoding"} body=[45 bytes]
2024-11-25T09:13:12.119056Z  INFO vector::topology::builder: Healthcheck passed.
2024-11-25T09:13:15.955440Z DEBUG sink{component_kind="sink" component_id=honeycomb component_type=honeycomb}: vector::utilization: utilization=0.07850103649827872
2024-11-25T09:13:16.313445Z DEBUG sink{component_kind="sink" component_id=honeycomb component_type=honeycomb}:request{request_id=1}:http: vector::internal_events::http_client: Sending HTTP request. uri=https://api.honeycomb.io/1/batch/xxx method=POST version=HTTP/1.1 headers={"x-honeycomb-team": "xxx", "accept-encoding": "zstd,gzip,deflate,br", "user-agent": "Vector/0.42.0 (x86_64-unknown-linux-musl 3d16e34 2024-10-21 14:10:14.375255220)"} body=[55593 bytes]
2024-11-25T09:13:16.313475Z DEBUG sink{component_kind="sink" component_id=honeycomb component_type=honeycomb}:request{request_id=1}:http: hyper::client::pool: reuse idle connection for ("https", api.honeycomb.io) uri=https://api.honeycomb.io/1/batch/xxx method=POST version=HTTP/1.1 headers={"x-honeycomb-team": "xxx", "accept-encoding": "zstd,gzip,deflate,br", "user-agent": "Vector/0.42.0 (x86_64-unknown-linux-musl 3d16e34 2024-10-21 14:10:14.375255220)"} body=[55593 bytes]
2024-11-25T09:13:16.313475Z DEBUG sink{component_kind="sink" component_id=honeycomb component_type=honeycomb}:request{request_id=1}:http: hyper::client::pool: reuse idle connection for ("https", api.honeycomb.io)

Example Data

No response

Additional Context

No response

References

No response