Chronicle compression doesn't set the content-encoding header

[10x-mattsearle]

A note for the community

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Problem

When using Google Chronicle as a log sink and enabling gzip compression vector doesn't set the content-encoding header in the API request and the chronicle API therefore responds with a 400 response because it cannot parse the compressed data as json.

Configuration

acknowledgements:
  enabled: true
api:
  enabled: true
data_dir: /var/lib/vector
healthchecks:
  enabled: true
  require_healthy: true
sources:
  aws_config:
    decoding:
      codec: bytes
    sqs:
      max_number_of_messages: 1
      queue_url: https://sqs.eu-west-1.amazonaws.com/xxxxxxxxxxx/aws_config_notifications_queue
    type: aws_s3

sinks:
  chronicle:
    compression: gzip
    credentials_path: /etc/chronicle/google_sa.json
    customer_id: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
    encoding:
      codec: json
    inputs:
    - aws_config
    log_type: AWS_CONFIG
    region: eu
    type: gcp_chronicle_unstructured

Version

0.43.0

Debug Output

No response

Example Data

No response

Additional Context

No response

References

No response