Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support {{topic}} variable with {{key}} in file template pattern #178

Closed
stephen-harris opened this issue Jun 19, 2023 · 0 comments · Fixed by #167
Closed

Support {{topic}} variable with {{key}} in file template pattern #178

stephen-harris opened this issue Jun 19, 2023 · 0 comments · Fixed by #167

Comments

@stephen-harris
Copy link
Contributor

This is a feature request to support use of {{key}} in conjunction with {{topic}} variable.

Currently two modes of grouping are supported:

  • grouping by the topic, partition, and timestamp;
  • grouping by the key.

In our context we have a connector responsible for backing-up multiple compacted topics to an S3 bucket. While this could be achieved through multiple connectors, but given the large number of topics we were hoping to use connectors to provide a logical grouping of topics. However, as keys may exist across topics, to do this we would need the topic name in the S3 key.

@jeqo jeqo transferred this issue from Aiven-Open/s3-connector-for-apache-kafka Jul 7, 2023
@jeqo jeqo closed this as completed in #167 Aug 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant