Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[feature] fallback to kafka topic name #67

Closed
hpgrahsl opened this issue Jan 21, 2019 · 1 comment
Closed

[feature] fallback to kafka topic name #67

hpgrahsl opened this issue Jan 21, 2019 · 1 comment
Assignees

Comments

@hpgrahsl
Copy link
Owner

The current collection/topic aware configuration options expect to provide an explicit mapping between any kafka topic name and its corresponding mongodb collection name. Besides, there is the option to specify a default mongodb collection name in case no explicit mapping is provided but this means that all sink records of potentially different kafka topics are written to the same mongodb collection. Complex connector pipelines would almost always specify proper mappings for all kafka topics in question especially since most likely different sink connector behaviour is needed for processing the contained sink records.

However, for very simple pass-through kinds of streaming ETL pipelines from kafka to mongodb it might be useful to allow to fallback not to a single default topic name but to use the name of the kafka topic as is for the mongodb collection name.

@hpgrahsl
Copy link
Owner Author

implemented in #68

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant