-
-
Notifications
You must be signed in to change notification settings - Fork 36
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
add send_batch method to kafka mod #324
base: main
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was just wondering how you were getting on with this the other day!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good!
Please also add docs for this to docs/reference/kumo.kafka/build_producer.md
You can use {{ since('dev') }}
in the section that you add there to indicate that this is available in dev
.
Thanks!
crates/mod-kafka/Cargo.toml
Outdated
rdkafka.workspace=true | ||
serde.workspace=true | ||
tokio.workspace=true | ||
tracing.workspace=true |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we can trim this out with the API change to return the errors instead of logging them
tracing.workspace=true |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
good catch, I'll remove it
Add a new method to the kafka mod to also send with rdkafka's internal batching. The return is a bit strange because I couldn't figure out how to send the failed indexes as LuaError that could then be used in the lua part if it was caught with a pcall...