Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simple producer benchmark #1310

Merged
merged 5 commits into from
Aug 25, 2024
Merged

Simple producer benchmark #1310

merged 5 commits into from
Aug 25, 2024

Conversation

svroonland
Copy link
Collaborator

It's not much, but it gives us something to compare against.

Relates to #1304

Copy link
Collaborator

@erikvanoosten erikvanoosten left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great start! I propose we:

  • move construction of ProducerRecord to outside the benchmark
  • in the benchmark we publish 100 chunks of 5000 small messages

WDYT?

Copy link
Collaborator

@erikvanoosten erikvanoosten left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just realized that there is one super-tiny thing we can improve: use a binary key/value. Because we now using string, there is the tiny overhead of converting string to byte array. I am not sure if you want to put in the effort, and this is most probably already good.

@erikvanoosten
Copy link
Collaborator

FYI, I have removed the first run of this new bench from the charts at https://zio.github.io/zio-kafka/dev/bench/ because it was producing a lot less data and is therefore not comparable with later runs.

@svroonland
Copy link
Collaborator Author

I just realized that there is one super-tiny thing we can improve: use a binary key/value. Because we now using string, there is the tiny overhead of converting string to byte array. I am not sure if you want to put in the effort, and this is most probably already good.

I think it doesn't really matter since this isn't currently a benchmark about how much performance we can squeeze out of the Producer. It's just to have a reference for future changes.

@svroonland svroonland merged commit 5674dbd into master Aug 25, 2024
14 checks passed
@svroonland svroonland deleted the producer-benchmark branch August 25, 2024 07:12
@erikvanoosten
Copy link
Collaborator

erikvanoosten commented Aug 25, 2024

You're right, good point 👍

@svroonland
Copy link
Collaborator Author

Would be nice to have a "max performance" benchmark though, using the *async methods, batching and clever pipelining.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants