-
Notifications
You must be signed in to change notification settings - Fork 138
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
FromJavaConsumer: how to synchronize access to the java consumer #1276
Comments
zio-kafka protects against concurrent access using a Semaphore. We could perhaps allow the user to pass their own Semaphore in |
@tnielens What is your use case? Would you be helped if all operations of the java-kafka consumer are available on the zio-kafka consumer? |
My usecase is passing the kafka consumer to a One workaround I'm trying now is introducing a |
I think that is okay! The zio-kafka consumer already runs on its own thread because almost every call to the standard consumer blocks! If it blocks a little bit longer, nobody will notice! We could improve somewhat by moving this logic into zio-kafka, because then access would be shared over a single, fair semaphore. But to be honest, since you have a perfectly good workaround, I am not sure it's worth it to put this into the library. |
#694 introduced a
fromJavaConsumer
constructor. But it isn't clear how to synchronize access to the consumer once handed to zio-kafka. My use case is the same as the one reported in #694. I have some logic working on the java consumer to check for the consumer lag, health and other metrics. These checks run periodically but I need to coordinate the accesses with zio-kafka. As far as I understand, zio-kafka doens't provide that ability and I'll face concurrency issues if both my checks and zio-kafka run tasks against the consumer concurrently.The text was updated successfully, but these errors were encountered: