-
Notifications
You must be signed in to change notification settings - Fork 138
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support for multiple subscriptions on the same Consumer #517
Conversation
* Remove subscribeAnd / SubscribedConsumer
@vigoo @adamgfraser @guizmaii Could I get your opinion on this interface change? |
@svroonland Can you rebase your branch, please? |
The only merge conflict is in some docs. Let's gather some feedback on these changes first before merging. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
by lack of understanding - given the description of this pr - and difference between: restartable plainStream
, partitionedStream
and non restartable partitionedAssignmentStream
i would expect the type signatures and/or their documentation to differ more prominently - am i missing something ?
zio-kafka/src/main/scala/zio/kafka/consumer/InvalidSubscriptionUnion.scala
Outdated
Show resolved
Hide resolved
zio-kafka/src/main/scala/zio/kafka/consumer/InvalidSubscriptionUnion.scala
Outdated
Show resolved
Hide resolved
I tried to find a way to be more typesafe: no more runtime error possible with the "union" operation, but I didn't find any way because in the end you're just composing streams 🤔 |
… `Serde.byteArray` as these deserializers are not doing anything anyway (#654)
zio-kafka/src/main/scala/zio/kafka/consumer/internal/Runloop.scala
Outdated
Show resolved
Hide resolved
zio-kafka/src/main/scala/zio/kafka/consumer/internal/Runloop.scala
Outdated
Show resolved
Hide resolved
zio-kafka/src/main/scala/zio/kafka/consumer/internal/Runloop.scala
Outdated
Show resolved
Hide resolved
Here's a proposal to improve a bit the readability: #663 Can you rebase your branch also, please :) |
…cala Co-authored-by: Jules Ivanic <[email protected]>
…cala Co-authored-by: Jules Ivanic <[email protected]>
zio-kafka/src/main/scala/zio/kafka/consumer/internal/Runloop.scala
Outdated
Show resolved
Hide resolved
…cala Co-authored-by: Jules Ivanic <[email protected]>
With this PR, a single Consumer instance and its backing apache kafka
Consumer
instance, can support multiple topic/pattern/manual subscriptions (as long as they are of the same type).Inspired by https://doc.akka.io/docs/alpakka-kafka/current/consumer.html#sharing-the-kafkaconsumer-instance
plainStream
/partitionedStream
/partitionedAssignmentStream
.subscribeAnd
,SubscribedConsumer
,SubscribedConsumerFromEnvironment
.plainStream
/partitionedStream
/partitionedAssignmentStream
.subscribe
andunsubscribe
have been removed from theConsumer
trait in favour of scoped subscriptions.plainStream/partitionedStream
, the stream may be restarted thanks to the scoped subscriptions. This was not possible previously, requiring to make a new Consumer instance (or perhaps unsubscribe & resubscribe).Implements #217.