Skip to content
This repository has been archived by the owner on Dec 14, 2022. It is now read-only.

[Feature] Support Pulsar message deduplication on Pulsar sink #459

Open
syhily opened this issue Nov 9, 2021 · 0 comments
Open

[Feature] Support Pulsar message deduplication on Pulsar sink #459

syhily opened this issue Nov 9, 2021 · 0 comments
Labels
compute/data-processing type/feature Indicates new functionality

Comments

@syhily
Copy link
Contributor

syhily commented Nov 9, 2021

Pulsar supports a built-in message deduplication policy.
The design documentation is PIP-6.
The usage cookbook is listed on the pulsar website.

We can implement this feature on the pulsar sink connector. Users can have an exactly-once semantic without requiring the transaction which supports the Pulsar prior 2.9.0.
@syhily syhily added compute/data-processing type/feature Indicates new functionality labels Nov 9, 2021
@syhily syhily changed the title [Feature] Support Pulsar message deduplication [Feature] Support Pulsar message deduplication on Pulsar sink Nov 10, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
compute/data-processing type/feature Indicates new functionality
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@syhily