Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

*: Support NSQ in ArgoLabs Dataflow #1381

Open
alexec opened this issue Oct 19, 2021 · 0 comments
Open

*: Support NSQ in ArgoLabs Dataflow #1381

alexec opened this issue Oct 19, 2021 · 0 comments

Comments

@alexec
Copy link

alexec commented Oct 19, 2021

In Dataflow, we currently have support for Kafka, Jetstream, NATS Streaming and other sources and sinks. It would be amazing to add support for NSQ too. We know that we plenty of use cases in the OSS community and this would get a lot of love.

The Dataflow team are not NSQ experts. My ask is - would anyone be interested in implementing the changes needed?

Here is a pull request for Jetstream, breaking it down:

  1. Add manifests to install a dev NSQ into Kubernetes cluster.
  2. Add APIs to specify a NSQ source and sink (e.g. the URL, authentication).
  3. Implement the source and sink.
  4. Write test infrastructure (e.g. check the right number of messages get written to a sink).
  5. Write tests (e.g. does the right number of message get written, even if the pod is deleted).
  6. Write examples.
  7. Write docs.

I'll raise a ticket against the main repo, and the Go client repo (there might be engineers there with the exact skills needed).

@mreiferson mreiferson changed the title Support NSQ in ArgoLabs Dataflow *: Support NSQ in ArgoLabs Dataflow Oct 24, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants