Websocket/HTTP Server as a source?

[autolyticus]

Hi! First of all, thanks for open-sourcing this project. It's super cool! I feel really fortunate running across this project as I was planning to work on something very similar, and I hadn't realised what a complex undertaking it was going to be, so I feel relieved that something like what I envisioned is already out there.

I've just been playing around with it and I had a naive question regarding how to set up an endpoint and to use Rayvens as a server (ws:// or wss:// preferred, but http is fine as well) for listening for publishers from outside the cluster. For eg. Ray provides the Ray serve API which can be used to set up routes and pass on parameters to other actors.

I also understand that Camel-K itself comes with support for a websocket server, however, my naive approach at trying to implement this as a generic source didn't quite work out

source_config = dict(kind='generic-source',
                     uri="websocket://0.0.0.0:9292")

(StreamActor pid=572) [Kamel subprocess] Caused by: org.apache.camel.NoSuchEndpointException: No endpoint could be found for: websocket://0.0.0.0:9292, please check your classpath contains the needed Camel component jar.

I'm sure I must be missing something, but even if the code was correct, does rayvens come with support for exposing ports/services based on this as an ingress resource? Or do I have to manually configure this on k8s?

Is there a recommended way to set up a server for listening to events other than polling as a client? Is this even supported or am I barking up the wrong tree?

[tardieu]

It should be possible to setup a server (websocket) or otherwise to receive events to feed into Ray.

Rayvens today does not help setup a Kubernetes or ingress to expose this server. This is manual. This is definitely something we could look into supporting as an option when running Ray on Kubernetes.

Kamel is intended to dynamically fetch the libraries (jars from maven) that are required to run a particular source/sink. It looks like this is not happening here. I have to investigate.

[autolyticus]

Thanks for the clarification. Please do let me know if you need any other details from my end for investigating!