Re-investigate gRPC streams as a transport option

[tigrannajaryan]

The initial design chose WebSocket over gRPC and listed a few reasons. There are a few arguments that may warrant re-evaluating this option.

1. There was a report from people who run WebSockets in production extensively about possible interoperability problems.
2. We are considering using OpAMP for managing Otel SDKs, which already have gRPC as dependency (because they implement OTLP), so the argument that gRPC is a complicated dependency may not be applicable.
3. We are considering adding plain HTTP as transport and use in Otel SDKs, which makes gRPC dependency totally unnecessary in this case.
4. While working on the reference implementation I tried multiple Go WebSocket libraries and have somewhat mixed feelings about their quality.

This is a call for comments. I want to know what other people think about WebSocket vs gRPC.

[blumamir]

2. We are considering using OpAMP for managing Otel SDKs, which already have gRPC as dependency (because they implement OTLP), so the argument that gRPC is a complicated dependency may not be applicable.

Since OTLP is already implemented in Plain HTTP and gRPC, I feel it is more natural and consistent to follow the same patterns for opamp as well.

This is a call for comments. I want to know what other people think about WebSocket vs gRPC.

For reference - envoy proxy has similar APIs to distribute configurations which are implemented in gRPC:
https://www.envoyproxy.io/docs/envoy/latest/configuration/overview/xds_api#config-overview-management-server

Are there any other similar tools that implement a control plane API similar to opamp that can be used as references?

[jpeach]

Since OTLP is already implemented in Plain HTTP and gRPC, I feel it is more natural and consistent to follow the same patterns for opamp as well.

Yup, I'd also like to see this just use OTLP for the protobuf transport.

[tigrannajaryan]

There is a known issue with gRPC: in Collector we were unable to implement an OTLP server that accepts both plain HTTP and gRPC in the same port when mTLS is used: open-telemetry/opentelemetry-collector#1256

This alone may be enough reason to stay on WebSocket, which we already implemented to work nicely on the same port as plain HTTP (unless someone finds the solution for the problem).

[tigrannajaryan]

After looking more into the issue with gRPC co-existence with plain HTTP on the same port I am convinced that we don't want to have the same problem in OpAMP.

Arguments in favour of keeping WebSocket:

• We already have a WebSocket implementation in Go that works reasonably well.
• There are other confirmed implementations that did not find any issues with WebSocket.
• We now have plain HTTP transport as an option for implementations that don't want/need WebSocket transport.

Given the above I am going to close this issue since I don't think there are good reasons to reconsider our choice at this point.

If anyone disagrees and is willing to investigate gRPC we still have time until OpAMP spec is marked stable. Feel free to reopen this issue and comment.

[tigrannajaryan]

I had people asking about the possibility to add a gRPC transport, particularly to help with adoption by organizations that have policies around preferring gRPC transport.

We discussed this topic in the Workgroup meeting today and decided that we are not against having a gRPC transport for OpAMP in the future. When/if there is sufficient demand for gRPC we can consider adding it as one more transport type.

I am going to re-open this issue to signal our readiness to consider and discuss this in the future. Feel free to comment on this thread.

[tigrannajaryan]

@open-telemetry/opamp-spec-approvers @open-telemetry/opamp-go-approvers

I have recently discovered that https://github.com/gorilla/websocket which I believe is the most popular and most complete WebSocket implementation in Go is no longer maintained. The standard library's WebSocket is in a worse shape. It is very unfortunate but it seems WebSockets are not seeing the love they deserve, especially in Golang which is a priority for us to implement in Collector.

I wonder if this is yet another argument in favour of exploring gRPC transport a bit more, perhaps have an experimental implementation in https://github.com/open-telemetry/opamp-go and if it works well consider making it part of the official spec.

[shenqidebaozi]

@tigrannajaryan I recently looked into OpAMP and tried the BindPlane product. I have some thoughts regarding this:

For the OpAMP protocol, we can consider using gRPC stream. This way we can directly leverage the gRPC + Protobuf architecture to implement the interaction between client and server, without needing to handle Protobuf encoding/decoding in the HTTP protocol. An even better point is that based on gRPC code generation, we can quickly build multi-language clients and servers, without repeatedly dealing with WebSocket interaction logic in different languages. The spec becomes the code.

Regarding the mTLS issue, we can consider having the service provide different protocols through multiple ports. This is not a fatal problem.

I will continue to research during my free time to try implementing client server interaction through gRPC

[tigrannajaryan]

It would be valuable to see an experimental gRPC implementation of OpAMP. I would expect an implementation that uses a gRPC stream and sends AgentToServer/ServerToAgent messages over the stream, mimicking the OpAMP/WebSocket semantics precisely.

Caveat: this does not mean that if there is such experimental implementation proposed then it will be accepted to OpAMP spec, however it means that we will consider such proposal carefully. It is still possible that the proposal will be rejected.

[shenqidebaozi]

@tigrannajaryan Of course. This does not mean that it has been determined that it can be implemented as a standard, it simply brings more possibilities to the standard.