Support for Kotlin native / multiplatform

[sebleclerc]

Great to see an official Kotlin gRPC library!

I wonder if support for Kotlin native is planned?

Thanks!

[LouisCAD]

Support for Kotlin/JS would also be nice!

[lowasser]

We would certainly like to do this in the long term, but a unified Kotlin solution for protos that is compatible with the Java API, which is a pretty enormous project by itself, would have to come first. I'm also not aware yet of networking APIs that have a common Kotlin surface between JVM, JS, and native.

[koperko]

Not sure what level of API do you require to implement gRPC protocol, nor what API granularity it actually offers, but ktor.io offers a solution for a common networking interface with platform specific dependencies to actually do the work..

[sebleclerc]

@lowasser thanks for the reply! Please keep us posted 😄

And as @koperko mentioned, ktor.io is the most used networking Kotlin Multiplatform library

[oianmol]

March 2022, Will we have this ever?

[jamesward]

There are a number of pieces that need to come together for this but first we need to get non-JVM Kotlin protos and that is something we are investigating.

[bubenheimer]

protos are not essential in terms of a must-have requirement for cross-platform grpc-kotlin. Especially when talking cross-platform there are alternatives like kotlin's own serialization, and that can leverage Kotlin's own protobuf plugin, which I assume is cross-platform. I'm not arguing against a non-JVM protobuf-kotlin, but I don't think it should be the cause for holding back grpc-kotlin. There may even be other cross-platform protobuf libs that could be leveraged initially.

Personally I'd be thrilled to have a non-protobuf grpc-kotlin that uses Kotlin's own serialization, to have a very clean, simple stack. When you're talking Kotlin having a clean technology stack is kind of a big deal.

[jamesward]

Hmmm... I'm not aware of gRPC working without protos. Are you saying that is the case?

[bubenheimer]

grpc-java is designed to be pluggable with alternative encodings. protobuf is the default and the common choice, but other encodings are possible. I believe the same is true for grpc-kotlin.

Here is my earlier blurb on it with an embedded link to a grpc-java blog post: #255

So far I have not made the switch to non-protobuf grpc-kotlin, because I already have all the protobuf-kotlin conversion layers in place. If I started from scratch I would absolutely go non-protobuf, because it would be a major architectural benefit. I've used protobuf on Android for many years, and doing anything halfway serious means pain. I understand that Google promotes protobuf even in the Android world, but from what I've seen that is based more on inexperience and a focus on Google technologies. It works fine for toy projects, but for anything serious it would not be my first choice except if I had to take non-Kotlin clients into account. My first choice is Kotlin serialization, because it's pretty seamless.

For multiplatform kotlin-grpc, going non-protobuf would make even more sense, for architectural simplification, and because I am less likely to have non-Kotlin clients.

[jamesward]

Thanks for the details on that. Super helpful. Just so I better understand all this... Why would you rather do multiplatform grpc-kotlin instead of ktor + Kotlin serialization or rsocket, etc?

[bubenheimer]

Here's where I'm coming from, hope it helps:

Primarily I need bidi streaming. I suppose there are many competing technologies in that realm these days. I decided on grpc for my main project years ago (grpc-java back then). I know the ins and outs of grpc fairly well, so to switch now I'd need a pretty good reason. Kotlin serialization is a more recent development, and it helped make my Android architecture much cleaner in general once I resolved to put in the groundwork, so it's my serialization technology of choice.

Websockets with ktor or something else might work, but my impression was that I might have an easier time punching through firewalls and having my Android app work in weird network configurations around the globe if I stick with something standardized like http/2. Also want something that is likely to have very long-term support to avoid having to replace the current technology gimmick next year.

Bare HTTP/2 or HTTP/3 could work, with okhttp, netty, ktor, or something else, but I'm guessing I'd still have to build my own APIs on top to make it work well. Don't want to reinvent the wheel. Minimize code maintenance.

rsocket looks very interesting. If I were to make a technology decision again today it would likely be a strong contender. I guess I'd use it on top of HTTP/2 or HTTP/3. If rsocket is still around by the time HTTP/3 is supported everywhere I may switch.

Edit: as for multiplatform, it's a forward-looking potential bonus for my existing app, opening the doors to more easily expand it to iOS.

[jamesward]

Thank you for the added details! I really appreciate you taking the time.

[luca992]

There are a number of pieces that need to come together for this but first we need to get non-JVM Kotlin protos and that is something we are investigating.

What about this project? https://github.com/streem/pbandk

[libill]

There are a number of pieces that need to come together for this but first we need to get non-JVM Kotlin protos and that is something we are investigating.

What about this project? https://github.com/streem/pbandk

I used wire in my kotlin multiplatform mobile project.How about wire? https://github.com/square/wire

[TimOrtel]

I have implemented a project that allows you to make GRPC calls from Kotlin Multiplatform code. Check it out if you're interested. https://github.com/TimOrtel/GRPC-Kotlin-Multiplatform

[BppleMan]

I think it is very necessary to add support for kotlin-native. I'm trying to develop a tool which uses flutter as gui but kotlin-native as backend service. Therefore, in the interaction between flutter and kotlin, I either choose ktor+kotlinx.serializer for inter-process communication in the form of http+json. There is another way is protobuf+grpc, when I try to use flutter with protobuf and grpc, I think this is my preferred way, but after my investigation, protoc does not support kotlin-native, and grpc does the same, which makes I'm a little sad, I really hope grpc at least supports the proto plugin for kotlinx.serializer.

[BppleMan]

Thanks for the details on that. Super helpful. Just so I better understand all this... Why would you rather do multiplatform grpc-kotlin instead of ktor + Kotlin serialization or rsocket, etc?

There is no good rpc implementation for ktor+kotlinx.serialization, which can only be based on http+json communication. It's hard to call a remote method like grpc. As far as I know, rsocket is just a low-level transport protocol. They have rsocket-kotlin + rsocket-rpc-kotlin, but in the end the upper layer implementation still relies on protobuf, so the problem is back to the original point.

[arncore]

I want to summarize this thread because there are multiple strategies to achieve the goal set forth in this issue and tbh it's incredibly confusing tackling both kotlin/native and kotlin/multiplatform under the same issue. They're different kotlin projects and one encompasses the other.

1. Support kotlin/jvm, kotlin/native kotlin/js etc. separately. This means that grpc has to support proto data model generation for each kotlin project independent of each other. This means that the grpc project controls the implementation of the protocol on http/2 for all those projects as well. This is the more difficult route but offers more control of the implementation for each platform.

2. grpc-kotlin supports kotlin/multiplatform instead and that will encompass all platforms supported by kotlin. This is a slightly easier route as it places the burden of cross-platform implementations on kotlin authors instead. However, this means you have to scrap grpc-kotlin as it is because it is tied to the java code gen (currently but obviously not for long hopefully) and furthermore the protocol implementation on top of http/2 is reliant on netty (another java specific client). To fully support kotlin/multiplatform my understanding currently is that it means integrating fully with ktor. Unless grpc-kotlin authors (you guys) write a grpc "ktor" which makes no sense, superficially speaking. Once you have that, grpc stubs will not use netty but ktor. Ktor allows you to use any engine/client they support and that can be configured depending on platform. Currently all engines except CIO, Apache and Android support http/2. Android folks can use okhttp so all is not lost in that regard. This path has its cons since those clients may not be as optimized as netty is/allow low level modifications like netty does. I would like to hear more about this if possible, as in, what would it take to get rid of netty?

Recognizing the cons of this path I still believe that supporting kotlin/jvm first and then looking at native or multiplatform long term is flawed and I'm not sure the work done on grpc-kotlin(/jvm) is reusable. At least from my point of view and I would like to be corrected if that's not the case. It's understandable as to why you're chosing that based on what was said and also since kotlin/multiplatform is still in beta. Coroutines were also experimental/nonexistent in native until recently as well. As others have said that depending on protobuf is no longer required now that kotlin serialization is stable. This means that the kotlin grpc code gen can create data models which use kotlin serialization. This allows them to serialize to whatever format kotlin serializations supports. This leads to:

2a. grpc-kotlin deviates from current protobuf compiler code generators, making new ones that generate kotlin serializable models (which can serialize to protobuf and other formats).

2b. grpc-kotlin does not deviate and relies on protobuf compiler to generate said models and has to wait for protobuf project to add those generators.

Based on all this, like I said above, I am not sure continuing to support jvm full steam ahead is ideal (and would like to hear more of your plans). Not because jvm is less important but because building on top of kotlin multiplatform not only ensures jvm support for grpc but also future proofs it. Kotlin is headed to multiplatform and that's where the future is. You do not want to miss that train even if it means decoupling from netty. As you may have seen even Graal has issues with nettys reflection usage. That path will most likely also lead you to ditch netty as well.

P.S
Kotlin has some level of interop with C. Could that be leveraged considering grpc-c++ exists?

[andrewparmet]

Protokt now supports Kotlin multiplatform with version 1.0.0-alpha.5:

• It generates its own types which are platform agnostic with runtimes for JVM and JS (further extensible to native but not yet implemented)
• It uses this project to generate RPC stubs for the JVM
• It uses its own generator for similar RPC stubs on JS and bundles a gRPC runtime wrapping grpc-js
• It is thoroughly integration tested

We welcome feedback and experimentation.

[sebaslogen]

There is another alternative to make gRPC work on Kotlin multiplatform.

It is described in this article (I'm not the author of the article): https://prog.world/how-to-use-grpc-client-in-a-kotlin-multiplatform-mobile-project/

What it says in a nutshell:

1. Generate swift and kotlin data models from proto files. The kotlin code generator must not include any java dependency: so far this worked fine with Wire and PBandK libraries.
2. Include the kotlin generated data models in the Kotlin multiplatform module
3. Create a kotlin interface in KMP module to call the native grpc network clients
4. Implement this interface in Android and iOS using native libraries. I tried so far with success: Wire, gRPC-Swift and Connect-kotlin/Swift
5. Once the client returns a parsed/deserialized Swift data model, get the byte-array version of the response and ask the KMP module to parse it again using the kotlin version of the data models
6. Both Android and iOS clients return to the KMP code the parsed kotlin version of the data models

Here I have a couple of working sample implementations:

• With gRPC-Swift + wire (android client not yet implemented): sebaslogen/artai@main...wire-rpc-experiment
• With connectRpc + pbandk: sebaslogen/artai@main...pbandk-with-connect-swift-poc-experiment