Plugin interface: Language binding

[hosie]

Need to decide the means and methods by which new / custom / 3rd party implementations of privacy preserving contracts can program their domain specific behaviour and how it would be bound to the paladin runtime. This issue it intended to explore options for the programming language(s) to support and once that has been decide, we can consider the lifecycle for installation, registration, versioning & update etc...

Golang
The majority of Paladin, and the host process, is golang so it would be a relatively trivial effort to define a golang based plugin interface.

However, for many of the target audience golang might not be a viable option. For example, for zero knowledge proof verifiers, outside of Circom, most toolkits are Rust based.

Java
We do need to support embedding Java for the programmable private EVM layer but it is not clear the there would a signifiant enough demand for a general Java based plugin interface to justify the complexity of engineering required to do so and embrace challenges like class loading isolation etc... And there is almost no known support for ZKP in java.

WebAssembly
We do need to support embedding of Web Assembly modules in the ZKP tokens provider because the prover circuits are compiled to WebAssembly. It seems like an obvious good idea to provide a web assembly based plug point specifically for ZK proofs.

This may also be a convenient option for all hooks because many modern languages can be compiled to Web Assembly and it is trivial to ensure isolation between modules from different providers and the core engine code and memory.

This is worth an investigation to see what our options are for web assembly.

Remote
We could define an HTTP based / WebHook plug point so that the extensions could be implemented in any language that can run an HTTP server or can be deployed as e.g. lambda functions or serverless workloads. This seems to be a necessary option to provide but may not be sufficient for all use cases and we still need to evaluate some of the above options - or alternatives - for local binding.

[hosie]

Investigation on WebAssembly

There are a few implementations of Web Assembly frameworks for golang. https://wazero.io/ and https://wasmtime.dev/ seem particularly active and popular projects.

From experimentation it does seem very straight forward to run modules under these framework.

One big drawback of WebAssembly is the fact that the only datatypes it supports on its interface is numbers (int and floating point). Given that we are likely to require things like strings, arrays and complex object structures, this is a big inhibitor.

There are a couple of options to get beyond basic numbers

• Component model is an emerging standard. Not yet W3C recommended and not supported by all frameworks ( wasmtime supports it but wazero does not) .
• Frameworks exist (e.g. https://extism.org/ ) , built on top of WebAssembly. These frameworks tend to be opinionated so some further investigation would be required to test whether those opinions are friendly to paladin's needs and constraints
• Roll our own framework. This is a non trivial undertaking so should be considered only as a last resort if we decide we absolutely must have general WASM plugability and none of the above options are acceptable.

My current belief is that Component model is the horse to back long term although we might see some churn while the community figures out the final version of the spec that will be standardised.

[hosie]

After speaking to @onelapahead , I believe that for remote binding, it is worth considering gRPC vs JSON HTTP.

We may also find that protobufs could come in useful for the "roll your own framework" solution for flexibility on the Web assembly interface, if we need it.

[hosie]

Component model ... wasmtime supports it

after various attempts to prove this out, I now believe that the go implementation of wasmtime does not support component model yet. Component model support is relatively new to the core wasmtime implementation ( which is Rust) and then it will need to be exposed on the CAPI and then the folks maintaining wasmtime-go will need to do some work to enable it there. So I think we need to rule this out in the short term.

[hosie]

So current focus of investigation is

• WASM specifically to call ZKP circuits
• gRPC and/or protocol buffers for local binding (and possibly also remote)

[peterbroadhurst]

I've been looking at Java and Golang sharing a process, and multiple places point me at a problem on Windows currently where a Golang parent process cannot start a JVM on Windows:
golang/go#58542

On the other hand, it seems like -buildmode=c-shared is supported for Windows DLLs (which a long time ago when I was last looking wasn't true).

So the experiment I want to do is:

• JVM parent process
• Call go function over JNI via DLL
• Start gRPC server in Goroutine
• Connect to gRPC server from calling Java process

This would make java "special" compared to Web Assembly, by needing to be the parent, but given the nature of Java that doesn't seem a big problem.

Our progression to a gRPC based world might over time for high performance scenarios lead us to multi-process scalable runtimes anyway.

[peterbroadhurst]

Progress update on experiment:

• Example Gradle/Intellij project in Java: https://github.com/kaleido-io/paladin/tree/domains/prototype/babel
• ... calling Golang function over JNI: https://github.com/kaleido-io/paladin/tree/domains/prototype/gable

Tested on Mac with all the Gradle build time things sorted for library paths, and then using DYLD_LIBRARY_PATH at runtime.

# Build the go shared library
cd prototype/gable
make
cd ../..
# Build and run the Java project
cd prototype/babel
gradle build
DYLD_LIBRARY_PATH=$PWD/../gable java -cp ./build/classes/java/main "-Djava.library.path=./build/libs/paladin/shared"  io.kaleido.Main

Output should be:

Hello From C++ World!
Hello from Golang world

[peterbroadhurst]

More progress on builds with gRPC and protobufs work from @hosie merged into both the Golang and Java sides.
Attempting to use UNIX Domain Sockets in Java + Netty on Mac (which is fine on the Golang side):

Caused by: java.lang.IllegalStateException: Only supported on Linux

Code link for posterity:
https://github.com/kaleido-io/paladin/blob/556267ba2db11719d79b1e7ecc8d6bc27fcb5346/prototype/babel/src/main/java/io/kaleido/Main.java#L47-L51

[peterbroadhurst]

Update: might be that it's kqueue vs. epoll rather than UDS that's the problem in Netty.

[peterbroadhurst]

Ok ✅ - there's some Mac build specifics now in the Gradle that will need reconciling for happy cross-platform life for developers, but ping and pong all good.

time="2024-07-08T18:42:23-04:00" level=info msg="server starting at unix socket /var/folders/tb/b88h058x6675g5g_dkgsy8jm0000gn/T/paladin12056374242347222721.sock" pid=91606
time="2024-07-08T18:42:23-04:00" level=info msg="server listening at /var/folders/tb/b88h058x6675g5g_dkgsy8jm0000gn/T/paladin12056374242347222721.sock" pid=91606
Jul 08, 2024 6:42:24 PM io.netty.bootstrap.AbstractBootstrap setChannelOption
WARNING: Unknown channel option 'SO_KEEPALIVE' for channel '[id: 0x37e62edd]'
time="2024-07-08T18:42:24-04:00" level=info msg="Received event id:\"0cf33e78-87ce-4d36-873d-85c3beb8072c\" type:\"ping\" [ping]" contractId=
Response in Java RKHVbDopv6HfzewJeyUd2 [ack] to 0cf33e78-87ce-4d36-873d-85c3beb8072c
time="2024-07-08T18:42:24-04:00" level=info msg="EOF - exiting" contractId=
Response stream in Java shut down

Note WARNING: Unknown channel option 'SO_KEEPALIVE' for channel is a Netty weirdness that's been documented, when using UDS with gRPC.

https://github.com/kaleido-io/paladin/blob/0e494d7a5b8bd5be84d1479e02af2d4b7b257dcd/prototype/babel/src/main/java/io/kaleido/Main.java#L46

[peterbroadhurst]

Note we were missing DCO signoff for some commits in the domains branch, so that's been fixed - but we lost some timestamps.

For posterity the commit head of the original branch before the rewrite was: https://github.com/kaleido-io/paladin/tree/0e494d7a5b8bd5be84d1479e02af2d4b7b257dcd

[peterbroadhurst]

@hosie and I have both done a bunch of independent research into where WASM/WASI fits into a ZKP code iteroperability story, as a way to plug in advanced cryptography modules that use languages like Rust - particularly as Circom is used in zeto.

We particularly looked at what it would take to try and create a standardized Paladin interface to/from WASM. Either:

1. By putting a gRPC client inside a WAS module
   • This does not seem possible with the state of the art in WASM module compilers and WASI, particularly as wasi-cloud-core deferred the inclusion of gRPC to a separate spec it seems probably there's not a momentum there either
2. By creating a standardized Paladin function interface for WASM modules to implement
   • The WIT / component-model workstream is one option here, but immature
   • Building our own equivalent of WASI on top of a framework like wasmer in Go is the other, and by far it seems the most adopted approach (if you rule out using stdin/stdout as a comms boundary)

This is demonstrated best, by what Circom recommends in it's Go client https://github.com/iden3/go-rapidsnark/blob/9323fbec34a32d2d108b655d119c0f422f84afb0/witness/wasmer/circom2witnesscalc.go#L157-L165 - which itself demonstrates how tightly coupled the Go code and the WASM code become.

Moreover, it shows that Circom generates this opinionated WASM from its compiler - so it wouldn't plug in directly anyway.

Our conclusion is that right now the coupling between the WASM code, and the Go code, really needs to be tight - and as such we should:

• Support in Paladin dynamically loaded Go modules
• Zeto will be an example of that type of go module
• Zeto will use WASM, but indirectly through the Circom API wrappers

[peterbroadhurst]

To avoid complexity in dev environments, we are moving from JNI to JNA as validated with this commit:
8efd1ae

[hosie]

decision has been made to use C shared library (from java JNA) or java jar file loading and always use GRPC for calling functions / passing data