WASM/WASI and NanoVMs - For App developers and Devops Engineers

[iamshreeram]

Instead of running the apps on Nanos VM, can we not run the apps directly in WASM runtime or WASI? After recent innovation in webassembly space (WASI, WAGI, Webassembly Cloud, etc. - socket access, file IO ), Would Unikernel / Nanosvm still be relevant?

We would be able to run wasm on wasm runtime with all the benefits that Nanos provide. How would you compare WASM and NanosVM.

Can you put some benefits of running WASM modules on NanosVM?

Thanks!

[francescolavra]

I would say running an application on a sandboxed WASM runtime is conceptually not much different from using a container-based approach: in both cases you have an intermediate layer between the host OS and the application, which means that any vulnerability in this intermediate layer can potentially result in the application being able to break into the host OS. And we know that container implementations like Docker, with all their good features and benefits, have historically suffered from many such vulnerabilities. On the other hand, when running an application as a unikernel, there is basically no host OS to break into, and you benefit from the lack of intermediate software layers in terms of simplicity, minimized attack surface, and performance.
I would also add that the ability of Nanos to run unmodified Linux binary executables is a valuable feature that you can't have with WASM.

[iamshreeram]

Thank you for providing the details, @francescolavra .

Based on my understanding, Nanos/Unikernels offer a secure and efficient way to package and run applications by eliminating intermediate layers like WASM runtime or Docker engine. They achieve this by removing unnecessary libraries and utilities from the container's operating system.

Given the advancements in WASM/WASi, do you foresee Nanos/Unikernels replacing WASM/WASi, or will they coexist? If they coexist, wouldn't adding WASM introduce another layer of abstraction within Nanos? Would it be more beneficial to run unikernels directly from a native runtime?

Here are a few questions to further explore this topic:

1. How does running an app on Nanos provide performance benefits compared to WASM / WASM inside Nanos?
2. Currently, WASM can run on a shredded container without a complete OS, reducing the potential attack vectors. If Kubernetes can orchestrate WASM directly on worker nodes instead of running it within a container, would that be advantageous?
3. Containers have a robust ecosystem for orchestration (e.g., Kubernetes) and low-overhead monitoring systems (eBPF). In contrast, Unikernels have limited orchestration tools and monitoring capabilities. How does this impact the choice between Nanos/Unikernels and Running WASM in K8s?
4. Considering the complexity of porting an application to Nanos and the lack of tooling for orchestration and monitoring, how does this affect overall management?

I look forward to your insights on these points. Thank you.

[francescolavra]

I definitely think unikernels can coexist with WASM, meaning you can run a WASM runtime as a unikernel program, as long as the runtime implementation is suitable (most notably, it must run as a single process). The additional layer of abstraction provided by the WASM runtime compared to running a native app is the same you would have when using a general-purpose OS, so most advantages and disadvantages of running WASM vs native apps apply regardless of the underlying OS.
As for the other questions:

1. You can expect the same performance benefits as you would have when using a general-purpose OS
2. I'm not familiar with how Kubernetes orchestrates WASM workloads, but using worker nodes instead of containers is in line with the unikernel philosophy, so yes, there are advantages in executing WASM workloads this way, just like with other types of workloads
3. The presence of a large ecosystem for orchestration and monitoring is a direct consequence of the popularity of container implementations; once unikernels gain momentum and their popularity grows, their ecosystem will develop as well
4. I wouldn't say porting an application to Nanos is complex, in many cases there is no porting at all, you just take a Linux binary and run it on Nanos as it is. The Ops tool automates things like creating Nanos images with the shared libraries necessary to run a given binary, and deploying these images to the cloud of your choice, and also has monitoring capabilities such as checking the status of VM instances and retrieving their serial log, so I would say it's a good starting point from which a robust ecosystem for unikernels can grow

[eyberg]

I missed this but my two cents:

1. wasm and unikernels are not at the same level of the stack - you can run many of the popular wasm runtimes inside of Nanos such as Wasmer https://repo.ops.city/v2/packages/eyberg/wasmer/4.2.2/x86_64/show and WasmEdge https://repo.ops.city/v2/packages/girishramnani/wasmedge/0.9.1/x86_64/show and wasmtime - that is to say it's less competitive and more complementary (at least in this regard). To pain the picture further you could run firecracker that runs Nanos that runs wasmer that runs some wasm payload.

2. Wasm itself is not quite enough to provide the same functionality that a traditional webserver like app needs - whether it's crypto w/TLS or threads or sockets - it needs to leverage other frameworks/tools to provide these as it doesn't inherently provide them itself. There is some movement in the ecosystem to provide this but neither wasi nor wasm provide it today. Wasix is one project looking at this.

3. wasm with its flat memory model suffers from some severe security issues including ones we specifically address in https://github.com/nanovms/nanos/blob/master/SECURITY.md . The sandbox itself might be 'good' but what happens to the application inside of the sandbox is kinda of wild west. Even if you run the wasm under a wasm runtime inside of Nanos it is still susceptible to security issues whereas if you just run the executable under Nanos (or Linux) it is not. For instance both Nanos and Linux respect the concept of read-only memory which wasm does not. Both linux and nanos will segfault if you try to write to 0x0 but not wasm. This can be very problematic.

[nuke-web3]

Found this great set of answers searching on this topic, also found https://www.nanovms.com/learn/wasm-vs-unikernels that has some notable comparisons mentioned (but not detailed) to add: