feat: run move2kube in wasm

[HarikrishnanBalagopal]

LFX link

https://mentorship.lfx.linuxfoundation.org/project/c2b5f721-2666-4d9e-85d6-7bedae27e144

Move2Kube is a command-line tool for automating creation of Infrastructure as code (IaC) artifacts. It has inbuilt support for creating IaC artifacts for replatforming to Kubernetes/OpenShift. We want to compile targetting WASM/WASI and run the resulting WASM module in the browser. This will help up showcase Move2Kube for demos and allow users to quickly try out Move2Kube without having to install it or any of its dependencies.

• Expected Outcome:
  • Run Move2Kube CLI in the browser using WebAssembly.

Prerequisites

Please familiar yourself with the topics and try compiling simple Golang programs to WASI (preferably those that use the filesystem) with TinyGo so that you are familiar with the workflow and challenges involved.

• Some in-depth knowledge of WASM/WASI and Golang/TinyGo.
• (Optional) In-depth knowledge of Move2Kube will also help a lot but we can help guide you through that.

Getting Started

• If you are familiar with Golang you can start by trying to build the Move2Kube CLI https://github.com/konveyor/move2kube
  Follow the tutorial on how to use it to do a transformation https://move2kube.konveyor.io/tutorials/cli
• If you are not familiar with WASM and WASI, there are lots of tutorials online. I suggest the TinyGo tutorial https://tinygo.org/docs/guides/webassembly/ and the MDN tutorial https://developer.mozilla.org/en-US/docs/WebAssembly
• For a more complete example of a Golang program running in the browser you can take a look at https://github.com/HarikrishnanBalagopal/asciidonut
• Virtual file system in the browser https://developer.mozilla.org/en-US/docs/Web/API/FileSystem and JS libraries for a virtual filesystem with different backends (local storage, indexed db, etc.):
  • https://github.com/jvilk/BrowserFS
  • https://github.com/streamich/memfs
  • https://github.com/filerjs/filer
  • https://github.com/isomorphic-git/lightning-fs
  • (Rust) https://github.com/kateinoigakukun/wasi-vfs
• Access to actual file system in the browser https://developer.mozilla.org/en-US/docs/Web/API/File_System_API
• (Optional) WASI FS proposal https://github.com/WebAssembly/wasi-filesystem
• (Optional) WASI support in Go compiler all: add GOOS=wasip1 GOARCH=wasm port golang/go#58141
• (Optional) Logrus added support for compiling to WASI with Go version 1.21 (will be released in the next version 1.9.4) Add WASI wasip1 support sirupsen/logrus#1388
• (Optional) Superset of WASI that adds more functionality https://wasix.org/
• (Optional) WASI support in the browser using a JS library https://github.com/bjorn3/browser_wasi_shim. Demo app https://github.com/HarikrishnanBalagopal/test-wasi-fs-browser
• Go compiler recently added support for some parts of WASI in 1.21 and expanded this support in 1.22 https://go.dev/blog/wasi

Challenges

This is not an exhaustive list, just the ones we have found so far.

• Some networking functions need to be stubbed in TinyGo similar to https://github.com/tinygo-org/tinygo/pull/3571/files
• Need to look at how to have a proper filesystem in WASI for our use-case.
• Any usage of sys/unix and syscall packages might need to be removed/made conditional in Move2Kube https://pkg.go.dev/golang.org/x/sys/unix
• List of packages supported by TinyGo is limited https://tinygo.org/docs/reference/lang-support/stdlib/
• build: wasip1 is broken on wasmer runtime golang/go#59907 (comment)

Tasks

Preview Give feedback

1. Run Move2Kube CLI in the browser using WASM/WASI

[Prakhar-Agarwal-byte]

I read the description of this issue from LFX project details, I am interested in working on this one.

[VaibhavMalik4187]

Hello, I'm interested in working on this project.

[HarikrishnanBalagopal]

Hello, I'm interested in working on this project.

@Prakhar-Agarwal-byte @VaibhavMalik4187

Thanks for your interest. For future reference, we welcome everyone to follow the getting started steps and familiarize themselves with the project itself and relevant topics like WASM/WASI. However since this issue is part of the LFX project we can't assign it to anyone in particular until the candidate selection is over to avoid any conflict of interest issues.

Feel free to start working on the issue as that will help you write a good proposal for the LFX candidate selection process.

[HarikrishnanBalagopal]

Working through some errors when compiling Move2Kube to Wasm using TinyGo compiler:

Done:

1. Fixed by adding build constraints to exclude the ArgoCD files

prakhar@prakhar-laptop:~/move2kube$ CGO_ENABLED=0 tinygo build -o wasm.wasm -target wasm .
package [github.com/konveyor/move2kube](http://github.com/konveyor/move2kube)
        imports [github.com/konveyor/move2kube/cmd](http://github.com/konveyor/move2kube/cmd)
        imports [github.com/konveyor/move2kube/lib](http://github.com/konveyor/move2kube/lib)
        imports [github.com/konveyor/move2kube/transformer](http://github.com/konveyor/move2kube/transformer)
        imports [github.com/konveyor/move2kube/environment](http://github.com/konveyor/move2kube/environment)
        imports [github.com/konveyor/move2kube/environment/container](http://github.com/konveyor/move2kube/environment/container)
        imports [github.com/docker/docker/pkg/archive](http://github.com/docker/docker/pkg/archive)
        imports [golang.org/x/sys/unix](http://golang.org/x/sys/unix): build constraints exclude all Go files in /home/prakhar/go/pkg/mod/golang.org/x/[email protected]/unix

2. Fix by stubbing the net.LookupHost function in TinyGo

prakhar@prakhar-laptop:~/move2kube$ CGO_ENABLED=0 tinygo build -o wasm.wasm -target wasm .
# [github.com/Masterminds/sprig](http://github.com/Masterminds/sprig)
../go/pkg/mod/github.com/!masterminds/[email protected]+incompatible/network.go:9:18: undefined: net.LookupHost

3. Error:

prakhar@prakhar-laptop:~/move2kube$ CGO_ENABLED=0 /home/prakhar/tinygo/build/tinygo build -o wasm.wasm -target wasm .
# [github.com/go-git/go-git/v5](http://github.com/go-git/go-git/v5)
../go/pkg/mod/github.com/go-git/go-git/[email protected]/worktree_js.go:14:30: undefined: syscall.Stat_t

Fix by using the target wasi or wasipi instead of wasm

$ CGO_ENABLED=0 /home/prakhar/tinygo/build/tinygo build -o wasm.wasm -target wasip1 .

https://github.com/tinygo-org/tinygo/blob/731532cd2b6353b60b443343b51296ec0fafae09/src/syscall/syscall_libc_wasi.go#L234-L251
https://tinygo.org/docs/guides/webassembly/wasi/

To Do:

prakhar@prakhar-laptop:~/move2kube$ CGO_ENABLED=0 /home/prakhar/tinygo/build/tinygo build -o wasm.wasm -target wasi .
# [google.golang.org/grpc/internal/resolver/dns](http://google.golang.org/grpc/internal/resolver/dns)
../go/pkg/mod/google.golang.org/[email protected]/internal/resolver/dns/dns_resolver.go:164:90: undefined: net.SRV
../go/pkg/mod/google.golang.org/[email protected]/internal/resolver/dns/dns_resolver.go:83:36: undefined: net.DefaultResolver
../go/pkg/mod/google.golang.org/[email protected]/internal/resolver/dns/dns_resolver.go:104:14: undefined: net.Resolver
../go/pkg/mod/google.golang.org/[email protected]/internal/resolver/dns/dns_resolver.go:281:29: undefined: net.DNSError

[HarikrishnanBalagopal]

Switched to a different approach. Previously we used a subtractive approach where we removed code to get Move2Kube to compile to WASM.
New approach is an additive approach where we start from scratch and add code/functionality to an already working WASM binary.
Code: https://github.com/HarikrishnanBalagopal/move2kube-wasm
Running at: https://haribala.dev/move2kube-wasm/

[HarikrishnanBalagopal]

Moved the code from https://github.com/HarikrishnanBalagopal/move2kube-wasm into this repo on a new branch https://github.com/konveyor/move2kube/tree/wasm

[HarikrishnanBalagopal]

Completed in https://github.com/konveyor/move2kube/tree/wasm branch. Deployed at https://move2kube.konveyor.io/experimental