Cross-compiling with Zig Build

[qwtel]

A niche that txiki fills that's currently not possible any other way is small, self-contained executables using JS. A console.log("hello world") with Deno or Bun compile will be ~50MB, while tjs can be as small as ~1MB with sqlite3, wasm and mimalloc disabled.

What's missing is the ability to cross-compile executables for distribution. To that end, I've ported the build system of tjs and its main dependencies to Zig Build which allows me to produce binaries for all major OSs and architectures in one build step on one machine. See here for example: Build step and corresponding GH Release:

File Name	Size
aarch64-linux-gnu.zip	1.59 MB
aarch64-linux-musl.zip	1.67 MB
aarch64-macos.zip	1.55 MB
aarch64-windows.zip	1.53 MB
x86_64-linux-gnu.zip	1.66 MB
x86_64-linux-musl.zip	1.73 MB
x86_64-macos.zip	1.56 MB
x86_64-windows.zip	1.64 MB

I've also added a parameter to tjs compile that allows me to create platform-specific executables from a JS bundle from any machine, e.g. tjs compile -x ./zig-out/aarch64-windows/tjs.exe my-bundle.js my.exe. It seems qjs byte code is platform independent (?) so all that was needed was appending the compiled bytecode after the magic string. Very cool!

Unfortunately it's not all sunshine and roses. Because I don't need network and FFI for my use case, I've removed curl and libffi from the project, which would probably be the hardest dependencies to cross compile. Also, there are some missing header files for macOS in Zig, so currently macOS builds only work on macOS... The github runner is set to macos for now.

If that sounds interesting, you can browse the code here: https://github.com/qwtel/zig-build-txiki. There are some unrelated changes in there, like modular sqlite3 and wasm, and msvc compatibility (I've went down that rabbit hole before getting zig build to work) and it's generally in a bit of a messy state.

EDIT: Also, here are the sub modules:

• https://github.com/qwtel/zig-build-quickjs
• https://github.com/qwtel/zig-build-libuv
• https://github.com/qwtel/zig-build-mimalloc
• https://github.com/qwtel/zig-build-wasm3 (wasm3 already has a zig build file, just added a 1liner there)

[saghul]

👋 Hey there! This is really cool, thanks for sharing!

A niche that txiki fills that's currently not possible any other way is small, self-contained executables using JS.

Yes!

while tjs can be as small as ~1MB with sqlite3, wasm and mimalloc disabled.

How much size do you lose by dropping mimalloc? Generally speaking, I'd like txiki to be as small as possible, but not sacrificing "core" features. Of course that's a very loose term! :-)

The bytecode is cross-platform indeed, so as you already proved, cross--compiling exeecutables is within arms reach.

Right now the "problem" is our dependency on libcurl. Since we dynamically link to it the executable is not 100% standalone. Once I remove it (it will take a while, but the idea is to use libwebsockets) we could in principle create fully static builds with musl and attempt cross-compilation as you did.

Thanks for your work, it will certainly be a stepping stone!

[qwtel]

How much size do you lose by dropping mimalloc?

Not much, around 100KB. The one that really makes a difference is sqlite, removing it shaves off ~1MB. The smallest binary I can get is 1.3MB. Removing wasm3 doesn't reduce the size much either, but it does fix the build for arm-linux-gnueabihf

The bytecode is cross-platform indeed.

Thanks for confirming. It has worked in practice but wasn't sure if there might be exceptions.

Right now the "problem" is our dependency on libcurl.

I see, good thing I didn't try and make it work with zig build if it's going away.

Thanks for your work, it will certainly be a stepping stone!

Unfortunately my fork is drifting further away, but I couldn't resist adding a module written in zig now that I have it in my repo...

[saghul]

You got me thinking :-) Maybe not in the direction you originally intended, but still!

There could be value in an even smaller txiki.js build. I see 2 options, both of which are doable:

• Create a polyfills bundle for QuickJS, which can be pre-imported in the CLI, this would provide a lot of missing APIs
• Create some "minimal" build flag for txiki.js, where we keep it to the bare minimum: no curl, no wasm, no sqlite, just scripting and fs operations mostly

Doing the latter is a bit annoying but doable, and I think the current size is somewhat reasonable.

Making the polyfills reusable sounds like a nice first step, and other QuickJS based runtimes could also use them...

Regarding the standalone binary generation, I believe it would still be doable with the current scheme, by downloading the required arch binary from the releases page. I'll look into that at some point!