Cross compilation

[sromovskypeter]

Hello,

Is it possible to do cross-compilation with node-gyp? I want to compile native addon for linux while running node-gyp build on Windows machine.
I am able to build addon on each of the platforms, but I would like to do cross compilation.

Thank You,
Peter

[bnoordhuis]

It's not explicitly supported at the moment and there are a few places in node-gyp where it assumes that the host arch/os == the target arch/os although you can probably work around that without too much effort.

More importantly, add-ons need to be compatible with cross-compiling. Add-ons with simple binding.gyp files probably are but more complex ones need not be.

[cdaringe]

@bnoordhuis, i have a similar question. I've been toying with electron-rebuild. as you may be aware, my node version !== electron's node version, so electron-rebuild has to recompile some modules against it's version. I see the rebuild module doing something like npm rebuild --target=TARGET_VERSION --arch=ARCH_VERSION. That doesn't seem to reliably product builds to my expectation. e.g. node v0.12.7 has process.versions.modules == 14, while my electron's === 44. the above command still seems to build against 14, vs 44 like I expect.

Do you have any tips on how I can best use npm to recompile against a target version? Assume that I have the target header files already downloaded. I need to use npm vs explicit node-gyp as there may be other lifecycle hooks that must be run for a module at install.

I will gladly add to the README with pertinent docs if you can assist!

[bnoordhuis]

@cdaringe You probably need to specify --nodedir /path/to/sources, otherwise it's assumed you're targeting the same binary that is used to execute node-gyp.

[cdaringe]

@bnoordhuis, thanks, I'll check it out. I did trace my original problem to the fact that some npm lifecycle scripts were conditionally rebuilding based on process.xzy, where the process was the sys process vs. the electron node process. I updated my stuff to rebuild using electron's node by appending env: ATOM_SHELL_INTERNAL_RUN_AS_NODE. yahtzee!

[ricardoespsanto]

+1

[PaulBGD]

Has any progress been made on this? Also is changing the OS inside node-gyp enough to make it cross compile?

[tyduptyler13]

Well, I need to cross compile, so I'll be messing around with node-gyp or alternatively using raw gyp. I'll let you know how it goes.

[rlabrecque]

Any update on this @tyduptyler13 ?

[tyduptyler13]

I have yet to prove this but I am pretty sure you can pull it off if you compile node itself from the platform you are compiling on as well. The reason being, it needs the same mangling scheme as your addon. I will get around to doing this soon. I am currently moving and starting a new job so I'm a bit busy lately.

It would be nice if there is a way to create a translation layer between windows and linux compiled code so that you can still do the dynamic linking at runtime (how node calls your stuff)

[PaulBGD]

@tyduptyler13 So uh.. webassembly?

[piranna]

@tyduptyler13 So uh.. webassembly?

That's would be cool :-P

[tyduptyler13]

Web assembly is a target of mine, it usually has the same mangling scheme as clang as it sort of emulates the behavior of a compiled C/C++ application.

Another point I can make is that if your addon only exposes C style anythings, you can totally get away with cross platform compiles because C doesn't get its names mangled. Hence the "extern C" stuff in lots of the old school addons.

I was exploring to see if node-gyp could just do some of this mangling management, but I haven't gotten deep enough to say yet. Either its loader could handle these weird schemes internally when selecting the C++ name OR it could try cheating and use ASM flags to manually set the name. The ASM would probably be easier, but could have issues if you don't handle all of the weirdness with namespaces, class names, templates, inheritance, etc.

I'll keep you guys updated if I ever get my shit compiling cross platform.

Also, side note: tempted to create a node driven build platform that sort of follows Gulp but would be more flexible in what it can compile. The goal is to allow node to compile c/c++/etc. That way I can remove gyp entirely and simplify the build process. Effectively it would be among gyp, make, cmake, and autotools type tools. This would definitely increase cross platform support. Would any of you be interested in using such a thing?

[piranna]

This would definitely increase cross platform support. Would any of you be interested in using such a thing?

Probably me, to port all my NodeOS custom build bash scripts :-) I was thinking also to do it myself, since I got them to have a somewhat similar structure and checkings (download sources, configure, make, make install), just only I don't have time, but would be a matter of porting them. You can take a look at them to know how they are organized and we could chat about moving it forward :-D

[mjangir]

Hi, my electron application has node ref and ffi modules for dll linking and it works perfectly fine on windows with electron-rebuild and after electron-builder but when I clone the source code on Ubuntu 18 and run the electron-rebuild the created exe file says %1 is not win32 application.

Could anybody please help me out in this? I'm struggling a lot to get it worked.

Thanks in advance.

[Not4me]

@mjangir You need to recompile your dll to .so for Linux and electron app will be work fine.

It maybe be better to create dinamic linking library for all support platforms, then create a nodejs addon with your export functions. Then you can compile the addon for the node or the Electron App.

If you have a question about creating add-ons, you can ask them

[rvagg]

I'm going to close this because I think a pull request by someone who wants this is going to be the only way forward here. Otherwise this issue is just going to stay open perpetually with occasional drop-ins. This will require external help to get it moving anywhere if it's still needed.

[artus9033]

@rvagg I was able to successfully rebuild native modules, even with electron-forge & electron-rebuild that utilize node-gyp under the hood (see electron/rebuild#378).
Cross-compilation from other host architectures seems to be easily achievable by prepending CC=arm-linux-gnueabihf-gcc CXX=arm-linux-gnueabihf-g++ CC_host="gcc -m32" CXX_host="g++ -m32" env variables before invoking node-gyp and requires minimal system configuration, only the installation of the target toolchain. How about merging this approach to the tool, so that it automatically sets the proper variables?

[aaomidi]

@artus9033 with what you're suggesting how would that work for building on Linux to support Windows platforms?

[cclauss]

@rvagg wrote:

a pull request by someone who wants this is going to be the only way forward

@artus9033 wrote:

Cross-compilation from other host architectures seems to be easily achievable

Is there a pull request as @rvagg requested?

[artus9033]

@artus9033 with what you're suggesting how would that work for building on Linux to support Windows platforms?

I haven't tested it yet, but when I find time, I will give x86_64-w64-mingw32-gxx a shot, set up analogically to the approach I described above. I suspect the difference would be that, when compiling from Linux to Windows, additionally the mingw-runtime package should be installed on the host to provide standard Windows headers.

@rvagg wrote:

a pull request by someone who wants this is going to be the only way forward

@artus9033 wrote:

Cross-compilation from other host architectures seems to be easily achievable

Is there a pull request as @rvagg requested?

@cclauss I will try to prepare one and reach out then, for now I just wanted to share this approach, as at the moment I don't have enough time to try to implement this as part of node-gyp.

[Animeshz]

Did anybody got any updates? I want to know where/how to get the shared libraries to link to the N-API functions while cross-compiling.

[AubinMahe]

I've writing an N-API extension on Node.js 14.16.0
Works well under Linux x86-64, compiled with GNU-gcc 8.3.0 but requires VisualStudio 2019 to address Windows server 2016.
When cross compiling from Linux to target Windows with mingw x64, Node says : the extension does not self register.
Why the same code base (very near of the samples in official documentation) compiled or cross-compiled acts differently?
If necessary, I can push a minimalist reproducible code.