Static build

[piranna]

Extracted code for static build from #571 to make it easier to review and
approve. It's a squashed version with the final code, for references on the
development history and commit logs go to the original issue.

[zbjornson]

Transferring from original PR: This doesn't build on Windows because there are shell scripts in binding.gyp, static.gyp and canvas.gypi. There's discussion in #754 about replacing has_lib.sh with something sturdier: either letting the compiler find the libs or porting it to JS.

[piranna]

Well, this doesn't build on Windows, but node-canvas doesn't do it too. I
think the change of the scripts should be done in another pull-request. By
the way, you could try to use WSL...

El 19/9/2016 1:51, "Zach Bjornson" [email protected] escribió:

Good chunk to split out :)

Transferring from original PR: This doesn't build on Windows because there
are shell scripts in binding.gyp, static.gyp and canvas.gypi. There's
discussion in #754 #754
about replacing has_lib.sh with something sturdier: either letting the
compiler find the libs or porting it to JS.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#813 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAgfvs_BA0ZdPf1yLx7QmHY_GvgAl01aks5qrc6YgaJpZM4J_4GX
.

[zbjornson]

Not sure I understand your reply. node-canvas builds on Windows currently.

[piranna]

Then what's the problem with bash? The 'has_lib.sh' script was already
there, I have only added some entries for the aditional libs that were used
globally before...

El 19/9/2016 8:09, "Zach Bjornson" [email protected] escribió:

Not sure I understand your reply. node-canvas builds on Windows currently.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#813 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAgfvgyRxpDCe8bRrO7zSy2CUMmjR3GVks5qricngaJpZM4J_4GX
.

[zbjornson]

has_lib.sh isn't used on Windows. Look at the top of the existing binding.gyp file; there's a check for Windows.

[piranna]

Ok, now I understand it. Fine, I have three options: state that static
builds doesn't work on Windows, add support for it, or port first the shell
scripts to Node.js... What do you think?

El 19/9/2016 8:19, "Zach Bjornson" [email protected] escribió:

has_lib.sh isn't used on Windows. Look at the top of the existing
binding.gyp file; there's a check for Windows.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#813 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAgfvgrI5f4b6I7aRe3wfim8UEsyNQuMks5qrimAgaJpZM4J_4GX
.

[zbjornson]

Supporting Windows somehow would be nice, but better to get the advice of the official maintainers...

[piranna]

I agree.

El 19/9/2016 8:25, "Zach Bjornson" [email protected] escribió:

Supporting Windows somehow would be nice, but better to get the advice of
the official maintainers...

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#813 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAgfvhDWYJ0bsEYfTN1CJFP12WgWENuNks5qrirygaJpZM4J_4GX
.

[chearon]

It's kind of unfortunate that it has to have so many gyp files and custom headers to get this to build right statically. Really increases the complexity of the build. Nice work, though, I never got this far when I tried it

My solution here was to build normally and then copy the shared libraries into the Release folder by looking at what canvas.node links to

[piranna]

This is not so much complex, it's only that Cairo has a lot of
dependencies, but my code dowload them too. Final static .node file has the
advantage that's independent of the Linux subsystem and can be published
with prebuild, and also ststic build fixes the compilation problems with
OSX and maybe Windows because it doesn't depend on external libraries.

El 19/9/2016 15:20, "Caleb Hearon" [email protected] escribió:

It's kind of unfortunate that it has to have so many gyp files and custom
headers to get this to build right statically. Really increases the
complexity of the build. Nice work, though, I never got this far when I
tried it

My solution here https://github.com/chearon/node-canvas-prebuilt was to
build normally and then copy the shared libraries into the Release folder
by looking at what canvas.node links to

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#813 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAgfvgirhB3La20-SGU06kg6flqiajcWks5qrowygaJpZM4J_4GX
.

[piranna]

Regarding the dynamic libraries, I proposed to Node.js dev something
similar to what you do, only that by distributing the dynamic libraries on
independent Node.js module dependencies so they could be reused and shared
by diferent modules, but seems they didn't liked the idea... :-(

El 19/9/2016 15:41, escribió:

This is not so much complex, it's only that Cairo has a lot of
dependencies, but my code dowload them too. Final static .node file has the
advantage that's independent of the Linux subsystem and can be published
with prebuild, and also ststic build fixes the compilation problems with
OSX and maybe Windows because it doesn't depend on external libraries.

El 19/9/2016 15:20, "Caleb Hearon" [email protected] escribió:

It's kind of unfortunate that it has to have so many gyp files and custom
headers to get this to build right statically. Really increases the
complexity of the build. Nice work, though, I never got this far when I
tried it

My solution here https://github.com/chearon/node-canvas-prebuilt was to
build normally and then copy the shared libraries into the Release folder
by looking at what canvas.node links to

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#813 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAgfvgirhB3La20-SGU06kg6flqiajcWks5qrowygaJpZM4J_4GX
.

[piranna]

Please consider this for 2.0.

[piranna]

Any update on this one?

[piranna]

Could this be reviewed by any of the project maintainers? It has started to get some conflicts with master but didn't get yet any news about its mergeability status... :-/

[LinusU]

I'm sorry for the delay on this, it's a rather large change and I haven't got much time. I'll try to get some open source work done this weekend so hopefully I can look thru this...

[piranna]

Cool :-) If you need some help don't doubt to ask :-)

[rgbkrk]

Would love to see prebuilds for node-canvas. ❤️

[piranna]

Code rebased with changes on master.

[markusthoemmes]

@piranna Do I understand this PR right as in it generates a statically linked version of node-canvas without dependencies on the packages (libpng etc.) at runtime?

[piranna]

@piranna Do I understand this PR right as in it generates a statically linked version of node-canvas without dependencies on the packages (libpng etc.) at runtime?

Yes, that's it, there's no runtime dependencies.

It checks on build time if Cairo library is available to do a dynamic build as usual, and if not, it does a static one. Static build can be forced too by passing the --has_cairo=false flag.

[markusthoemmes]

FWIW, this constantly fails to build on my Mac, exits with the following log:

tar: Unrecognized archive format
tar: Error exit delayed from previous errors.
gyp: Call to 'static/ensure_deps.sh' returned exit status 1 while in binding.gyp. while trying to load binding.gyp

[markusthoemmes]

This link is no longer valid as zlib has updated to 1.2.11.

For sanity, this should be pointing to sourceforge as well, as zlib doesn't seem to keep history of their old releases.

https://sourceforge.net/projects/libpng/files/zlib/1.2.8/zlib-1.2.8.tar.gz works.

[piranna]

I'm just working on that, I'll push a commit with the fixes when it's ready, sorry for the inconvenience.

[chearon]

Does everyone really think it's a good idea to be tracking 3rd-party header files and hard-coded links to external tarballs? And a custom gyp file for each dependency? This is really complicating an already complicated build. The approach @eonarheim outlined in #641 doesn't change the build.

If this does get merged, there's also the other half of the problem, how does @LinusU publish the binaries to NPM?

[piranna]

If this does get merged, there's also the other half of the problem, how does @LinusU publish the binaries to NPM?

prebuild-install is your friend :-)

[chearon]

prebuild-install is your friend :-)

OK, but it still has to be done on each of the 3 platforms. So assuming we do that through CI servers, someone has to set up each of those

[piranna]

I have just send an email to @bnoordhuis asking for info about how we can be able to add those gyp files in the gypified organization.

[rgbkrk]

Always amazing to see this PR continuing on. 😄

[piranna]

Always amazing to see this PR continuing on.

What do you mean?

[rgbkrk]

I just happened to notice there were more commits and wasn't merged. Sorry for the notification spam.

[piranna]

I just happened to notice there were more commits and wasn't merged. Sorry for the notification spam.

No problem, just wanted to keep my pull-requests updated to make them easier to be merged someday :-) Or at least get some feedback about what does they need to be improved.

[piranna]

I've seen by chance what was the issue here and fixed it, and now tests are passing on Windows, Linux and macOS. Windows has disabled support for static builds, probably it could be possible too but I don't use Windows so I can't try.

[piranna]

Code sync'ed with latest master branch and all tests are fixed and passing in all platforms. Can this pull-request be reviewed and merged? :-)

[d4tocchini]

@piranna, that's some serious gyp!

In case others like myself are seeking to quarantine node-gyp nuances in favor of CC flavored cli and make/cmake/etc builds, here's a PoC node-gyp wrapper, node-gypcc, that conforms to gcc/clang/emcc cli. It's early (released <24hrs ago, PRs welcome). Hope it helps, cheers!

[LinusU]

I have just send an email to bnoordhuis asking for info about how we can be able to add those gyp files in the gypified organization.

Did anything come off this? I'm still very hesitant to adding the vast amount of header files that this patch currently brings 😅

[piranna]

I'm still very hesitant to adding the vast amount of header files that this patch currently brings

What issues do you see?

[LinusU]

What issues do you see?

It brings in ~6000 new lines of code that we need to maintain. I'm not familiar with the process enough, which is really the biggest problem.

e.g. HAVE_SCHED_H is not defined, but sched.h seems to be present in recent Linux versions, why was this not set? I'm absolutely not saying that it should be set, I don't know anything about this. But I'm just not sure why we have 6000 lines of hardcoded options, and wether that will work for every platform.

Another example is that it hardcodes __attribute((visibility("hidden"))), is this standard C++ or is specifically gcc or llvm required now? 🤔

Will this work with Apple aarch64 platform? Or arm64 Linux? Or is this something that we now need to add support for manually? 🤔

[piranna]

I'm just not sure why we have 6000 lines of hardcoded options, and wether that will work for every platform

I just generated the config files with the default options for all the needed libraries, similar to how it was done in other ones. Must to admit I'm not fully happy with having so much hardcoded options too, but seems it's the only way. Maybe they can be generated on build time too, but I'm not sure.

Another example is that it hardcodes __attribute((visibility("hidden"))), is this standard C++ or is specifically gcc or llvm required now?

Will this work with Apple aarch64 platform? Or arm64 Linux? Or is this something that we now need to add support for manually?

Honestly, no idea... :-(

[piranna]

In any case, some of my other PRs regarding screen drawring are more useful and easier to merge, maybe we can review them first and later retake this one, in case it's still needed, what do you think?

[piranna]

I'm talking about https://github.com/Automattic/node-canvas/pulls?q=is%3Aopen+is%3Apr+author%3A%40me+sort%3Aupdated-desc, but more specifically, #1543.

[piranna]

In any case, maybe this PR can be splitted in two, one for the gyp config files to allow static vs. dynamic libraries, and another one with the specific config changes, in case it's still needed.

[zbjornson]

Despite all the hard work, unfortunately I don't really see a good path forward still.

• This is a lot of build code to maintain.
• The config files are not universal. I pointed out a few examples in Static build #813 (comment). I don't know how Node.js deals with this with its dependencies, or if their deps happen to not use autotools/config headers.
• We'll still have the issue that users have with musl vs. glibc, and with various version of glibc (old CentOS).
• The links in https://github.com/Automattic/node-canvas/pull/813/files#diff-c90228f531302537249e40c7bb66cb4c9663cc8afe2815338f3139501af11c78 are bound to break. When that happens, the only way we could fix it would be publishing a new version; i.e. code depending on older versions of node-canvas would spontaneously break. Node.js has the source code of its dependencies in its git tree, which is safer both for that reason and in case malicious code is published at an existing URL.
• Fixable: using -march=native to make prebuilds will result in binaries that don't run on older machines.
• Fragments the build: Windows would still be dynamic, Linux (and macOS?) static.

I don't really have suggestions either unfortunately. It would be worth looking at how Firefox deals with this scenario since they use a lot of the same dependencies.