add OS X build

[chearon]

This is a small part of a larger project I'm working on to automate build & deploy binaries to NPM. I thought it would be good to get this in sooner since it's sort of an independent part

There was a bug in Travis (well, really in Node on OS X) that I had to wait to be fixed till I could create this

One downside is that the builds take stupid long now.

• master build on my fork - 11m 36s
• add-os-build build on my fork - 28m 7s 😖

I have seen it sometimes go faster

Unrelated sorta:

I've been getting really close to having the Windows version of pango work with MSYS2 instead of a downloaded ZIP from the wiki. When I get that done, the path is clear to having a Windows build on AppVeyor, but it will change the wiki and the binding.gyp significantly!

[LinusU]

this should be indented by two spaces

[LinusU]

Could you please rebase on master now that I landed #797. Hopefully this should speed up the build quite a lot.

Stole your GCC update also, sorry about that, wanted it as a separate commit. It had no impact on test duration btw

[LinusU]

Oh, and forgot to mention

This is a small part of a larger project I'm working on to automate build & deploy binaries to NPM.

🙌 🙌 🙌 🙌

[chearon]

Hopefully this should speed up the build quite a lot.

Yeah that'll let more of them go in parallel - I think sometimes Travis is just slow though, this PR built in 8 minutes. I'll rebase in a sec

[chearon]

ok good to go

[zbjornson]

... the path is clear to having a Windows build on AppVeyor,

Doesn't use the MSYS2 approach, but an AppVeyor setup that uses the ZIP is here if it's any use for you: #672

[LinusU]

Before:

Elapsed time 2 min 36 sec

After:

Elapsed time 9 min 59 sec

This is unfortunate, would it make sense to only build OS X on Node 4 & 6? I imagine those being the targets (LTS and Current) that we should build for if we start publish binaries?

Regarding the tests, I don't think that many people are using Node.js 0.x on OS X. Also, if it works on Linux it'll probably work on OS X as well...

[chearon]

Thing is, all of the Nodes have different ABIs. So if I do finish automated binary deploys, it will have to build each one anyways... I could look at making the build faster in other ways, brew takes 2 seconds so maybe if it just didn't run brew update first or something

@zbjornson oh I didn't realize there was work done on a Windows build, is the only thing blocking that really just getting an Automattic account? Aside from that I think we should promote MSYS/MinGW, took me a while to figure out but it ended up being simple

[zbjornson]

Is it possible to cache whatever brew updates when you run brew update? Circle CI also provides four concurrent linux containers for free (but no free OS X containers) -- that would help for the linux build matrix at least, but linux users probably have the lowest barrier to installing.

Re: my PR, yeah, just the Appveyor account, and the JPEG tests fail without #458 merged. Agreed it would be nice to use the newer distributions, but almost definitely need to provide precompiled binaries so people don't have to install both Visual Studio and MSYS & friends.

[chearon]

@zbjornson I've been trying caches as per your recommendation, I tried:

• cache /usr/local/Cellar - doesn't work because you have to cache sym links, so:
• cache cellar and /usr/local/<library names> - doesn't work because some formula (glib) install to like 30 directories
• cache ~/Library/Caches/Homebrew (cached .tar.gz files) and /usr/local/Library/Taps (cached taps from brew update)

I really thought that last one would work but somehow the cache ends up being 4GB.

The biggest impact though is that OS X builds don't start until all the Linux ones are done, and that they seem to take 1 minute to boot. I'm kind of running out of ideas :/

[zbjornson]

Boo... nice effort though.

Maybe it's worth considering using circle for linux then so Linux completes quickly and the OS X builds start right away on Travis?

[chearon]

I looked at their pricing earlier and it seemed like you could only get 1 VM and 1 concurrent build for free. Maybe another CI server is the way to go though.

I can shave off 30 seconds by caching the kegs and figure out a way to get Homebrew to stop calling fc-cache.

Or maybe an 8-9 minute build isn't worth a ton more trouble?

[chearon]

sooooooooo I did end up figuring out how to cache Homebrew downloads (it's in the rebased commit), it cuts off 20s. They key was $HOMEBREW_CACHE. next time a commit is made on this one it would be slightly faster

I'm throwing in the towel here tho, I spent way too much time on this and ultimately Travis's OS X VMs are just kinda slow

An alternate if @LinusU doesn't want to merge this is that we could somehow keep Linux-only for CI/tests and have separate/slower builds for making and releasing the binaries? Anyone have any ideas on how to do that?

[zbjornson]

Props again for the dedication.

I can't find any way to have separate CI configs (a fast test matrix one that runs automatically, and a slow build one that is triggered manually), aside from using two different CI services. Another possibility could be using $TRAVIS_BRANCH to only run OS X on master.

This repo isn't terribly high traffic as far as pushes go, so maybe the slowdown doesn't matter anyway. The benchmarks were in there until recently and they took about this long.

[chearon]

Oh yeah, something like that could work, it will still launch the OS X VM before it can even check that though.

I'm thinking about closing this and having build+deploy to NPM as a separate, outside project somehow. I could have it mirror this repo, it publishes builds to NPM as canvas-prebuilt or something or other. Won't reduce all the build issues people have here but if it works well it could be merged in here later

[chearon]

I just published canvas-prebuilt which you can use as a drop-in replacement for canvas on all 3 platforms. These binaries are deployed to GitHub by the CI servers after successful builds on a tag. The NPM library just wraps a copy of canvas and uses node-pre-gyp to place the binaries into it.

I have to manually update the version number along with canvas releases, but they auto-build and bundle shared libraries on all platforms (notes) and as a bonus the Windows build uses MSYS2 so you get all features. Would love to hear feedback from you guys and maybe this could eventually become a PR.

One thing I did to speed up the OSX builds is just iterate through Node versions and run nvm manually, that way you use one boot instead of 5