Run CI on Windows with Appveyor

[doug-wade]

We currently run our CI build on Linux, but it would be sweet to run them on Windows too with Appveyor. Unfortunately, the build is currently broken on Windows, so we'll have to get a Windows machine to work with and fix the build before we turn on the CI.

[k-ode]

Mocha is not installed in any node_modules folder. I checked the docs for Lerna, and this seems to be by design. From their readme:

Note that devDependencies providing "binary" executables that are used by npm scripts still need to be installed directly in each package where they're used.

They recommend pulling up the dependency to the root repo.

Not sure why it works for you on Linux, maybe you have mocha installed globally?

[gigabo]

@kimgronqvist - The mocha dep actually can't be pulled up to the repo root because it's being used as an executable in an npm script. But it looks like the dependency is there, so... Maybe it just failed to build? Posted some more error output in #534.

[k-ode]

Looking at the logs, there's an issue with node-gyp. There's instructions for setting it up properly for Windows here: https://github.com/nodejs/node-gyp

I have node-gyp configured already, and didn't get any errors during bootstrap. But when I tried to run the tests, it couldn't find Mocha because of how Lerna handles devDependencies that run as scripts.

When I run npm install manually in babel-plugin-react-server, it finds Mocha fine. However, at that point the tests complain about a missing src folder.

[gigabo]

Oh, man... don't tell me lerna run doesn't work on windows. 😬

[doug-wade]

complains about a missing source folder

this source folder?

there's an issue with node-gyp

Looking at the link you included, it seems like we'd need to have admin permissions on the ci instance to install windows-build-tools, which I don't think we have. The Appveyor vms do have visual studio 2015 preinstalled, so I'll try adding

npm config set python python2.7
npm config set msvs_version 2015

as part of init and see if that helps mocha build.

[doug-wade]

I tried both installing windows-build-tools and the manual configuration steps, but the error doesn't seem to change. Reading the error text a little more carefully

C:\projects\react-server\node_modules\bufferutil>if not defined npm_config_node_gyp (node "C:\Program Files (x86)\nodejs\node_modules\npm\bin\node-gyp-bin\\..\..\node_modules\node-gyp\bin\node-gyp.js" rebuild )  else (node "" rebuild ) 
Building the projects in this solution one at a time. To enable parallel build, please add the "/m" switch.
  bufferutil.cc
  win_delay_load_hook.c
C:\Program Files (x86)\nodejs\node_modules\npm\node_modules\node-gyp\src\win_delay_load_hook.c(34): error C2373: '__pfnDliNotifyHook2': redefinition; different type modifiers [C:\projects\react-server\node_modules\bufferutil\build\bufferutil.vcxproj]
  C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\include\delayimp.h(134): note: see declaration of '__pfnDliNotifyHook2'
gyp ERR! build error 
gyp ERR! stack Error: `msbuild` failed with exit code: 1
gyp ERR! stack     at ChildProcess.onExit (C:\Program Files (x86)\nodejs\node_modules\npm\node_modules\node-gyp\lib\build.js:276:23)

it seems like we might be hitting nodejs/node-gyp#972, so I think I'm going to try a different version of node maybe?

[doug-wade]

Neither 4.4.7 or 6.3.1 resolves the issue. I wonder, considering how long after the error is emitted, whether the C:\Program Files (x86)\nodejs\node_modules\npm\node_modules\node-gyp\src\win_delay_load_hook.c(34): error C2373: '__pfnDliNotifyHook2': redefinition; different type modifiers [C:\projects\react-server\node_modules\zombie\node_modules\ws\node_modules\utf-8-validate\build\validation.vcxproj] C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\include\delayimp.h(134): note: see declaration of '__pfnDliNotifyHook2' error is really the root cause of 'mocha' is not recognized as an internal or external command, operable program or batch file. Strangely, even after I change the test target to ./node_modules/.bin/mocha --compilers js:babel-register, the error is unchanged, which makes me feel like something is going on that I don't understand.

[knoxzin1]

I managed to run the tests here, the node-gyp problem is related to npm version not node, you'll need to update lerna too in order to run the tests on windows.

[doug-wade]

That's awesome @knoxzin1 ! What version of npm are you using? I see that lerna 26 is the version of lerna that worked for you. I'm going to pull your commit into my branch to try to maintain authorship.

The test failures are discouraging, though -- looks like the color chooser isn't stable across OSes. I feel like the tests are kind of hacky, though -- the guarantee isn't that the colors will be stable across systems and runs, but instead visually distinct. We'll need to change the assertions to be more savvy about how they check that the modules were tagged correctly.

[knoxzin1]

@doug-wade I'm using the latest npm version 3.10.6

[k-ode]

Upgrading to lastest Lerna and npm, I can see that it didn't actually link dependencies before - it just failed silently.

However, now I get this error message (during linking):

Errored while running BootstrapCommand.execute
Error: Command failed: npm run prepublish
(node:4488) fs: re-evaluating native module sources is not supported. If you are using the graceful-fs module, please update it to a more recent version.

npm ERR! Windows_NT 6.3.9600
npm ERR! argv "C:\\Program Files\\nodejs\\node.exe" "C:\\Users\\kg\\AppData\\Roaming\\npm\\node_modules\\npm\\bin\\npm-cli.js" "run" "prepublish"
npm ERR! node v6.3.1
npm ERR! npm  v3.10.6
npm ERR! code ELIFECYCLE
npm ERR! [email protected] prepublish: `gulp build`
npm ERR! Exit status 1

Everything seems to be working on the build server though, so it's most likely something specific on my system.

[drewpc]

Can we restart this after #833 is merged? The dependency node-inspector is what was causing the build to fail before--no node-gyp. That dependency was removed a few weeks ago.

[doug-wade]

I rebased onto #833 and kicked off the build again: doug-wade#14

🤞

[drewpc]

@doug-wade 🥁 🥁 🥁 and?

[doug-wade]

All the Bad Things ™️; EPERM: operation not permitted, rename 'C:\Users\appveyor\AppData\Roaming\npm-cache\source-map\0.5.6\package.tgz.2807194044' -> 'C:\Users\appveyor\AppData\Roaming\npm-cache\source-map\0.5.6\package.tgz'. I'm not sure who's renaming tarballs. I know asini does hoisting, but I thought it was by moving the root package.json, writing a new package.json, running the install and then replacing the package.json. If it actually fiddles around with the npm-cache, that'd be a good candidate. If its not, that would point to npm, but iirc the npm-cli has extensive testing on Windows, so that would surprise me. Maybe a race condition with running multiple installs simultaneously causes one process to move the tarball before the other?

Even more puzzling is the parent: 'babel-cli' part -- why is babel renaming tarballs? Feels like a 🐟

Here's the full error trace:

npm ERR! Windows_NT 6.3.9600
npm ERR! argv "C:\\Program Files (x86)\\nodejs\\node.exe" "C:\\Program Files (x86)\\nodejs\\node_modules\\npm\\bin\\npm-cli.js" "install"
npm ERR! node v4.4.7
npm ERR! npm  v2.15.8
npm ERR! path C:\Users\appveyor\AppData\Roaming\npm-cache\source-map\0.5.6\package.tgz.2807194044
npm ERR! code EPERM
npm ERR! errno -4048
npm ERR! syscall rename
npm ERR! Error: EPERM: operation not permitted, rename 'C:\Users\appveyor\AppData\Roaming\npm-cache\source-map\0.5.6\package.tgz.2807194044' -> 'C:\Users\appveyor\AppData\Roaming\npm-cache\source-map\0.5.6\package.tgz'
npm ERR!     at Error (native)
npm ERR!  { [Error: EPERM: operation not permitted, rename 'C:\Users\appveyor\AppData\Roaming\npm-cache\source-map\0.5.6\package.tgz.2807194044' -> 'C:\Users\appveyor\AppData\Roaming\npm-cache\source-map\0.5.6\package.tgz']
npm ERR!   errno: -4048,
npm ERR!   code: 'EPERM',
npm ERR!   syscall: 'rename',
npm ERR!   path: 'C:\\Users\\appveyor\\AppData\\Roaming\\npm-cache\\source-map\\0.5.6\\package.tgz.2807194044',
npm ERR!   dest: 'C:\\Users\\appveyor\\AppData\\Roaming\\npm-cache\\source-map\\0.5.6\\package.tgz',
npm ERR!   parent: 'babel-cli' }
npm ERR! 
npm ERR! Please try running this command again as root/Administrator.
npm ERR! Please include the following file with any support request:
npm ERR!     C:\projects\react-server\npm-debug.log
npm ERR! Windows_NT 6.3.9600
npm ERR! argv "C:\\Program Files (x86)\\nodejs\\node.exe" "C:\\Program Files (x86)\\nodejs\\node_modules\\npm\\bin\\npm-cli.js" "run" "bootstrap-no-i"
npm ERR! node v4.4.7
npm ERR! npm  v2.15.8
npm ERR! code ELIFECYCLE
npm ERR! @ bootstrap-no-i: `asini bootstrap`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the @ bootstrap-no-i script 'asini bootstrap'.
npm ERR! This is most likely a problem with the  package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     asini bootstrap
npm ERR! You can get information on how to open an issue for this project with:
npm ERR!     npm bugs 
npm ERR! Or if that isn't available, you can get their info via:
npm ERR! 
npm ERR!     npm owner ls 
npm ERR! There is likely additional logging output above.
npm ERR! Please include the following file with any support request:
npm ERR!     C:\projects\react-server\npm-debug.log
npm ERR! Windows_NT 6.3.9600
npm ERR! argv "C:\\Program Files (x86)\\nodejs\\node.exe" "C:\\Program Files (x86)\\nodejs\\node_modules\\npm\\bin\\npm-cli.js" "i"
npm ERR! node v4.4.7
npm ERR! npm  v2.15.8
npm ERR! code ELIFECYCLE
npm ERR! @ postinstall: `npm run bootstrap-no-i`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the @ postinstall script 'npm run bootstrap-no-i'.
npm ERR! This is most likely a problem with the  package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     npm run bootstrap-no-i
npm ERR! You can get information on how to open an issue for this project with:
npm ERR!     npm bugs 
npm ERR! Or if that isn't available, you can get their info via:
npm ERR! 
npm ERR!     npm owner ls 
npm ERR! There is likely additional logging output above.
npm ERR! Please include the following file with any support request:
npm ERR!     C:\projects\react-server\npm-debug.log
npm ERR! Windows_NT 6.3.9600
npm ERR! argv "C:\\Program Files (x86)\\nodejs\\node.exe" "C:\\Program Files (x86)\\nodejs\\node_modules\\npm\\bin\\npm-cli.js" "run" "bootstrap"
npm ERR! node v4.4.7
npm ERR! npm  v2.15.8
npm ERR! code ELIFECYCLE
npm ERR! @ bootstrap: `npm i`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the @ bootstrap script 'npm i'.
npm ERR! This is most likely a problem with the  package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     npm i
npm ERR! You can get information on how to open an issue for this project with:
npm ERR!     npm bugs 
npm ERR! Or if that isn't available, you can get their info via:
npm ERR! 
npm ERR!     npm owner ls 
npm ERR! There is likely additional logging output above.
npm ERR! Please include the following file with any support request:
npm ERR!     C:\projects\react-server\npm-debug.log
Command exited with code 1

See more at https://ci.appveyor.com/project/doug-wade/react-server/build/1.0.114

[drewpc]

Try changing node versions: nodejs_version: "7". The builds install fine for me, it's the tests + CRLF/LF issues that are the problem. The Babel tests rely on generating a file and comparing it with an expected.js file in the repo--which has CRLF. The generated file has LF. Here's the Appveyor output: https://ci.appveyor.com/project/drewpc/react-server/build/1.0.1

If we change the settings and have git checkout every file with LF automatically, lint blows up:
https://ci.appveyor.com/project/drewpc/react-server/build/1.0.1

Either we go through each test and make them all work on multiple platforms (huge pain) or we disable lint somehow on the Windows tests.

[doug-wade]

We should definitely support nodes 4 and 6, as they are still supported, so I'd be resistant to running CI exclusively on node 7.

Either we go through each test and make them all work on multiple platforms (huge pain) or we disable lint somehow on the Windows tests.

I think I lean towards the "go through each test and make them all work on multiple platforms" approach here. My goal is not only to test that React Server instances can run on Windows machines, but also to make sure that developers on Windows machines can contribute to React Server -- I think I might even value making it possible to develop on Windows more than making it possible to run on Windows. Being able to run the tests and linter locally are a big part of a good developer experience.

[doug-wade]

I just rebased and re-ran the tests https://ci.appveyor.com/project/doug-wade/react-server/build/1.0.124. Looks like the colors are operating system dependent.