[1.4.2.1-rc.0] meteor todos client tests fail with select: Invalid argument

[hwillson]

Migrating this over from meteor/meteor#8016.

---

(originally reported by @hexsprite)

Noticed this in my project and was able to reproduce using the meteor-todos project.

Just upgrade to rc.0 and you'll see that when running npm test the client tests fail. In fact, it seems the process exits unceremoniously somehow since the output is cut off.

I20161104-09:06:23.738(-7)? --------------------------------
I20161104-09:06:23.738(-7)? ----- RUNNING CLIENT TESTS -----
I20161104-09:06:23.738(-7)? --------------------------------
I20161104-09:06:23.738(-7)? 
I20161104-09:06:23.738(-7)? select: Invalid argument
....  80 repeated snipped ....
I20161104-09:06:23.738(-7)? select: Invalid argu
➜  todos git:(master) ✗ echo $?
0

[hwillson]

I can't seem to re-create this issue with Meteor 1.4.2.3 (OSX 10.11.5). Issue meteor/meteor#8016 had a fair number of up votes though, so if anyone is able to reproduce this issue with the current version of todos please let me know. For now I'll consider this issue closed, but let me know if it should be re-opened. Thanks!

[dburles]

Strangely, replacing the factory package caused this exact issue! 😕

[hwillson]

Interesting - I'll take a look - thanks!

[weilitao]

The bug is still exists. Todo app and my app all have the same issue when running npm test, npm test-watch is ok.

OSX 10.12.1
Meteor 1.4.2.3
phantomjs 2.1.1

[hwillson]

I've looked into this a bit further. This doesn't appear to be a dispatch:mocha-phantomjs issue, like DispatchMe/meteor-mocha-phantomjs#33 suggests. I'd like to say it's a phantomjs issue (something like ariya/phantomjs#13231), but I'm not 100% convinced of that yet either. I've been debugging the dispatch:phantomjs-tests package to get access to the original phantomjs process while running tests, and haven't quite found the real issue yet. A few troubleshooting notes:

• Running meteor reset allows meteor test --once --driver-package dispatch:mocha-phantomjs to complete successfully the first time it's run. Each subsequent run fails with the select: Invalid argument error message (until meteor reset is run again).

• select: Invalid argument is coming from phantomjs' stderr.

• The message select: Invalid argument really looks/sounds like an error message being thrown from a shell script; something coming from using Bash's select command for example.

I'll look into this more shortly.

[robsecord]

+1
Experiencing the same issue when upgrading to Meteor 1.4.2.3 from 1.4.2 (OSX 10.12.1)

[awatson1978]

I've been debugging this all day, and am of the opinion that the number of packages in the .meteor/packages directory can semi-reliably reproduce the error.

We have a test harness for running QA scripts across the Clinical Meteor distro, and the way the harness and packages are configured, we have about 40 small packages, each with a different SimpleSchema in it. Anyhow, I can reliably reproduce the error with different combinations of packages, which are all effectively exactly the same. It happens consistently when I add the 47th package.

I was thinking "that's totally random, and couldn't be related to the Todos example'. But then I looked, and Todos actually has 49 packages. Probably one or two of them somehow aren't getting loaded during the meteor test command.

Anyhow, some sort of buffer overflow. Could be amount of memory the packages are using; number of files on disk; time it takes to load them, etc. But PhantomJS doesn't like it when there are too many packages, and burps this select: Invalid argument error.

[gsabran]

For me replacing dispatch:mocha-phantomjs by dispatch:mocha fixed the issue

[awatson1978]

Unfortunately, it dispatch:mocha doesn't run the tests on client. :/

[aldeed]

Try with dispatch:[email protected], which uses latest phantomjs. Seems to have fixed this for me in another app, but it's possible that something else I did fixed it.

[hwillson]

Thanks @aldeed - updating to dispatch:[email protected] didn't help unfortunately.

@awatson1978 - I definitely think you're on to something! I looked into why meteor npm test always works after running a meteor reset. Instead of running a full reset, if you just delete your apps .meteor/local/plugin-cache it will work also. I tested just removing some of the packages in the plugin-cache directory, and testing failed; I then removed most of them (but left about 40) and it worked. Very interesting!

For those who want a really quick workaround (hack), run your tests as (from your app root):

rm -Rf .meteor/local/plugin-cache; meteor npm test

Needless to say this is quite slow (since we're killing the plugin cache), but it works until we can get a better fix in place.

[iMagdy]

Clearing cache didn't work for me unfortunately.
dispatch:[email protected]

[timbrandin]

I have the same issue, also the cache is always cleared when we run it fresh on a test docker.

[hwillson]

@iMagdy @timbrandin Sorry to hear that! We definitely still need to find the source of the memory leak. I'll try to look into this more today. Thanks!

[timbrandin]

For me it's not a huge ordeal, we just started adding tests and we're not depending on them yet. But thx for looking into it so soon.

[timbrandin]

Shouldn't this be solved in: DispatchMe/meteor-mocha-phantomjs#33 ?

[hwillson]

Quite possibly, but I haven't pinpointed the source of the problem yet. I know phantomjs is running out of memory, but I haven't been able to track down what's causing that to happen. It's quite possible some combination of things that the todos app exposes is causing that leak, so it's difficult to move this issue over to the https://github.com/DispatchMe/meteor-mocha-phantomjs repo until we know more.

[hwillson]

Hi all - I recently discussed this issue with @abernix and be brought up some good points about the stability of phantomjs v2.x, and mentioned trying the last 1.9.x version. I just tried replacing the 'phantomjs-prebuilt': '2.1.13' dependency in meteor-phantomjs-tests with 'phantomjs': '1.9.19', and everything works properly again. I'll try opening a new issue about this in the phantomjs repo, but with 1666 open issues and an approximate issue resolution time of 56 days, I'm really not optimistic we'll get a timely response.

In the meantime, I'll package up a forked version of dispatch:mocha-phantomjs that people can drop in to verify that this does indeed fix the issue. Once verified, we can put some more through into how to address this moving forward. Thanks!

[abernix]

I don't recall if the issue I was running into was the same or not, but I spent a lot of time debugging it. My comment on the (private) commit was:

Phantom 2 just seems to evalulate the page before it's ready.  It might be a problem with Qt, but I don't have time to check.

As you can see by my misspelling of "evaluate", I didn't have much time but had got so far as to thinking it was a problem with Qt, a PhantomJS dependency. Ultimately, I moved to 1.9.x as well and have been there ever since.

[JoshuaColeman]

I tested downgrading meteor-phantomjs-tests to 'phantomjs': '1.9.19' here:
DispatchMe/meteor-phantomjs-tests@master...JoshuaColeman:downgrade-phantomjs

The result (on OSX) was that client tests reported running without any output, and an 'undefined' number of errors. I added some extra logging around the call to phantomjs and found that the 1.9.19 phantomjs crashes with a SIGSEV fault. I tested directly calling phantomjs and sure enough: Segmentation fault: 11

This is a known issue documented here: ariya/phantomjs#14558

Any other ideas on how to resolve this?

[hwillson]

Ouch - I just updated to macOS Sierra and now it's failing for me as well. Using phantomjs 1.9.19 worked for me on El Capitan, but it now looks like this is another dead end. Back to the drawing board ...

[NickBusey]

meteor reset seems to be quite a reliable fix for me. To the point that editing the test line in package.json to "test": "meteor reset; meteor test --once --driver-package dispatch:mocha-phantomjs",, while far from ideal, does work repeatably.

[0o-de-lally]

See fix: meteor/meteor#8225.
Updating to meteor update --release 1.4.3-rc.3 seems to fix this. Also in DispatchMe/meteor-mocha-phantomjs#33

[hwillson]

Woohoo - awesome news @keyscores! Thanks for testing it out and posting back here!

[0o-de-lally]

Sadly updating to dispatch:mocha-phantomjs@=0.1.9 causes the error to return on my machine. Though 0.1.7 works on my local. @hwillson can you confirm?

[benjamn]

@keyscores Since meteor update --release 1.4.3-rc.xxx doesn't permanently change the default version of Meteor, it's easy to end up using the latest official version (1.4.2.6) by accident. Are you sure you're running with meteor --release 1.4.3-rc.4 ... (or in an app that has [email protected] in its .meteor/release file)?

[0o-de-lally]

@benjamn Yep I'm pretty sure I'm at the right version. Here are my steps:

// ## passing ##
> unzip todos-master.zip 
> cd todos-master
> meteor npm install
> meteor update --release 1.4.3-rc.4
> cat .meteor/release 
// [email protected]
> meteor list 
// dispatch:mocha-phantomjs          0.1.7
> meteor test --full-app --once --driver-package dispatch:mocha-phantomjs --release 1.4.3-rc.4
// OK tests passing smoothly

// ## failing ##
> meteor update dispatch:mocha-phantomjs 
// dispatch:mocha-phantomjs  upgraded from 0.1.7 to 0.1.9
// dispatch:phantomjs-tests  upgraded from 0.0.5 to 0.0.7
> meteor test --full-app --once --driver-package dispatch:mocha-phantomjs --release 1.4.3-rc.4
// select: Invalid argument 
// select: Invalid argument 
// continues 200x

[aldeed]

The only real difference is the bump from using phantomjs-prebuilt 2.1.7 to 2.1.13, but I don't actually think that phantomjs has fixed select: Invalid argument. I think it appears randomly sometimes and other times not, possibly related to memory or number of packages as others have suggested. If you rerun with both versions multiple times, I'm guessing you'll see it once in a while regardless of version.

[0o-de-lally]

@aldeed I've run through the steps above again. It's very consistent. I'm not exactly sure what is being lost from upgrading from dispatch:mocha-phantomjs 0.1.7 to 0.1.9 . Have you been able to reproduce?

[0o-de-lally]

@benjamn have you been able to reproduce my steps above?

[derouck]

I bumped into this "select: Invalid argument" error as well for a different usecase (PDF rendering). But the solution I found might be interesting. The problem occurred for me when there were images loaded from the same server (src="Meteor.absoluteUrl()/....). Earlier I didn't have problems with that case, but with latest Meteor update this broke the PDF rendering.

I could solve it by making the path to the image relative and by setting up a base path for these files in the phantom.js settings. I have no idea if this finding is relevant for the tests issue, maybe it helps to point you in the right direction?

[lorensr]

Issue consistently persists for me in:

[email protected]
[email protected]
dispatch:[email protected] – 0.1.9

> git clone [email protected]:meteor/todos.git
> meteor update --release 1.4.3.2
> meteor npm install
> meteor list | grep phantom
dispatch:mocha-phantomjs          0.1.7

> meteor test --full-app --once --driver-package dispatch:mocha-phantomjs
PASS

> meteor test --full-app --once --driver-package dispatch:mocha-phantomjs
FAIL (and fails subsequent times)

> rm -Rf .meteor/local/plugin-cache
> meteor test --full-app --once --driver-package dispatch:mocha-phantomjs
PASS (and fails subsequent times)

Same behavior after meteor update dispatch:mocha-phantomjs to 0.1.9.

[aldeed]

We need to update todos app to use the latest dispatch:mocha instead of dispatch:mocha-phantomjs, and use one of the other supported browsers instead of phantomjs. dispatch:mocha-phantomjs is deprecated. Phantom is just too broken. Any volunteers?

Edit: Scratch that, just noticed the commit from @lorensr. Cool!

[NickBusey]

I made similar changes to my app, and testing seems to work fine now. I'd say this is closed.

[robsecord]

For our test environment, Nightmare works well for the developers who are running MacOS. However, we run into problems when using Nightmare with our Linux-based Dev and CI servers, in which case we need to run Phantom in order to have our tests pass. This creates an inconsistent testing environment between Dev and CI. For now, we are using such a workaround, but I would not consider this issue closed.

[hwillson]

Merging #222 auto-closed this issue, but #198 (comment) still needs to be investigated.