Processes seem to run sequentially for specs, but in parallel for features

[sciprog]

I've followed the installation and setup procedure to the letter. The parallel:create task successfully detects 4 cores, and builds 4 test databases. parallel:prepare runs without a hitch. I see a dramatic improvement in the runtime of my cucumber features, and my system monitor shows all 4 cores chewing away at the tests. However, when I run the specs with "rake parallel:spec", or "parallel_spec" the tests consistently run in slightly more time than it takes to run them sequentially (ie. "rake spec"). Additionally, the system monitor shows all 4 cores fire up during the initialization of the specs, but only 1 core is active while the tests are actually running. Although 4 processes are created, according to the output "4 processes for 41 specs, ~ 10 specs per process", it appears as if these processes are then run sequentially, rather than in parallel. I see the rspec summary output for the first process before the second process begins. I haven't been able to find other posts about this issue, which makes me suspect that I've somehow messed up the configuration, but I have not been able to find the problem. Any thoughts?
Thank you.

[grosser]

can you recreate the issue with a very simple setup (e.g. empty folder with 4 test files that each run sleep 10) ?

[sciprog]

Thanks, that shed some light on the situation. With 4 test files each running sleep 10, I'm seeing a 10 second increase, over baseline, in the runtime of the tests. When I switch out the sleep statement with a call to a naive fibonacci calculator, I can see all 4 cores operating at 100% while the tests are running. Maybe the parallelization is working correctly, but I've got a bottleneck somewhere that's keeping the CPU waiting between tests, making it appear to be idle. However, the same bottleneck would exist during sequential execution, so I'd expect see at least some improvement when I parallelize. I'll keep looking. Thanks again for the help.

[grosser]

maybe try to install parallel_tests as plugin and hack around/scatter some puts to see if the code truly runs in parallel

[sciprog]

Got it sorted out. The bottleneck was caused by my DatabaseCleaner configuration. I changed the strategy from truncation to transaction, and now my tests run like they're on fire! Thanks for all the help :-)

[grosser]

sounds very hard to figure out, could you add a bit of into to the wiki, so other people do not run into the same trap

[sciprog]

Done. I'm afraid I didn't uncover the root of the problem, so my entry doesn't include much in the way of specifics, but I think it's enough to put others on the right track.

[astjohn]

Any thoughts for a mongoid user? DatabaseCleaner requires truncation for mongoid.

[tazsingh]

@astjohn You'd need separate Mongo databases in order to run parallel tests.

[astjohn]

@tazsingh, You can run separate databases for each thread easily, but that still won't help for between tests. The default setup when using DatabaseCleaner is to purge the database before each test using either transaction or truncation method, but with mongoid, you only have truncation.

[tazsingh]

@astjohn Sorry, I'm not following you. If you have a separate database for each thread/process, then each test can purge/truncate the database and it won't affect the other tests that are running in parallel (assuming your tests are not dependent on each other and could run in random order).

[astjohn]

@tazsingh Exactly. This original issue was started because truncating was creating a bottleneck, and with mongoid / databasecleaner, truncating is your only option. Read above comment before this issue was closed. I'm wondering if anyone else out there has experienced a similar bottleneck with mongoid or not?

[tazsingh]

@astjohn Ah thanks for the clarification, just did a quick read initially. Having used this gem with Mongoid, we haven't noticed the bottleneck and have noticed significant speed improvements :)