travis-ci: builds have become very slowwwww...

[grondo]

... in fact, so slow that, when dependencies have to be built (i.e. when pushing to a new branch for the first time), a large percentage of the builds do not finish before the ~50min overall travis-ci timeout.

e.g. see a build from one my branches here

I seem to remember these builds taking on the order of ~24min in the past, and normal builds using the cache took ~8min, whereas now they take ~20min on average.

I don't know if the issue has arisen after Travis-CI switched to containers based on trusty instead of precise, or if it has something to do with a switch to GCE, or if they tweaked the config of their containers.
There is an open issue that claims trusty builds are slower, but unfortunately not much activity so far.

To mitigate the timeouts, which cause build failures, there are probably multiple things to try:

• Since we moved to a new version of the OS, see if some of the dependencies we're building using src/test/travis-dep-builder.sh can instead be provided by apt packages. If we could at least get a version of MPI from a package that would have a big impact (MPI build seems to be one of the larger time sinks in building up the dependency cache)
• add -j2 to all make invocations (this caused strange errors on precise builds, and I'm not sure how many vcpu the travis-ci containers actually have anyway)
• Upload built dependencies to an S3 bucket or other publicly accessible uri so we can build once and use the products from multiple branches
• I think travis allows docker to run inside their containers. We could upload a docker image pre-installed with all dependencies and docker run make distcheck.

[grondo]

Reverting to VM builders as in PR #1167 seems to have mitigated much of the slowness in the travis builders. Closing this for now. Just reopen or open a new issue to discuss further testsuite efficiency improvements.