Add basic tests

[chorrell]

Test image builds, and ideally, basic commands. Ideally this help us catch issues before making a pull request to the official image repo.

A recent example of where we got burned by this: docker-library/official-images/pull/630

[tianon]

You might be interested in checking out https://github.com/docker-library/official-images/tree/master/test (including docker-library/official-images#602 + docker-library/official-images#603). 👍

[chorrell]

Yes! I've been looking at that and was thinking about setting up a Jenkins job to run the tests after a git push. Some basic tests of npm would be good too (npm install foo, npm install -g foo, etc).

[tianon]

🤘

[chorrell]

I have a prototype/demo test suite for node:0.12: https://github.com/chorrell/docker-node/tree/serverspec-testing.

It uses the serverspec gem and the docker-api gem. If you have bundler already installed, the gems can be installed with:

cd test; bundle

If you have docker running, and those gems installed, you just run the tests with:

cd test; rake

That will build the 0.12 image(tagged as node:0.12), run tests against a container, then delete the image. I've been running this on my Mac with boot2docker. Works pretty well so far.

The tests are very simple at this point, and I'm not testing all the variants (like on-build etc) but this looks really promising and something that could easily be run via Jenkins for pull requests and such.

[retrohacker]

Cross posting from nodejs/docker-iojs#46

---

So far, I've created a tool for automated testing of Docker images: https://github.com/wblankenship/dante

And put together some smoke tests for the Docker image, which follows the work of rvagg: https://github.com/wblankenship/docker-iojs/blob/tests/tests/iojs/Dockerfile

These tests are executed according to the inventory file here: https://github.com/wblankenship/docker-iojs/blob/tests/inventory.yml

The tests currently do not run the Node.js project's test suite, which may be a good idea.

We have an open pull request for the build group to provision us a Jenkins server that we can wire this workflow up to: nodejs/build#168

The Jenkins slave should trigger the test on a new PR to this repo. The test is run by simply calling dante in the base of the project directory. If the tests pass, dante will exit with a status 0, else it will exit with a status of 1. dante creates tagged images for each build for historical reference. This will chew up HDD space over time so we will need some sort of garbage collection as well.

[chorrell]

TIL that Travis-CI can do a simple build test for us (by running ./test-build.sh) by running a Docker image builds.

I did a test over here:

• https://github.com/chorrell/docker-node/tree/travis-ci
• https://travis-ci.org/chorrell/docker-node/builds/120693505

And it pretty much works as expected. I think for a simple build test that should be sufficient and we can explore some deeper testing later.

So next step is to get Travis-Ci enabled for this GitHub repo.