Use Docker for isolated local testing

CONTRIBUTING.md

@@ -25,21 +25,62 @@ This document is a guide for those maintaining Aruba, and others who would like
   bump version in a commit by itself so we can ignore when we merge your change)
 * Send us a pull request.
 
-## Bootstrap environment
+## Running tests using Docker (recommended!)
 
-To get started with `aruba`, you just need to bootstrap the environment by
-running the following command.
+NOTE: Since custom user shell/app settings can break tests, it's recommended
+to use Docker for running tests. The advantages are:
+
+1. You still can run tests based on modified sources.
+2. Almost as fast as running tests on your host.
+3. Guarantees tests won't be affected by your setup.
+4. You can avoid installing all the test-related gems on your system.
+5. You can test various combinations of different versions of Ruby and tools.
+6. It's easier to reproduce problems without messing around with your own system.
+7. You can share your container with others (so they can see the reproduced scenario too).
+
+To run tests using Docker, just run:
+
+    # Automaticaly build Docker image with cached gems and run tests
+    bundle exec rake docker:build && bundle exec rake docker:run
+
+or using Docker Compose:
+
+    # Automaticaly build Docker image with cached gems and run tests
+    docker-compose run tests
+
+And if you get any failures on a fresh checkout, report them first so they're
+fixed quickly.
+
+You can also run specific tests/scenarios/commands, e.g.:
+
+    # Run given cucumber scenario in Docker using rake
+    bundle exec rake "docker:run[cucumber features/steps/command/shell.feature:14]"
+
+or
+
+    # Run given cucumber scenario in Docker using Docker Compose
+    docker-compose run tests bash -l -c cucumber features/steps/command/shell.feature:14
+
+## Running tests locally (mail fail depending on your setup)
+
+First, bootstrap your environment with this command:
 
     # Bootstrap environment
     script/bootstrap
 
-## Running tests
+(This will check system requirements and install needed gems).
 
-Make sure you bootstrap the environment first.
+Then, you can run the tests.
 
     # Run the test suite
     script/test
 
+Or, you can run a specific test or scenario, e.g.
+
+    # Run only selected Cucumber scenario
+    script/test cucumber features/steps/command/shell.feature:14
+
+
 ## Installing your own gems used for development
 
 A `Gemfile.local`-file can be used to have your own gems installed to support
@@ -66,7 +107,7 @@ Now release it
     git commit -m "Release X.Y.Z"
     rake release
 
-Now send a PR to https://github.com/cucumber/website adding an article about the with details of the new release and merge it - an aruba maintainer should normally allowed to merge PRs on `cucumber/website`. A copy of an old announcement can be used as basis for the new article. After this send an email with the link to the article to [email protected]. 
+Now send a PR to https://github.com/cucumber/website adding an article about the with details of the new release and merge it - an aruba maintainer should normally allowed to merge PRs on `cucumber/website`. A copy of an old announcement can be used as basis for the new article. After this send an email with the link to the article to [email protected].
 
 ## Gaining Release Karma
 

Dockerfile

@@ -0,0 +1,41 @@
+FROM cucumber/aruba:latest
+
+USER root
+
+# Zsh (just for the sake of a handful of Cucumber scenarios)
+RUN apt-get update -qq && apt-get -y install zsh --no-install-recommends && rm -rf /var/lib/apt/lists/* /var/cache/apt/archives/*.deb /var/cache/apt/archives/partial/*.deb /var/cache/apt/*.bin
+
+# Python (just for the sake of a handful of Cucumber scenarios)
+RUN apt-get update -qq && apt-get -y install python --no-install-recommends && rm -rf /var/lib/apt/lists/* /var/cache/apt/archives/*.deb /var/cache/apt/archives/partial/*.deb /var/cache/apt/*.bin
+
+# Java (for javac - also for just a few Cucumber scenarios)
+RUN apt-get update -qq && apt-get -y install openjdk-7-jdk --no-install-recommends && rm -rf /var/lib/apt/lists/* /var/cache/apt/archives/*.deb /var/cache/apt/archives/partial/*.deb /var/cache/apt/*.bin
+
+# Cache needed gems - for faster test reruns
+ADD Gemfile Gemfile.lock aruba.gemspec /home/guest/cache/aruba/
+ADD lib/aruba/version.rb /home/guest/cache/aruba/lib/aruba/version.rb
+RUN chown -R guest:guest /home/guest/cache
+USER guest
+
+# Add gemrc for possible gem mirror config
+# ADD docker.gemrc /home/guest/.gemrc
+
+# NOTE: It's best to setup your own gem mirror/server, because:
+#
+# 1. Faster gems installing (especially useful when rebuilding the Docker image)
+# 2. Less internet bandwidth used
+# 3. Lesser load on rubygems.org servers
+#
+# Info: http://guides.rubygems.org/run-your-own-gem-server/#
+
+RUN echo '---\n\
+:sources:\n\
+  - http://172.17.0.1:8808\n\
+gem: "--no-ri --no-rdoc --source http://172.17.0.1:8808"\n'\
+>> /home/guest/.gemrc
+
+# Actually download
+RUN bash -l -c "cd cache/aruba && bundle install"
+
+
+WORKDIR /home/guest/aruba

Dockerfile.base

@@ -0,0 +1,37 @@
+FROM ubuntu:14.04
+MAINTAINER Aruba Maintainers <[email protected]>
+
+# Packages needed to install RVM and run Bundler gem commands
+RUN apt-get update -qq && apt-get -y install ca-certificates curl git-core --no-install-recommends && rm -rf /var/lib/apt/lists/* /var/cache/apt/archives/*.deb /var/cache/apt/archives/partial/*.deb /var/cache/apt/*.bin
+
+# Create guest user early (before rvm) so uid:gid are 1000:000
+RUN useradd -m -s /bin/bash guest
+
+# Temporarily install RVM as root - just for requirements
+RUN gpg --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3
+RUN curl -L get.rvm.io | bash -s stable
+RUN bash -l -c 'rvm requirements 2.2.1'
+RUN bash -l -c 'echo yes | rvm implode'
+
+# Fix locale
+ENV DEBIAN_FRONTEND noninteractive
+RUN dpkg-reconfigure locales && locale-gen en_US.UTF-8 && /usr/sbin/update-locale LANG=en_US.UTF-8
+RUN echo 'en_US.UTF-8 UTF-8' >> /etc/locale.gen && locale-gen
+ENV LC_ALL C.UTF-8
+ENV LANG C.UTF-8
+ENV LANGUAGE C.UTF-8
+
+USER guest
+ENV HOME /home/guest
+WORKDIR /home/guest
+
+# Install RVM as guest
+RUN gpg --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3
+RUN /bin/bash -l -c "echo 'gem: --no-ri --no-rdoc' > ~/.gemrc"
+RUN curl -L get.rvm.io | bash -s stable
+RUN /bin/bash -l -c "rvm install 2.3.0 && rvm cleanup all"
+RUN /bin/bash -l -c "gem install bundler --no-ri --no-rdoc"
+
+ONBUILD ENV USER guest
+ONBUILD WORKDIR /home/guest
+ONBUILD RUN /bin/bash -l -c "source /home/$USER/.rvm/scripts/rvm"

Rakefile

@@ -61,9 +61,7 @@ namespace :docker do
     application_version = args[:version]
     docker_file = 'Dockerfile'
 
-    cmdline = []
-    cmdline << 'docker'
-    cmdline << 'build'
+    cmdline = %W(docker build)
     cmdline << '--no-cache=true' if nocache == 'true'
 
     %w(http_proxy https_proxy HTTP_PROXY HTTPS_PROXY).each do |var|
@@ -85,19 +83,17 @@ namespace :docker do
   task :run, :command do |_, task_args|
     command = task_args[:command]
 
-    args =[]
-    args << '-it'
-    args << '--rm'
-    args << "--name #{container_name}"
-    args << "-v #{File.expand_path('.')}:/srv/app"
-
-    cmdline = []
-    cmdline << 'docker'
-    cmdline << 'run'
-    cmdline.concat args
+    cmdline = %W(docker run -it --rm --name #{container_name} -w /home/guest/aruba)
+    cmdline << "-v #{File.expand_path('.')}:/home/guest/aruba"
     cmdline << image_name
-    cmdline << command if command
+    if command
+      cmdline << "bash -l -c #{Shellwords.escape(command)}"
+    else
+      cmdline << "bash -l -c './script/bootstrap && ./script/test'"
+    end
 
+    STDOUT.puts "Running Docker with arguments:"
+    STDOUT.puts cmdline.inspect
     sh cmdline.join(' ')
   end
 end

docker-compose.yml

@@ -0,0 +1,24 @@
+# To build the base image (official Aruba base for this), run:
+#
+#   $ docker-compose build base
+#
+# To run the tests, run:
+#
+#   $ docker-compose run tests
+#
+
+version: '2'
+services:
+  base:
+    image: 'cucumber/aruba'
+    build:
+      context: .
+      dockerfile: Dockerfile.base
+
+  tests:
+    container_name: 'cucumber-aruba-1'
+    build: .
+    working_dir: /home/guest/aruba
+    command: bash -l -c "./script/bootstrap && ./script/test"
+    volumes:
+      - .:/home/guest/aruba

script/bootstrap

@@ -5,21 +5,27 @@ set -e
 info_msg="\e[0;32m[INFO]\e[39;49m"
 error_msg="\e[0;31mFAILED\e[39;49m"
 
-function output_error_log {
-  [[ -f error.log ]] && ( cat error.log >&2; rm error.log)
+# TODO: these are Unix specific
+TMPDIR="${TMPDIR:=/tmp}"
+logfile=`mktemp -t aruba_tests.XXXXXXXXXX`
+
+output_error_log() {
+  [[ -f "$logfile" ]] && ( cat "logfile" >&2; rm -f "$logfile")
 }
 
-echo -ne "$info_msg Checking if ruby installed? "
-which 'ruby' >/dev/null 2>error.log || ( echo -e "$error_msg\n\nCould not find \`ruby\`. Please install ruby or add it to PATH"; output_error_log; exit 1 )
-echo OK
+check_for_binary() {
+  name="$1"
 
-echo -ne "$info_msg Checking if bash installed? "
-which 'bash' >/dev/null 2>error.log || ( echo -e "$error_msg\n\nCould not find \`bash\`. Please install bash or add it to PATH"; output_error_log; exit 1 )
-echo OK
+  echo -ne "$info_msg Checking if $name installed? "
+  which "$name" >/dev/null 2>"$logfile" || ( echo -e "$error_msg\n\nCould not find \`$name\`. Please install $name or add it to PATH"; output_error_log; exit 1 )
+  echo OK
+}
 
-echo -ne "$info_msg Checking if zsh installed? "
-which 'zsh' >/dev/null 2>error.log || ( echo -e "$error_msg\n\nCould not find \`zsh\`. Please install zsh or add it to PATH"; output_error_log; exit 1 )
-echo OK
+check_for_binary "ruby"
+check_for_binary "bash"
+check_for_binary "zsh"
+check_for_binary "python"
+check_for_binary "javac"
 
 echo -e "$info_msg rubygem \"bundler\" "
 gem install bundler

script/test

@@ -1,3 +1,8 @@
-#!/bin/sh
+#!/usr/bin/env bash
 
-bundle exec rake test $*
+if [ "$RUN_IN_DOCKER" == '1' ]; then
+  bundle exec rake docker:run[$*]
+else
+  bundle exec rake test $*
+  echo bundle exec rake test $*
+fi