A tutorial introduction for clojure.test users

This tutorial shows a clojure.test user how to migrate to Midje. That migration can be gradual: Midje coexists with clojure.test, so you can use both at the same time, even in the same file.

A sample project

If you want to follow along with this tutorial, you can fetch this project:

% git clone https://github.com/marick/midje-clojure-test-tutorial.git

The project is about a whimsical little function called migrate that "moves" key/value pairs from one map to another. Migrate calls are written left to right:

(migrate left-map :key-to-migrate right-map)

... and the result is a description of the two new maps plus a set (to be described later):

user=> (migrate {:KEY 1} :KEY {})
{:new-left {}, :clashes #{}, :new-right {:KEY 1}}

You can migrate more than one key:

user=> (migrate {:stay 1, :MOVE-THIS 2, :AND-THIS 3} :MOVE-THIS :AND-THIS {})
{:new-left {:stay 1}, :clashes #{}, :new-right {:MOVE-THIS 2, :AND-THIS 3}}

In the case of key clashes, the migration isn't done and the clashing keys are noted:

user=> (migrate {:*clash* "stays", :MOVE 2} :*clash* :MOVE {:*clash* "also* stays"})
{:new-left {:*clash* "stays"}, 
 :clashes #{:*clash*}, 
 :new-right {:MOVE 2, :*clash* "also* stays"}}

Tests and facts

migrate has tests. You can find their source here. Here are the first two of them:

(deftest migration
  (testing "Migration produces a new left and right map"
    (is (= {:new-left {} :clashes #{} :new-right {:a 1}}
           (migrate {:a 1} :a {}))))
  (testing "multiple keys can be moved at once"
    (is (= {:new-left {} :clashes #{} :new-right {:a 1 :b 2}}
           (migrate {:a 1, :b 2} :a :b {})))))

We're going to translate those tests into Midje notation. To do that, you need to add Midje to your project.clj: [midje "1.5.0"]. Midje typically goes in the :dev dependencies. The tutorial project.clj already has Midje added.

You then use midje.sweet in your namespace:

  (:use clojure.test      ;; No harm in retaining this
        midje.sweet       ;; <<<<  
        migration.core)

(The name midje.sweet is a historical artifact: Midje used to have three distinct top-level namespaces.)

Now you can write Midje code that looks structurally very like the clojure.test tests:

(facts "about migration"
  (fact "Migration produces a new left and right map"
    (migrate {:a 1} :a {}) 
    => {:new-left {} :clashes #{} :new-right {:a 1}})
  (fact "multiple keys can be moved at once"
    (migrate {:a 1, :b 2} :a :b {}) 
    => {:new-left {} :clashes #{} :new-right {:a 1 :b 2}}))

Other than using different names than "deftest" and "testing", the only difference is that the (is (= expected actual)) form is replaced by a non-Lispy actual => expected pseudo-form. That's done for two reasons:

1. People who grew up with languages written left to right and top to bottom think of time flowing that way. (Look at your German or Spanish or English calendar.) Since the calculation of the actual result precedes the comparison to the expected result, the actual result should come first. (If I remember correctly, Kent Beck once told me that putting the expected result on the left was an implementation accident in Sunit, the Smalltalk model for Junit.)

2. Look at how Clojure books explain Clojure functions. Here's the notation used by The Joy of Clojure:

   

   Notice that it's left-to-right. (In longer examples, they put the expected result below the calculation, just as I did in the translation above.) Notice also that their notation uses an arrow. Arrows are popular in examples (see also). That's natural, since arrows are both associated with the movement of time and are also used to point to something interesting (in this case, the result of the computation).

   You'd recoil from a Clojure book that wrote its examples like this:

   (is (= [1 2 3] (conj [1 2] 3)))

   I believe you should have the same reaction to test suites written like that: a slavish adherence to Lisp style in tests incorrectly exalts purity over user-friendliness. (Code style is a different matter.) The Midje syntax is definitely more work for me, its main implementer, but why should you care about that?

A lot of the structure in the previous Midje example is optional. For example, here's a minimalist version:

(fact
  (migrate {:a 1} :a {}) => {:new-left {} :clashes #{} :new-right {:a 1}}
  (migrate {:a 1, :b 2} :a :b {}) => {:new-left {} :clashes #{} :new-right {:a 1 :b 2}})

Names

The arrow pseudo-forms are called checkables. They're what Midje executes and checks. I chose fact or facts to name the form that wraps checkables because I'm fond of thinking of my tests as grand claims of eternal truth about my program, with the checkables serving as examples that compel belief in those claims (since grand universal claims can't be checked programmatically).

Other people think that strings like "Migration produces a new left and right map" are nothing more than doc strings and that the real facts are the checkables themselves.

And other people think all this talk of facts is silly, and that "fact" is a misspelling of "test".

Midje works fine with any of those interpretations.

Running tests

When you run the tests with lein tests, Midje failures are printed. To demonstrate that, I changed migrate to be buggy. (If you're following along, I changed the drop 1 to a drop 2.) To avoid the clutter of many failures, I also commented out the clojure.test tests. Here's the output:

(I used a screen shot here to emphasize that Midje by default uses terminal colors in its output.)

Unfortunately, even though Midje found a failure, clojure.test's failure count didn't include it. For that reason, I recommend you use Midje's own Leiningen plugin. To install it, add this to your :user profile in ${HOME}/.lein/profiles.clj:

{:plugins [... [lein-midje "3.2.1"] ...]}

To show how lein-midje works, I added back a clojure.test test so that there are both midje and clojure.test failures. Here they are:

Notice that both Midje and clojure.test output appears (colored so that failures stand out). Both Midje's and clojure.test's failure counts are reported. They are also both used to construct the exit status, which is 0 if there were no problems, a non-zero number otherwise. (Strictly, the exit status is the number of failures, up to 255. So in this case, the exit status is 3.)

Because Clojure and Midje's startup time is slow, you will probably prefer to use autotest, in which Midje watches your project for changed files. When it sees a change, it reloads the changed file and all files that depend on it (either directly or indirectly). In the following example, I started autotest on a buggy version of migrate, made (and saved) a syntax error trying to fix it, and then really fixed it. (I also removed the clojure.test test to keep the output from flooding the screen.)

I personally prefer to start autotest from within the repl, using Midje's Repl Tools. That makes it easy to fluidly switch between test-driven development and repl-driven development:

Checkers

Here's the earlier translation of the clojure.test tests again:

(facts "about migration"
  (fact "Migration produces a new left and right map"
    (migrate {:a 1} :a {}) 
    => {:new-left {} :clashes #{} :new-right {:a 1}})
  (fact "multiple keys can be moved at once"
    (migrate {:a 1, :b 2} :a :b {}) 
    => {:new-left {} :clashes #{} :new-right {:a 1 :b 2}}))

I think they're unclear. These tests have nothing to do with clash-handling, but the :clashes key is just sitting there in the expected value, distracting the reader from the tests' actual purpose. To help avoid such distractions, Midje provides various checkers that do a better job of checking than pure equality can. For example, consider this:

  (fact "Migration produces a new left and right map"
    (migrate {:a 1} :a {}) => (contains {:new-left {} :new-right {:a 1}}))

contains is a checker that lets you ignore extra values. It lets you defer discussion of the :clashes key until later, allowing the reader to focus on what's special about this particular actual and expected value.

contains is no sort of magic. Whenever a function appears on the right-hand-side of an arrow, it's treated specially. The actual result is given to the function. If it returns a value that counts as true in Clojure, all is well. Otherwise, Midje signals a failure. Here's a simple example:

user=> (fact 3 => even?)

FAIL at (NO_SOURCE_FILE:2)
Actual result did not agree with the checking function.
        Actual result: 3
    Checking function: even?

Midje comes with a variety of predefined checkers useful for testing.

Extended equality

Functions on the right-hand side of a checkable are one example of Midje's extended equality. Midje defaults to ordinary Clojure equality, but various types on the right-hand side are special-cased. Functions (which you just saw) are the most important case. Second to them is regular expressions. On the right-hand side, regexps are interpreted to mean a comparison using re-find:

user=> (fact "aaab" => #"a+b")
true

Extended-equality is pervasive in Midje. For example, here's a way of checking a regular expression against a sequence of values:

user=> (fact ["ab" "hi, mom!" "aaab"] => (has every? #"a+b"))

FAIL at (NO_SOURCE_FILE:1)
Actual result did not agree with the checking function.
        Actual result: ["ab" "hi, mom!" "aaab"]
    Checking function: (has every? #"a+b")
false

Tabular tests

Clojure.test has a way of writing tests in a tabular form:

(are [x] (= (+ x x) (* x x))
     0
     2)

In Midje, you'd write this:

(tabular
  (fact (+ ?x ?x) => (* ?x ?x))
  ?x
  0
  2)

Moving beyond what clojure.test can do

There are, roughly, two styles of test-driven design. One is bottom up, where you construct working simple functions and then write other functions that use them. (This is reminiscent of traditional Lisp repl-driven development). The other style is top down (best described in Growing Object-Oriented Software, Guided by Tests, one of the strong early inspirations for Midje).

Clojure.test supports the first. Midje supports both. Since there are various paths through this user documentation, I'll point you to this introduction if you're interested in learning about how Midje views the top-down approach in a functional language.

In summary

With Midje, I've aimed to support bottom-up design, top-down design, repl-driven development, and (most importantly) a smooth switching between them all. I believe you can get the long-term value of repeatable tests without sacrificing the immediate feedback we're used to from the repl. To judge how well I've met my goals, you'll have to put Midje to use.

Anything else?

If you're a clojure.test user and you'd like this tutorial (or the whole user guide) to cover anything else, send mail to [email protected].