Skip to content
This repository has been archived by the owner on Nov 9, 2022. It is now read-only.

Unit Testing

Jump to bottom
David Cassel edited this page Nov 10, 2017 · 22 revisions

The Roxy framework includes a unit testing component. Unit testing provides several benefits:

  • Good tests give you confidence that your code does what you think it does
  • Tests show other developers how you expect your code will be used.
  • Testing ensures that new code does not reintroduce old bugs
  • Unit tests allow you to easily automate the tedious job of careful testing

If you're working with --app-type=rest, then take a look at Unit Testing REST applications after reading this wiki.

Configuring and Seeing tests

Several options are available to you to configure your unit tests, some of which you will find commented out in deploy/build.properties:

test-content-db
test-modules-db
test-port
do-not-deploy-tests

test-content-db defines the content database that tests will run against. If not defined, you cannot use unit testing. You may point it to an existing database if you like.

test-modules-db defines the modules database that tests will run against. If not defined, the modules database for your application will be used. When defined, modules related to unit testing will only be deployed to this test modules database.

test-port defines the modules database that tests will run against. If not defined, you cannot use unit testing. You may point it to an existing appserver only if your unit tests will use the same databases as that existing appserver.

do-not-deploy-tests allows you to define a comma separated list such as stage,prod of environments which unit testing will not be deployed to.

The minimal configuration you need to define to make unit testing work is to define test-content-db and test-port. Once these two properties are defined, point your browser to:

http://{server}:{test-port}/test/

If you had already deployed your source code, you'll need to do so again to get the testing code in place:

ml {env} deploy modules

Filename Conventions

By convention, a test suite corresponds to a library module in your application. To get started, create a directory under src/test/suites/ named for your library module.

Inside your test suite, you can create four specially-named files:

  • setup.xqy/setup.sjs - This module will be run before each test in your suite. Here you might insert a document into the test database that each of your tests will modify.
  • teardown.xqy/teardown.sjs - This module will run after each test in your suite. You might use this module to remove the document inserted by setup.xqy/setup.sjs.
  • suite-setup.xqy/suiteSetup.sjs - Run once when your suite is started. You can use this to insert some data that will not be modified over the course of the suite's tests.
  • suite-teardown.xqy/suiteTeardown.sjs - Run once when your suite is finished, to clean up after the suite's tests.

You create your test modules in the test suite directory. Typically, a module has responsibility for testing a particular function.

You can also create subdirectories in your test suite. The testing component will ignore these, so they are a good place for supporting files, like test data. Test data should be placed in a subdirectory called test-data.

As an example, consider a hypothetical library module that converts Comma Separated Values (CSV) to XML called csv-lib.xqy. Let's suppose it has a function called convert. We might test that with files like the following:

  • suites/csv-lib/suite-setup.xqy
  • suites/csv-lib/convert.xqy
  • suites/csv-lib/suite-teardown.xqy
  • suites/csv-lib/test-data/td.xqy

Why put test data into a separate module and not into suite-setup.xqy/suiteSetup.sjs? Because this way, setup, teardown and the test(s) can all refer to the same data source, making it easier to update the tests.

Assert Functions

The testing component has a helper library that provides several assert functions:

  • assert-true($supposed-truths as xs:boolean*)
  • assert-true($supposed-truths as xs:boolean*, $msg as item()*)
  • assert-false($supposed-falsehoods as xs:boolean*)
  • assert-equal($expected as item()*, $actual as item()*)
  • assert-not-equal($expected as item()*, $actual as item()*)
  • assert-exists($item as item()*)
  • assert-all-exist($count as xs:unsignedInt, $item as item()*)
  • assert-not-exists($item as item()*)
  • assert-at-least-one-equal($expected as item()*, $actual as item()*)
  • assert-same-values($expected as item()*, $actual as item()*) - Return true if and only if the two sequences have the same values, regardless of order.
  • assert-meets-minimum-threshold($expected as xs:decimal, $actual as xs:decimal+)
  • assert-meets-maximum-threshold($expected as xs:decimal, $actual as xs:decimal+)
  • assert-throws-error($function as xdmp:function)
  • assert-throws-error($function as xdmp:function, $error-code as xs:string?)
  • assert-throws-error($function as xdmp:function, $params as item()*, $error-code as xs:string?)
  • assert-http-get-status($url as xs:string, $options as element(xdmp-http:options), $status-code)

It is good practice to use a specific assert function. So rather than:

test:assert-equal(fn:true(), $actual)`

use this instead:

test:assert-true($actual)`

Using specific asserts makes your intentions more clear to developers who read your test code.

Loading Data

The Roxy Unit Test framework simplifies loading data for your tests. Files to be loaded can be placed in a sub-directory of your test suite called test-data. The test helper provides a function which will automatically use this directory as the location of test files to be loaded.

 test:load-data-file(<name-of-file-to-be-loaded-without-path>,database id, <URI>)

example:

 test:load-test-file("test-article.xml", xdmp:database(), "/test-article.xml")

The first parameter should be set to the filename without any path elements. The database id can be determined by using the API xdmp:database(). You can also specify the name of the database as in input string to this function if need be. The URI will be the URI of the file in the database.

This should be included in the appropriate suite or test setup xquery file. In the corresponding teardown, remove the data files by referring to the URI.

let $name := "/test-article.xml" return xdmp:document-delete($name)

When loading data you

  • create the data in the test-data subdir mentioned above
  • then load the data (and any test modules) by running ml <env> deploy modules. This copies all data to the Server
  • at runtime the test data is loaded into your test-content-db from the modules database or filesystem if you are running out of the filesystem.

Unit Testing Patterns

Testing with Server-side JavaScript

Roxy can unit test XQuery with XQuery, SJS with XQuery, and SJS with SJS. No testing XQuery with SJS yet. The examples below assume that /lib/simple.sjs exports a function called addOne().

module.exports = {
  addOne: addOne
};

function addOne(value) {
  return value + 1;
}

Test Server-side JavaScript with XQuery

You can write unit tests in XQuery to test code written in JavaScript. The one tricky part is that you can't import an .sjs module (such as the code you want to test) in XQuery. You can use xdmp:javascript-eval() to get around this.

xquery version "1.0-ml";

import module namespace test="http://marklogic.com/roxy/test-helper" at "/test/test-helper.xqy";

let $actual := xdmp:javascript-eval(
  "var simple = require ('/lib/simple.sjs');

   simple.addOne(1);")

return (
  test:assert-equal(2, $actual)
)

This is very similar overall to testing XQuery with XQuery, except that generating the $actual response happens in an eval.

Testing Server-side JavaScript with Server-side JavaScript

There are a couple differences when using SJS to test SJS code. Below is an example.

var test = require('/test/test-helper.xqy');
var simple = require('/lib/simple.sjs');

function addOnePlusOne() {
  var actual = simple.addOne(1);
  return [
    test.assertEqual(2, actual),
    test.assertNotEqual(5, actual)
  ];
};

function addOnePlusTwo() {
  var actual = simple.addOne(2);
  return test.assertEqual(3, actual);
};

[].concat(
  addOnePlusOne(),
  addOnePlusTwo()
)

We use require to import both the XQuery library module with the testing functions and the SJS module that defines the function we want to test.

Two things to note: first, wrap your tests in Immediately Invoked Function Expressions, or else call them explicitly after defining them. Although a function name is not technically necessary in an IIFE, it's useful when looking at the stack trace if a test fails. Either way, you must return the assert results and capture them. (Otherwise, the counts of successes and failures will be inaccurate.)

Second, as with XQuery-based tests, you must return the results of the assert functions in order to get test counts. If you use more than one assert with a test, put them in an array. Note that you need a flat array, hence the use of [].concat.

If you use IIFE, you can structure it like this:

let results = (function testOne() {
  // do stuff
  return [
    test.assert...(...),
    test.assert...(...)
  ];
})();

results = results.concat(
  (function testTwo() {
    // do stuff
    return [
      test.assert...(...),
      test.assert...(...)
    ]
  })();
);
Clone this wiki locally