Unit Testing

EnergyPlus Unit Testing

Unit testing has been added to EnergyPlus with tests located in the tst/EnergyPlus/unit subdirectory. A small set of "starter" tests has been committed to demonstrate some of the key usage.

Googletest (gtest)

We are using Googletest, also known as gtest. It offers a lot of nice capabilities and a low burden on developers to add new tests. Some of the features are:

• New tests are added by just putting the test file in the unit test directory (and adding the file to the CMakeLists.txt if building with CMake -- until we automate that)
• "Death" tests to check that exceptions or assertions should fire with certain inputs
• Tolerant equality checks for floating point values
• Simple setup/teardown support for tests with dependencies
• Filtering so that only certain tests are run (good while trying to correct test failures)
• Nice colored reports

Googletest documentation can be found here.

Writing Tests

Writing tests is simple:

• In CMake check the BUILD_TESTING option to add unit tests to the configuration
• Add a file (if it doesn't exist) with a name of the form MyClass.unit.cc or MyFunction.unit.cc
  • Copying an existing file is a quick way to get the boilerplate structure
• Add the file name to the test_src list in CMakeLists.txt in the unit directory
• Include the necessary headers for the code being tested
• Add using declarations to simplify the names you must type
• Add tests, where each test can include as few or as many checks as you want

The body of the simplest kind of test looks like this:

TEST( TestCaseName, TestName )
{
  ... // Test body goes here
}

Each test in a file should have the same ''TestCaseName'' and a different ''TestName''. CamelCase without underscores is suggested to avoid internal gtest problems at macro expansion time.

Tests may include some application object declarations, operations, and function calls interspersed with some gtest check statements, such as EXPECT_EQ( a, b );, which says that a and b should be equal. If the check fails then the test fails and gtest will report the test and line number where the failure occurred.

Running Tests

The tests will build into a single executable that runs all the tests in the same folder with other build targets. EnergyPlus_tests.exe reports test results to standard output.

To run a subset of the tests you can add a --gtest-filter argument such as:

EnergyPlus_tests.exe --gtest-filter="MyClassTest.*"

to run all tests within one test case. Other wildcard usage is supported to run, for example, a set of tests with similar test names.

Getting Fancier

There are a bunch of different checks beyond EXPECT_EQ. Some of other ones are EXPECT_NE, EXPECT_LT, EXPECT_GT, EXPECT_LE, EXPECT_GE, EXPECT_TRUE, and EXPECT_FALSE. There are checks that are tolerant for floating point values such as EXPECT_DOUBLE_EQ to allow for some roundoff and since precision can vary across platforms/compilers/builds.

To run tests on components that need some external state set up the tests use a class where the setup normally happens in the constructor and the teardown (to restore the prior state) happens in the destructor. The DataPlant.unit.cc file has an example of setup/teardown. Note that TEST_F is then used for the tests instead of TEST.

Strategy

Some strategies for growing the unit testing coverage of EnergyPlus:

• Tackle the easier code first:
  • Code with fewer dependencies, esp. on global data: setting up the state to run meaningful tests of functions with many dependencies is hard to do, which is why some codes use "mock" objects just for testing.
  • Simple functions with clearly defined inputs, outputs, and actions
• Skip hard-to-test functions that are expected to be refactored soon into OO code
• Add unit tests for any new classes and functions
• Most EnergyPlus classes are just containers at this point so there isn't much to test but as functions are brought into the classes tests should be added
• For code that is too hard to unit test currently at least use assertions to verify pre and post conditions
• As you work on code think about refactoring it to make it unit test friendly

Tips

• "Death" tests are nice but can slow down unit test runs on Windows (badly with MinGW GCC) and cause problems when multithreading test runs due to forking on Linux. Probably best to stay away from them or put them in a separate set of tests to be run manually, unthreaded only.