A Java API for conducting unit testing. Prints code coverage and result reports, and also verifies the correctness of the tested code.
Our solution has been designed to offer robust code testing and coverage reporting in a lightweight package. The core Tester class is under 800 lines of code; this keeps the overhead of running the tester minimal. Our solution also exceeds the project’s requirements by providing support for many other objects out of the box. We accomplish this by utilizing the toString functionality of objects to distinguish them. If, however, an object’s toString has not been implemented, our solution provides an interface, the Identifiable interface, to distinguish objects. Any class that implements this lightweight interface can be used in our Tester - even if it is stored in an array.
The class diagram describing our solution has been provided at the end of this README.
Our API consists of three main calls: input(); load(); and log(). The other functions exposed by our API act mostly in support of these, or are used for result and coverage printing. A data flow diagram for each of these three calls has been provided at the end of this README.
The use and implementation of each of these three calls is outlined below:
The input() API call stores a input/output pair that will be tested in the method in which the call is made. If multiple input/output pairs are expected for a given method, then each one requires a separate input() call. An example of this can be seen below.
public boolean isEven(int numToCheck) throws IOException, UnIdentifiableException {
Tester.INSTANCE.input(2, true);
Tester.INSTANCE.input(3, false);
Tester.INSTANCE.input(4, true);
Tester.INSTANCE.load(numToCheck);
// divide the number by 2 and no remainder exists, the number is even
if (numToCheck % 2 == 0) {
Tester.INSTANCE.log(true);
return true;
} else {
Tester.INSTANCE.log(false);
return false;
}
}When the input() call is made, the tester generates an MD5 hash using runtime information combined with the expected input. The exact components that are used to create the hash can be seen on the right side of the input() section above. This hash is then mapped to a Result object containing information about that expected input, including the matching result, time/date information, etcetera in a (thread safe) HashMap data structure. This HashMap contains the hashes of all such expected inputs for the Tester instance.
The input() call also plays a key role in the tracking of code coverage information. It uses the expected result that is mapped to each input to create a hash of the possible route through the code. The exact components that are used to create this hash can be seen on the left side of our data flow diagram. For example, in isEven(), the MD5 hashes of the two possible paths (true and false) would be mapped into a HashMap containing all yet-unseen values. The expected output values that remain in this HashMap when the coverage report is generated are assumed to be uncovered.
The load() API call is made using the ‘actual’ input data passed into the method at runtime. An example of this API call being used is given at the bottom of isEven().
When the load() call is made, an MD5 hash is created using runtime information combined with a binary representation of the actual object passed to load as a parameter. Since the same runtime information is collected and used in the hash as in input, if the loaded object is identical to one specified in input(), then the MD5 hash generated by load should be identical to what was generated by input earlier. The expected input HashMap is queried using the hash; if the hash is truly identical, the HashMap should return information containing the expected result for the loaded input. Thanks to the O(1) search time of HashMaps, this querying stage scales extremely well. If null is returned from the query, the loaded value is assumed to be unexpected (i.e. there is no matching input statement for it). We chose to store this value as an unexpected value in our Tester, since it may help programmers diagnose bugs in certain situations.
Hopefully, most loaded values will have been expected. The matching output value will be hashed yet again with runtime information and a unique identifier number which differentiates each loaded value. This hash is then mapped to the same result object, but in a different map containing all the inputs awaiting log() calls with their expected return values. This is the Expected Results HashMap and can be seen at the bottom of the load() section of our data flow diagram.
The log() function is used to register return values with the tester, and is usually made at the end of the unit being tested. An example of log() being used can be seen in isEven().
When the log() call is made, a hash is again generated using runtime information and the unique identifier generated by the earlier load() call. The exact contents of the MD5 hash are detailed in the left-side MD5 hash in the log() section of the data flow diagram. The Expected Results HashMap is then queried using that hash. If an expected result is returned from the query, it is compared to the logged value. If there is a match, the result is marked as passed. Otherwise, if there is no match, it is marked as a failure.
The log() call is also responsible for updating the code coverage information stored in the tester. It uses the logged result (again combined with runtime information) to create an MD5 hash. This hash is used to query the unseen values HashMap. If there is a matching result in the unseen values HashMap, it is moved to a HashMap containing the seen values. The contents of these two maps are used to generate the code coverage report for this tester.
We have included a small test class named TesterHealthCheck. It performs a number of operations designed to test that each requirement is met, and also checks if native MD5 libraries are enabled for the platform running the Tester code.
The source of the test program is located at unit-test-api/src/com/clratm/unittest/TesterHealthCheck.java.
Our unit tester utilizes a fast MD5 library. We packaged the code along with our tester code, so no additional steps are necessary to import this library. There are a number of pre-compiled native libraries that we also distribute with our tester, located in unit-test-api/lib.
This software is licensed under the MIT license, available in the LICENSE file.