SBML Test Suite release 3.3.0

At long last, we are pleased to announce the full release of version 3.3.0 of the SBML Test Suite. This release brings many updates and some changes. A summary of the changes is listed below; more details can be found in the NEWS.md file.

Installers

The release page on GitHub has the following ready-to-run installers for different operating systems:

O.S.	Architecture	Format	File
macOS	64-bit	runnable installer	SBML Test Runner 3.3.0 macOS installer.pkg
Windows	64- or 32-bit	runnable installer	Setup_SBMLTestRunner-3.3.0-Windows.exe
Linux	64-bit	runnable installer	SBMLTestRunner-3.3.0-linux-x64-installer.run
Linux	32-bit	runnable installer	SBMLTestRunner-3.3.0-linux-installer.run
Linux	64-bit	Debian package	SBMLTestRunner_3.3.0-0.linux_amd64.deb

In addition, archives of the test cases are available for separate download. Note that the SBML Test Runner includes the semantic cases internally so you do not need to download them. Only download the test case archives if you want to run the stochastic or syntactic test cases using your own framework or software, or need the semantic test cases separately for some reason.

Test case archive	Included in Test Runner?	File
Stochastic tests	⬜︎	sbml-stochastic-test-cases-2017-12-12.zip
Syntactic tests	⬜︎	sbml-syntactic-test-cases-2017-12-12.zip
Semantic tests	⬛︎	sbml-semantic-test-cases-2017-12-12.zip

Changes in release 3.3.0

• There is a new SBML Test Runner installer for Mac OS X. The installation is now distributed as an OS X .pkg installer file. By default, it will install the SBML Test Runner into the /Applications folder on the user's computer.

• This release contains hundreds of new semantic test cases, numbers 01219–1780, written by Lucian Smith. They test many more SBML features and rules. Important note: tests 01234–01292 specifically test models that follow the release candidate specification of SBML Level 3 Version 2, which encode elements and situations that were not present or illegal in SBML Level 3 Version 1. If your simulator uses libSBML, it will need version 5.15.0 or later of libSBML, though the latest (5.16.0) must be used to correctly validate some models. Please see the file NEWS.md (provided with the distribution) for more details about the changes to the test cases. Finally, as before, the SBML Test Runner only uses the semantic test cases (and bundles them), but stochastic and syntactic test cases are also available as separate downloadable archives.

• A few things have been cleaned up in past tests. Examples include model identifiers of a few test models that were misleading (now fixed), and a few tests that were designed to test species in multiple compartments were at some point accidentally changed so that all the species were put into a single compartment (they have been redistributed appropriately now).

• With the addition of FBC v2 tests, the packagesPresent line in the .m file can now include fbc_v1 or fbc_v2, depending on which version of the test it is. The original fbc tag is still present. In addition, a new FBC test tag fbc:NonStrict is now provided for FBC v2 models where the strict flag has been set to false.

• The Test Runner's algorithm for comparing test results and displaying the differences between expected and actual result values now properly ignores column ordering. Also, it now considers it an error if the results do not have the expected number of rows. Thanks to Leandro Watanabe and Chris Myers (U. Utah) and Lucian Smith for reporting the problems.

• The Test Runner now allows the combination of filters that include, simultaneously, tags, problematic test cases, and only supported test cases. Thanks to Chris Myers for raising the issue and testing the implementation.

• The SBML Test Runner now monitors for file changes in the currently-displayed test case. If the application's output .csv file is changed outside the runner (e.g., by manually editing the file), it should now notice the changes and update the display. In addition, the Test Runner also monitors the test case settings file, to detect changes that developers may make while experimenting with test case definitions. Note that for architectural reasons, only the case being displayed is so monitored. A setting in the preferences dialog allows this feature to be turned on and off.

• This release of the SBML Test Runner works around a crashing bug caused by an SWT-GTK issue that manifests itself on Ubuntu 16.

• This release of the SBML Test Runner works around a font scaling issue that manifests itself on Windows when using display scaling of 150%.

• The default value of the "Always check for new versions of test cases when application starts" preference option is now "yes" instead of "no" for new users. Users who have used the previous version of the Test Runner will not be affected; this only affects new users of the Test Runner.

• The File → Exit menu item should now work on Windows. Thanks to Thomas Hamm for reporting that it did nothing before. Missing keyboard shortcuts on Windows are now available in the latest version.

• The Test Runner features various other small fixes and improvements.

• The SBML Test Suite home repository has been moved to GitHub. New development will take place in the branch called ‘develop’. Each time we make a new release, we will merge the changes into branch ‘master’ and start the next new developments in ‘develop’.

• The issue tracker for the Test Suite is now the GitHub tracker: https://github.com/sbmlteam/sbml-test-suite/issues. Lucian Smith migrated the previous issues from the SourceForce tracker, so they are now in the GitHub tracker, for history. We thank the guides by Thomas Zajac and Chris Mungall for helpful info about how to achieve this migration.

More information

Please see the SBML Test Suite for general information about the SBML Test Suite, and the GitHub Page for the SBML Test Suite for the software source code.

This software is released under the terms of the LGPL 2.1, but it incorporates code and other materials from other software. Please see the LICENSE.html file for more detailed information about license and distribution terms.