Release Notes
New features
-
Make directory prefix for topology files configurable (#3317)
-
Allow partition/environment extras to be used also as feature constraints in
valid_systems
andvalid_prog_environs
(#3316) -
Show the last N lines in test failures (#3291)
-
New configuration parameter for specifying commands to install reframe remotely for topology detection (#3281)
-
Add a
check_fail_phase
log formatting specifier (#3292) -
Allow users define custom log handlers and attach them to the framework (#3274)
-
Add new
--retries-threshold
option (#3262) -
Introduce permanent result storage and support performance results comparisons between runs (#3227)
- Add a configuration option to enable/disable results storage (#3295)
- Support session annotations with a new
--session-extras
option (#3266) - Expose SQLite connection timeout as configuration parameter (#3253)
- Disable results storage by default (#3313)
- Update
--table-format
arguments (#3307) - Use builtin
json
package for decoding testcase data (#3296) - Various enhancements to the results DB feature (#3283)
- Allow setting the file mode of the SQLite database (#3260)
- Change default values for
--performance-report
and--list-stored-sessions
options (#3258) - Fix query error when aggregation groups and presentation columns are disjoint (#3294)
- Fix DST skew in
CMPSPEC
period calculations (#3288) - Optimize testcase queries over time period (#3256)
- Ignore records with unset performance when grouping test cases (#3247)
-
Add a perflog entry also when sanity fails (#3189)
-
Skip by default flexible tests when not enough nodes found (#3173)
For the new feature of permanent results storage and performance analytics, please have a look at the tutorial and the manpage for details.
Enhancements
- Report the undefined parameters if an instantiation of an abstract test or fixture is attempted (#3293)
- Warn instead of failing when a filter expression is invalid for test (#3310)
- Issue a more compact message when JSON schema validation fails (#3314)
- Do not issue warning when skipping tests during the init phase (#3286)
- Reorganize the structure of the manpage (#3267)
- Improve performance of module backend when detecting module conflicts (#3263)
- Improve naming of automatically generated tests (#3265)
Bug fixes
- Add job options from the command-line to processor auto-detection remote job (#3318)
- Handle
None
cpu model name returned fromarchspec
(#3199) - Constraint the lxml version on aarch64 (#3193)
Test library
- Fix compilation of gpu burn benchmark with hip (#3232)
- Added MetalWalls scalability test for
mw
(#3233)