-
Notifications
You must be signed in to change notification settings - Fork 5
/
README.tests
97 lines (78 loc) · 4.22 KB
/
README.tests
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
Working with the gfs2-utils test suite
--------------------------------------
Overview
--------
The test suite in the tests directory of the gfs2-utils source tree is based on
the Autotest framework provided by Autoconf. The basic idea is that the
testsuite.at file is the main source file for the tests, written in m4 and
generating a bourne shell script called testsuite, which we run with 'make check'.
When run, the test suite sources tests/atconfig and tests/atlocal for
configuration and then runs the whole suite or a specified set of tests,
depending on options passed to the test suite using TOPTS. For example, to see
a list of available options, use 'make check TOPTS=-h' and to see a numbered
list of the available tests, use 'make check TOPTS=-l'.
A subset of the available tests can be run using keywords and/or by specifying
the test ID numbers in TOPTS, e.g. make check TOPTS='-k mkfs 24 25'
Test output is captured and, if a test fails, a log is kept in
tests/testsuite.dir/$n/testsuite.log where $n is the test's ID number. Failed
tests can be re-run using make check TOPTS='--recheck' although it's better to
re-run the entire suite after fixing tests as a fix for one test could break
another.
Smoke/regression tests
----------------------
A number of GFS2-specific convenience macros have been defined in testsuite.at
to make defining new tests quick and easy. Also, some variables have been
defined in atlocal.in so that full paths to programs do not have to be included
in each test. Configuration should be specified in atlocal.in as atconfig is
generated by the configure script and atlocal is generated from atlocal.in at
build time.
To keep the test suite organised, the testsuite.at file sources the actual
tests from other files, e.g. mkfs.at.
A single test, specified as a test group in Autotest terms, follows the form
AT_SETUP([Test title])
...test goes here...
AT_CLEANUP
so, when adding tests, this is generally all that is required unless the tests
do not fit into an existing category, in which case AT_BANNER can be used to
group them, and they can be organised into a new .at file and sourced from
testsuite.at.
As the tests can be run individually, any new tests which require the dummy
volume $GFS_TGT to be present should call GFS_TGT_REGEN before attempting to
use it.
Documentation for Autotest, including the AT_* macros used to define tests, can
be found in the autoconf manual at:
http://www.gnu.org/software/autoconf/manual/index.html
Unit tests
----------
A unit test framework for each of the main utils and libgfs2 is included in its
source directory. These tests are based on the libcheck C unit test library and
are built and run by 'make check'. Generally, there should be one check "Suite"
defined for each source file of the util, named with a check_ prefix,
containing one or more test cases defined for each externally-visible function.
For example:
check_libgfs2 (main file: check_libgfs2.c)
|--------Suite: rgrp.c (in check_rgrp.c)
| |---------- Test case: test_rbm_find_good
| |-----------Test case: test_rbm_find_bad
|--------Suite: meta.c (in check_meta.c)
| |-----------Test case: test_lgfs2_meta
etc. etc.
The build rules for the unit tests are separated into a checks.am file in each
source directory. Documentation for writing libcheck unit tests can be found
at:
https://libcheck.github.io/check/doc/check_html/index.html
Autoconf's automatically generated test driver script only prints one line for
each unit test executable run instead of one line per test case. To view
details of unit test failures, see the generated .log and .trs files.
Generating coverage reports
---------------------------
Test coverage instrumentation can be enabled using the --enable-gcov option at
the configure stage. Once the tools have been built and run with this option
enabled, coverage data will be written to files in the source directories for
use by tools such as gcov or lcov. For example, to generate a HTML report of
testsuite code coverage, using lcov, in a directory named 'coverage':
./autogen.sh
./configure --enable-gcov
make check
lcov --directory . -c -o gfs2-utils.info
genhtml -o coverage gfs2-utils.info