Skip to content

Commit

Permalink
add test harness/test doubles, functional test ot be removed, issues #35
Browse files Browse the repository at this point in the history
 , #65
  • Loading branch information
mariojmdavid committed Nov 23, 2021
1 parent b23d717 commit cdaec1c
Showing 1 changed file with 54 additions and 7 deletions.
61 changes: 54 additions & 7 deletions content/06.quality_criteria.md
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,53 @@ early detection of failures in the development cycle.
* **[QC.Uni04.1]** When working on automated testing, the use of testing doubles is
RECOMMENDED to mimic a simplistic behavior of objects and procedures.

## 4.7. Functional Testing [QC.Fun]
## **PROPOSAL**

## 4.6. Test Harness [QC.Har]

In software development, a test harness
[@url:https://searchsoftwarequality.techtarget.com/definition/test-harness],
is a collection of software and test data used by developers to unit test software models during
development. A test harness will specifically refer to test doubles, which are programs
that interact with the software being tested. Once a test harness is used to execute a test, they
can also utilize a test library to generate reports.

It is also a simple form of Integration Testing, where interaction and integration with external
components are substituted by a Double.

Test Double is a generic term for any case where you replace a production object for testing
purposes. There are various kinds of double [@isbn:9780131495050]:

* Dummy objects are passed around but never actually used. Usually they are just used to fill
parameter lists.

* Fake objects actually have working implementations, but usually take some shortcut which makes
them not suitable for production (an InMemoryTestDatabase is a good example).

* Stubs provide canned answers to calls made during the test, usually not responding at all to
anything outside what's programmed in for the test.

* Spies are stubs that also record some information based on how they were called. One form of this
might be an email service that records how many messages it was sent.

* Mocks are pre-programmed with expectations which form a specification of the calls they are
expected to receive. They can throw an exception if they receive a call they don't expect and
are checked during verification to ensure they got all the calls they were expecting.

As such the following criteria is defined for Test Harness:

* **[QC.Har01]** When working on automated testing, the use of Test Doubles
is RECOMMENDED to mimic a simplistic behavior of objects and procedures.

* **[QC.Har02]** Test Doubles SHOULD reside in the software component repository code
base but separated from the main code.

* **[QC.Har03]** Regression testing, that checks the conformance with previous tests, is
covered at this stage by executing the complete set of Test Doubles available.

* **[QC.Har04]** Test Doubles and regression, MUST be checked on change basis.

## 4.XX. Functional Testing [QC.Fun] - **TO BE REMOVED**

Functional testing is a type of black-box testing.
It involves the verification of the software component’s
Expand All @@ -115,24 +161,25 @@ functionality that the software component exposes, leaving apart any internal
design analysis or side-effects to external systems.

* **[QC.Fun01]** Functional testing SHOULD tend to cover the full set of functionality that
the software component claims to provide.
the software component claims to provide. **[SvcQC.Fun01]**

* **[QC.Fun02]** Functional tests SHOULD be checked automatically with the exception of those
functionality that require human interaction, such as Graphical User
Interfaces (GUI).
Interfaces (GUI). **[SvcQC.Fun02]**

* **[QC.Fun03]** When working on automated testing, the use of testing doubles is RECOMMENDED
to mimic a simplistic behavior of objects and procedures.
**TEST HARNESS**

* **[QC.Fun04]** Functional tests SHOULD reside in the software component repository code
base but separated from the main code.
base but separated from the main code. **[SvcQC.Fun03]** **TEST HARNESS**

* **[QC.Fun05]** Regression testing, that checks the conformance with previous tests, is
covered at this stage by executing the complete set of functional tests available.
covered at this stage by executing the complete set of functional tests available. **TEST HARNESS**

* **[QC.Fun06]** Functional and regression testing MUST be checked on change basis.
* **[QC.Fun06]** Functional and regression testing MUST be checked on change basis. **TEST HARNESS**

* **[QC.Fun07]** Functional and regression testing coverage MAY NOT be suitable for automated testing.
* **[QC.Fun07]** Functional and regression testing coverage MAY NOT be suitable for automated testing. **REMOVE**

## 4.8. Test-Driven Development [QC.Tdd]

Expand Down

0 comments on commit cdaec1c

Please sign in to comment.