From b26c8d8bb49797fe35470dbd85dbb3035fa961ef Mon Sep 17 00:00:00 2001 From: Gilles Peskine Date: Wed, 4 Sep 2019 19:26:17 +0200 Subject: [PATCH 1/6] Create a driver interface test strategy document Just the structure for now, no actual content. --- .../testing/driver-interface-test-strategy.md | 24 +++++++++++++++++++ 1 file changed, 24 insertions(+) create mode 100644 docs/architecture/testing/driver-interface-test-strategy.md diff --git a/docs/architecture/testing/driver-interface-test-strategy.md b/docs/architecture/testing/driver-interface-test-strategy.md new file mode 100644 index 000000000000..5db91888c559 --- /dev/null +++ b/docs/architecture/testing/driver-interface-test-strategy.md @@ -0,0 +1,24 @@ +# Mbed Crypto driver interface test strategy + +This document describes the test strategy for the driver interfaces in Mbed Crypto. Mbed Crypto has interfaces for secure element drivers, accelerator drivers and entropy drivers. This document is about testing Mbed Crypto itself; testing drivers is out of scope. + +The driver interfaces are standardized through PSA Cryptography functional specifications. + +## Secure element driver interface + +The secure element driver interface (SE interface for short) is defined by [`psa/crypto_se_driver.h`](../../../include/psa/crypto_se_driver.h). This is an interface between Mbed Crypto and one or more third-party drivers. + +TODO + + +## Accelerator driver interface + +The accelerator driver interface is defined by [`psa/crypto_accel_driver.h`](../../../include/psa/crypto_accel_driver.h). + +TODO + +## Entropy driver interface + +The entropy driver interface is defined by [`psa/crypto_entropy_driver.h`](../../../include/psa/crypto_entropy_driver.h). + +TODO From 92bcfdbb6652c6065a9a953d9abc94950d0a7e4e Mon Sep 17 00:00:00 2001 From: Gilles Peskine Date: Wed, 4 Sep 2019 19:26:50 +0200 Subject: [PATCH 2/6] Write secure element driver interface test strategy --- .../testing/driver-interface-test-strategy.md | 92 ++++++++++++++++++- 1 file changed, 91 insertions(+), 1 deletion(-) diff --git a/docs/architecture/testing/driver-interface-test-strategy.md b/docs/architecture/testing/driver-interface-test-strategy.md index 5db91888c559..d8dade6eb0c0 100644 --- a/docs/architecture/testing/driver-interface-test-strategy.md +++ b/docs/architecture/testing/driver-interface-test-strategy.md @@ -8,8 +8,98 @@ The driver interfaces are standardized through PSA Cryptography functional speci The secure element driver interface (SE interface for short) is defined by [`psa/crypto_se_driver.h`](../../../include/psa/crypto_se_driver.h). This is an interface between Mbed Crypto and one or more third-party drivers. -TODO +The SE interface consists of one function provided by Mbed Crypto (`psa_register_se_driver`) and many functions that drivers must implement. To make a driver usable by Mbed Crypto, the initialization code must call `psa_register_se_driver` with a structure that describes the driver. The structure mostly contains function pointers, pointing to the driver's methods. All calls to a driver function are triggered by a call to a PSA crypto API function. + +### SE driver interface unit tests + +This section describes unit tests that must be implemented to validate the secure element driver interface. Note that a test case may cover multiple requirements; for example a “good case” test can validate that the proper function is called, that it receives the expected inputs and that it produces the expected outputs. + +Many SE driver interface unit tests could be covered by running the existing API tests with a key in a secure element. + +#### SE driver registration + +* Test `psa_register_se_driver` with valid and with invalid arguments. +* Make at least one failing call to `psa_register_se_driver` followed by a successful call. +* Make at least one test that successfully registers the maximum number of drivers and fails to register one more. + +#### Dispatch to SE driver + +For each API function that can lead to a driver call (more precisely, for each driver method call site, but this is practically equivalent): + +* Make at least one test with a key in a secure element that checks that the driver method is called. A few API functions involve multiple driver methods; these should validate that all the expected driver methods are called. +* Make at least one test with a key that is not in a secure element that checks that the driver method is not called. +* Make at least one test with a key in a secure element with a driver that does not have the requisite method (i.e. the method pointer is `NULL`) but has the substructure containing that method, and check that the return value is `PSA_ERROR_NOT_SUPPORTED`. +* Make at least one test with a key in a secure element with a driver that does not have the substructure containing that method (i.e. the pointer to the substructure is `NULL`), and check that the return value is `PSA_ERROR_NOT_SUPPORTED`. +* At least one test should register multiple drivers with a key in each driver and check that the expected driver is called. This does not need to be done for all operations (use a white-box approach to determine if operations may use different code paths to choose the driver). +* At least one test should register the same driver structure with multiple lifetime values and check that the driver receives the expected lifetime value. + +Some methods only make sense as a group (for example a driver that provides the MAC methods must provide all or none). In those cases, test with all of them null and none of them null. + +#### SE driver inputs + +For each API function that can lead to a driver call (more precisely, for each driver method call site, but this is practically equivalent): + +* Wherever the specification guarantees parameters that satisfy certain preconditions, check these preconditions whenever practical. +* If the API function can take parameters that are invalid and must not reach the driver, call the API function with such parameters and verify that the driver method is not called. + +#### SE driver outputs + +For each API function that leads to a driver call, call it with parameters that cause a driver to be invoked and check how Mbed Crypto handles the outputs. + +* Correct outputs. +* Incorrect outputs such as an invalid output length. +* Expected errors (e.g. `PSA_ERROR_INVALID_SIGNATURE` from a signature verification method). +* Unexpected errors. At least test that if the driver returns `PSA_ERROR_GENERIC_ERROR`, this is propagated correctly. + +Key creation functions invoke multiple methods and need more complex error handling: + +* Check the consequence of errors detected at each stage (slot number allocation or validation, key creation method, storage accesses). +* Check that the storage ends up in the expected state. At least make sure that no intermediate file remains after a failure. + +#### Persistence of SE keys + +The following tests must be performed at least one for each key creation method (import, generate, ...). + +* Test that keys in a secure element survive `psa_close_key(); psa_open_key()`. +* Test that keys in a secure element survive `mbedtls_psa_crypto_free(); psa_crypto_init()`. +* Test that the driver's persistent data survives `mbedtls_psa_crypto_free(); psa_crypto_init()`. +* Test that `psa_destroy_key()` does not leave any trace of the key. + +#### Resilience for SE drivers + +Creating or removing a key in a secure element involves multiple storage modifications (M1, ..., Mn). If the operation is interrupted by a reset at any point, it must be either rolled back or completed. + +* For each potential interruption point (before M1, between M1 and M2, ..., after Mn), call `mbedtls_psa_crypto_free(); psa_crypto_init()` at that point and check that this either rolls back or completes the operation that was started. +* This must be done for each key creation method and for key destruction. +* This must be done for each possible flow, including error cases (e.g. a key creation that fails midway due to `OUT_OF_MEMORY`). +* The recovery during `psa_crypto_init` can itself be interrupted. Test those interruptions too. +* Two things need to be tested: the key that is being created or destroyed, and the driver's persistent storage. +* Check both that the storage has the expected content (this can be done by e.g. using a key that is supposed to be present) and does not have any unexpected content (for keys, this can be done by checking that `psa_open_key` fails with `PSA_ERRROR_DOES_NOT_EXIST`). + +This requires instrumenting the storage implementation, either to force it to fail at each point or to record successive storage states and replay each of them. Each `psa_its_xxx` function call is assumed to be atomic. + +### SE driver system tests + +#### Real-world use case + +We must have at least one driver that is close to real-world conditions: + +* With its own source tree. +* Running on actual hardware. +* Run the full driver validation test suite (which does not yet exist). +* Run at least one test application (e.g. the Mbed OS TLS example). + +This requirement shall be fulfilled by the [Microchip ATECC508A driver](https://github.com/ARMmbed/mbed-cryptoauthlib). + +#### Complete driver + +We should have at least one driver that covers the whole interface: + +* With its own source tree. +* Implementing all the methods. +* Run the full driver validation test suite (which does not yet exist). +A PKCS#11 driver would be a good candidate. It would be useful as part of our product offering. ## Accelerator driver interface From 545c28bf706382eb36d64838da95170e28c835f7 Mon Sep 17 00:00:00 2001 From: Gilles Peskine Date: Wed, 4 Sep 2019 19:41:16 +0200 Subject: [PATCH 3/6] Fix URL of ATECC driver --- docs/architecture/testing/driver-interface-test-strategy.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/architecture/testing/driver-interface-test-strategy.md b/docs/architecture/testing/driver-interface-test-strategy.md index d8dade6eb0c0..2f31c001035a 100644 --- a/docs/architecture/testing/driver-interface-test-strategy.md +++ b/docs/architecture/testing/driver-interface-test-strategy.md @@ -89,7 +89,7 @@ We must have at least one driver that is close to real-world conditions: * Run the full driver validation test suite (which does not yet exist). * Run at least one test application (e.g. the Mbed OS TLS example). -This requirement shall be fulfilled by the [Microchip ATECC508A driver](https://github.com/ARMmbed/mbed-cryptoauthlib). +This requirement shall be fulfilled by the [Microchip ATECC508A driver](https://github.com/ARMmbed/mbed-os-atecc608a/). #### Complete driver From 8b193c10cac024b7e33320818cbb3b156efb8d1e Mon Sep 17 00:00:00 2001 From: Gilles Peskine Date: Thu, 5 Sep 2019 17:58:13 +0200 Subject: [PATCH 4/6] Check inputs too --- docs/architecture/testing/driver-interface-test-strategy.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/architecture/testing/driver-interface-test-strategy.md b/docs/architecture/testing/driver-interface-test-strategy.md index 2f31c001035a..d6769da0b3d6 100644 --- a/docs/architecture/testing/driver-interface-test-strategy.md +++ b/docs/architecture/testing/driver-interface-test-strategy.md @@ -41,6 +41,7 @@ For each API function that can lead to a driver call (more precisely, for each d * Wherever the specification guarantees parameters that satisfy certain preconditions, check these preconditions whenever practical. * If the API function can take parameters that are invalid and must not reach the driver, call the API function with such parameters and verify that the driver method is not called. +* Check that the expected inputs reach the driver. This may be implicit in a test that checks the outputs if the only realistic way to obtain the correct outputs is to start from the expected inputs (as is often the case for cryptographic material, but not for metadata). #### SE driver outputs From 1ff67cc65cf08ebbdff01fa46dd412dfee795f89 Mon Sep 17 00:00:00 2001 From: Gilles Peskine Date: Mon, 9 Sep 2019 18:25:13 +0200 Subject: [PATCH 5/6] Build the driver interface test strategy document --- docs/architecture/Makefile | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/architecture/Makefile b/docs/architecture/Makefile index f763c9c54849..5bde640447e7 100644 --- a/docs/architecture/Makefile +++ b/docs/architecture/Makefile @@ -4,6 +4,7 @@ default: all all_markdown = \ mbed-crypto-storage-specification.md \ + testing/driver-interface-test-strategy.md \ # This line is intentionally left blank html: $(all_markdown:.md=.html) From 4b3db7382d0a5596a53b67a2a1ea0006d66fe38b Mon Sep 17 00:00:00 2001 From: Gilles Peskine Date: Mon, 9 Sep 2019 18:26:20 +0200 Subject: [PATCH 6/6] Add "clean" rule --- docs/architecture/Makefile | 3 +++ 1 file changed, 3 insertions(+) diff --git a/docs/architecture/Makefile b/docs/architecture/Makefile index 5bde640447e7..258abcdb0ae3 100644 --- a/docs/architecture/Makefile +++ b/docs/architecture/Makefile @@ -18,3 +18,6 @@ all: html pdf $(PANDOC) -o $@ $< .md.pdf: $(PANDOC) -o $@ $< + +clean: + rm -f *.html *.pdf