PSA compliance tests suite #9312

orenc17 · 2019-01-09T15:02:56Z

Description

Add PSA compliance tests suite.
Note: this PR has been expanded to contain the entire suite.
This PR include tests for:

PSA internal-trusted-storage.
PSA Crypto
PSA Attestation

Relies on PRs
#9708 (merged)
#9795 (merged)
#9668 (merged)
#9822 (merged)
upcoming mbedTLS release
Note: this PR will not build without these PRs

Pull request type

[ ] Fix
[ ] Refactor
[ ] Target update
[ ] Functionality change
[ ] Docs update
[X] Test update
[ ] Breaking change

Reviewers

@alzix @jaypit02 @dreemkiller

jaypit02

Added review comments

jaypit02 · 2019-01-09T17:50:54Z

TESTS/psa-compliance/test_s001/main.c

+
+const psa_api_t psa_api = {
+    .framework_version     = pal_ipc_framework_version,
+    .version               = pal_ipc_version,


These structures are common to all tests. Therefore, these can be moved to a common file. Moving to common file is more scalable to consume any addition/deletion of an element.

Also you may want add copyright header to such files.

I prefer to generate the structure for each test
That way it won't mistakenly compiled to an mbed-app and waste flash

after a check I've moved the struct to the framework directory

jaypit02 · 2019-01-09T17:58:42Z

components/TARGET_PSA/spm/spm_client.h

    size_t in_len,
-    const psa_outvec_t *out_vec,
+    const psa_outvec *out_vec,


As per the latest PSA FF spec, out_vec parameter is no more "const". It is now:

Suggested change

const psa_outvec *out_vec,

psa_outvec *out_vec,

jaypit02 · 2019-01-09T18:03:05Z

features/frameworks/TARGET_PSA/mbed_lib.json

@@ -0,0 +1,26 @@
+{
+    "name": "psa-compliance",
+    "config": {


FYI- These macros definition will be available in pal_config.h in PSA compliance test suite release. Once you have pal_config.h, this json may not be required.

This is the "mbed way" for configuration
We could add platform specific configuration in the future through this file

jaypit02 · 2019-01-09T18:15:41Z

TESTS/psa-compliance/test_s001/test_s001.c

+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+**/
+#define ITS_TEST


Is is possible to maintain test specific mbed_lib.json to pass ITS_TEST macro?
Idea is to avoid test editing.

alzix

test_s001 name does not describe it actually tests PSA ITS implementation.
please rename TESTS/psa-compliance/test_s001 to TESTS/psa-compliance/psa-ist-s001 and fix the importer

alzix · 2019-01-31T08:29:53Z

@orenc17 - ping

cmonr · 2019-01-31T16:51:00Z

Making a note here. I think this now relies on #9192 instead.

mikisch81 · 2019-01-31T17:16:15Z

@cmonr actually like many others, it depends on #9529

0xc0170 · 2019-02-13T13:44:37Z

@orenc17 What is the status for this PR? The dependencies were integrated, weren't they?

orenc17 · 2019-02-13T13:50:07Z

the porting is being continued by another team.. i believe they use this PR as a base

cmonr · 2019-02-13T19:13:10Z

@orenc17 They'll still hit the same problem once they introduce a PR.

0xc0170 · 2019-02-19T14:57:50Z

the porting is being continued by another team.. i believe they use this PR as a base

Shall this be closed?

cmonr · 2019-02-19T19:41:03Z

@orenc17 This became unblocked once #9529 came in.

Any updates?

amiraloosh · 2019-02-19T20:19:27Z

Yes, we will continue with this PR. The PSA team will update tomorrow.

NirSonnenschein · 2019-02-21T18:16:54Z

Hi @cmonr , @jaypit02 @alzix,
This PR has been forced pushed with multiple changes (multiple additional tests added for attestation and crypto modules as well as its), please re-review the new version.

NirSonnenschein · 2019-02-21T18:23:14Z

FYI @netanelgonen

cmonr · 2019-02-22T02:01:40Z

Checking in, this is still waiting on two other PRs before it can progress, correct?

0xc0170 · 2019-02-28T12:40:20Z

Very last dependency here: #9668 - is now in CI. Will this fix also astyle issues in Travis?

NirSonnenschein · 2019-02-28T13:23:06Z

@0xc0170 , this should fix most of the issues, I'm adding an astyle ignore rule to ignore the style of the imported compliance tests shortly.

alekla01 · 2019-03-06T19:19:11Z

exporter likely needs to be restarted, as it probably has incorrectly status as pending after the license issues.

cmonr · 2019-03-06T19:25:29Z

CI job restarted: jenkins-ci/mbed-os-ci_exporter

cmonr · 2019-03-06T19:58:29Z

Restarted CI.

Was getting odd null pointer exception issue when restarting single job.

mbed-ci · 2019-03-06T22:58:46Z

Test run: SUCCESS

Summary: 13 of 13 test jobs passed
Build number : 8
Build artifacts

NirSonnenschein · 2019-03-07T08:26:16Z

features/frameworks/TARGET_PSA/pal/pal_mbed_os_crypto.cpp

+{
+#ifndef PSA_ATTESTATION_DISABLED
+    const uint8_t private_key_data[] = {
+       0x49, 0xc9, 0xa8, 0xc1, 0x8c, 0x4b, 0x88, 0x56,


This is a hard-coded attestation key used for testing the attestation feature. it was randomly generated.
The specific key chosen shouldn't matter to the test it just needs a key to be injected before it is run (in practice each decide is expected to have it's own randomly generated key).

cmonr · 2019-03-06T23:48:22Z

features/frameworks/TARGET_PSA/val_greentea.cpp

+    if (IS_TEST_FAIL(status) || IS_TEST_SKIP(status))
+    {
+        GREENTEA_TESTSUITE_RESULT(false);
+        return;


Seems odd that this was needed.

if you mean the explicit return, you are right, it should not be needed.
It is likely an artifact from the previous implementation of the function. - Fixed

cmonr · 2019-03-06T23:52:30Z

features/frameworks/TARGET_PSA/val_greentea.cpp

+    bool continue_test = true;
+
+    test_info.test_num = test_num;
+    if (boot.state == BOOT_NOT_EXPECTED || boot.state == BOOT_EXPECTED_CRYPTO)


I'm probably missing something, but how would this ever not be true?

boot_t boot isn't static.

good point this check should be removed - fixed

cmonr · 2019-03-06T23:56:07Z

features/frameworks/TARGET_PSA/val_greentea.cpp

+        mbed_val_print(PRINT_ERROR, "Call to dispatch SF failed. Status=%x\n", status_of_call);
+    }
+
+    pal_ipc_close(*handle);


Why should the handle be closed here instead of outside of the function?

this function is called from a pointer to function in the original attestation test framework (prior to our adaptation to greentea) in the struct val_api_t. The original implementation had the calling semantic that this function frees the handle inside and we preserved this for future compatibility (had we implemented the test framework we would have done many things very differently).

cmonr · 2019-03-06T23:57:22Z

features/frameworks/TARGET_PSA/val_greentea.cpp

+}
+
+
+}


} // extern "C"

good point, fixed

cmonr · 2019-03-07T00:11:50Z

features/frameworks/TARGET_PSA/pal/pal_mbed_os_crypto.cpp

+#endif
+
+#if defined(MBEDTLS_ENTROPY_NV_SEED) || defined(COMPONENT_PSA_SRV_IPC)
+        inject_entropy();


Seems weird that this and the following line of code are indented.

you are right, however this file has since been replaced (git move) with the file pal_mbed_os_intf.cpp which has been refactored and no longer contains this anomaly

cmonr · 2019-03-07T00:16:52Z

@orenc17 A couple of questions/nits, but just looking for answers before merging.

orenc17 · 2019-03-07T00:31:27Z

@cmonr i've handed over the PR to @NirSonnenschein
i'm no longer working on this PR, we kept it open for convenience

NirSonnenschein

Thanks @cmonr

NirSonnenschein · 2019-03-07T08:17:42Z

features/frameworks/TARGET_PSA/pal/pal_mbed_os_crypto.cpp

+#endif
+
+#if defined(MBEDTLS_ENTROPY_NV_SEED) || defined(COMPONENT_PSA_SRV_IPC)
+        inject_entropy();


you are right, however this file has since been replaced (git move) with the file pal_mbed_os_intf.cpp which has been refactored and no longer contains this anomaly

NirSonnenschein · 2019-03-07T08:20:41Z

features/frameworks/TARGET_PSA/val_greentea.cpp

+    if (IS_TEST_FAIL(status) || IS_TEST_SKIP(status))
+    {
+        GREENTEA_TESTSUITE_RESULT(false);
+        return;


if you mean the explicit return, you are right, it should not be needed.
It is likely an artifact from the previous implementation of the function. - Fixed

NirSonnenschein · 2019-03-07T08:23:29Z

features/frameworks/TARGET_PSA/val_greentea.cpp

+    bool continue_test = true;
+
+    test_info.test_num = test_num;
+    if (boot.state == BOOT_NOT_EXPECTED || boot.state == BOOT_EXPECTED_CRYPTO)


good point this check should be removed - fixed

NirSonnenschein · 2019-03-07T08:26:16Z

features/frameworks/TARGET_PSA/pal/pal_mbed_os_crypto.cpp

+{
+#ifndef PSA_ATTESTATION_DISABLED
+    const uint8_t private_key_data[] = {
+       0x49, 0xc9, 0xa8, 0xc1, 0x8c, 0x4b, 0x88, 0x56,


This is a hard-coded attestation key used for testing the attestation feature. it was randomly generated.
The specific key chosen shouldn't matter to the test it just needs a key to be injected before it is run (in practice each decide is expected to have it's own randomly generated key).

NirSonnenschein · 2019-03-07T08:57:18Z

features/frameworks/TARGET_PSA/val_greentea.cpp

+        mbed_val_print(PRINT_ERROR, "Call to dispatch SF failed. Status=%x\n", status_of_call);
+    }
+
+    pal_ipc_close(*handle);


this function is called from a pointer to function in the original attestation test framework (prior to our adaptation to greentea) in the struct val_api_t. The original implementation had the calling semantic that this function frees the handle inside and we preserved this for future compatibility (had we implemented the test framework we would have done many things very differently).

NirSonnenschein · 2019-03-07T08:57:56Z

features/frameworks/TARGET_PSA/val_greentea.cpp

+}
+
+
+}


good point, fixed

@cmonr

Thanks @cmonr.

NirSonnenschein · 2019-03-07T10:34:16Z

restarted CI on review fixes

alekla01 · 2019-03-07T12:57:51Z

Restarted jenkins-ci/exporter

mbed-ci · 2019-03-07T13:19:08Z

Test run: FAILED

Summary: 1 of 13 test jobs failed
Build number : 10
Build artifacts

Failed test jobs:

jenkins-ci/mbed-os-ci_exporter

NirSonnenschein · 2019-03-07T13:57:30Z

CI has passed again on the CR changes, @cmonr please take a look, if all is ok we can proceed.

cmonr · 2019-03-07T15:13:55Z

#9312 (comment)

@NirSonnenschein It would be good to capture this as a comment in the file, but that can be added in a seperate PR.

Not going to block the PR on that.

jaypit02 reviewed Jan 9, 2019

View reviewed changes

cmonr added needs: review needs: preceding PR labels Jan 10, 2019

alzix suggested changes Jan 13, 2019

View reviewed changes

alzix mentioned this pull request Jan 13, 2019

Update PSA defines and headers #9214

Closed

0xc0170 added needs: work and removed needs: review labels Jan 14, 2019

cmonr removed the needs: preceding PR label Feb 13, 2019

NirSonnenschein added release-version: 5.12.0-rc1 needs: preceding PR needs: review do not merge and removed needs: work labels Feb 20, 2019

alekshex approved these changes Feb 24, 2019

View reviewed changes

cmonr added the risk: R label Feb 25, 2019

NirSonnenschein mentioned this pull request Feb 26, 2019

add initial version of PSA compliance test documents ARMmbed/mbed-os-5-docs#966

Merged

0xc0170 added needs: CI and removed needs: review labels Mar 6, 2019

cmonr mentioned this pull request Mar 6, 2019

Fix for secure partition #9939

Merged

cmonr added ready for merge and removed needs: CI labels Mar 6, 2019

cmonr reviewed Mar 7, 2019

View reviewed changes

cmonr added needs: review and removed ready for merge labels Mar 7, 2019

NirSonnenschein reviewed Mar 7, 2019

View reviewed changes

removed redundant code + added comment

4edb6da

Thanks @cmonr.

0xc0170 added needs: CI and removed needs: review labels Mar 7, 2019

0xc0170 added needs: review and removed needs: CI labels Mar 7, 2019

cmonr approved these changes Mar 7, 2019

View reviewed changes

NirSonnenschein added ready for merge and removed needs: review labels Mar 7, 2019

cmonr merged commit a87c7c8 into ARMmbed:master Mar 7, 2019

cmonr removed the ready for merge label Mar 7, 2019

PSA compliance tests suite #9312

PSA compliance tests suite #9312

Conversation

orenc17 commented Jan 9, 2019 • edited by cmonr Loading

Description

Pull request type

Reviewers

jaypit02 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alzix left a comment

Choose a reason for hiding this comment

alzix commented Jan 31, 2019

cmonr commented Jan 31, 2019

mikisch81 commented Jan 31, 2019

0xc0170 commented Feb 13, 2019

orenc17 commented Feb 13, 2019

cmonr commented Feb 13, 2019

0xc0170 commented Feb 19, 2019

cmonr commented Feb 19, 2019

amiraloosh commented Feb 19, 2019

NirSonnenschein commented Feb 21, 2019

NirSonnenschein commented Feb 21, 2019

cmonr commented Feb 22, 2019

0xc0170 commented Feb 28, 2019

NirSonnenschein commented Feb 28, 2019

alekla01 commented Mar 6, 2019

cmonr commented Mar 6, 2019

cmonr commented Mar 6, 2019

mbed-ci commented Mar 6, 2019

This comment was marked as resolved.

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cmonr commented Mar 7, 2019

orenc17 commented Mar 7, 2019

NirSonnenschein left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

NirSonnenschein commented Mar 7, 2019

alekla01 commented Mar 7, 2019

mbed-ci commented Mar 7, 2019

NirSonnenschein commented Mar 7, 2019

cmonr commented Mar 7, 2019

orenc17 commented Jan 9, 2019 •

edited by cmonr

Loading