Record the outcome of each test case in compat.sh #7136
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
yanrayw
added
enhancement
component-tls
size-s
yanrayw
added
needs-review
yanrayw
force-pushed
the
5692-record-compatsh-test-cases
branch
from
920112b to
a0a6bad
Compare
3 tasks
yanrayw
force-pushed
the
5692-record-compatsh-test-cases
branch
3 times, most recently
from
fa455cd to
77b85f7
Compare
added 4 commits
February 28, 2023 14:29
Signed-off-by: Yanray Wang <[email protected]>
If the environment variable MBEDTLS_TEST_OUTCOME_FILE is set, the test outcome file records each test case in a single line with the format of PLATFORM;CONFIGURATION;compat;TEST CASE DESCRIPTION;RESULT;[CAUSE] - CONFIGURATION comes from MBEDTLS_TEST_CONFIGURATION to record configuration of each test case - PLATFORM is either set by users or calculated from test platform - RESULT is one of PASS, FAIL or SKIP. If test case fails, srv_out/cli_out follows as FAILURE CAUSE. Signed-off-by: Yanray Wang <[email protected]>
Signed-off-by: Yanray Wang <[email protected]>
Adjust where to set absolute path for MBEDTLS_TEST_OUTCOME_FILE as it's supposed to set its absolute path after all possible value assignment. Signed-off-by: Yanray Wang <[email protected]>
yanrayw
force-pushed
the
5692-record-compatsh-test-cases
branch
from
77b85f7 to
5b33f64
Compare
lpy4105
reviewed
Feb 28, 2023
tom-daubney-arm
added
historical-reviewing
tom-daubney-arm
removed
the
needs-reviewer
tom-daubney-arm
approved these changes
Jul 6, 2023
|
CI failures appear to be bugs in the CI. |
tom-daubney-arm
added
approved
|
(I've asked someone who's now on the non-roadmap team to do the backport) |
|
@gilles-peskine-arm Has anyone planned to do the backport? If no, I'll create one. |
|
Hi @yanrayw , I can do the backport :) |
|
Great! Thank you! @tgonzalezorlandoarm |
3 tasks
tgonzalezorlandoarm
removed
the
needs-backports
3 tasks
|
Just wanted to say thank you to everyone involved: this will make our life easier in driver-only cipher testing! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Fix: #5692
For
compat.sh, ifMBEDTLS_TEST_OUTCOME_FILEis set, the outcome of each test case is recorded in a outcome file with the format ofIn addition,
--append-outcomeis introduced to append test records to the outcome file if set.Gatekeeper checklist