sanitycheck: parse testcase names correctly #21191
Conversation
nashif
force-pushed
the
sanity_parse_cases
branch
from
522e804
to
9faa168
Compare
nashif
changed the title
We were parsing random FAIL messages from the output of test runs ad testcases and capturing them in the xml output. Now we only parse the name if it starts with test_. Fixes zephyrproject-rtos#21162 Signed-off-by: Anas Nashif <[email protected]>
nashif
force-pushed
the
sanity_parse_cases
branch
from
9faa168
to
d261637
Compare
| @@ -126,7 +126,7 @@ class Test(Harness): | |||
|
|
|||
| def handle(self, line): | |||
| match = result_re.match(line) | |||
| if match: | |||
| if match and match.group(2): | |||
There was a problem hiding this comment.
@nashif First, I thought that it can help. But then I decided to check our regular expression. Now I don't think that change can help get rid of the incorrect results. Incorrect text comes from the group(3)
With that fix we check according to the full match and match(2). Even if the line has an additional unnecessary text inside -regular expression will be matched. Please check my idea on that webpage https://regex101.com/ You can insert there our regular expression (PASS|FAIL|SKIP) - (test_)?(.*) with test string for example FAIL - test_kernel_3 garbage here
You can see that group 1 is FAIL, group 2 is test_, group 3 is kernel_3 garbage here.
I think necessary to add a check if group(3) has a space symbols inside that part of the string. If no spaces, that part is a part of the test case name. If space(s) detected, something went wrong, maybe additional incorrect text inside that part of string. I think, necessary to edit regular expression. I will continue to think about it.
We were parsing random FAIL messages from the output of test runs ad
testcases and capturing them in the xml output. Now we only parse the
name if it starts with test_.
Fixes #21162