forked from HDFGroup/hdf5
-
Notifications
You must be signed in to change notification settings - Fork 2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Correct Reporting of Multi-Threaded Test Outcomes #15
Open
mattjala
wants to merge
12
commits into
LifeboatLLC:1_14_2_multithread
Choose a base branch
from
mattjala:mt_test_reporting
base: 1_14_2_multithread
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Correct Reporting of Multi-Threaded Test Outcomes #15
mattjala
wants to merge
12
commits into
LifeboatLLC:1_14_2_multithread
from
mattjala:mt_test_reporting
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mattjala
force-pushed
the
mt_test_reporting
branch
from
November 25, 2024 20:39
e10a923
to
6ef874f
Compare
mattjala
force-pushed
the
mt_test_reporting
branch
3 times, most recently
from
November 26, 2024 16:15
9cf1378
to
cc9369b
Compare
mattjala
force-pushed
the
mt_test_reporting
branch
from
December 4, 2024 14:54
0c2fa79
to
e729420
Compare
mattjala
force-pushed
the
mt_test_reporting
branch
2 times, most recently
from
December 30, 2024 15:01
6d97430
to
1a70c83
Compare
mattjala
force-pushed
the
mt_test_reporting
branch
from
January 3, 2025 14:57
1a70c83
to
5109153
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This consists of the following major aspects:
Since subtests aren't exposed directly to testframe, several changes had to be made to support including them in the results while also avoid duplication during multithreading:
(Sub-)Test count and outcomes are recorded incrementally during runtime by TESTING/TESTING_2/H5_FAILED/SKIPPED/PASSED, and not displayed until after the subtest has finished in each thread. This is to prevent erroneous reporting of success in the case that it passes in the main thread and fails in another thread.
To pass information between testframe and the MT tests, some of these fields have been duplicated in the TestThreadArgs struct.
All subtests are run in a row during test execution, so correct reporting requires waiting until after the subtests are complete to display the test information, which in turn requires storing the test description until it is to be displayed.
These probably should have been subtests in the first place, but it's now necessary to handle this right since the behavior between the top-level tests and subtest output is different.
Rework INCR_TEST_STAT into INCR_*_COUNT macros that manage the new threadlocal test/subtest information
Factor out the display portions of several test macros to use from testframe
Rename
H5_API_TEST_FILENAME_MAX_LENGTH
->H5_TEST_FILENAME_MAX_LENGTH
since it's no longer exclusive to the API testsSuppress some special output in group/dataset tests on non-main threads
Replace non-atomic handling of potentially atomic num_errs_g
Known issue: If a subtest has any mid-test output, the output will be displayed before the subtest's description
This is a dependency for the following PRs:
MT VL Testing - mattjala#10
Make H5VL Multi-Thread Safe - mattjala#9
Implement virtual locks on IDs - mattjala#7