Record test start/end events for data driven tests #631
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
singhsarab
reviewed
Sep 23, 2019
singhsarab
reviewed
Sep 23, 2019
src/Adapter/MSTest.CoreAdapter/Execution/TestExecutionManager.cs
Outdated
Show resolved
Hide resolved
singhsarab
reviewed
Sep 23, 2019
src/Adapter/MSTest.CoreAdapter/Execution/TestExecutionManager.cs
Outdated
Show resolved
Hide resolved
singhsarab
reviewed
Sep 23, 2019
singhsarab
reviewed
Sep 23, 2019
src/Adapter/MSTest.CoreAdapter/Execution/TestExecutionRecorderWrapper.cs
Show resolved
Hide resolved
NGloreous
commented
Sep 26, 2019
NGloreous
commented
Sep 26, 2019
NGloreous
commented
Sep 26, 2019
NGloreous
commented
Sep 26, 2019
NGloreous
commented
Nov 19, 2019
| CollectionAssert.AreEqual(expectedTestCaseStartList, this.frameworkHandle.TestCaseStartList); | ||
| CollectionAssert.AreEqual(expectedTestCaseEndList, this.frameworkHandle.TestCaseEndList); | ||
| CollectionAssert.AreEqual(expectedResultList, this.frameworkHandle.ResultsList); | ||
| CollectionAssert.Contains(this.frameworkHandle.TestCaseStartList, "PassingTest"); |
There was a problem hiding this comment.
@jayaranigarg I added back the start/end events in TestExecutionManager. The side effect is that we now have duplicate start/end events for non-data driven tests. It's a bit odd but shouldn't effect execution because vstest ignores duplicate events. I don't think there is much I can do here because TestMethodInfo doesn't really know if the test is being executed more than once.
jayaranigarg
approved these changes
Nov 29, 2019
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a fix for #340 so that data collectors can get start/end events for individual execution of data driven tests. Also solves to allow the custom test execution extension to run a test multiple times and get multiple start/end events.
Solving for #464 might be a better approach but seems like a much bigger change.
One side effect of this change is ClassInitialize happens before any start event gets fired. IMO this is fine, it always seemed weird that this execution would be lumped into the first test that happened to execute. This also make it so ClassInitialize is now identical to ClassCleanup which doesn't get captured.