Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Visual Studio test reports as source #10009

Closed
3 tasks done
denniebouman opened this issue Oct 11, 2024 · 0 comments · Fixed by #10286
Closed
3 tasks done

Support Visual Studio test reports as source #10009

denniebouman opened this issue Oct 11, 2024 · 0 comments · Fixed by #10286
Assignees
Labels
Source(s) New, enhanced, or removed metric source

Comments

@denniebouman
Copy link
Member

denniebouman commented Oct 11, 2024

Describe the solution you'd like
Add support for Visual Studio test reports (.trx).
The Visual Studio test result file (or TRX) is the default format for publishing test results.
https://learn.microsoft.com/en-us/dotnet/core/testing/unit-testing-platform-extensions-test-reports#visual-studio-test-reports

Additional context
Similar to the JUnit XML report, support the following metrics:

  • Source up-to-dateness
    • from TestRun / Times
  • Test cases
    • from Results / UnitTestResult
    • combine with TestDefinitions / UnitTest / TestCategory (for Jira ticket number(s))
    • since the testresult is present in UnitTestResult, correlate with each other based on the executionId and Execution id
  • Tests
    • from Results / UnitTestResult

Add parameter for "test result".

Example:

<?xml version="1.0" encoding="utf-8"?>
<TestRun id="d9a53ba2-3fe2-49a9-93fa-68fd6c421e20" name="abcdef@BLA-24 2024-09-12 11:33:30" runUser="BLA\abcdef" xmlns="http://microsoft.com/schemas/VisualStudio/TeamTest/2010">
  <Times creation="2024-09-12T11:33:30.3272909+02:00" queuing="2024-09-12T11:33:30.3272909+02:00" start="2024-09-12T11:33:27.0973122+02:00" finish="2024-09-12T11:33:30.8892628+02:00" />
  <TestSettings name="default" id="01faf8da-6375-4cfb-9ceb-53350be32406">
    <Deployment runDeploymentRoot="abcdef_XYZ-BLA-24_2024-09-12_11_33_30" />
  </TestSettings>
  <Results>
    <UnitTestResult executionId="268fd488-12c7-4107-80d8-5df1200cd637" testId="63eb0c90-d1fc-a21e-6fc0-3974b0cc65db" testName="BestaandeZaakOpenen2" computerName="XYZ-BLA-24" duration="00:00:01.4635437" startTime="2024-09-12T11:33:29.3938415+02:00" endTime="2024-09-12T11:33:30.8644329+02:00" testType="13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b" outcome="Passed" testListId="8c84fa94-04c1-424b-9868-57a2d4851a1d" relativeResultsDirectory="268fd488-12c7-4107-80d8-5df1200cd637">
      <Output>
        <StdOut>
        </StdOut>
      </Output>
    </UnitTestResult>
    <UnitTestResult executionId="daf369f6-7c54-482d-a12c-68357679bd78" testId="446a0829-8d87-1082-ab45-b2ab9f846325" testName="BestaandeZaakOpenen" computerName="XYZ-BLA-24" duration="00:00:01.7342127" startTime="2024-09-12T11:33:27.3275629+02:00" endTime="2024-09-12T11:33:29.3909874+02:00" testType="13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b" outcome="Passed" testListId="8c84fa94-04c1-424b-9868-57a2d4851a1d" relativeResultsDirectory="daf369f6-7c54-482d-a12c-68357679bd78">
      <Output>
        <StdOut></StdOut>
      </Output>
    </UnitTestResult>
  </Results>
  <TestDefinitions>
    <UnitTest name="BestaandeZaakOpenen" storage="c:\XYZ\frontendtests.dll" id="446a0829-8d87-1082-ab45-b2ab9f846325">
      <TestCategory>
        <TestCategoryItem TestCategory="FeatureTag" />
        <TestCategoryItem TestCategory="ScenarioTag1" />
        <TestCategoryItem TestCategory="JIRA-225" />
      </TestCategory>
      <Execution id="daf369f6-7c54-482d-a12c-68357679bd78" />
      <TestMethod codeBase="C:\XYZ\FrontendTests.dll" adapterTypeName="executor://mstestadapter/v2" className="ClassName" name="BestaandeZaakOpenen" />
    </UnitTest>
    <UnitTest name="BestaandeZaakOpenen2" storage="c:\XYZ\frontendtests.dll" id="63eb0c90-d1fc-a21e-6fc0-3974b0cc65db">
      <TestCategory>
        <TestCategoryItem TestCategory="FeatureTag" />
        <TestCategoryItem TestCategory="ScenarioTag2" />
        <TestCategoryItem TestCategory="JIRA-225" />
      </TestCategory>
      <Execution id="268fd488-12c7-4107-80d8-5df1200cd637" />
      <TestMethod codeBase="C:\XYZ\FrontendTests.dll" adapterTypeName="executor://mstestadapter/v2" className="ClassName" name="BestaandeZaakOpenen2" />
    </UnitTest>
  </TestDefinitions>
  <TestEntries>
    <TestEntry testId="63eb0c90-d1fc-a21e-6fc0-3974b0cc65db" executionId="268fd488-12c7-4107-80d8-5df1200cd637" testListId="8c84fa94-04c1-424b-9868-57a2d4851a1d" />
    <TestEntry testId="446a0829-8d87-1082-ab45-b2ab9f846325" executionId="daf369f6-7c54-482d-a12c-68357679bd78" testListId="8c84fa94-04c1-424b-9868-57a2d4851a1d" />
  </TestEntries>
  <TestLists>
    <TestList name="Results Not in a List" id="8c84fa94-04c1-424b-9868-57a2d4851a1d" />
    <TestList name="All Loaded Results" id="19431567-8539-422a-85d7-44ee4e166bda" />
  </TestLists>
  <ResultSummary outcome="Completed">
    <Counters total="2" executed="2" passed="2" failed="0" error="0" timeout="0" aborted="0" inconclusive="0" passedButRunAborted="0" notRunnable="0" notExecuted="0" disconnected="0" warning="0" completed="0" inProgress="0" pending="0" />
    <Output>
      <StdOut>Test Parallelization enabled for C:\XYZ\FrontendTests.dll (Workers: 24, Scope: ClassLevel)&#xD;
</StdOut>
    </Output>
  </ResultSummary>
</TestRun>
@denniebouman denniebouman added the Feature New, enhanced, or removed feature label Oct 11, 2024
@fniessink fniessink moved this from Inbox to To be refined in Quality-time backlog Oct 11, 2024
@fniessink fniessink added Source(s) New, enhanced, or removed metric source and removed Feature New, enhanced, or removed feature labels Nov 14, 2024
@fniessink fniessink changed the title Add support for Visual Studio test reports (.trx) Support Visual Studio test reports as source Nov 14, 2024
@fniessink fniessink moved this from Refinement in progress to Ready in Quality-time backlog Nov 14, 2024
@fniessink fniessink self-assigned this Nov 14, 2024
@fniessink fniessink moved this from Ready to Development in progress in Quality-time backlog Nov 14, 2024
fniessink added a commit that referenced this issue Nov 14, 2024
Allow for using Visual Studio test reports (.trx) as source for the metrics 'tests', 'test cases', and 'source up-to-dateness'.

Closes [#10009]
@fniessink fniessink linked a pull request Nov 14, 2024 that will close this issue
fniessink added a commit that referenced this issue Nov 14, 2024
Allow for using Visual Studio test reports (.trx) as source for the metrics 'tests', 'test cases', and 'source up-to-dateness'.

Closes [#10009]
fniessink added a commit that referenced this issue Nov 15, 2024
Allow for using Visual Studio test reports (.trx) as source for the metrics 'tests', 'test cases', and 'source up-to-dateness'.

Closes [#10009]
fniessink added a commit that referenced this issue Nov 15, 2024
Allow for using Visual Studio test reports (.trx) as source for the metrics 'tests', 'test cases', and 'source up-to-dateness'.

Closes [#10009]
fniessink added a commit that referenced this issue Nov 15, 2024
Allow for using Visual Studio test reports (.trx) as source for the metrics 'tests', 'test cases', and 'source up-to-dateness'.

Closes [#10009]
fniessink added a commit that referenced this issue Nov 15, 2024
Allow for using Visual Studio test reports (.trx) as source for the metrics 'tests', 'test cases', and 'source up-to-dateness'.

Closes [#10009]
fniessink added a commit that referenced this issue Nov 15, 2024
Allow for using Visual Studio test reports (.trx) as source for the metrics 'tests', 'test cases', and 'source up-to-dateness'.

Closes [#10009]
@github-project-automation github-project-automation bot moved this from Reviewing in progress to Merged in Quality-time backlog Nov 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Source(s) New, enhanced, or removed metric source
Projects
Status: Release candidate released
Development

Successfully merging a pull request may close this issue.

2 participants