Maven@3 does not fail task on test failures

[mercer]

Question, Bug, or Feature?
Type: Bug

Enter Task Name: Maven@3

Environment

Azure Pipelines:
- account name: censored

• Agent - Hosted or Private:
  • private: ubuntu 18.04

Issue Description

1. junit test fails
2. but task does not fail

- task: Maven@3
  inputs:
    mavenPomFile: '$(system.defaultWorkingDirectory)/E2e/pom.xml'
    javaHomeOption: 'JDKVersion'
    jdkArchitectureOption: 'x64'
    goals: 'clean verify'
    options: '-Dtest=".*ApiTest" -Dsurefire.rerunFailingTestsCount=3'
    publishJUnitResults: true
    testResultsFiles: '$(system.defaultWorkingDirectory)/E2e/target/surefire-reports/TEST-*.xml'
  displayName: 'Execute All API Integration Tests'
  timeoutInMinutes: 10

Build is green after this.

Task logs

N/A did not collect

[vladislav-ryzhov]

Hi @mercer !
Could you please enable system diagnostic by adding System.Debug: trueto variables in your YAML file (screenshot #1) and send the pipeline logs? (screenshot #2)
Could you tell us which version of the azure devops server you are using? On prem or cloud?
If there is some sensitive information and you don't want to share it publicly, you can send it to me by email [email protected].
Please feel free to ask any questions about the process of getting logs about the pipeline.

[mercer]

I'm afraid I don't have that setup at hand anymore.

Please try to reproduce with the data I provided.

1. create a fresh java project + pipeline with a task configured as in the ticket (or similarly)
2. have a test that passes and a test that fails
3. execute the pipeline, test results are published
4. however, the pipeline is green

Expected behavior is that the pipeline fails when the test is red. Notice that PublishTestResults@2 manages to fail the pipeline just fine.

[vladislav-ryzhov]

Hello @mercer
We have a similar ticket #14514, and it looks like you have the same problems.
We've already fixed this issue, and I think this will fix your issue as well.
The changes will be rolled out in a few weeks.
Let me close this ticket. Please feel free to ask any further questions.