sf project deploy validate does not tell why validation failed

[MarcDBehr]

Summary

When running a "sf project deploy validate" command, if the validation fails, it does not say why it failed (if the reason was Code Coverage Failure at least)

Steps To Reproduce:

Attempt to run "sf project deploy validate" in a scenario where there are missing apex tests.

Expected result

Some type of error message stating why the deployment failed should be provided

Actual result

*** Validating Deployment with SOAP API v57.0 ***
Deploy ID: 0AfDR00002bckRW0AY
Status: Failed | ████████████████████████████████████████ | 93/93 Components

Test Results Summary
Passing: 16
Failing: 0
Total: 16
Time: 10807
Error (1): Failed to validate the deployment (0AfDR00002bckRW0AY).

or with the --json flag:

{
  "code": 1,
  "context": "DeployMetadataValidate",
  "commandName": "DeployMetadataValidate",
  "message": "Failed to validate the deployment (0AfDR00002bckVY0AY).",
  "name": "FailedValidationError",
  "status": 1,
  "stack": "FailedValidationError: Failed to validate the deployment (0AfDR00002bckVY0AY).\n    at Messages.createError (/usr/local/lib/node_modules/@salesforce/cli/node_modules/@salesforce/core/lib/messages.js:393:16)\n    at DeployMetadataValidate.run (/usr/local/lib/node_modules/@salesforce/cli/node_modules/@salesforce/plugin-deploy-retrieve/lib/commands/project/deploy/validate.js:59:28)\n    at processTicksAndRejections (node:internal/process/task_queues:96:5)\n    at async DeployMetadataValidate._run (/usr/local/lib/node_modules/@salesforce/cli/node_modules/@oclif/core/lib/command.js:117:22)\n    at async Config.runCommand (/usr/local/lib/node_modules/@salesforce/cli/node_modules/@oclif/core/lib/config/config.js:329:25)\n    at async run (/usr/local/lib/node_modules/@salesforce/cli/node_modules/@oclif/core/lib/main.js:89:16)",
  "exitCode": 1,
  "warnings": []
}

Looking in the syetm and the Deployment Status page shows that there was insufficient code coverage in the org

System Information

• Which shell/terminal are you using? (e.g. bash, zsh, powershell 5, powershell 7, cmd.exe, etc.)
  bash

• If you are using sfdx

  • Run sfdx version --verbose --json

• If you are using sf

  • Run sf version --verbose --json

$ sf version --verbose --json
{
  "cliVersion": "@salesforce/cli/1.82.0",
  "architecture": "darwin-x64",
  "nodeVersion": "node-v16.13.2",
  "osVersion": "Darwin 22.5.0",
  "shell": "bash",
  "rootPath": "/usr/local/lib/node_modules/@salesforce/cli",
  "pluginVersions": [
    "@oclif/plugin-autocomplete 2.3.0 (core)",
    "@oclif/plugin-commands 2.2.15 (core)",
    "@oclif/plugin-help 5.2.9 (core)",
    "@oclif/plugin-not-found 2.3.24 (core)",
    "@oclif/plugin-plugins 3.1.2 (core)",
    "@oclif/plugin-search 0.0.17 (core)",
    "@oclif/plugin-update 3.1.16 (core)",
    "@oclif/plugin-version 1.3.4 (core)",
    "@oclif/plugin-warn-if-update-available 2.0.37 (core)",
    "@oclif/plugin-which 2.2.21 (core)",
    "@salesforce/cli 1.82.0 (core)",
    "apex 2.2.22 (core)",
    "auth 2.7.17 (core)",
    "data 2.3.20 (core)",
    "deploy-retrieve 1.11.0 (core)",
    "info 2.6.16 (core)",
    "limits 2.3.17 (core)",
    "login 1.2.11 (core)",
    "org 2.9.3 (core)",
    "packaging 1.16.11 (user)",
    "schema 2.3.10 (core)",
    "settings 1.4.10 (core)",
    "sobject 0.1.22 (core)",
    "source 2.10.12 (core)",
    "telemetry 2.2.0 (core)",
    "templates 55.4.18 (core)",
    "trust 2.4.20 (core)",
    "user 2.3.14 (core)"
  ]
}```


### Additional information

_Feel free to attach a screenshot_.

[github-actions]

Thank you for filing this issue. We appreciate your feedback and will review the issue as soon as possible. Remember, however, that GitHub isn't a mechanism for receiving support under any agreement or SLA. If you require immediate assistance, contact Salesforce Customer Support.

[git2gus]

This issue has been linked to a new work item: W-13546168

[cristiand391]

Hi @MarcDBehr

I'm able to repro the issue, thanks!

if you pass in the --verbose flag to project deploy validate you will see an additional table showing the code coverage results:

Apex Code Coverage
 Name                 % Covered Uncovered Lines
 ──────────────────── ───────── ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
 FileUtilities        0%        3,8,9,10,11,12,13,15,18,22,23,24,25,26,28,29,30
 GeocodingService     0%        5,8,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,28,29,30,31,32,33,34,35,36,38,39,40,41,42,44,48,50
 PagedResult          0%        3,6,9,12
 PropertyController   0%        2,3,16,25,26,27,28,29,30,31,32,33,35,36,38,39,40,41,45,46,47,48,49,50,52,67,68,69,70,71,72,75,76,78,87,88,92,97,98,101,103,104,107,110
 SampleDataController 0%        3,4,5,6,7,9,10,11,14,15,20,21,22,23,25,28,29,34,35,36,37,39,40,43,44,49,50,51,52,54,57,58,59,60

Test Results Summary
Passing: 0
Failing: 0
Total: 0
Error (1): Failed to validate the deployment (0Af0300000z5TcWCAU).

I think we could add a msg that suggest people to enable the verbose mode when there's a code coverage warning in the response payload.

Also confirmed deploy details aren't included in the json output even with --verbose.

[MarcDBehr]

Hi @cristiand391,

Not sure if the --verbose would have helped me. My results showed that all 16 tests actually passed, but the coverage was coming up at 65%. I guess if I wanted to try to do the math based on the results to see that I was below 75%, it might have helped, but I think that it would be better if the 'insufficient code coverage' message were returned rather than a vague "Failed to validate the deployment" message.

[cristiand391]

@MarcDBehr

I guess if I wanted to try to do the math based on the results to see that I was below 75%, it might have helped, but I think that it would be better if the 'insufficient code coverage' message were returned rather than a vague "Failed to validate the deployment" message.

Agree. I looked at the response and it also includes that warning about code coverage < 75% so the command can print that too 👍🏼