Test-VSTeamYamlPipeline fails if no pipeline runs on master/main exist

[hvl71]

Hi

We encounter an issue when trying to validate a yaml pipeline using the Test-VSTeamYamlPipeline cmdlet.

We run:

Test-VSTeamYamlPipeline -PipelineId 1315 -ProjectName "MyProject" -FilePath C:\src\sandbox\azure-pipelines.yml

Result:

WARNING: Unable to resolve the reference 'refs/heads/main' to a specific version. Verify the reference exists in the
source repository.
Test-VSTeamYamlPipeline : Unable to resolve the reference 'refs/heads/main' to a specific version. Verify the reference
 exists in the source repository.
At line:1 char:1
+ Test-VSTeamYamlPipeline -PipelineId 1315 -ProjectName "MyProject" -File ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Write-Error], WriteErrorException
    + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,Test-VSTeamYamlPipeline

We see the behaviour on both versions we have tried (7.2.0 and 7.4.0)

We have observed that our PipelineId only has pipeline runs on our development branch. We have never run the pipeline on our master branch (we don't have a main branch).

If we validate the yaml agains another PipelineId which has had a pipeline run against the relevant master branch, the yaml validates fine.

Steps to reproduce
Validate a known valid yaml pipeline against a PipelineId that have no pipeline runs on master or main in Azure Devops Service

Expected behavior
Yaml validates without errors or fails with more clear error message about reason

Actual behavior:

WARNING: Unable to resolve the reference 'refs/heads/main' to a specific version. Verify the reference exists in the
source repository.
Test-VSTeamYamlPipeline : Unable to resolve the reference 'refs/heads/main' to a specific version. Verify the reference
 exists in the source repository.
At line:1 char:1
+ Test-VSTeamYamlPipeline -PipelineId 1315 -ProjectName "MyProject" -File ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Write-Error], WriteErrorException
    + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,Test-VSTeamYamlPipeline

Environment data: Windows 10
Server: Azure DevOps Service

Get-VSTeamAPIVersion:

Billing                     : 5.1-preview.1
Build                       : 5.1
Core                        : 5.1
DistributedTask             : 6.0-preview
DistributedTaskReleased     : 5.1
ExtensionsManagement        : 6.0-preview
Git                         : 5.1
Graph                       : 6.0-preview
HierarchyQuery              : 5.1-preview
MemberEntitlementManagement : 6.0-preview
Packaging                   : 6.0-preview
Pipelines                   : 5.1-preview
Policy                      : 5.1
Processes                   : 6.0-preview
Release                     : 5.1
ServiceEndpoints            : 5.0-preview
TaskGroups                  : 6.0-preview
Tfvc                        : 5.1
VariableGroups              : 5.1-preview.1
Version                     : VSTS

$PSVersionTable

Name                           Value
----                           -----
PSVersion                      5.1.19041.1237
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.19041.1237
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1

[SebastianSchuetze]

Thanks for the report @hvl71 it checked it and it is per se not a bug with the cmdlet, but implemented incomplete in regards to the API.

Also documenting it for myself here. So, I had the following setup:

• repo with a main and a dev branch
• an azure pipeline exists pointing to a yaml file in the dev branch
• yaml file does not exist on main
• I run the cmdlet

Following error I get:

PS F:\repos\general_github\vsteam> Set-VSTeamDefaultProject -Project 'YAML Tests'

PS F:\repos\general_github\vsteam> Test-VSTeamYamlPipeline -PipelineId 41        

WARNING: An error occurred while loading the YAML build pipeline. File /azure-pipelines.yml not found in repository https://dev.azure.com/razorspoint-test/YAML%20Tests/_git/YAML%20Tests branch refs/heads/main version 96e7436b59b550a411dc5f6c7dc42f7a5f8416bb.

Warning is different but comes down to the same problem.

When calling the following API endpoint:
https://docs.microsoft.com/en-us/rest/api/azure/devops/pipelines/preview/preview?view=azure-devops-rest-6.1

You can give the pipeline Id. If you don't specify anything else, it tries to do the test against the branch which is named "main" of the repo. Additionally, I tried to switch the default branch to dev and then run it. It still didn't work.

• Then I also deleted the main branch and then I got your message.
• The cmdlet is not setting any default branch here.
• The AzDO API is in the backend setting it to an expected default branch, which is in your and my case main.
• It's independent if this branch exists. That also means that the API is not even checking what the default branch in the repo is

How to resolve it?

1. It is a bug in the API in your case (if you only have a dev branch and that is the default one) so I will report it so they may be able to fix it (e.g. take the current default branch if nothing is given)
2. Will add a Parameter Branch to the cmdlet which creates a resources JSON object in the body like described in the API. Checked it even how the GUI does it when trying to do a validation:

and it sends an object like the following which is mostly empty and always seems to have only one resource in it:

{
   "resources": {
      "pipelines": {},
      "repositories": {
         "self": {
            "refName": "dev"
         }
      },
      "builds": {},
      "containers": {},
      "packages": {}
   },
   "templateParameters": {},
   "previewRun": true,
   "yamlOverride": "YAML Code here"
}

[SebastianSchuetze]

I reported that problem to the support:
https://developercommunity.visualstudio.com/t/api-to-preview-pipeline-run-takes-non-existing-bra/1635377