Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Passing test run parameters not working and breaks old method for overriding run settings #2377

Closed
TroyWalshProf opened this issue Mar 27, 2020 · 2 comments
Assignees
Milestone

Comments

@TroyWalshProf
Copy link

Description

With the newest release of VSTest I can no long pass in test run parameters.
This hack working a week ago:
dotnet vstest tests.dll --TestCaseFilter:TestCategory=Smoke --Settings:Parallel.RunSettings -- TestRunParameters/Parameter[contains(@name,'Browser')]/@value="Chrome"
So now I get the following error:

The test run parameter argument 'TestRunParameters.Parameter(name=Browser,' is invalid. Please use the format below.
Format: TestRunParameters.Parameter(name="", value="")

Given the error I should be able to do this instead:
dotnet vstest tests.dll --TestCaseFilter:TestCategory=Smoke --Settings:Parallel.RunSettings -- TestRunParameters.Parameter(name="Browser",value="Chrome")
but I still get the same parameter argument error.

Note* This is almost certainly related to @singhsarab changes in #2251

Steps to reproduce

  1. Create a simple test
  2. Run
    dotnet vstest tests.dll -- TestRunParameters.Parameter(name="Browser",value="Chrome")
    or
    "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe" vstest tests.dll -- TestRunParameters.Parameter(name="Browser",value="Chrome")
  3. Note the error VSTest returns

Expected behavior

There would be some way pass in test run parameters

Actual behavior

No matter what I do I get the following error:

The test run parameter argument 'TestRunParameters.Parameter(name=Browser,' is invalid. Please use the format below.
Format: TestRunParameters.Parameter(name="", value="")

Diagnostic logs

See: https://github.com/TroyWalshProf/ActionsSample/runs/539569841?check_suite_focus=true

Environment

  1. GitHub actions using windows-latest build agent
  1. Local Windows 10 using latest VS 2019 Enterprise install
  2. Using VSTest console directly or "dotnet vstest"
@nohwnd
Copy link
Member

nohwnd commented Mar 28, 2020

@TroyWalshProf please see #862 (comment) for some workarounds and explanation. You will need to do some escaping, which will also depend on whether you run the commands from cmd or powershell. Because vstest does not route via msbuild you should be able to escape it like this, and in the same way for vstest.console.

# powershell
dotnet vstest tests.dll -- 'TestRunParameters.Parameter(name=\"Browser\",value=\"Chrome\")'

or

REM cmd
dotnet vstest tests.dll -- TestRunParameters.Parameter(name=\"Browser\",value=\"Chrome\")

We will be working on improving this, but can't promise any date.

@nohwnd
Copy link
Member

nohwnd commented Mar 30, 2020

Tracking in #862

@nohwnd nohwnd closed this as completed Mar 30, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants