Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix plus tests #1886

Merged
merged 5 commits into from
Jun 16, 2016
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion Tasks/VsTest/Strings/resources.resjson/en-US/resources.resjson
Original file line number Diff line number Diff line change
Expand Up @@ -32,5 +32,14 @@
"loc.input.help.configuration": "Configuration against which the tests should be reported. If you have defined a variable for configuration in your build task, use that here.",
"loc.input.label.publishRunAttachments": "Upload Test Attachments",
"loc.input.help.publishRunAttachments": "Opt in/out of publishing test run level attachments.",
"loc.messages.VstestReturnCode": "Vstest exited with return code: %d"
"loc.messages.VstestReturnCode": "Vstest exited with return code: %d",
"loc.messages.NoMatchingTestAssemblies": "No test assemblies found matching the pattern: %s",
"loc.messages.VstestNotFound": "Vstest of version %d not found.",
"loc.messages.VstestFailed": "Vstest failed with error. Check logs for failures. There might be failed tests",
"loc.messages.NoResultsToPublish": "No results found to publish.",
"loc.messages.ErrorWhileReadingRunSettings": "Error occured while reading run settings file. Error : %s",
"loc.messages.RunInParallelNotSupported": "Run in Parallel is not supported with testsettings file.",
"loc.messages.FailedToSetRunInParallel": "Failed to set run in parallel. Invalid run settings file.",
"loc.messages.UpdateOneOrHigherRequired": "Install Visual Studio 2015 Update 1 or higher on your build agent machine to run the tests in parallel.",
"loc.messages.ErrorOccuredWhileSettingRegistry": "Error occured while setting registry key, Error: %s"
}
213 changes: 111 additions & 102 deletions Tasks/VsTest/task.json
Original file line number Diff line number Diff line change
@@ -1,106 +1,106 @@
{
"id": "EF087383-EE5E-42C7-9A53-AB56C98420F9",
"name": "VSTest",
"friendlyName": "Visual Studio Test",
"description": "Run tests with Visual Studio test runner",
"helpMarkDown": "[More Information](https://go.microsoft.com/fwlink/?LinkId=624539)",
"category": "Test",
"visibility": [
"Build",
"Release"
],
"author": "Microsoft Corporation",
"version": {
"Major": 1,
"Minor": 0,
"Patch": 41
},
"demands": [
"vstest"
],
"minimumAgentVersion": "1.89.0",
"groups": [
{
"name": "executionOptions",
"displayName": "Execution Options",
"isExpanded": true
},
{
"name": "advancedExecutionOptions",
"displayName": "Advanced Execution Options",
"isExpanded": false
},
{
"name": "reportingOptions",
"displayName": "Reporting Options",
"isExpanded": false
}
],
"inputs": [
{
"name": "testAssembly",
"type": "string",
"label": "Test Assembly",
"defaultValue": "**\\*test*.dll;-:**\\obj\\**",
"required": true,
"helpMarkDown": "Test binaries to run tests on. Wildcards can be used. For example, `**\\*test*.dll;-:**\\obj\\**` for all dlls with test in name while excluding files in any sub-directory named obj.",
"groupName": "executionOptions"
},
{
"name": "testFiltercriteria",
"type": "string",
"label": "Test Filter criteria",
"defaultValue": "",
"required": false,
"helpMarkDown": "Additional criteria to filter tests from Test assemblies. For example: `Priority=1|Name=MyTestMethod`",
"groupName": "executionOptions"
},
{
"name": "runSettingsFile",
"type": "filePath",
"label": "Run Settings File",
"defaultValue": "",
"required": false,
"helpMarkDown": "Path to runsettings file to use with the tests. Use `$(Build.SourcesDirectory)` to access the Project folder.",
"groupName": "executionOptions"
},
{
"name": "overrideTestrunParameters",
"type": "string",
"label": "Override TestRun Parameters",
"defaultValue": "",
"required": false,
"helpMarkDown": "Override parameters defined in the TestRunParameters section of runsettings file. For example: `AppURL=$(DeployURL);Port=8080`",
"groupName": "executionOptions"
},
{
"name": "codeCoverageEnabled",
"type": "boolean",
"label": "Code Coverage Enabled",
"defaultValue": "False",
"required": false,
"helpMarkDown": "Collect code coverage information from the Test run.",
"groupName": "executionOptions"
},
{
"name": "runInParallel",
"type": "boolean",
"label": "Run In Parallel",
"defaultValue": "false",
"required": false,
"helpMarkDown": "Enable parallel execution of your tests.",
"groupName": "executionOptions"
"id": "EF087383-EE5E-42C7-9A53-AB56C98420F9",
"name": "VSTest",
"friendlyName": "Visual Studio Test",
"description": "Run tests with Visual Studio test runner",
"helpMarkDown": "[More Information](https://go.microsoft.com/fwlink/?LinkId=624539)",
"category": "Test",
"visibility": [
"Build",
"Release"
],
"author": "Microsoft Corporation",
"version": {
"Major": 1,
"Minor": 0,
"Patch": 41
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

version?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will change

},
{
"name": "vsTestVersion",
"type": "pickList",
"label": "VSTest version",
"defaultValue": "14.0",
"required": false,
"helpMarkDown": "The version of VSTest to use.",
"groupName":"advancedExecutionOptions",
"options": {
"latest": "Latest",
"demands": [
"vstest"
],
"minimumAgentVersion": "1.89.0",
"groups": [
{
"name": "executionOptions",
"displayName": "Execution Options",
"isExpanded": true
},
{
"name": "advancedExecutionOptions",
"displayName": "Advanced Execution Options",
"isExpanded": false
},
{
"name": "reportingOptions",
"displayName": "Reporting Options",
"isExpanded": false
}
],
"inputs": [
{
"name": "testAssembly",
"type": "string",
"label": "Test Assembly",
"defaultValue": "**\\*test*.dll;-:**\\obj\\**",
"required": true,
"helpMarkDown": "Test binaries to run tests on. Wildcards can be used. For example, `**\\*test*.dll;-:**\\obj\\**` for all dlls with test in name while excluding files in any sub-directory named obj.",
"groupName": "executionOptions"
},
{
"name": "testFiltercriteria",
"type": "string",
"label": "Test Filter criteria",
"defaultValue": "",
"required": false,
"helpMarkDown": "Additional criteria to filter tests from Test assemblies. For example: `Priority=1|Name=MyTestMethod`",
"groupName": "executionOptions"
},
{
"name": "runSettingsFile",
"type": "filePath",
"label": "Run Settings File",
"defaultValue": "",
"required": false,
"helpMarkDown": "Path to runsettings file to use with the tests. Use `$(Build.SourcesDirectory)` to access the Project folder.",
"groupName": "executionOptions"
},
{
"name": "overrideTestrunParameters",
"type": "string",
"label": "Override TestRun Parameters",
"defaultValue": "",
"required": false,
"helpMarkDown": "Override parameters defined in the TestRunParameters section of runsettings file. For example: `AppURL=$(DeployURL);Port=8080`",
"groupName": "executionOptions"
},
{
"name": "codeCoverageEnabled",
"type": "boolean",
"label": "Code Coverage Enabled",
"defaultValue": "False",
"required": false,
"helpMarkDown": "Collect code coverage information from the Test run.",
"groupName": "executionOptions"
},
{
"name": "runInParallel",
"type": "boolean",
"label": "Run In Parallel",
"defaultValue": "false",
"required": false,
"helpMarkDown": "Enable parallel execution of your tests.",
"groupName": "executionOptions"
},
{
"name": "vsTestVersion",
"type": "pickList",
"label": "VSTest version",
"defaultValue": "14.0",
"required": false,
"helpMarkDown": "The version of VSTest to use.",
"groupName": "advancedExecutionOptions",
"options": {
"latest": "Latest",
"14.0": "Visual Studio 2015",
"12.0": "Visual Studio 2013"
}
Expand Down Expand Up @@ -175,6 +175,15 @@
}
},
"messages": {
"VstestReturnCode": "Vstest exited with return code: %d"
"VstestReturnCode": "Vstest exited with return code: %d",
"NoMatchingTestAssemblies": "No test assemblies found matching the pattern: %s",
"VstestNotFound": "Vstest of version %d not found.",
"VstestFailed": "Vstest failed with error. Check logs for failures. There might be failed tests",
"NoResultsToPublish": "No results found to publish.",
"ErrorWhileReadingRunSettings": "Error occured while reading run settings file. Error : %s",
"RunInParallelNotSupported": "Run in Parallel is not supported with testsettings file.",
"FailedToSetRunInParallel": "Failed to set run in parallel. Invalid run settings file.",
"UpdateOneOrHigherRequired": "Install Visual Studio 2015 Update 1 or higher on your build agent machine to run the tests in parallel.",
"ErrorOccuredWhileSettingRegistry": "Error occured while setting registry key, Error: %s"
}
}
11 changes: 10 additions & 1 deletion Tasks/VsTest/task.loc.json
Original file line number Diff line number Diff line change
Expand Up @@ -175,6 +175,15 @@
}
},
"messages": {
"VstestReturnCode": "ms-resource:loc.messages.VstestReturnCode"
"VstestReturnCode": "ms-resource:loc.messages.VstestReturnCode",
"NoMatchingTestAssemblies": "ms-resource:loc.messages.NoMatchingTestAssemblies",
"VstestNotFound": "ms-resource:loc.messages.VstestNotFound",
"VstestFailed": "ms-resource:loc.messages.VstestFailed",
"NoResultsToPublish": "ms-resource:loc.messages.NoResultsToPublish",
"ErrorWhileReadingRunSettings": "ms-resource:loc.messages.ErrorWhileReadingRunSettings",
"RunInParallelNotSupported": "ms-resource:loc.messages.RunInParallelNotSupported",
"FailedToSetRunInParallel": "ms-resource:loc.messages.FailedToSetRunInParallel",
"UpdateOneOrHigherRequired": "ms-resource:loc.messages.UpdateOneOrHigherRequired",
"ErrorOccuredWhileSettingRegistry": "ms-resource:loc.messages.ErrorOccuredWhileSettingRegistry"
}
}
43 changes: 27 additions & 16 deletions Tasks/VsTest/vstest.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,8 @@ var fs = require('fs');
var xml2js = require('xml2js');

try {
tl.setResourcePath(path.join( __dirname, 'task.json'));
tl.setResourcePath(path.join(__dirname, 'task.json'));

var vsTestVersion: string = tl.getInput('vsTestVersion');
var testAssembly: string = tl.getInput('testAssembly', true);
var testFiltercriteria: string = tl.getInput('testFiltercriteria');
Expand Down Expand Up @@ -53,7 +53,7 @@ try {
}
else {
tl._writeLine("##vso[task.logissue type=warning;code=002004;]");
tl.warning("No test assemblies found matching the pattern: " + testAssembly);
tl.warning(tl.loc('NoMatchingTestAssemblies', testAssembly));
}
}
catch (error) {
Expand Down Expand Up @@ -108,8 +108,13 @@ function invokeVSTest(testResultsDirectory: string): Q.Promise<number> {
setRunInParallellIfApplicable(vsVersion);
setupRunSettingsFileForParallel(runInParallel, overriddenSettingsFile)
.then(function(parallelRunSettingsFile) {
var vsCommon = tl.getVariable("VS" + vsVersion + "0COMNTools");
var vstestLocation = path.join(vsCommon, "..\\IDE\\CommonExtensions\\Microsoft\\TestWindow\\vstest.console.exe");
var vstestPath = tl.getVariable("VSTest_" + vsVersion.toFixed(1));
if (!vstestPath) {
tl.warning(tl.loc('VstestNotFound', vsVersion));
defer.resolve(1);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why 1

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 means task fails

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

then the warning shuld be made error?


In reply to: 67283921 [](ancestors = 67283921)

return defer.promise;
}
var vstestLocation = path.join(vstestPath, "vstest.console.exe");
var vstest = tl.createToolRunner(vstestLocation);

testAssemblyFiles.forEach(function(testAssembly) {
Expand Down Expand Up @@ -164,7 +169,7 @@ function invokeVSTest(testResultsDirectory: string): Q.Promise<number> {
})
.fail(function(err) {
cleanUp(parallelRunSettingsFile);
tl.warning("Vstest failed with error. Check logs for failures. There might be failed tests");
tl.warning(tl.loc('VstestFailed'));
tl.error(err);
defer.resolve(1);
});
Expand Down Expand Up @@ -197,7 +202,7 @@ function publishTestResults(testResultsDirectory: string) {
}
else {
tl._writeLine("##vso[task.logissue type=warning;code=002003;]");
tl.warning("No results found to publish.");
tl.warning(tl.loc('NoResultsToPublish'));
}
}
}
Expand All @@ -214,7 +219,12 @@ function getFilteredFiles(filesFilter: string, allFiles: string[]): string[] {
function cleanUp(temporarySettingsFile: string) {
//cleanup the runsettings file
if (temporarySettingsFile && runSettingsFile != temporarySettingsFile) {
tl.rmRF(temporarySettingsFile, true);
try {
tl.rmRF(temporarySettingsFile, true);
}
catch (error) {
//ignore. just cleanup.
}
}
}

Expand Down Expand Up @@ -245,7 +255,7 @@ function overrideTestRunParametersIfRequired(settingsFile: string): Q.Promise<st
var parser = new xml2js.Parser();
parser.parseString(xmlContents, function(err, result) {
if (err) {
tl.warning("Error occured while reading run settings file. Error : " + err);
tl.warning(tl.loc('ErrorWhileReadingRunSettings', err));
tl.debug("Error occured while overriding test run parameters. Continuing...");
defer.resolve(settingsFile);
return defer.promise;
Expand Down Expand Up @@ -337,7 +347,7 @@ function setupRunSettingsFileForParallel(runInParallel: boolean, settingsFile: s
var defer = Q.defer<string>();
if (runInParallel) {
if (settingsFile && settingsFile.split('.').pop().toLowerCase() == "testsettings") {
tl.warning("Run in Parallel is not supported with testsettings file.");
tl.warning(tl.loc('RunInParallelNotSupported'));
defer.resolve(settingsFile);
return defer.promise;
}
Expand All @@ -363,14 +373,14 @@ function setupRunSettingsFileForParallel(runInParallel: boolean, settingsFile: s
var parser = new xml2js.Parser();
parser.parseString(xmlContents, function(err, result) {
if (err) {
tl.warning("Error occured while reading run settings file. Error : " + err);
tl.warning(tl.loc('ErrorWhileReadingRunSettings', err));
tl.debug("Error occured while setting run in parallel. Continuing...");
defer.resolve(settingsFile);
return defer.promise;
}

if (result.RunSettings === undefined) {
tl.warning("Failed to set run in parallel. Invalid run settings file.");
tl.warning(tl.loc('FailedToSetRunInParallel'));
defer.resolve(settingsFile);
return defer.promise;
}
Expand Down Expand Up @@ -432,16 +442,17 @@ function setRunInParallellIfApplicable(vsVersion: number) {
if (runInParallel) {
if (!isNaN(vsVersion) && vsVersion >= 14) {
var vs14Common = tl.getVariable("VS140COMNTools");
if ((vs14Common && pathExistsAsFile(path.join(vs14Common, "..\\IDE\\CommonExtensions\\Microsoft\\TestWindow\\TE.TestModes.dll"))) || vsVersion > 14) {
if (vsVersion > 14 || (vs14Common && pathExistsAsFile(path.join(vs14Common, "..\\IDE\\CommonExtensions\\Microsoft\\TestWindow\\TE.TestModes.dll")))) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is temodes.dll?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this dll determines if it is below update 1

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and we r sure it wont be removed going forward? we cant guarantee that correct?


In reply to: 67283885 [](ancestors = 67283885)

setRegistryKeyForParallelExecution(vsVersion);
return;
}
}
resetRunInParallel();
}
}

function resetRunInParallel() {
tl.warning("Install Visual Studio 2015 Update 1 or higher on your build agent machine to run the tests in parallel.");
tl.warning(tl.loc('UpdateOneOrHigherRequired'));
runInParallel = false;
}

Expand Down Expand Up @@ -497,11 +508,11 @@ function setRegistryKeyForParallelExecution(vsVersion: number) {
};
regedit.putValue(values, function(err) {
if (err) {
tl.warning("Error occured while setting registry key, Error: " + err);
tl.warning(tl.loc('ErrorOccuredWhileSettingRegistry', err));
}
});
} else {
tl.warning("Error occured while creating registry key, Error: " + err);
tl.warning(tl.loc('ErrorOccuredWhileSettingRegistry', err));
}
});
}
Expand Down
Loading