dotnet test error MSB4181: The "Microsoft.TestPlatform.Build.Tasks.VSTestTask" task returned false but did not log an error

[loop-evgeny]

When running NUnit tests using dotnet test I sometimes get the following error at the end of the test run:

C:\Program Files\dotnet\sdk\3.1.401\Microsoft.TestPlatform.targets(32,5): error MSB4181: The "Microsoft.TestPlatform.Build.Tasks.VSTestTask" task returned false but did not log an error. [C:\W ork\Play\TestFailureTest\TestFailureTest.csproj]

This seems to always happen when at least one test failed. I have occasionally seen the same error when all tests pass on our TeamCity build servers, but cannot reproduce that. Output with a failing test, with the attached trivial test project:

C:\W ork\Play\TestFailureTest>dotnet test
Test run for C:\W ork\Play\TestFailureTest\bin\Debug\netcoreapp3.1\TestFailureTest.dll(.NETCoreApp,Version=v3.1)
Microsoft (R) Test Execution Command Line Tool Version 16.7.0
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...

A total of 1 test files matched the specified pattern.
  X FailingTest [39ms]
  Error Message:
   This is a test failure message
  Stack Trace:
     at TestFailureTest.Tests.FailingTest() in C:\W ork\Play\TestFailureTest\Tests.cs:line 11


Test Run Failed.
Total tests: 1
     Failed: 1
 Total time: 0.9542 Seconds
C:\Program Files\dotnet\sdk\3.1.401\Microsoft.TestPlatform.targets(32,5): error MSB4181: The "Microsoft.TestPlatform.Build.Tasks.VSTestTask" task returned false but did not log an error. [C:\W ork\Play\TestFailureTest\TestFailureTest.csproj]

Happened both on Windows 7 and on a Linux (Mint 19) VM.

dotnet --info on Windows:

.NET Core SDK (reflecting any global.json):
 Version:   3.1.401
 Commit:    5b6f5e5005

Runtime Environment:
 OS Name:     Windows
 OS Version:  6.1.7601
 OS Platform: Windows
 RID:         win7-x64
 Base Path:   C:\Program Files\dotnet\sdk\3.1.401\

Host (useful for support):
  Version: 3.1.7
  Commit:  fcfdef8d6b

.NET Core SDKs installed:
  3.0.100 [C:\Program Files\dotnet\sdk]
  3.1.107 [C:\Program Files\dotnet\sdk]
  3.1.400 [C:\Program Files\dotnet\sdk]
  3.1.401 [C:\Program Files\dotnet\sdk]

.NET Core runtimes installed:
  Microsoft.AspNetCore.All 2.1.20 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.20 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.7 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.0.9 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.20 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.21 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.7 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 3.1.7 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

and on Linux:

.NET Core SDK (reflecting any global.json):
 Version:   3.1.401
 Commit:    39d17847db

Runtime Environment:
 OS Name:     linuxmint
 OS Version:  19
 OS Platform: Linux
 RID:         linuxmint.19-x64
 Base Path:   /usr/share/dotnet/sdk/3.1.401/

Host (useful for support):
  Version: 3.1.7
  Commit:  fcfdef8d6b

.NET Core SDKs installed:
  2.1.809 [/usr/share/dotnet/sdk]
  3.1.401 [/usr/share/dotnet/sdk]

.NET Core runtimes installed:
  Microsoft.AspNetCore.All 2.1.21 [/usr/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.21 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.7 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.1.21 [/usr/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.7 [/usr/share/dotnet/shared/Microsoft.NETCore.App]

TestFailureTest.zip

[NikolayPianikov]

@loop-evgeny could you please add args -d my.log and attach this file?

FYI: from a TeamCity build configuration you could publish it as build an artifact like my.log => .

[loop-evgeny]

Sure, here is the log I get from the attached trivial test project failing on Windows: diag-failure.log

I also have logs from a TeamCity build on Linux where this error occurs even though all tests passed, but don't want to attach those publicly. Happy to send them directly to a .NET developer if that would help.

Running .NET SDK 3.1.402 now on both the Windows machine and the Linux TeamCity build agent.

[danreg]

any updates on this? We have the same issue

[joeloff]

MSBuild logs this error when ever a task's Execute method returns false, but no error has been logged. There are other instances where this can occur as well, e.g. if a task is interrupted/cancelled (dotnet/msbuild#5508).

I did come across a fix in vstest for 16.8 (microsoft/vstest#2384 discusses the issue at length), PR https://github.com/microsoft/vstest/pull/2581/files (this is the issue @loop-evgeny previously mentioned.

From the initial post it seemed 16.7 was being used.

[milliorn]

C:\Program Files\dotnet\sdk\3.1.402\Microsoft.TestPlatform.targets(32,5): error MSB4181: The "Microsoft.TestPlatform.Build.Tasks.VSTestTask" task returned false but did not log an error.

I too am now getting this error with nunit.

It will error on Five()

using NUnit.Framework;

namespace Decipher
{
    public class Test
    {
        [Test]
        public void One()
        {
            Assert.That(Program.decipher("10197115121") == "easy");
        }

        [Test]
        public void Two()
        {
            Assert.That(Program.decipher("98") == "b");
        }

        [Test]
        public void Three()
        {
            Assert.That(Program.decipher("122") == "z");
        }

        [Test]
        public void Four()
        {
            Assert.That(Program.decipher("1229897") == "zba");
        }

        [Test]
        public void Five()
        {
            Assert.That(Program.decipher("97989910010110210310410510610710810911011111211311411511611711811912012112297") == "abcdefghijklmnopqrstuvwxyza");
        }

        [Test]
        public void Six()
        {
            Assert.That(Program.decipher("10297115106108102108971061151041029897107106115981001029710711510010298") == "fasjlflajshfbakjsbdfaksdfb");
        }

        [Test]
        public void Seven()
        {
        }
    }
}

[danreg]

MSBuild logs this error when ever a task's Execute method returns false, but no error has been logged. There are other instances where this can occur as well, e.g. if a task is interrupted/cancelled (dotnet/msbuild#5508).

I did come across a fix in vstest for 16.8 (microsoft/vstest#2384 discusses the issue at length), PR https://github.com/microsoft/vstest/pull/2581/files (this is the issue @loop-evgeny previously mentioned.

From the initial post it seemed 16.7 was being used.

Do we have to wait for the next Core 3 Release to get this fix?

[kichalla]

We have been seeing similar error in our build pipelines. We are using 3.1.402 SDK.

[joeloff]

@nohwnd if the fix in vstest you merged went into the 16.8 branch, I assume that means it's only available in 16.8 and not necessarily tied to a specific SDK build? We plan on updating 16.8 with a new 3.1.4xx SDK later this month. Or would this be a fix that supports all the SDKs in 16.8 (3.1 and 5.0)?

[oca159]

How can i fix the issue?, i can't wait until the next release this month. My builds are failing in CodeBuild and i don't know why, I am seeing the same message. Here is the dotnet info:

Running command dotnet --info

.NET Core SDK (reflecting any global.json):
Version: 3.1.401
Commit: 39d17847db
 
Runtime Environment:
OS Name: ubuntu
OS Version: 18.04
OS Platform: Linux
RID: ubuntu.18.04-x64
Base Path: /root/dotnet/sdk/3.1.401/
 
Host (useful for support):
Version: 3.1.7
Commit: fcfdef8d6b

.NET Core SDKs installed:
3.1.401 [/root/dotnet/sdk]
 
.NET Core runtimes installed:
Microsoft.AspNetCore.App 3.1.7 [/root/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.NETCore.App 3.1.7 [/root/dotnet/shared/Microsoft.NETCore.App]

[nohwnd]

@rainersigwald before trying to push fix for this through net3.1 servicing, do you see any workaround for this? I could not find any that would allow me to suppress the error externally.

[BrandesEric]

I'm looking for a workaround as well, is there an easy fix for this?

[nohwnd]

There is not, or at least not any that I know of. The MSBxxxx errors cannot be suppressed by any parameters to MSBuild, so the only way to do this externally would be to build our version of the task and then use the extension point in the task to replace the DLL with another. Which is a hacky solution at best.

You could also grab the net5.0 RC2 SDK that has this fixed and use that to build and test your project, while still targetting netcoreapp3.1 in your csproj, but that is not great either.

[AlbertoBN]

Everything is red in our CI thanks to this problem. And you cannot expect every company to take risks on RC versions of the next SDK. Any updates on this?

[nohwnd]

Well it writes that error only when there are failed (or cancelled) tests doesn't it? Or how does this make all your builds fail?

[NikolayPianikov]

@nohwnd msbuild returns 1 instead 0 when all tests are passed

[AlbertoBN]

Tests tend to fail you know. This is how I know someone messed up. The fact that this step fails when tests fails hinders proper reporting afterwards. The CI does not report failed tests but failed builds.

[danreg]

@nohwnd Our Team City Build is failing. We have some failing muted tests, but they are executed anyway and now our build is failing all the time...
We can't wait until .net5 is released...

[NikolayPianikov]

@danreg try changing the property one of build steps exited with an error (e.g non-zero exit code in command line runner) in Failure Conditions for your build configuration with test steps.

[danreg]

@NikolayPianikov I can't do that. The build should fail, when tests are failing. But it must not fail, when muted tests are failing.

[NikolayPianikov]

@danreg Sorry I am not sure I understand. You could do the following:

• Set the property one of build steps exited with an error (e.g non-zero exit code in command line runner) in Failure Conditions to false
• Mute a build problem like ... VSTestTask task returned false but did not log an error

That is all. After that you could mute or unmute your tests and everything should work as expected.

[totollygeek]

I am getting the same error, trying to run tests inside dotnet 3.1 sdk latest Docker container. I also have some additional exception, which I am not sure if it is related or helpful, but here it is:

testbox    | Unhandled exception. System.Xml.XmlException: The ' ' character, hexadecimal value 0x20, cannot be included in a name.
testbox    |    at System.Xml.XmlDocument.CheckName(String name)
testbox    |    at System.Xml.XmlElement..ctor(XmlName name, Boolean empty, XmlDocument doc)
testbox    |    at System.Xml.XmlDocument.CreateElement(String prefix, String localName, String namespaceURI)
testbox    |    at System.Xml.XmlDocument.CreateElement(String name)
testbox    |    at Microsoft.VisualStudio.TestPlatform.CommandLine.Processors.Utilities.LoggerUtilities.AddLoggerToRunSettings(String loggerIdentifier, Dictionary`2 loggerParameters, IRunSettingsProvider runSettingsManager)
testbox    |    at Microsoft.VisualStudio.TestPlatform.CommandLine.Processors.EnableLoggerArgumentExecutor.Initialize(String argument)
testbox    |    at Microsoft.VisualStudio.TestPlatform.CommandLine.Processors.ArgumentProcessorFactory.<>c__DisplayClass20_0.<WrapLazyProcessorToInitializeOnInstantiation>b__0()
testbox    |    at System.Lazy`1.PublicationOnlyViaFactory(LazyHelper initializer)
testbox    |    at System.Lazy`1.CreateValue()
testbox    |    at System.Lazy`1.get_Value()
testbox    |    at Microsoft.VisualStudio.TestPlatform.CommandLine.Executor.GetArgumentProcessors(String[] args, List`1& processors)
testbox    |    at Microsoft.VisualStudio.TestPlatform.CommandLine.Executor.Execute(String[] args)
testbox    |    at Microsoft.VisualStudio.TestPlatform.CommandLine.Program.Main(String[] args)
testbox    | /usr/share/dotnet/sdk/3.1.402/Microsoft.TestPlatform.targets(32,5): error MSB4181: The "Microsoft.TestPlatform.Build.Tasks.VSTestTask" task returned false but did not log an error. [**REDACTED**.csproj]

[totollygeek]

Okay, I have figured my problem out. Apparently it just highlights that errors inside dotnet test will report MSB4181.
My problem was the logger I was passing to dotnet test.
It looked like this: --logger "trx; LogFileName=output.trx"
This character between trx; and LogFileName..., which I guess turned up there due to some wrong copy-paste I did, was causing that issue. Maybe a different problem. Might be worth handling that situation a bit better, but I guess this has to be put inside different issue?

[ErikSchierboom]

As a workaround, I've installed the 3.1.302 SDK and added the following global.json to my project:

{
  "sdk": {
    "version": "3.1.302",
    "rollForward": "disable"
  }
}

[mwpowellhtx]

@ErikSchierboom Great observation, 👓 thank you. If we want to ensure that our MSBuild custom task, i.e. tooling, runs properly, then we can deploy that with a runtimeconfig.json instead of chasing the issue from deployment to deployment?

[mwpowellhtx]

@totollygeek From what I've been able to determine, the issue is not contained to just dotnet test, as I have encountered similar MSB4181 behavior for my BumpAssemblyVersions tooling. It does seem to be "trying" to run the tooling, but the integration, passing of elements to the tooling, if you will, from MSBuild to the tooling is failing, for whatever reason, outside the scope of our tooling itself. Apart from the workaround above, that is, potentially.

[jvandertil]

As a workaround, I've installed the 3.1.302 SDK and added the following global.json to my project:

{
  "sdk": {
    "version": "3.1.302",
    "rollForward": "disable"
  }
}

@ErikSchierboom Wouldn't this downgrade the .NET Core version to 3.1.6? There have been 2 security updates published since then. Looks like SDK 3.1.109 might be a better choice

[ErikSchierboom]

@jvandertil Aha, fair point.

[marinovdh]

In my case I could fix it by adding NuGet package Microsoft.NET.Test.Sdk to my solution's test project...

[nohwnd]

@marinovdh that sounds odd, as the task responsible for the message is in a totally different part than the package you installed. But maybe I am overlooking something. Do you have a repro?

[evilguest]

I have the same diag stably reproduced with XUnit when:

1. A Debug.Assert() fails within a test
2. A native code throws an exception and host crashes.
   I believe, in both cases XUnit must catch the issue and properly log a failed test. At least in case Templates: Deterministic builds should be on by default #1, because otherwise it renders the whole test project useless: it stops running more tests as soon as it encounters a first assertion failure.

[nohwnd]

@evilguest you see that message because that is just a different way to fail the VSTest msbuild task.

But you can work around your particular problem, if you install a recent version of Microsoft.NET.Test.SDK and if your project is .NET Core. Because in .NET Core we translate Debug.Assert to an exception so it should not fail your whole project run. You will still see the MSB message till it is fixed, but your whole run won't fail.

[evilguest]

Thanks, will try that. At the moment I have no issues, since the bug causing the assertion failure has been fixed. Will consider this change upon the next occurence. As for the second issue - it is still actual, since it doesn't even indicate the test that fails. I had to triangulate the offender until located the specific call (the issue was the call convention mismatch on Unix)

[danreg]

Just want to share that after .Net5 migration everything works again.

[mviswan]

I had to pull in below nuget packages to get around this issue.

<PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.4.0" />
<PackageReference Include="xunit.runner.visualstudio" Version="2.4.1" />