-
Notifications
You must be signed in to change notification settings - Fork 263
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Cannot execute tests when DisableAppDomain is true #1598
Comments
Hi @avivanoff, |
@engyebrahim, as I said, I could not create a sample project, even though I tried using the exact same settings of our failing one. How can we proceed? Is there a way to produce a detailed log a can send to you? |
Hi @avivanoff, Could you please follow the steps described here https://github.com/microsoft/vstest/blob/main/docs/troubleshooting.md#visual-studio and share the logs with us? It should produce some logs (probably better to clean the folder then run your tests to have the issue). |
@Evangelink, I got the logs. I did not remove any possibly sensitive information, so where can I send them? |
You can send them to my email amauryleve[at]microsoft[dot]com |
@Evangelink, logs are sent. Let me know if this is enough. |
This might be because of multiple test projects (with different targets / or adapter versions) deploying to same directory. |
@Haplois, we use Central Package Management, so all adapters are the same version. |
@avivanoff - if you provide a small repro, it would be much easier to debug. Even if you're using the same version of the adapters in all your test projects, if you output all your tests in same directory and they are in different TFNs, that might cause this issue. I'm sure after reviewing the logs @Evangelink will know more. |
@Haplois, I did mention in the original description that I was not able to create a sample project with repro this time. I also mentioned that assembly in question is being loaded from the wrong location - test adapter path instead of test host executable path. |
Sorry I read issue around 2am, missed that part. If you send your logs to me I can check the issue, but I'm no longer affiliated with Microsoft. My Twitter DMs are open. (https://twitter.com/medenibaykal) |
@avivanoff are you able to provide your CI setup? How are you running tests(manual vstest.console.exe, VSTest@2 task etc...with some specific runsettings/testadapter path list?)? Are you using Azure DevOps, is so which agent version? |
@MarcoRossignoli, any other information I can provide to help you figure out the issue? |
@avivanoff is the build machine a dedicated one and it's cleaned at any test run or is it a shared machine with something else? I want to understand if the machine configuration you've specified is complete and we can try to repro it but it means that we don't have to have other software installed on the agent. |
The problem exists on both VM-based interactive and Docker-based pipeline agents. Yes, it is VSTest@2 task. But I can also reproduce the issue by manually running the tests. |
@Evangelink, @MarcoRossignoli, any updates? |
Hi @avivanoff sorry for the late answer, we're working on the repro, no new updates so far. |
@MarcoRossignoli, anything? Anything I can do to provide more info? |
@engyebrahim, @Evangelink, @Haplois, @MarcoRossignoli, as more and more of our pipelines start failing, I was finally able to create a simple sample project TestProject.zip.
|
Can you try to cleanup all bin/obj and bump the version to <ItemGroup>
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.5.0" />
<PackageReference Include="MSTest.TestAdapter" Version="3.0.2" />
<PackageReference Include="MSTest.TestFramework" Version="3.0.2" />
</ItemGroup> I've been able to run it in my local
|
That worked. Not sure how I missed these. But the problem still exists on our build servers. Not sure what to do. |
Just to confirm that when running tests locally with the exact same solution/setup the tests are ok but on CI they are failing, right? If that's the case, that would seem to indicate that there is some difference of setup between the local and the CI. There is one thing that is puzzling me though. In your first comment you are saying that Would it be possible for us to have access to your real solution setup? I will ask my manager to see if we have some kind of private support path. |
|
@Evangelink, any updates? |
Hi @avivanoff, Sorry for the delay. We weren't able to reproduce the issue in any way. If you can always reproduce the issue, the easiest would be to have access to the repro (or the source code or dll). You can either send them here, by email to me (amauryleve[at]microsoft[dot]com) or use https://developercommunity.visualstudio.com/home where you can upload privately items. |
@Evangelink, I got the email, but there are no attachments or anything else that points to the preview packages. |
Maybe our antivirus is doing some cleanup. |
@Evangelink, it worked. |
AWESOME! Thank you so much for your patience, I will merge the related PR and we will do the release next week. |
@Evangelink, any ETA? |
@avivanoff, release is now done. Please let us know if there is any other problem! |
@Evangelink, unfortunately, the problem is still there. I am trying to see where my verification went wrong. Did you try 3.0.3 with the sample binaries I sent you? |
@avivanoff, No I haven't tried again as you confirmed the preview package was correct. I am testing now... |
@Evangelink, I attached another set of files to reproduce the issue to https://developercommunity.visualstudio.com/t/Cannot-execute-tests-when-DisableAppDoma/10363055. These are some of my findings:
In both cases the unit test assembly is loaded from TestResults\Deploy_XXX\Out, while in the first case dependencies are loaded from Bin folder, and in the second case they are loaded from TestResults\Deploy_XXX\Out. |
@avivanoff thanks for the extra sample. I have been doing more tests with your binaries and got the same error with all versions of MTest from 2.2.4 to 3.0.2 (I haven't tested previous versions). Could you please let me know if you did some ugprade (if so what was the previously working version?)? Or is it a new test? PS: I confirm your observations with the 2nd test result (except that tests don't work for me because I don't have the database). |
@Evangelink, the tests always worked because they were executing in app domains. It is when I disabled the app domain the tests started to fail. |
Thanks for the confirmation! Looks like this bug has always been present (when AppDomains are disabled). I will do another round of tests to be sure I didn't miss anything using your 2 samples and then will proceed with the new release either today or tomorrow |
@Evangelink, any updates on testing and 3.0.4 ETA? |
cc @engyebrahim who is handling this release. |
Hello @avivanoff, The release is in progress right now and will let you know after it's done. |
@Evangelink, @engyebrahim, so far, all test runs that used to fail are succeeding. Thank you. |
@avivanoff Glad to hear! Thank you for your patience and for using MSTest. |
Describe the bug
An exception occurred while invoking executor 'executor://mstestadapter/v2': Could not load file or assembly
Steps To Reproduce
Steps are the same as in #1493.
The problem happens with both 3.0.2 and 4.0.0-preview-20221222-4.
Unfortunately, this time I cannot create a simple project to re-create the problem, but it is reproducible on our pipelines. One thing I noticed Fusion is showing
Microsoft.TestPlatform.CoreUtilities
is being loaded from test adapter path instead ofvstest.console.exe
location. SettingsDisableAppDomain
to false fixes the problem.The log:
AB#1827543
The text was updated successfully, but these errors were encountered: