Debugging guide

[jnm2]

Please provide a single step-by-step guide to hitting your first breakpoint in the VS adapter.
I'm stuck and the root of the repo is the first place I look for such things. Debugging.md or Getting Started.md or something similar?

I've tried:

• Uninstalling my existing NUnit VS extension out of the non-experimental instance
• Installing the debug extension to the non-experimental instance
• Defining LAUNCHDEBUGGER
• Switching to the Debug configuration
• Attaching the debugger additionally to vstest.executionengine.x86.exe, multiple vstest.discoveryengine.x86.exe, and vstest.console.exe
• Building the VSIX project (which doesn't happen automatically because it's not the startup project)

And I still can't get any NUnit code to be loaded. Breakpoints are disabled and the Modules window has no NUnit assemblies. I can't figure out a way to install the debug VSIX to the experimental instance, but I would expect that to happen automatically when I start debugging? It doesn't show up in Extensions; is it supposed to?

(This is VS2017.)

I can't imagine it's as complex as all this? What am I missing?

[jnm2]

I finally got it to work, though it ate more time than I'm prepared to admit.

Steps seem to be:

• Switch build configuration to Debug
• Set the startup project to NUnit3TestAdapterInstall so that the VSIX ends up getting installed in the experimental VS instance when you run- probably especially important if you switch build configurations
• Remove this section from NUnit3TestAdapter.csproj and put it in NUnit3TestAdapterInstall.csproj:

  <PropertyGroup>
    <!-- Common debugging support -->
    <StartAction>Program</StartAction>
    <StartProgram>$(DevEnvDir)\devenv.exe</StartProgram>
    <StartArguments>/rootsuffix Exp</StartArguments>
  </PropertyGroup>

Once that's done (and perhaps this should be done in the repo), each time you want to start debugging:

0. Set a breakpoint
1. Start debugging
2. In the experimental VS instance that comes up, open a project
3. In the debugging VS instance, attach to the vstest.discoveryengine.x86.exe process if your breakpoint is in test discovery code. There will be two such processes, the older one from your debugging VS instance, the newer from the experimental VS instance. There's no quick way to tell them apart so attach to both.
4. Cause test discovery to happen in the experimental VS instance by running all tests.
5. If your breakpoint is not hit, VS is likely to have ended the vstest.discoveryengine.x86.exe process and started a new one. Go to step 3.

Hopefully someone can refine this process because it's a bit hit-or-miss for me.

[CharliePoole]

Unfortunately, our assumption up to now has been that only folks with experience in debugging extensions would be doing this. It sounds as if it's new to you and - again unfortunately - there's a lot of smoke and mirrors involved.

The copying of entries from one project to another makes no sense to me. In the past, I had the install project set up to start the experimental instance but somebody changed it to the adapter project. So long as you set the correct project as the startup project, everything works. You can do this in the properties dialog, without editing any csproj files.

The steps you are following are very close to how I did it when I first started working on the extension but is a bit too complicated for my taste now. With Terje's help, here's what I now do:

1. Build in debug - of course.
2. Uncomment the launch debugger define in either the discovery or the execution class, depending on which stage I need to debug.
3. Run without debug, so that the experimental instance is started
4. Open a project in the experimental instance, generally NUnit3TestDemo, which is designed for the purpose. Run all tests if I'm debugging execution.
5. When the debugger is launched in the initial VS instance, make sure my breakpoints are set and continue to debug.

I hope this helps. What are you working on? I'm trying to get the adapter release out, so this may not be the best time to start working in it unless your work is just held over till the next quarter.

[CharliePoole]

@OsirisTerje Where do you think a documentation page like this belongs? My first reaction is that it belongs in the documentation wiki rather than in a file in the root of the project. IMO such files should just give the barest outline and shouldn't have to teach people how to work on extensions. A docs page could go into more depth and would serve all our extensions. The README in each project could link to the right page. What do you think?

[jnm2]

Unfortunately, our assumption up to now has been that only folks with experience in debugging extensions would be doing this. It sounds as if it's new to you and - again unfortunately - there's a lot of smoke and mirrors involved.

I have debugged extensions in the past. Debugging test adapters is a bit more specialized than normal extensions, you'll have to admit- for one thing, it's split across processes which sometimes recycle.

The copying of entries from one project to another makes no sense to me. In the past, I had the install project set up to start the experimental instance but somebody changed it to the adapter project. So long as you set the correct project as the startup project, everything works. You can do this in the properties dialog, without editing any csproj files.

This is part of what I was saying- that's not actually the case. You have to build the NUnit3TestAdapterInstall project to register the extension in the experimental instance, and that does not happen if you just go with the NUnit3TestAdapter project. Also, even if you do a one-time build to get that out of the way, it'll bite you again if you switch build configurations and the VSIX path changes.

Uncomment the launch debugger define in either the discovery or the execution class, depending on which stage I need to debug.

That's exactly the sort of process that would be good to document. I didn't find that.
Also, if I follow your steps, it tries to launch the debugger in a new instance of Visual Studio 2015 and there is no 2017 option. I'll have to figure that out next, but a tip would be nice.

Also: can I leave my NUnit VS adapter installed and still have it replaced by the debug VSIX in the experimental VS instance? It's nice to read the answer rather than taking 5-10 minutes to test it and hope you aren't making some other mistake which invalidates your findings.

I hope this helps. What are you working on? I'm trying to get the adapter release out, so this may not be the best time to start working in it unless your work is just held over till the next quarter.

Just debugging enough to diagnose #296

[OsirisTerje]

@CharliePoole Imho the documentation itself should be in the wiki, but we could add a readme file with links to the appropriate sections in the wiki, to be more helpful.
Debuggings as you do is the way I do it, except that I dont even bother with experimental instance. With 2017 however, the install process is worse, since it requires all instances to be closed down, so I have stopped debugging with the vsix, and just uses the nuget package itself. When my test project hits the breakpoint, I just attach to the VS that has the source code for the adapter. No extra steps should be needed. Has worked fine for me. But, I will document this up.

@jnm2 About your questions, I'll try to answer them

VSIX running: Uninstall all existing vsix adapters, they are just going to be a nuisance. The way VS searches for the adapters is just weird.
Selecting 2015/2017: Yeah, noticed this too. It responds to the 2017 preview, but not the RC. Not sure why yet, but assumed it had to do with 2017 being RC. I'll ping the product group and ask. For now, just have the source open in 2015 and debug there. There are no 2017 features being used yet that excludes the 2015 (except for the vsix, but you dont need that to work to debug the code)

Also, make sure you have an easy way to kill the vstest processes. I keep a short cmd on my desktop, with the appropriate taskkill command:

[OsirisTerje]

@jnm2 Which exact version number do you have on your VS 2017 ? (See Help/About)

[jnm2]

@OsirisTerje 15.0.26020.0

[CharliePoole]

@jnm2 Looks like Terje has filled in a lot of stuff. Just a few comments...

• You're right... the adapter is a bit strange to work with compared to regular extensions.
• I always rebuild the solution before debugging so I don't have the problem you described. I should have listed that as a step, but it's just a habit I have with practically every project.
• I build in VS2017 and run the experimental instance using VS2015. The debugger opens in VS2017 of course. This is left over from when we built the adapter with VS2015 - only a few months ago. We can change it to start under VS2017 as soon as it is able to work.
• Don't have any vsix version of the adapter installed if you are working on the adapter. Use nuget if you need it in production projects on the same machine. I don't have this issue myself, because I don't use the adapter.

@OsirisTerje I'll set up a page in the right place to get us started.

[OsirisTerje]

There is now a section in the docs, pointing to this post with complete steps.