Implement and test script-style use of the VTS

[dcuccia]

Discussed in #70

Main Idea

In order to lay the groundwork for future Jupyter Notebook and .NET Polyglot Notebook sample code, as envisioned in #70, this ticket is to create sample C# "scripts" (top-level statements) that mimic the existing VTS Matlab scripts. In addition to unblocking the future work, at the end of this, anyone more comfortable working out of the VS or VS Code IDE should be able to use these scripts directly for their work if they desire.

Details

Design details:

1. Create these in a new project within the main Vts solution, with project name & namespace Vts.Scripting. This will ensure building the solution catches any build-level regressions to the scripts that might occur when the VTS library changes.
2. Use C# top-level statements combined with SmallSharp for simplifying running each script independently.
3. Make the code as "distraction free" as possible, removing as much unhelpful boilerplate code as possible, as these will be mirrored in other toolchains and languages.
4. Scripts will directly reference the Vts.csproj project within the Vts solution, to facilitate full use of the tools for design-time compilation error checking and debugging. This will differ from the future "external" Notebooks, which will resolve Vts.dll from Nuget.
5. Add a Vts.Scripting.Test project to cover the new script samples to catch/flag any runtime issues/bugs that will necessarily need updates to the libraries in the other repos

Acceptance Criteria

• Project should run "out of the box" with either VS or VS Code, requiring only a recently released version of .NET installed
• Should not require "classic" .NET Framework installed to run
• Should not concern the user with unnecessary details (e.g. throw logging errors, etc).
• Test project should successfully execute each script without errors

[lmalenfant]

To follow naming convention of the other projects, the test application should be named Vts.Scripting.Test and it needs to be created now alongside this project. It is much easier to write unit tests at the same time as the code being tested.

As per our SonarCloud analysis, at least 80% code coverage is what we require for new code.

[dcuccia]

Sounds good, thanks @lmalenfant !

[dcuccia]

@hayakawa16 I wonder if I could enlist your help. I just flushed out a pMC scripting demo (corresponding to Demo 6 in the Matlab Monte Carlo demos), and for some reason, all three OPs appear to be post-processed to the same result. This is the file:

https://github.com/VirtualPhotonics/VTS/blob/72-net-polyglot-notebook-and-jupyter-notebook-documentation-and-samples/src/Vts.Scripting/MonteCarlo-06-pMCPostProcessor.cs

Would you mind doing a sanity check for me, and maybe comparing with Matlab? (I don't have Matlab anymore...too expensive). You can check it out under branch #72, and be sure to check that when you open MonteCarlo-06-pMCPostProcessor in Visual Studio, the built-in nuget tool SmallSharp changes the "launchSettings" to specify that file (a work-around to have multiple top-level programs in the same .csproj). If not, you can manually change it in the drop down. It should look like this, and then you can click the solid "play" button (on the left) to run that script:

[hayakawa16]

Hi dcuccia, First, I ran Example 6 in vts_mc_demo.m in the MATLAB interop download. This is the plot I get:

First question: are you plotting on a log scale because with only 1000 photons being run, the changes might be hard to see on regular scale. Maybe this solves your question. If not, I will try to checkout your branch and see what happens.

[dcuccia]

Thanks Carole. I see - I didn't actually check to see if they were identical. Will do that and also run more photons and report back. (Yes, though, that was a log plot).

[hayakawa16]

Let me know how it goes. I can try to pull your branch later this week.

[dcuccia]

Following the "number 6" matlab example I simulated mua = 0.01, musp = 1.0 for the baseline, and then updated the perturbation numbers to mua = 0.5/mm and 2.0/mm for the "half" and "double" cases (just to be extreme), and I still see fully overlapping plots:

Maybe if you have time @hayakawa16 you could run this as well to spot any issues with the logic? Probably a bug in how I've set up the pMC, but I don't "see" it (probably too close!).

[hayakawa16]

Hi, I checked out your branch and opened it visual studio. I set Vts.Scripting to the startup project. Then clicked on MonteCarlo-060pMCPostProcessor.cs. I see this title in the run window like your image. I click the green right arrow but get an exception: The database file could not be found. I think I'm doing something wrong.

[hayakawa16]

One thing I see just visually. On line 41, the database set is pMCDiffuseTransmittance. In Example 6, the database is set to pMCDiffuseReflectance. I see that the slab is 100mm thick and this may be the problem.

[dcuccia]

Ah, I'm sorry - I did fix that bug locally but hadn't pushed yet. These are created with DiffuseReflectance as the virtual boundary.

[hayakawa16]

I can change on my side. Ah, now I'm getting a plot! Okay I can try to debug.

[hayakawa16]

I figured it out. Do you want me to push my changes or would you rather I describe here what I did.

[dcuccia]

Yes, please go ahead!

[hayakawa16]

Committed and pushed.

[hayakawa16]

Just as an aside, I made the fix but didn't realized I didn't have your commit on Aug 4, so I pulled and then pushed my fix. Now in VS I can't seem to get MonteCarlo-06-pMCPostProcesor to show in run window. So I wasn't able to run it after the pull.

[dcuccia]

@hayakawa16 thanks - I see that I had the wrong type of photon database (maybe we should have better compile-time constraints). Anyway, that all worked out great, and on to the next thing...

No rush on this, but when you can, please check out and run my example 07 and let me know what you think. I realized that we didn't have a pMCForwardSolver, so I made one (for the demo only, at this point). I still need to work on the plot formatting, but the Lev-Marq optimizer seems to do it's job....

[hayakawa16]

That result look good. I see though that you are including all rho bins and remember that the final rho bin gets everything reflected beyond that bin added to it (so that total diffuse reflectance can be checked and validated). So when I run inversions, I define my rho bins so that I can exclude the final bin in the fitting. Surprisingly your result seems to accomodate that! But in general best not to include final bin.

[hayakawa16]

I just ran the vts_mc_demo.m Ex. 7 and it appears we run with that last rho bin included too!

[dcuccia]

Excellent, thank you. And good point - thanks for the reminder on that last bin. Perhaps we should update all to remove the last bin.

In the meantime, I've added Example 08, and noticed that the "unit test" criteria are strangely, barely not passing. The issue is the middle (complex) number for ROfRhoAndOmega.

[dcuccia]

Here's where it stands:

[dcuccia]

Update: I've added MC09, so that's all of the MC demos with the two caveats above (final bin issue/alignment in MC07, and accuracy check in MC08). Some plot formatting would be nice too.

[hayakawa16]

I see problem. The var omegaRange needs to start at 0.05 (not 0..5).

[hayakawa16]

I pushed fix.

[dcuccia]

Oops! Thank you!

[dcuccia]

Got started on forward solvers:

[dcuccia]

Heat map plots working:

[hayakawa16]

Nice! I see you're using Plotly. Just curious, what are advantages of this package over other python visualization libraries?

[dcuccia]

Plotly is pretty common in Python science and engineering work, which is why I thought it would be a good common denominator between the C# examples and Python ones (via pythonnet). It's built on JavaScript and is the backbone of tools like Dash, so it's got a lot of support/community (to the level that there's now Plotly.net that wraps it for F# and C#). Given, I don't have a ton of experience with it yet, so it will take a little digging to get the plot formatting just right.

FWIW, per the above plot in FS03 I took some liberty in the C# to plot the heat map of internal fluence(rho, z) at 1000nm. The corresponding example in Matlab instead plots fluence(rho=0, z) vs wavelength. Thoughts on which is a better "slice" to show at the end of the computation?

[hayakawa16]

Hard to know what would be best slice. I think having comments about order of indices (column major) so that the user can dissect themselves is always good. I noticed from the code, that the plot above is for the last wavelength, however the plot doesn't list that wavelength. If it was shown then user would know which is the fixed variable and that the data used for the plot actually has that many independent variables. I'm not sure how flexible the plotting is but if "Phi(rho,z,820nm)" could be shown or something like that.

[dcuccia]

Thanks Carole. I went ahead and adapted based on your suggestions:

(as you can see, I also now have correctly cartesian depth/height, but have still to figure out how to get rid of the "gray" space)

[hayakawa16]

That looks great! I like the "square" aspect ratio too.

[dcuccia]

Thanks @hayakawa16! Two next questions for you:

1. I'm looking at Examples 12-16 in vts_solver_demo.m and feeling like there's a bit of redundancy with previous examples. Do these correspond to particular course material and/or were they based on user questions (don't remember having so many R(fx) samples before)
2. are the Matlab scrips in short_course_monte_carlo.m still used (e.g. part of the course educational material)? Definitely worth porting if they are, but if the modern material is somewhere else, maybe better to focus my energy elsewhere. Corollary: are there other separate scripts used for the course material that might be good candidates for adding (and ultimately putting into formatted C#/Python notebooks)?

[hayakawa16]

Hi, to answer your questions:

1. the examples added at the end of vts_solver_demo.m were typically written to help a user asking for help with a particular combination of inputs.
2. yes, short_course_monte_carlo_lab.m is still used (was just used in this years short course). For the course, we used the WPF GUI, Matlab Interop (short_course_monte_carlo_lab.m) and MCCL (infile_one_layer_FluenceOfRhoAndZ_RadianceOfRhoAndZAndAngle.txt). The new instrumentation labs added this year were written in Matlab.

[dcuccia]

Thanks!

the examples added at the end of vts_solver_demo.m were typically written to help a user asking for help with a particular combination of inputs.

Ok, thinking ahead, maybe it would be worth it to curate/order the samples to teach the use and breadth of the Vts, and separate those from one-off demonstrations (which are super valuable, but don't need to be "supported" in the sense that they don't need to be continually tested, refactored, and synchronized with the matching notebooks). I think that's the beauty of the notebooks, which can be shared (and searched) in Q&A Discussion threads here in Github.

yes, short_course_monte_carlo_lab.m is still used (was just used in this years short course). For the course, we used the WPF GUI, Matlab Interop (short_course_monte_carlo_lab.m) and MCCL (infile_one_layer_FluenceOfRhoAndZ_RadianceOfRhoAndZAndAngle.txt). The new instrumentation labs added this year were written in Matlab.

Awesome - I'll translate short_course_monte_carlo_lab.m for the scripts!

[lmalenfant]

@dcuccia I was thinking about the next project with the Python scripts and I think we should do it the same way as this one in case you discover any more bugs in the VTS code. We can create the Vts.Scripting.Python (I'm open to other naming suggestions) project in the VTS repository and then once we are ready with the code we can either merge it to master and then move it out later or create the next VTS release then and move it out into a separate Repo referencing the VTS NuGet package.

[dcuccia]

@lmalenfant I like the idea, and would hope we can still commit to getting them segregated to their own projects in the relatively near-term, because as you suggested earlier, the audience for the different languages could vary significantly. Practical question - do you want me to pull the PR and continue work here, or can we merge this one and create a continuation ticket (my preference)?

Re: naming/organization, I suggest we re-organize the root 'Matlab' folder into a sub-folder called examples. Then, we can have all sorts of demos of use of the framework under that umbrella, including Scripting/Matlab, Scripting/Python, Scripting/C# (Polyglot Notebooks), maybe even Scripting/JavaScript. :)

[lmalenfant]

Yes, as soon as it is complete (or as complete as it needs to be to be useful) we will move it to its own repo. We can release to NuGet any time going forward. The next phase should have its own issue and branch, let’s get your first PR reviewed separately. Nothing is preventing you from either branching from master or your branch (if you need code from there) and starting work on the next piece, while we review your PR. It might take me a few days. Message ID: ***@***.***>

[dcuccia]

Sounds good, thanks!

[dcuccia]

I made some notes on the PR itself, but just to follow up to the above, this is now ready for review.