Improve APBS test cases with intermediate result analysis

[sobolevnrm]

Many of the APBS test cases produce results at the intermediate stages of the calculations that can provide additional information for ensuring agreement between versions (and identifying the source of disagreement when it occurs). In addition to using the final output (e.g., from the PRINT statement), we should include this intermediate output in our test cases' checking behavior.

[keith923]

This is a great idea. I'm not sure how to implement it though. On one hand, if all of the intermediate calculations are not available via .in file commands, then we'll need to add support for those. The bigger issue in my mind is: where do we get the ground truth values to compare against?

These may both be non-issues, and I just don't know it.

[sobolevnrm]

Hi Keith --

There is intermediate output from various ELEC blocks that could be parsed
by the test script.

On Mon, Aug 31, 2015 at 3:21 PM, Keith T. Star [email protected]
wrote:

This is a great idea. I'm not sure how to implement it though. On one
hand, if all of the intermediate calculations are not available via .in
file commands, then we'll need to add support for those. The bigger issue
in my mind is: where do we get the ground truth values to compare against?

These may both be non-issues, and I just don't know it.

—
Reply to this email directly or view it on GitHub
#64 (comment)
.

[keith923]

And for ground truth? Where can I go to find the expected values for each of the intermediate steps, for each of the tests?

[sobolevnrm]

This is primarily a regression test, so referring to results from past
versions of the code is sufficient.

On Tue, Sep 1, 2015 at 6:03 AM, Keith T. Star [email protected]
wrote:

And for ground truth? Where can I go to find the expected values for each
of the intermediate steps, for each of the tests?

—
Reply to this email directly or view it on GitHub
#64 (comment)
.

[kmonson]

Did you mean to assign this to Keith?

On Tue, Dec 15, 2015 at 4:25 PM, Liz [email protected] wrote:

Assigned #64 #64
to @kmonson https://github.com/kmonson.

—
Reply to this email directly or view it on GitHub
#64 (comment)
.

[kozlac]

@sobolevnrm were you thinking on something like writing the intermediate charge or pot from the elec block to files and then compare them to previous apbs versions?

[sobolevnrm]

I think writing the intermediate energies in mg-para and mg-auto
calculations would be a good start.

On Wed, Dec 30, 2015 at 11:51 AM Juan [email protected] wrote:

@sobolevnrm https://github.com/sobolevnrm were you thinking on
something like writing the intermediate charge or pot from the elec block
to files and then compare them to previous apbs versions?

—
Reply to this email directly or view it on GitHub
#64 (comment)
.

[lizutah]

Reading intermediate results and comparing them, but need to modify all the test cases and run against builds. Still in progress (moving to next milestone).

[kozlac]

Finish the python code to extract intermediate energies and added the the expected values to the cfg file as well (34edf403d74cd0acd4061b44d117647cc6acb29d)[64edf40]). Do we want to keep this issue open in case we add more test instances or can we close it?

[lizutah]

we can close it!