PyNEST test result parsing confused by NEST output

[heplesser]

The code parsing the output generated by nosetests to provide a progress indication during testing is confused by output generated by NEST, since the parser relies on lines of the form TestABC ... ok. If NEST input intervenes before ok (or any other result), no progress will be displayed, confusing the user (see also #677).

We thus should revise output parsing, but that should first happen once we have decided how to test in the future #761.

[gtrensch]

@heplesser After some investigation I found that the NEST stderr output interferes with the nosetests messages. nosetests comes with several plugins, e.g to capture stdout. The capture-plugin is on per default. For stderr a similar plugin (nose-capturestderr) is available but needs to be installed manually: https://pypi.python.org/pypi/nose-capturestderr/1.2 I ran some tests and it works quite well in my local environment. If stderr is the only source of the corrupted nosetests output, it should be easily solvable.

nosetests takes the first comment line of a test for reporting. This also causes a strange output. E.g.
The models should behave as iaf_cond_* if a == 0., b == 0. and ... ok

[heplesser]

@gtrensch Thanks for investigating! I would not want to make the correct operation of the NEST test suite dependent on the installation of an additional Pyton package. But that package is only a single file of code that appears to be quite directly based on similar code in nose. Could we integrate this capture technique directly in NEST, so that we can capture stderr without the need for installing additional third-party software?

I will schedule a discussion of #761 for the next Open Developer VC.

[gtrensch]

@heplesser I see two aspects which I would handle seperatly:

• the bug we want to fix for the 2.12.1 release which is due to missing nosetests functionality (stdout is captured while stderr is not)
• the question whether we stay with nosetests and perhaps move to nose2 or swich to a different Python test framework

Addressing the question of the test framework we want to use, will very likely also solve the stderr-issue. Depending on the descision we make there might be some additional effort.

However, as you stated, we should carry out this discussion (#761) in the next Open Developer VC !