Turn examples into simple unit tests.

[ari-koivula]

We already have @example('.calc 1 + 1'). This syntax could be expanded to not only show the input, but also the output like: @example('.calc 1 + 1', '2'). Then all that is left is to make @example construct the needed mock objects to feed to the callable, and check if it replied or msg'd the output.

This decorator would also generate the needed objects for unittest to find, so that you could simply put

if __name__ == '__main__':
    unittest.main()

to the bottom of the file and get some simple unit tests.

This would also complement issue #268, as the same examples could be used in that framework. But being able to simply run a file after modifications to check for dumb mistakes is very valuable. The dummy-triggers and bots could also be used in more complicated test cases.

I'm going to start working on this, but what do you guys think? Should the decorator be called something other than example? Is this too simplistic to be useful? Should I target something other that python's unittest module?

[elad661]

I think we should have another decorator, because we need way more than just one testcase per callable (for example, to test both database timezone and user-specified timezone in clock.py

[ari-koivula]

How about selecting the first example as default and extending the syntax of help to show more examples? Like:

    <user> .help calc
    <Willie> e.g. for ".calc 1 + 1", I would respond with "2" (example 1 of 2)
    <user> .help calc 2
    <Willie> e.g. for ".calc", I would respond with "Nothing to calculate." (example 2 of 2)

[ari-koivula]

I'm thinking of using nose instead of unittest, because a lot of these examples are io-bound and nose can run tests in multiple threads. It's an optional dependency, so not a big deal.

[elad661]

This sounds like a great idea!

We should also consider improving the help system by adding a search feature... for example

<user> .help calculate
<willie> search results: .calc, .wa

[ari-koivula]

Actually, screw nose, pytest is much better. Everything just works, unlike the buggy mess that is nose. It's looking pretty great right now but I don't have time to work more on this today.