ch-test is unwieldy

[lucaudill]

Recent work on ch-test (#1899, for example) has made it clear that the file has become difficult to work with or expand in any significant way. Parts of the code are awkward and difficult to read, and it feels increasingly hard to add to it without worsening this problem. As I see it, there are two possibilities to address this. We could either refactor the code, or re-write it in Python.

My experience with Bash makes me think that refactoring the existing code could very well be harder than re-writing it in an easier language like Python. On top of this, I don't see many obvious benefits to keeping ch-test in Bash, aside from sticking to the way we've been doing things. There is the argument that ch-test being in Bash makes it more portable, but Reid pointed out that any system that can run Bash can also probably run Python 3. Furthermore, I don't know how relevant this argument is from the user's perspective. The test suite is almost certainly more important for devs, and I'm skeptical that a significant portion of our user base works on systems that don't have Python.

Here's my argument for re-writing ch-test in Python.
Pros:

• It'll probably be easier than a Bash refactor, as I said
• It'll make the test suite more approachable for new devs
• Experienced devs will have an easier time understanding and contributing to the code
• Calling BATS in Python will be just as easy as in Bash, so we can keep all our .bats files for now
• It could be a step toward re-writing the entire test suite in Python, eliminating BATS as a dependency

Cons:

• It might be an issue for portability, but I'm skeptical.