Move test execution to runner file, refactor output

[dylanhitt]

Closes #121.

TODO:

• Migrate runtime.Start to it's own file runner.go
• Stream TestResults to output in runtime
• Aggregate TestResults to a single struct Result
• Add FileName to TestCase struct when executing against a dir
• Define a single OutputWriter to be used in both Runtime and for out.PrintSummary
• Test runner.go
• Converge struct Results in Test_Command when executing against a dir
• Switch output to go/template

Checklist

• Added unit / integration tests for windows, macOS and Linux?
• Added a changelog entry in CHANGELOG.md?
• Updated the documentation (README.md, docs)?
• Does your change work on Linux, Windows and macOS?

[dylanhitt]

Mainly just trying to organize my thoughts.

I elected to implement runner.go in the runtime package do to moving runner into its own package would require a refactor of runtime. Pulling the execution out currently causes a circular dependency as all the executors reside in runtime. runtime being broken up would beneficial as well, but is out of the scope of this PR and the issue associated

We may benefit creating a Test_Command struct. We could store command ctx, OutputWriter, and test filters. We could provide a validation function make sure the command is valid. We could also use output to print logs from the Test_Command as well. Currently I just add OutputWriter as a global var.

--dir has a special case where we need to pass FileName to TestCase or TestResult. There's really no place to do this on TestResult as execution has been moved to runner.go. I elected to implement suite.ParseYAML(content, fileName) giving us the option to pass TestCase FileName depending on the needs of app.Test_Command

[dylanhitt]

@SimonBaeumer when you get a chance would you mind taking a look. My above comment explains some of my decisions. I'm hesitant to write tests. I'm not a big fan of my go/template implementation, it's kinda tough to implement the output colors and still get the benefit of using template. I will implement tests based upon how your feedback goes.

I do enjoy moving execution to runner.go as passing the the channel doesn't feel very forced anymore, as well simplifying output. We had a concern of printing the results in runtime, however with say a json implementation with an http endpoint we can delay printing/posting until all tests are finished.

Let me know what you think, Im looking forward to your feedback. I hope all is well

Cheers

[SimonBaeumer]

Hey @dylanhitt!
Nice to see your pull request, it looks really good to me for a first approach!

[SimonBaeumer]

If you need any help on formatting issues let me know - I can help you with the annoying tasks ;)

[codeclimate]

Code Climate has analyzed commit 5ead497 and detected 0 issues on this pull request.

The test coverage on the diff in this pull request is 93.3% (50% is the threshold).

This pull request will bring the total coverage in the repository to 92.2% (0.4% change).

View more on Code Climate.

[dylanhitt]

@SimonBaeumer I believe this ready for review

[SimonBaeumer]

This looks really good to me!
I only requested some minor changes. 👍
A point we can discuss another time is to move the aggregation and composition of the suites to the runtime package, but atm it is not necessary in my opinion.

Would be very happy to merge this :)

[dylanhitt]

@SimonBaeumer should be good to go now. I don't know if you want an update to the changelog since this is a just refactor. Unless you plan on incrementing versions. If so let me know.

[SimonBaeumer]

Merge! :)