Test Output, Run and RunAsync

Test output is written to Console during Run() or RunAsync().
Each step is listed after the optional Scenario title, with the step type and its outcome.

Step outcomes are:

• [Passed]
• [Failed]
• [Inconclusive]
• [not run]

Outcomes per step will indicate which step is failing the test and which have run successfully.

Scenario: Login_WithSignedUpNewUser_Successful
  Given a new user context [Passed]
    And the user signs up [Passed]
  When a call is made to log the user in [Failed]
  Then the response is successful [not run]

This can also help diagnose remote test failures when looking at test run logs.

Scenario title

The Scenario line in the test output is added when the test starts with Scenario(). This will use the test method name as the scenario description by default. If a scenario title is added by supplying a string argument then it will be used instead.

Scenario()

Scenario: Login_WithSignedUpNewUser_Successful

Scenario("Login for a new user is successful")

Scenario: Login with a new signed up user is successful

Customizing test output

Supply an action of type Action<BddPipe.ScenarioResult> to the Run() call to customize the test output. This also will prevent the regular output code from running. Having an empty method will therefore mute the output.

.Run(scenarioResult =>
{

});

The current default console output code used by Run() is equivalent to the following:

if (!string.IsNullOrWhiteSpace(scenarioResult.Title))
{
  Console.WriteLine(scenarioResult.Description);
}

foreach (StepResult stepResult in scenarioResult.StepResults)
{
  Console.WriteLine(stepResult.Description);
}

A method WriteLogsToConsole exists to perform this, which optionally takes an Action<string> method if you wish to handle how a line is written (or collect lines that would have been written). The default Action<string> used otherwise is Console.WriteLine.

.Run(scenarioResult =>
{
   WriteLogsToConsole(scenarioResult);
});

Depending on the custom implementation, the test output can now be sent anywhere else, as well as also optionally performing the WriteLogsToConsole behaviour.

Custom output reuse

Run() is an extension method on Pipe<T>. You can therefore make your own extension method. Ensure it still calls Run() to evaluate the test result.

public static class CustomOutput
{
  public static BddPipeResult<T> RunCustom<T>(this Pipe<T> pipe) =>
    pipe.Run(scenarioResult =>
    {
      // implementation
    });
}

The extension can then be called from the test code in place of Run().

.And("the final step", () => { })
.RunCustom();

Run vs RunAsync

It is recommended to use Run when all steps are synchronous, and RunAsync when any steps are asynchronous.

After introducing any async steps, the steps from that point onward are all internally awaited to pass the result to the next step. The only place this doesn't happen is during a call to the synchronous Run which causes the code to block and evaluate the result. This approach uses the 'Thread Pool Hack' mentioned in Stephen Cleary, (July 2015). Async Programming - Brownfield Async Development. This may or may not matter to you and has a few disadvantages mentioned in the article. The following code is used to block and evaluate the result:

public static R RunAndWait<R>(Task<R> fn) =>
    Task.Run(() => fn)
        .GetAwaiter()
        .GetResult();

RunAsync returns a Task instead that can be awaited, or returned from a test method returning Task (shown below), which means nothing will block at all.

[Test]
public Task Test()
{
    return Scenario()
        // steps
        .RunAsync();
}

[Test]
public Task Test() =>
    Scenario()
        // steps
        .RunAsync();