Skip to content

Commit

Permalink
Re-organize and improve output check scenarios
Browse files Browse the repository at this point in the history
- Move scenarios for checking success + output to a separate feature
- Unify order and implementation of scenarios for checking output,
  stderr and stdout
  • Loading branch information
mvz committed Nov 15, 2024
1 parent f6b5d16 commit 1460be7
Show file tree
Hide file tree
Showing 4 changed files with 231 additions and 246 deletions.
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
Feature: Checking success and output together

In order to specify expected output and success
As a developer using Cucumber
I want to use the "it should pass with" step family

Background:
Given I use a fixture named "cli-app"

Scenario: Match passing exit status and partial output
Given a file named "features/output.feature" with:
"""cucumber
Feature: Run command
Scenario: Run command
When I run `echo hello world`
Then it should pass with:
\"\"\"
hello
\"\"\"
"""
When I run `cucumber`
Then the features should all pass

Scenario: Match passing exit status and exact output
Given a file named "features/output.feature" with:
"""cucumber
Feature: Run command
Scenario: Run command
When I run `ruby -e 'puts "hello\nworld"; exit 0'`
Then it should pass with exactly:
\"\"\"
hello
world
\"\"\"
"""
When I run `cucumber`
Then the features should all pass

Scenario: Match failing exit status and partial output
Given a file named "features/output.feature" with:
"""cucumber
Feature: Run command
Scenario: Run command
When I run `ruby -e 'puts "hello\nworld"; exit 1'`
Then it should fail with:
\"\"\"
hello
\"\"\"
"""
When I run `cucumber`
Then the features should all pass

Scenario: Match failing exit status and exact output
Given a file named "features/output.feature" with:
"""cucumber
Feature: Run command
Scenario: Run command
When I run `ruby -e 'puts "hello\nworld"; exit 1'`
Then it should fail with exactly:
\"\"\"
hello
world
\"\"\"
"""
When I run `cucumber`
Then the features should all pass

Scenario: Match failing exit status and output with regex
Given a file named "features/output.feature" with:
"""cucumber
Feature: Run command
Scenario: Run command
When I run `ruby -e 'puts "hello\nworld"; exit 1'`
Then it should fail with regex:
\"\"\"
hello\s*world
\"\"\"
"""
When I run `cucumber`
Then the features should all pass

Scenario: Detect output from all processes
Given a file named "features/output.feature" with:
"""cucumber
Feature: Run command
Scenario: Run command
When I run `echo hello`
And I run `echo good-bye`
Then the stdout should contain exactly:
\"\"\"
hello
good-bye
\"\"\"
"""
When I run `cucumber`
Then the features should all pass
Original file line number Diff line number Diff line change
Expand Up @@ -14,35 +14,11 @@ Feature: All output of commands which were executed
Scenario: Run command
When I run `echo hello world`
Then the output should contain "hello world"
And the output should not contain "good-bye"
"""
When I run `cucumber`
Then the features should all pass

Scenario: Detect absence of inline subset of output
Given a file named "features/output.feature" with:
"""cucumber
Feature: Run command
Scenario: Run command
When I run `echo hello world`
Then the output should not contain "good-bye"
"""
When I run `cucumber`
Then the features should all pass

Scenario: Failed detection of inline subset of output
Given a file named "features/output.feature" with:
"""cucumber
Feature: Run command
Scenario: Run command
When I run `echo hello world`
Then the output should contain "goodbye world"
"""
When I run `cucumber`
Then the features should not all pass with:
"""
expected "hello world" to string includes: "goodbye world"
"""

Scenario: Detect subset of output with multiline string
Given a file named "features/output.feature" with:
"""cucumber
Expand All @@ -53,57 +29,73 @@ Feature: All output of commands which were executed
\"\"\"
hello
\"\"\"
And the output should not contain:
\"\"\"
good-bye
\"\"\"
"""
When I run `cucumber`
Then the features should all pass

Scenario: Detect absence subset of output with multiline string
Scenario: Detect exact multiline output
Given a file named "features/output.feature" with:
"""cucumber
Feature: Run command
Scenario: Run command
When I run `ruby -e 'puts "hello\nworld"'`
Then the output should not contain:
When I run `ruby -e 'puts "hello\nworld"; warn "good-bye"'`
Then the output should contain exactly:
\"\"\"
hello
world
good-bye
\"\"\"
"""
When I run `cucumber`
Then the features should all pass

Scenario: Detect exact multiline output
Given a file named "features/output.feature" with:
"""cucumber
Scenario: Detect combined output from normal and interactive processes
Given an executable named "bin/aruba-test-cli" with:
"""
#!/usr/bin/env ruby
while input = gets do
break if "" == input
puts input
end
"""
And a file named "features/output.feature" with:
"""
Feature: Run command
Scenario: Run command
When I run `ruby -e 'puts "hello\nworld"'`
Then the output should contain exactly:
When I run `echo hello`
When I run `aruba-test-cli` interactively
And I type "good-bye"
And I type ""
Then the stdout should contain exactly:
\"\"\"
hello
world
good-bye
\"\"\"
"""
When I run `cucumber`
Then the features should all pass

Scenario: Failed detection of exact multi-line output
Scenario: Detect output from named source
Given a file named "features/output.feature" with:
"""cucumber
"""
Feature: Run command
Scenario: Run command
When I run `ruby -e 'puts "goodbye\nworld"'`
Then the output should contain exactly:
When I run `echo hello`
And I run `echo good-bye`
Then the output from "echo hello" should contain "hello"
And the output from "echo good-bye" should contain exactly "good-bye"
And the output from "echo hello" should contain exactly:
\"\"\"
hello
world
\"\"\"
"""
When I run `cucumber`
Then the features should not all pass with:
"""
expected "goodbye\nworld" to output string is eq: "hello\nworld"
Diff:
"""
Then the features should all pass

Scenario: Detect exact output with ANSI output
Given a file named "features/output.feature" with:
Expand Down Expand Up @@ -191,121 +183,6 @@ Feature: All output of commands which were executed
When I run `cucumber`
Then the features should all pass

Scenario: Match passing exit status and partial output
Given a file named "features/output.feature" with:
"""cucumber
Feature: Run command
Scenario: Run command
When I run `echo hello world`
Then it should pass with:
\"\"\"
hello
\"\"\"
"""
When I run `cucumber`
Then the features should all pass

Scenario: Match passing exit status and exact output
Given a file named "features/output.feature" with:
"""cucumber
Feature: Run command
Scenario: Run command
When I run `ruby -e 'puts "hello\nworld"; exit 0'`
Then it should pass with exactly:
\"\"\"
hello
world
\"\"\"
"""
When I run `cucumber`
Then the features should all pass

Scenario: Match failing exit status and partial output
Given a file named "features/output.feature" with:
"""cucumber
Feature: Run command
Scenario: Run command
When I run `ruby -e 'puts "hello\nworld"; exit 1'`
Then it should fail with:
\"\"\"
hello
\"\"\"
"""
When I run `cucumber`
Then the features should all pass

Scenario: Match failing exit status and exact output
Given a file named "features/output.feature" with:
"""cucumber
Feature: Run command
Scenario: Run command
When I run `ruby -e 'puts "hello\nworld"; exit 1'`
Then it should fail with exactly:
\"\"\"
hello
world
\"\"\"
"""
When I run `cucumber`
Then the features should all pass

Scenario: Match failing exit status and output with regex
Given a file named "features/output.feature" with:
"""cucumber
Feature: Run command
Scenario: Run command
When I run `ruby -e 'puts "hello\nworld"; exit 1'`
Then it should fail with regex:
\"\"\"
hello\s*world
\"\"\"
"""
When I run `cucumber`
Then the features should all pass

Scenario: Detect output from all processes
Given a file named "features/output.feature" with:
"""cucumber
Feature: Run command
Scenario: Run command
When I run `echo hello`
And I run `echo good-bye`
Then the stdout should contain exactly:
\"\"\"
hello
good-bye
\"\"\"
"""
When I run `cucumber`
Then the features should all pass

Scenario: Detect combined output from normal and interactive processes
Given an executable named "bin/aruba-test-cli" with:
"""
#!/usr/bin/env ruby
while input = gets do
break if "" == input
puts input
end
"""
And a file named "features/output.feature" with:
"""
Feature: Run command
Scenario: Run command
When I run `echo hello`
When I run `aruba-test-cli` interactively
And I type "good-bye"
And I type ""
Then the stdout should contain exactly:
\"\"\"
hello
good-bye
\"\"\"
"""
When I run `cucumber`
Then the features should all pass

Scenario: Check size of output
Given a file named "features/flushing.feature" with:
"""cucumber
Expand All @@ -316,16 +193,3 @@ Feature: All output of commands which were executed
"""
When I run `cucumber`
Then the features should all pass

Scenario: Detect output from named source
Given a file named "features/output.feature" with:
"""
Feature: Run command
Scenario: Run command
When I run `echo hello`
And I run `echo good-bye`
Then the output from "echo hello" should contain exactly "hello"
And the output from "echo good-bye" should contain exactly "good-bye"
"""
When I run `cucumber`
Then the features should all pass
Loading

0 comments on commit 1460be7

Please sign in to comment.