Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve zero tests messaging #1258

Merged
merged 3 commits into from
Aug 30, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions Changelog.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,11 @@
* [#1257](https://github.com/mbj/mutant/pull/1257)
Fix crash on numblock mutations.

* [#1258](https://github.com/mbj/mutant/pull/1258)
Add improved UI on detecting 0 tests.
This should be beneficial in onboarding scenarios or after manual
persistend rspec selections.

# v0.10.33 2021-08-25

* [#1249](https://github.com/mbj/mutant/pull/1249/files)
Expand Down
25 changes: 24 additions & 1 deletion lib/mutant/cli/command/environment/run.rb
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ class Environment
class Run < self
NAME = 'run'
SHORT_DESCRIPTION = 'Run code analysis'
SLEEP = 40
SLEEP = 60
SUBCOMMANDS = EMPTY_ARRAY

UNLICENSED = <<~MESSAGE.lines.freeze
Expand All @@ -16,6 +16,20 @@ class Run < self
See https://github.com/mbj/mutant#licensing
MESSAGE

NO_TESTS_MESSAGE = <<~'MESSAGE'
===============
Mutant found no tests. Mutation testing cannot be started.

This can have various reasons:

* You did not setup an integration, see:
https://github.com/mbj/mutant/blob/main/docs/configuration.md#integration
* You set environment variables like RSPEC_OPTS that filter out all tests.
* You set configuration optiosn like `config.filter_run :focus` which do
make rspec to not report any test.
===============
MESSAGE

# Test if command needs to be executed in zombie environment
#
# @return [Bool]
Expand All @@ -28,10 +42,19 @@ def zombie?
def action
soft_fail(License.call(world))
.bind { bootstrap }
.bind(&method(:validate_tests))
.bind(&Runner.public_method(:call))
.bind(&method(:from_result))
end

def validate_tests(environment)
if environment.integration.all_tests.length.zero?
Either::Left.new(NO_TESTS_MESSAGE)
else
Either::Right.new(environment)
end
end

def from_result(result)
if result.success?
Either::Right.new(nil)
Expand Down
2 changes: 1 addition & 1 deletion lib/mutant/reporter/cli/printer/mutation.rb
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ class Printer
class Mutation < self
NO_DIFF_MESSAGE = <<~'MESSAGE'
--- Internal failure ---
BUG: A generted mutation did not result in exactly one diff hunk!
BUG: A generated mutation did not result in exactly one diff hunk!
This is an invariant violation by the mutation generation engine.
Please report a reproduction to https://github.com/mbj/mutant
Original unparsed source:
Expand Down
2 changes: 1 addition & 1 deletion lib/mutant/reporter/cli/printer/mutation_result.rb
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ class MutationResult < self

NO_DIFF_MESSAGE = <<~'MESSAGE'
--- Internal failure ---
BUG: A generted mutation did not result in exactly one diff hunk!
BUG: A generated mutation did not result in exactly one diff hunk!
This is an invariant violation by the mutation generation engine.
Please report a reproduction to https://github.com/mbj/mutant
Original unparsed source:
Expand Down
57 changes: 39 additions & 18 deletions spec/unit/mutant/cli_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -679,10 +679,10 @@ def self.main_body
[
license_validation_event,
[:stderr, :puts, 'license-error'],
[:stderr, :puts, "[Mutant-License-Error]: Soft fail, continuing in 40 seconds\n"],
[:stderr, :puts, "[Mutant-License-Error]: Soft fail, continuing in 60 seconds\n"],
[:stderr, :puts, "[Mutant-License-Error]: Next major version will enforce the license\n"],
[:stderr, :puts, "[Mutant-License-Error]: See https://github.com/mbj/mutant#licensing\n"],
[:sleep, 40],
[:sleep, 60],
[
:load_config_file,
world
Expand Down Expand Up @@ -748,28 +748,49 @@ def self.main_body
end

context 'on runner success with unsuccessful result' do
let(:expected_exit) { false }
let(:runner_result) { right(env_result) }
context 'on alive mutations' do
let(:expected_exit) { false }
let(:runner_result) { right(env_result) }

let(:env_result) do
instance_double(
Mutant::Result::Env,
success?: false
)
end

let(:env_result) do
instance_double(
Mutant::Result::Env,
success?: false
)
let(:expected_events) do
[
*super(),
[
:stderr,
:puts,
'Uncovered mutations detected, exiting nonzero!'
]
]
end

include_examples 'CLI run'
end

let(:expected_events) do
[
*super(),
context 'on not having found tests' do
let(:tests) { [] }

let(:expected_events) do
[
:stderr,
:puts,
'Uncovered mutations detected, exiting nonzero!'
*super()[0..-2],
[
:stderr,
:puts,
Mutant::CLI::Command::Environment::Run::NO_TESTS_MESSAGE
]
]
]
end
end

include_examples 'CLI run'
let(:expected_exit) { false }

include_examples 'CLI run'
end
end

context 'with valid subject expression' do
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@
-----------------------
Killfork: #<InstanceDouble(Process::Status) (anonymous)>
--- Internal failure ---
BUG: A generted mutation did not result in exactly one diff hunk!
BUG: A generated mutation did not result in exactly one diff hunk!
This is an invariant violation by the mutation generation engine.
Please report a reproduction to https://github.com/mbj/mutant
Original unparsed source:
Expand Down