Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a configuration option for making not-implemented tests failures #123

Closed
jcornaz opened this issue Jun 28, 2021 · 3 comments
Closed
Assignees
Labels
enhancement Improvement of existing features or bugfix
Milestone

Comments

@jcornaz
Copy link

jcornaz commented Jun 28, 2021

If there is a mismatch between the *.feature file and the the steps defined, we'll see an output like.

[Cucumber v0.8.4]
Feature: Animal feature                                                                                                                                                                                           features/animal.feature:1:1 

 Scenario: If we feed a hungry cat it will no longer be hungry                                                                                                                                                    features/animal.feature:3:3 
  ✔ Given A hungry cat                                                                                                                                                                                            features/animal.feature:4:5 
  ✔ When I feed the cat                                                                                                                                                                                           features/animal.feature:5:5 
  - Then The cat is not hungry!!!!                                                                                                                                                                                features/animal.feature:6:5 
    ⚡ Not yet implemented (skipped)

[Summary]
1 features
1 scenarios (1 skipped, 0 passed)
6 steps (1 skipped, 2 passed)

And the command line returns 0 (success).

But I don't want to have this to pass the CI and sneak in the main branch.

Ideally if any test was not implemented, the command line should return a failure. Eventually this could be a configurable behavior.

@bbqsrc bbqsrc changed the title Consider not-implemented tests as failure Add a configuration option for making not-implemented tests failures Jul 19, 2021
@bbqsrc bbqsrc added enhancement Improvement of existing features or bugfix good first issue Good for newcomers help wanted Extra attention is needed labels Jul 19, 2021
@rob-mur
Copy link

rob-mur commented Sep 14, 2021

From a brief look at the documentation I think we want to change the following code to include skips if specified in the config:

impl Stats {
    /// Indicates this has failing states (aka failed or timed_out)
    pub fn failed(&self) -> bool {
        self.failed > 0 || self.timed_out > 0
    }
}

I'll put it on my list to look at hopefully in the next week or two, but this should be a super simple one to do if anyone has more time!

@tyranron
Copy link
Member

Now possible after merging #128

@ilslv we need a meaningful example in the Book for this too.

@tyranron tyranron added this to the 0.10 milestone Sep 29, 2021
tyranron added a commit that referenced this issue Sep 30, 2021
, #64, #102, #103, #123, #132)

Additionally:
- fix outputing English in writer::Basic instead of parsed keywords

Co-authored-by: Kai Ren <[email protected]>
@tyranron
Copy link
Member

Described in docs for Cucumber::fail_on_skipped() and mentioned in "Managing growth" chapter of the Book.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Improvement of existing features or bugfix
Projects
None yet
Development

No branches or pull requests

5 participants