-
-
Notifications
You must be signed in to change notification settings - Fork 3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Consistent handling of --forbid-only for suites and tests #4282
Conversation
@arvidOtt looks like this now conflicts with |
@mochajs/core This is one of those things that, I think, makes sense. However, it's a) a breaking change, and b) maybe not worth breaking. Opinions? |
Before we merge this--or any breaking change, really--we should answer these questions:
I'll try to answer these: This change ensures that, when used with
Generally, I don't see a lot of usage of Since the intent of |
@arvidOtt FWIW this needs conflict resolution again, but you may want to just wait until someone else reviews it. Or, if it's approved by others, I can resolve the conflicts myself and merge. |
Maybe one thing to highlight here is that the inconsistency in the behaviour of suites and tests also might "harm" users by unexpected behaviour. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code change looks good. I've got a small remark about reusing a sandbox
.
@boneskull I agree with you that the intent of --forbid-only
is to avoid false positives. I don't believe a lot of people are using it in any other way. In conclusion, I think releasing this in our next major version is a good idea.
If users are using --forbid-only
in conjunction with --grep
, they should be able to work around it by making multiple config's for example. We should clearly state so in the changelog, so users know what to do.
@nicojs Can you explain further? |
3894d12
to
5c58d0a
Compare
@arvidOtt I've rebased to fix the conflict; please look at the result. In another PR, I had created a |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree with this change.
We throw during the parsing phase - similar to linting - and grep
grips later during runtime. Parsing comes first and therefore has higher priority.
@boneskull that makes sense, thanks! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@arvidOtt thank you
Wanted to understand what @nicojs is recommending here insofar as documentation |
Sure, I'll write down my thoughts tomorrow. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry to come back to the sandbox, but the way it is now I see a problem with it. Sorry if I didn't comunicate myself clearly the first time.
Maybe something like this. BREAKING CHANGEFocussed suites (i.e. This is no longer OK to do: // foobar.spec.js
describe.only('foo', () => {
});
describe('bar', () => {
});
Work around If you want to keep using this setup, you can work around this issue by moving focussed suites to different files and configure mocha to exclude them in your CI. // only-foo.spec.js
describe.only('foo', () => {
});
// bar.spec.js
describe('bar', () => {
});
The same configuration could also be achieved with 2 configuration files. // .mocharc.json
{
"spec": "*.spec.js",
"exclude": "only-foo.spec.js",
"grep": "bar",
"forbid-only": true
}
// .mocharc.only.js
const config = require('./.mocharc.json');
config['forbid-only'] = false;
config.spec = ['**.spec.js'];
module.exports = config; npx mocha # normal run, without foo
npx mocha --config .mocharc.only.js # run with focussed foo |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Awesome! Thanks for the fast response. For me, this can land now 👍
Description of the Change
The handling of the
--forbid-only
option is currently inconsistent for theTest
andSuite
classes. Suites that are marked as only and excluded by--grep
or--fgrep
don't throw an error. Test that are marked as only but excluded bygrep
orf-grep
do throw an error.Following the discussion in #4256 it was suggested that due to the nature of the
--forbid-only
option it should not matter if a suite or test is excluded by a grep. An error should be thrown either way to protect the user from forgetting to remove.only()
statements from his test code. I moved the changes concerning this inconsistency, including a refactoring of the Suite class to use amarkOnly()
instance method, into this PR to separate them from the fix #4256 for bug #3840.Alternate Designs
An alternative to make it consistent would be to adapt the logic so it will be checked if test cases are excluded by
grep
orfgrep
as well. (see discussion in #4256)Why should this be in core?
It improves the consistency of the
--forbid-only
option and improves encapsulation between classes.Benefits
It improves the consistency of the
--forbid-only
option and improves encapsulation between classes.Possible Drawbacks
If the
--forbid-only
option is set and a test or suite is marked only but excluded by grep it will now throw an error.Applicable issues
#4256
#3840