Should auto-generated test cases fail by default?

[samcic]

Consider generating a new Ember service using ember generate service my-service (my comments here apply equally to most other kinds of resources...I'm using a service just as an example). Two files are created => the service file and the unit test file. The unit test file includes an example test case which looks something like this:

// Replace this with your real tests.
test('it exists', function(assert) {
  var service = this.subject();
  assert.ok(service);
});

If I do nothing else, this won't break anything. I might forget completely about this new service...running my application's test suite won't alert me to anything because this test just passes by default.

I'd like to raise the question of whether or not this is the most appropriate approach for application development. Although I can imagine that the default behavior for these scaffolded tests is a matter of taste, perhaps I can share what I feel are some benefits of having these tests fail by default:

1. Ideally, the developer will be following some kind of TDD approach and will focus on testing first. If not, a failing test will force them to think about testing when they otherwise may not have.
2. If I'm creating a large number of resources in one go (which I often do when scaffolding a large feature), it can be easy to lose track of which ones I haven't yet tested. Auto-failing tests will "remind" me in these situations.
3. Often I find myself adding a service, then realizing that there's a better way to code it. If I forget to delete the service file, an auto-failing test will remind me that I have redundant files/code that I can delete in this situation.
4. Put simply, it would give me extra "peace of mind". That is, I'm sure I'd be able to sleep better knowing that the tools I'm using are designed to be critical of what I'm doing, explicitly questioning and making me think about the quality and integrity of my app by default.

Concretely, my suggestion would simply be to change the scaffolded tests into something like this (self-documenting):

test('it exists', function(assert) {
  var service = this.subject();
  assert.ok(false,'The service "my-service" has not yet been tested. It is highly recommended that you test your code for reasons X,Y,Z...');
});

I'm sure few serious developers would argue against the benefits of needing to have automated testing in place for modern apps. In my opinion, auto-failing tests on resource-creation just seems to be a simple and intuitive way to draw attention to this need.

If the community is against this suggestion, I'd be curious to hear the justification. At this stage I'm struggling to think of practical reasons for not having failing tests by default.

[stefanpenner]

generators producing failing tests by default is not something we will do, this will great excess pain/noise, with little rewards.

I do see the value of red/green refactor cycles. I can think of two approaches.

1. use something like: https://github.com/JamesMGreene/qunit-pending and generating a pending test, this would be a sort of yellow test, pending implementation.
2. a community add-on that provides blueprints which generate failing tests, as an opt-in.

---

I am closing this, not because the discussion isn't good but it isn't quite the correct venue.

Typically these sorts of requests are to be provided via the https://github.com/ember-cli/rfcs, i noticed the CONTRIBUTING.MD doesn't say this clearly, I have updated it accordingly: fbbb354

I welcome an issue (or fleshed our RFC) on the RFC repo.