Feature Request: Add --runInBand to package.json options

[JamesMcGuigan]

This is a feature request.

I currently have a setup that involves jest tests asyncronously creating and re-aliasing elasticsearch indices and part of their setup and tear down routines.

The tests run perfectly fine if I specify jest --runInBand on the command line.

If I run them without --runInBand I get random test failures due to the inherent race conditions with running these tests asynchronously.

What I would like is an option in package.json:

{
  "jest": {
    "runInBand": true
  }
}

Such that the flag is automatically applied to jest (like all the other package.json settings), without needing to manually specify it on the command line each time.

The lack of this feature may potentially confuse other developers on my team who attempt to run the jest tests without being explicitly aware the tests are dependent on the --runInBand flag being present, thus observing a broken test suite.

package.json is the correct place to document this configuration dependancy

Thank you.

[cpojer]

I'd prefer not to do this. -i is an escape hatch. It would be better to find a way to run the tests in parallel.

…

________________________________ From: James McGuigan <[email protected]> Sent: Tuesday, March 28, 2017 6:24:32 AM To: facebook/jest Cc: Subscribed Subject: [facebook/jest] Feature Request: Add --runInBand to package.json options (#3215) This is a feature request. I currently have a setup that involves jest tests asyncronously creating and re-aliasing elasticsearch indices and part of their setup and tear down routines. The tests run perfectly fine if I specify jest --runInBand on the command line. If I run them without --runInBand I get random test failures due to the inherent race conditions with running these tests asynchronously. What I would like is an option in package.json: { "jest": { "runInBand": true } } Such that the flag is automatically applied to jest (like all the other package.json settings), without needing to manually specify it on the command line each time. The lack of this feature may potentially confuse other developers on my team who attempt to run the jest tests without being explicitly aware the tests are dependent on the --runInBand flag being present, thus observing a broken test suite. package.json is the correct place to document this configuration dependancy Thank you. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<#3215>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AAA0KI5INcWs1BFGWPxPeoYDHKjg1uQrks5rqCkQgaJpZM4Mq2Fy>.

[thymikee]

Create a npm script and don't think about it?

"test:es": "jest -i"

[jdx]

Just want to give my 2 cents on this one. We use jest for lots of integration tests in a project and set -i like above which works, but it's a bit annoying when you want to run jest --watch or any other one-off jest command. We've looked into parallelizing this suite, but it's very tough to do for us given the nature of the project. Most of our projects work fine parallelized, but this one is full of integration tests that install/uninstall plugins to a CLI that requires a lock over the system.

Yes we could maybe sandbox it out so it would run, but really, we'd rather just run the tests in serial and be done with it. I think it's a great idea to encourage developers to write tests such that they won't collide with one another, but it shouldn't be annoying to run jest without it.

Anyways, this is one man's opinion.

[thymikee]

@dickeyxxx you can put your configuration into a JS file and set process.env.CI=true since Jest 20, maybe that helps?

[jdx]

seems it still does the same thing: heroku/cli-engine@310aa52

$ jest --version
v20.0.0
$ jest
● Validation Warning:

  Unknown option "runInBand" with value true was found.
  This is probably a typing mistake. Fixing it will remove this message.

  Configuration Documentation:
  https://facebook.github.io/jest/docs/configuration.html


 RUNS  src/integration.test.js
 RUNS  src/namespaces.test.js
 RUNS  src/plugins/user.test.js

Test Suites: 0 of 16 total
Tests:       0 total
Snapshots:   0 total
Time:        0s, estimated 47s

[lgandecki]

This would be very useful for jest projects as well (if I have a runner for a e2e tests + unit tests, I want to be able to run e2e inBand, and unitTests in parallel, right now my only option to run all of them one or the other way)

[b4dnewz]

Hi, I know this issue is closed but there is a particular reason why the command options you can pass are different from the configuration options?

I mean, we can always use like this: jest --runInBand as thymikee said but let assume someone have this setup:

"scripts": {
    "test": "jest --runInBand"
 },
 "jest": {
    "bail": true,
    "verbose": true
 }

Compared to this:

 "jest": {
    "bail": true,
    "verbose": true,
    "runInBand": true
 }

It's more readable and manageable all in one place, this example is simple but can be more complex in certain projects.

[mgcrea]

Also another valid use case is when you are in a monorepo configuration and running jest globally with a projects config:

If one of your packages has to run serially, the only option is to run everything serially with --runInBand with a very poor performance. Would be great if each project could define if they have be run serially or not.

In an extreme case, even if all your packages must run serially, jest should still be able to run each projects in parallel.

[Adverbly]

The detectOpenHandles flag used by the cli is another example of a flag missing from config. Would indeed be very nice if all flags could be made config.

[SimenB]

In the case of detectOpenHandles it has an unfortunate performance overhead, so you don't really want it all the time.

It's designed to be used when needed, not all the time, so adding it as config doesn't really make sense.

---

As an aside, runners can say that they have to run serially, if that helps

[lukepighetti]

As an aside, runners can say that they have to run serially, if that helps

@SimenB how?

[SimenB]

See #5712

[capaj]

please consider reopening. Even though it has forced me to improve our test pipeline significantly(instead of passing runInBand everywhere I made sure we run each worker against his own DB), it would be very nice to have for projects which really need a serial test runner.

[AntonNarkevich]

If there is a CLI option there should be a config file equivalent.
The API is inconsistent otherwise.
Also forcing users to run tests in parallel is not a good thing. If you wanna flag that parallel execution is beneficial why not simply add an execution time hint?

[ranyitz]

@capaj & @AntonNarkevich As pointed out it's currently possible to configure a runner to run serially.

Here is a simple example:

extend the default jest-runner and only change its isSerial property.

// serial-jest-runner.js
const DefaultJestRunner = require('jest-runner');

class SerialJestRunner extends DefaultJestRunner {
  constructor(...args) {
    super(...args);
    this.isSerial = true;
  }
}

module.exports = SerialJestRunner;

// jest.config.js
{
  runner: './serial-jest-runner.js',
}

[olivierlacan]

So in order to configure Jest to do something it can do with a flag, you need to create a custom runner whose sole purpose at 11 lines of code is to do something a configuration like could do in 1 line? That's a bit much.

isSerial: true

I'd recommend avoiding spiraling config value synonyms. runInBand is fairly self-explanatory (and suggests its relationship with the runner) and implies a truthy default so --runInBand and runInBand: true should be equivalent. runInBand: false would be identical to omitting --runInBand as a flag. Unless isSerial has some semantic reason to exist which I'm missing, it should be deprecated and its value assigned to runInBand if that's not already the case.

[danielzgtg]

This seems to be available now but undocumented as { "maxWorkers": 1 }.

[trulyronak]

@danielzgtg seems like its gone - it really makes 0 sense at all why they wouldn't surface a cli feature in the config, when there's a clear reason to want to run certain tests in sequential order. Not everything is going to be parallelized, this is ridiculous.

[godrose]

Why is this closed? It's a valid request that's been ignored for several years.

[Aeolun]

Why is this closed? It's a valid request that's been ignored for several years.

Apparently the jest philosophy is that they know what we want better than we do?

I find it incomprehensible that that there are options that exist on the command line, but cannot be passed in the configuration file. It's the first time I've seen it, and I imagine it was actually more work to make it like this than it would be to allow everything in both command line and config.

Would a PR be accepted if I fix this?

[JamesMcGuigan]

The think the design philosophy was to encourage/enforce junior developers to write unit tests in an asynchronous manner and prevent people from complaining that the jest framework is slow because they took the easy way out by setting synchronous as default.

Though as we all become senior developers, we understand that valid edgecases do indeed exist, and the wisdom of "Learn the rules well so you know how to break them properly" becomes more apparent.

[danielzgtg]

Then apparently Jest's philosophy is that they want to make it hard for me to write end-to-end tests, which must not run in parallel for performance reasons

[AvantaR]

The think the design philosophy was to encourage/enforce junior developers to write unit tests in an asynchronous manner and prevent people from complaining that the jest framework is slow because they took the easy way out by setting synchronous as default.

Adding very handy option to config file (which is support in CLI), won't magically push junior developers to turn an asynchronous manner off. Can't understand why it's not supported in configuration file.

[Jaid]

For some reason, --runInBand affects my performance even in a positive way (tested in multiple projects). Combined with the better stability compared to parallel test runs (in some edge cases), there is no reason for me not to use --runInBand.

I would like to add it once in jest.config.json, so I don't have to append --runInBand to every single Jest call in my package.json scripts.

[dtiziani]

On ubuntu with a very decent machine (XPS 13 with 16gb RAM and SSD m2), testing a simple lambda projects (backend) with 10 tests, without --runInBand it freezes completly the machine.

Would be REALLY nice to have this under config file.

[elliot-nelson]

The think the design philosophy was to encourage/enforce junior developers to write unit tests in an asynchronous manner and prevent people from complaining that the jest framework is slow because they took the easy way out by setting synchronous as default.

I had to laugh at this because on every machine I can find, on every project I use Jest, using --runInBand or --maxWorkers=1 crushes the performance of parallel workers. Often it drops the total run time to less than half of the "supposedly better" parallel worker approach.

A config file entry seems just mandatory to me. Even better would be JEST_MAX_WORKERS=1 jest (that way I could just set it in my .zshrc and never deal with this again).

[s0ber]

It's so strange this is not a config option for such a long time. Will make a lot of things so much easier and reliable.

[jensbodal]

I mean it essentially is available, just set workers to 1, it does the same thing (#3215 (comment))

[elliot-nelson]

I mean it essentially is available, just set workers to 1, it does the same thing (#3215 (comment))

Yes, you can set workers to 1, but it's only supported as an immediate CLI parameter, not configuration (i.e. in the config file or as an environment variable). That's the complaint in this ticket.

[jensbodal]

I mean it essentially is available, just set workers to 1, it does the same thing (#3215 (comment))

Yes, you can set workers to 1, but it's only supported as an immediate CLI parameter, not configuration (i.e. in the config file or as an environment variable). That's the complaint in this ticket.

How so? You can just set in your jest config

{  
  maxWorkers: 1
}

And it runs in band. Maybe this wasn’t working before? I just tested again now and it works fine.

[g0di]

I mean it essentially is available, just set workers to 1, it does the same thing (#3215 (comment))

Yes, you can set workers to 1, but it's only supported as an immediate CLI parameter, not configuration (i.e. in the config file or as an environment variable). That's the complaint in this ticket.

How so? You can just set in your jest config

{  
  maxWorkers: 1
}

And it runs in band. Maybe this wasn’t working before? I just tested again now and it works fine.

This does not work if maxWorkers is set at project settings level, see following package.json file as example

{
  "name": "foobar",
  "version": "1.0.0",
  "jest": {
    "projects": [
      {
        "displayName": "e2e",
        "maxWorkers": 1
      }
    ]
  }
}

Moreover, I don't understand why this issue is closed. The runInBand option is not available through options as requested and there is no reason to not add it as any other config option. Did I miss something?

[adrian-gierakowski]

can we reopen this please? Thanks!

[sandijs-private]

Serverless events based Functions testing case makes this issue as a catastrophic one.
If you want code coverage, it all runs in parallel. You cannot test Cloud Functions.
Without code coverage, command line option allows to get single process. You can test Cloud Functions.
Cloud Function tests are asynchronous integration tests with several events chains being launched. Also there are database changes. Complex ones.
If the two tests run in parallel, you ruin the environment and the tests malfunction.
The world has changed.
Tests now must run differently than in year 2000 java programmers blogs.
Now the integration tests are the primary ones, and the unit tests are just a nice add-on.

[agirorn]

Why are maintainers of Jest taking a hard stance against adding the runInBand as a configuration variable?

I think we probably all agree that if you can, run your tests in parallel and having that as the default option is the best way. I also guess most of us try to have them running in parallel and try to have more unit tests than integration tests.

But then reality comes knocking, and you simply have to get stuff done, and you want to have tests, even if they are not the holy grail of tests, you still want to have them, so you got for an integration test, and since your project is maybe setup in a way, that you can't run the tests at the same time, you have to to use the dreaded runInBand. And this is simply the reality of many projects.

Now that you have started using runInBand, you want to make sure all the other developers you are working with can run these tests. The suit in parallels, so you decide to add runInBand: true to the config so that you do not have to hide this fact in the dreadful doc and make life nice for the other developers.

BUT NO: The devs that make the testing tool you're using deem this as BAD practice and want to make this as painful as they can

I'm sorry, but can we have a little more understanding here and try to make this tool support the needs of the projects and developers who are using jest by reopening this issue and hopefully adding the runInBand to the configuration file?

[lordphnx]

you want to make sure all the other developers you are working with can run these tests

In this case, I want a developer to be able to just right-click a test file in their IDE and it runs without hitch. Having a config flag would absolutely be great.