Skip to content

Commit

Permalink
Add toggle autoRun and coverage to TestExplorer context menu (#932)
Browse files Browse the repository at this point in the history
  • Loading branch information
connectdotz authored Nov 4, 2022
1 parent 8ffc9d9 commit d272bad
Show file tree
Hide file tree
Showing 27 changed files with 932 additions and 598 deletions.
61 changes: 52 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,7 @@ Content
- [Menu](#menu)
- [Troubleshooting](#troubleshooting)
- [Jest failed to run](#jest-failed-to-run)
- [Performance issue?](#performance-issue)
- [I don't see "Jest" in the bottom status bar](#i-dont-see-jest-in-the-bottom-status-bar)
- [What to do with "Long Running Tests Warning"](#what-to-do-with-long-running-tests-warning)
- [The tests and status do not match or some tests showing question marks unexpectedly?](#the-tests-and-status-do-not-match-or-some-tests-showing-question-marks-unexpectedly)
Expand Down Expand Up @@ -216,18 +217,22 @@ shows the autoRun will be triggered by either test or source file changes.
</details>

### How to use the Test Explorer?
Users with `vscode` v1.59 and `vscode-jest` v4.1 and up will start to see tests appearing in the test explorer automatically. Test explorer provides a "test-centric" view (vs. "source-centric" view in the editors), allows users to run/debug tests directly from the explorer (in addition to the inline debug codeLens), and provides a native terminal output experience (with colors!):
Users with `vscode` v1.59 and `vscode-jest` v4.1 and up will start to see tests appearing in the test explorer automatically. Test explorer provides a "test-centric" view, allows users to run/debug tests directly from the explorer (in addition to the inline debug codeLens), and provides a native terminal output experience (with colors!):

![test-explorer.png](images/test-explorer.png)
![TestExplorer-5.1.jpg](images/TestExplorer-5.1.jpg)

You can further customize the explorer with [jest.testExplorer](#testexplorer) in [settings](#settings).
<a id='how-to-toggle-auto-run'>**How to toggle autoRun for the workspace?**</a>
- In TestExplorer, click on the root of the test tree, i.e. the one with the workspace name and the current autoRun mode. You will see a list of buttons to its right.
- Click on the [autoRun](#autorun) button (see image above) to toggle it on or off.
- If autoRun is originally on, the button will turn it off and users can use the run menu (in both editor gutter and test explorer tree) to trigger test run(s).
- If the autoRun is originally off, the button will turn it on by restoring to your original autoRun setting, if it is not "off", otherwise it will switch to ["on-save"](#autorun) mode instead.

<a id='how-to-toggle-coverage'>**How to toggle test coverage for the workspace?**</a>
- In TestExplorer, click on the root of the test tree, i.e. the one with the workspace name and the current autoRun mode. You will see a list of buttons to its right.
- Click on the coverage button (see image above) to toggle on or off.
- The next test run (auto or manual) will start reporting test coverage.

However, test explorer is new and some features are still work-in-progress or not available yet:
- can't turn on/off coverage yet (pending on vscode API change)
- not able to accurately indicate run/debug eligibility on the item level, this means you might not be able to run/debug some items through run/debug buttons. (pending on vscode API change)
- the tests stats on the top of the explorer might not be accurate, especially for multiroot workspaces. (pending on vscode fix))
- for watch-mode workspaces, the run button is turned off since tests will be automatically executed.
- debug can only be executed for the test blocks, not on the file or folder level. (Please let us know if you have an use case otherwise)
You can further customize the explorer with [jest.testExplorer](#testexplorer) in [settings](#settings).

### How to see more debug info (self-diagnosis)?

Expand Down Expand Up @@ -337,6 +342,22 @@ for example:
</details>

##### autoRun

Performance and automation/completeness are often a trade-off. autoRun is the tool to fine-tune the balance, which is unique for every project and user.

![autoRun-tradeoff.jpg](images/autoRun-tradeoff.jpg)

Performance and automation are self-explanatory, "completeness" might not:
1. test coverage might not be complete as it only includes the tests that ran.
2. when changing the source or test code, you might not see all the tests broken until you run them explicitly.
3. tests with dynamic names (test.each with variables, template-literals, etc.) will not be translated; therefore, they can only be run through parent blocks (describe-with-static-name or test suite, etc.).

There are 2 ways to change autoRun:
1. Temporarily [toggle autRun on/off in TestExplorer](#how-to-toggle-auto-run)
2. Change "jest.autoRun" in `settings.json` file.

**autoRun Configuration**

```ts
AutoRun =
| "watch" | "off" | "legacy" | "on-save"
Expand Down Expand Up @@ -562,6 +583,28 @@ Sorry you are having trouble with the extension. If your issue did not get resol

There could be other causes, such as jest test root path is different from the project's, which can be fixed by setting [jest.rootPath](#rootPath). Feel free to check out the [customization](#customization) section to manually adjust the extension if needed.

### Performance issue?

The extension should be a thin wrapper on top of the jest process, i.e., it shouldn't use much more resources than the jest process itself.

Having said that, the sluggish performance for some projects/users is real, and we can help. The short answer is [try turning off autoRun in the explorer](#how-to-toggle-auto-run), which should usually show noticeable improvement.

The long answer is a bit more complicated:
- The jest/node/watchman might be slow due to code changes, your test setup, environment, etc. See [facebook/jest#11956](https://github.com/facebook/jest/issues/11956) for a glimpse of such examples. However, this issue should impact with or without this extension. There are many resources and tips online about optimizing jest performance; we will leave it at that.
- Depending on the degree of cross-dependency or your development habit (e.g., save frequently even before the code is complete), the autoRun system ( jest watchman "watch" or "on-save") might decide to run many more tests than you intended to. Imagine adding a single test could trigger 90% of all the tests in the project... yeah we have been there, and it's not fun. If that's you, try [toggling off autoRun in TestExplorer](#how-to-toggle-auto-run) and only trigger test-run when ready with the run button in the gutter or test tree.
- But keep in mind while performance is important, turning autoRun off or setting it to be less "complete" does come with a cost, such as incomplete coverage and missing-broken-tests-detection. Please read up on the [autoRun trade-off](#autorun) and experiment to find the one that works for you.
- Never say never; it is possible that we did something stupid. :cold_sweat: Feel free to log an issue if your performance awe still needs to be resolved after you patiently read and tried the above.

<details>

<summary>fine tune performance with autoRun demo</summary>

https://user-images.githubusercontent.com/891093/199872543-4f37de90-1e56-4e0d-8387-9af591264e13.mov

Every project and developer are different. Experiment and pick the autoRun setting that fits your style and preference!

</details>

### I don't see "Jest" in the bottom status bar
This means the extension is not activated.

Expand Down
Binary file added images/autoRun-tradeoff.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/testExplorer-5.1.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
42 changes: 42 additions & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -279,6 +279,26 @@
{
"command": "io.orta.jest.setup-extension",
"title": "Jest: Setup Extension"
},
{
"command": "io.orta.jest.test-item.auto-run.toggle-on",
"title": "Toggle AutoRun On",
"icon": "$(sync)"
},
{
"command": "io.orta.jest.test-item.auto-run.toggle-off",
"title": "Toggle AutoRun Off",
"icon": "$(sync-ignored)"
},
{
"command": "io.orta.jest.test-item.coverage.toggle-off",
"title": "Toggle Coverage Off",
"icon": "$(circle-slash)"
},
{
"command": "io.orta.jest.test-item.coverage.toggle-on",
"title": "Toggle Coverage On",
"icon": "$(color-mode)"
}
],
"menus": {
Expand Down Expand Up @@ -322,6 +342,28 @@
"command": "io.orta.jest.editor.run-all-tests",
"group": "Jest"
}
],
"testing/item/context": [
{
"command": "io.orta.jest.test-item.auto-run.toggle-off",
"group": "inline",
"when": "testId in jest.autoRun.on"
},
{
"command": "io.orta.jest.test-item.auto-run.toggle-on",
"group": "inline",
"when": "testId in jest.autoRun.off"
},
{
"command": "io.orta.jest.test-item.coverage.toggle-off",
"group": "inline",
"when": "testId in jest.coverage.on"
},
{
"command": "io.orta.jest.test-item.coverage.toggle-on",
"group": "inline",
"when": "testId in jest.coverage.off"
}
]
},
"keybindings": [
Expand Down
119 changes: 119 additions & 0 deletions src/JestExt/auto-run.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
import * as vscode from 'vscode';
import {
JestExtAutoRunConfig,
JestExtAutoRunSetting,
JestExtAutoRunShortHand,
OnSaveFileType,
OnStartupType,
} from '../Settings';
import { AutoRunMode } from '../StatusBar';

export class AutoRun {
private useOnConfig: boolean;
private readonly onConfig: JestExtAutoRunConfig;
private readonly offConfig: JestExtAutoRunConfig;

constructor(
setting: JestExtAutoRunSetting | null | undefined,
autoEnable?: boolean,
runAllTestsFirst?: boolean
) {
let config: JestExtAutoRunConfig;
if (!setting) {
config =
this.autoRunFromLegacySettings(autoEnable, runAllTestsFirst) ?? this.toAutoRun('default');
} else if (typeof setting === 'string') {
config = this.toAutoRun(setting);
} else {
config = setting;
}
if (this.isConfigOff(config)) {
this.offConfig = config;
this.useOnConfig = false;
// different from "default" (which is "watch") ? If user specifically set "off"
// and want to turn on at runtime, we choose the one setting closest (in turns of trade-off metric)
// to the original setting.
this.onConfig = this.toAutoRun('on-save');
} else {
this.onConfig = config;
this.useOnConfig = true;
this.offConfig = this.toAutoRun('off');
}
}

public get config(): JestExtAutoRunConfig {
return this.useOnConfig ? this.onConfig : this.offConfig;
}
public get isOff(): boolean {
return this.isConfigOff(this.config);
}
private isConfigOff(config: JestExtAutoRunConfig): boolean {
return config.watch === false && config.onSave == null && config.onStartup == null;
}
public get isWatch(): boolean {
return this.config.watch === true;
}
public get onSave(): OnSaveFileType | undefined {
return this.config.watch === false ? this.config.onSave : undefined;
}
public get onStartup(): OnStartupType | undefined {
return this.config.onStartup;
}

public get mode(): AutoRunMode {
return this.autoRunMode();
}
public toggle(): void {
this.useOnConfig = !this.useOnConfig;
}

private autoRunMode(): AutoRunMode {
if (this.config.watch === false && !this.config.onSave && !this.config.onStartup) {
return 'auto-run-off';
}
if (this.config.watch === true) {
return 'auto-run-watch';
}
if (this.config.onSave === 'test-src-file') {
return 'auto-run-on-save';
}
if (this.config.onSave === 'test-file') {
return 'auto-run-on-save-test';
}
return 'auto-run-off';
}

/**
* create a backward compatible runMode from the the legacy settings
*/
private autoRunFromLegacySettings(
autoEnable?: boolean,
runAllTestsFirst?: boolean
): JestExtAutoRunConfig | undefined {
if (autoEnable === false) {
return this.toAutoRun('off');
}
if (runAllTestsFirst === true) {
return this.toAutoRun('legacy');
}
}
private toAutoRun(shortHand: JestExtAutoRunShortHand): JestExtAutoRunConfig {
switch (shortHand) {
case 'legacy':
return { watch: true, onStartup: ['all-tests'] };
case 'default':
case 'watch':
return { watch: true };
case 'off':
return { watch: false };
case 'on-save':
return { watch: false, onSave: 'test-src-file' };
default: {
const message = `invalid autoRun setting "${shortHand}". Will use default setting instead`;
console.error(message);
vscode.window.showErrorMessage(message);
return this.toAutoRun('default');
}
}
}
}
14 changes: 10 additions & 4 deletions src/JestExt/core.ts
Original file line number Diff line number Diff line change
Expand Up @@ -514,13 +514,13 @@ export class JestExt {
}

private handleOnSaveRun(document: vscode.TextDocument): void {
if (!this.isSupportedDocument(document) || this.extContext.autoRun.isWatch) {
if (!this.isSupportedDocument(document) || this.extContext.settings.autoRun.isWatch) {
return;
}
const isTestFile = this.testResultProvider.isTestFile(document.fileName);
if (
this.extContext.autoRun.onSave &&
(this.extContext.autoRun.onSave === 'test-src-file' || isTestFile !== 'no')
this.extContext.settings.autoRun.onSave &&
(this.extContext.settings.autoRun.onSave === 'test-src-file' || isTestFile !== 'no')
) {
this.processSession.scheduleProcess({
type: 'by-file',
Expand Down Expand Up @@ -618,6 +618,12 @@ export class JestExt {
// restart jest since coverage condition has changed
this.triggerUpdateSettings(this.extContext.settings);
}
toggleAutoRun(): void {
this.extContext.settings.autoRun.toggle();

// restart jest since coverage condition has changed
this.triggerUpdateSettings(this.extContext.settings);
}

private setupStatusBar(): void {
this.updateStatusBar({ state: 'initial' });
Expand All @@ -628,7 +634,7 @@ export class JestExt {
if (this.coverageOverlay.enabled) {
modes.push('coverage');
}
modes.push(this.extContext.autoRun.mode);
modes.push(this.extContext.settings.autoRun.mode);

this.updateStatusBar({ state: 'initial', mode: modes, stats: emptyTestStats() });
}
Expand Down
Loading

0 comments on commit d272bad

Please sign in to comment.