Skip to content

Commit

Permalink
Add more details for matching against the baseline, and keeping the b…
Browse files Browse the repository at this point in the history
…aseline lean
  • Loading branch information
softius committed Aug 1, 2024
1 parent 4462ce6 commit 1a1cbba
Showing 1 changed file with 18 additions and 1 deletion.
19 changes: 18 additions & 1 deletion designs/2024-baseline-support/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,14 +77,31 @@ The suggested solution always compares against the baseline, given that the base

This will go through each result item and message, and check each rule giving an error (`severity == 2`) against the baseline. By design, we do not take warnings into consideration (regardless of whether quite mode is enabled or not), since warnings do not cause eslint to exit with an error code and they already serve a different purpose. If the file and rule are part of the baseline, means that we can remove and ignore the result message.

To implement this, we will need to adjust further `cli.js` and check if the baseline file exists - taking `--baseline-location` into consideration if exists, otherwise fallback to `.eslint-baseline.json`. This needs to take place right after the baseline is generated so that we take the baseline into consideration, if it was just generated. It also needs to take place before the error counting, so that the remaining errors are counted correctly.
To implement this, we will need to adjust further `cli.js` to adopt the following operations:

* Check if the `--baseline` option is passed
> * If yes, we need to generate the baseline based on `resultsToPrint`.
> * If no, we need to check if the baseline file already exists taking `--baseline-location` into consideration
* Assuming that a baseline was found or generated, we need to match the errors found against the baseline. In particular, for each error found:
> * We need to check whether both file and error are part of the baseline
> * If yes, we reduce the `count` by 1 and ignore the current error. If `count` has already reach zero we keep the error.
> * If no, we keep the error.
Note that `cli.js` the error detection in `cli.js` happens quite earlier before the error counting. This allow us to create the baseline, before it is time to count errors. Please refer to the last example of the "Implementation details".

We can also keep track of which errors from baseline were not matched, that is useful for the next section.

### Maintaining a lean baseline

When using the baseline, there is a chance that an error is resolved but the baseline file is not updated. This might allow new errors to creep in without noticing. To ensure that the baseline is always up to date, eslint can exit with an error code when there are ignored errors that do not occur anymore. To fix this, the developer can regenerate the baseline file.

Consider the following scenario:

* The developer executes `eslint --baseline ./src` which updates the baseline file.
* Then `eslint ./src` is executed which gives no violations, with an exit status of 0.
* The developer addresses an error violation. While the error is addressed is still part of the baseline.
* The developer then executes `eslint ./src` again. While it still gives no violations, it exits with a non-zero status code. That is to indicate that the baseline needs to be re-generated by executing `eslint --baseline ./src`.

To implement this, we will need to extend `applyBaseline` to return the unmatched rules. In particular, we will need to check if one or more rules are left unmatched, and exit with an error code. Depending on the verbose mode we can display the list of errors that were left unmatched.

### ESLint Cache
Expand Down

0 comments on commit 1a1cbba

Please sign in to comment.