[Bug]: Upgrading jest-preset-angular to 11.* makes it unusable

[yadue]

Version

11.1.0

Steps to reproduce

We have quite large app, after upgrading to 11.1.0 our tests increased by 2x in time only with jest-preset-angular upgrade.
I suspect that using lodash-es, lodash-decorators-esm, date-fns, @UPPY makes it longer with the latest version. I already tried several approaches:

• align our configuration to v11 changes
• using v11 with angular 12
• running on clean configurations taken from exaples/ dir

The result is almost always the same, run time increased 2x on ci environments but even worse on my local machine. With v10 warmup took more or less 15-20 seconds, with v11 warmup takes more than 140 - 200 seconds.

At this point using library v11 is complete blocker and blocks us from upgrading to angular 13.

Expected behavior

Performance should stay as it was in v10

Actual behavior

v11 increases built time more than 2x.

Additional context

Our tests with
Test Suites: 1 skipped, 349 passed, 349 of 350 total
Tests: 3 skipped, 1247 passed, 1250 total

v10.1 with clean cache:
Time: 181.311 s

With cache:
Time: 84.668 s

v11.1.0
Time: 421.699 s

With cache:
Time: 132.581 s

require('jest-preset-angular/ngcc-jest-processor');
const { pathsToModuleNameMapper } = require('ts-jest');
const { compilerOptions } = require('./tsconfig.json');

process.env.TZ = 'Europe/Warsaw';

module.exports = {
    preset: 'jest-preset-angular',
    testMatch: ['**/*.spec.ts'],
    maxWorkers: '50%',
    setupFilesAfterEnv: ['<rootDir>/jest.setup.ts'],
    moduleNameMapper: {
        '^lodash-es/(.*)$': 'lodash/$1',
        ...pathsToModuleNameMapper(compilerOptions.paths, { prefix: '<rootDir>/src/' }),
    },
    testResultsProcessor: './node_modules/jest-junit-reporter',
    collectCoverageFrom: ['src/**/*.ts'],
    coverageReporters: ['json', 'html'],
    transformIgnorePatterns: ['<rootDir>/node_modules/(?!lodash-es|lodash-decorators-esm|date-fns|@uppy/utils)'],
    globals: {
        'ts-jest': {
            stringifyContentPathRegex: '\\.(html|svg)$',
            isolatedModules: true,
            tsconfig: './tsconfig.spec.json',
        },
    },
};

Environment

System:
    MacOS 12.1
Binaries:
    Node 14.15.3
    NPM 6.14.9
Packages:
    Jest: 27.4.7
    @ngneat/spectator 7.2.0
    Angular 12.2.14 && Angular 13

[ahnpnl]

This is odd, we don’t change anything about compilation except using esbuild to process mjs files. Maybe you can try this to see if it helps #1309

a note is that, upgrading deps will cause old Jest cache to be invalid so the next run Jest will need to cache again everything.

[yadue]

@ahnpnl no change for me with this pr, also please validate the code, imo it never gets inside condition.

[yadue]

More details about our issue, it's about js files processing, more specifically about
https://github.com/thymikee/jest-preset-angular/blob/main/src/ng-jest-transformer.ts#L62
it transforms also all js files, previously it didnt. Comment says it should be faster but in our case it's not. From what I see it even doesn't check if the file has been previously processed so during our runtime I see the same files being processed over and over.

Now imagine you have 5 workers, every single js file from lodash-es is being processed 5 times by jest-preset-angular then 5 times by ts-jest. This makes process way longer.

[ahnpnl]

Thanks for the information, it makes sense to me. Would you please use https://github.com/thymikee/jest-preset-angular/tree/main/examples to make a reproduce case?

[yadue]

@ahnpnl Here's the repository:
https://github.com/yadue/jest-preset-angular-issue
I think its pretty simple, copy more and more components so initial warmup takes longer and longer

And I think I found true problem:
const configSet = this._createConfigSet(transformOptions.config);
regenerates with each file, also affects overall performance greatly, it could be cached as its not required to do this over and over. after changing locally results are:

==== original ng-jest-transformer
Time: 425.231 s

With cache:
Time: 99.716 s, estimated 411 s

==== configSet cached
Time: 132.717 s
Time: 134.964 s

With cache:
Time: 97.741 s

=== configSet cached, without parsing js files with esbuild
Time: 149.353 s
Time: 141.527 s

With cache:
Time: 96.139 s

So esbuild makes everything faster so the condition should stay as it was.

let configSet;
export class NgJestTransformer extends TsJestTransformer {
...
configSet = configSet ? configSet : this._createConfigSet(transformOptions.config);

fixes the issue

[ahnpnl]

the fix looks a bit similar to #1309 indeed. Feel free to open a PR :)

[seth-broweleit]

I just updated all my angular packages from 12.2.3 to 12.2.13. When I ran test I got this

    > jest --config ./src/jest.config.js     
    Error: Cannot find module 'esbuild'
    Require stack:
    - C:\code\EDM_DIG_UI\node_modules\jest-preset-angular\build\ng-jest-transformer.js
    - C:\code\EDM_DIG_UI\node_modules\jest-preset-angular\build\index.js
    - C:\code\EDM_DIG_UI\node_modules\@jest\transform\node_modules\jest-util\build\requireOrImportModule.js
    - C:\code\EDM_DIG_UI\node_modules\@jest\transform\node_modules\jest-util\build\index.js
    - C:\code\EDM_DIG_UI\node_modules\@jest\transform\build\ScriptTransformer.js
    - C:\code\EDM_DIG_UI\node_modules\@jest\transform\build\index.js
    - C:\code\EDM_DIG_UI\node_modules\jest-runtime\build\index.js
    - C:\code\EDM_DIG_UI\node_modules\@jest\core\build\cli\index.js
    - C:\code\EDM_DIG_UI\node_modules\@jest\core\build\jest.js
    - C:\code\EDM_DIG_UI\node_modules\jest\node_modules\jest-cli\build\cli\index.js
    - C:\code\EDM_DIG_UI\node_modules\jest\node_modules\jest-cli\build\index.js
    - C:\code\EDM_DIG_UI\node_modules\jest\node_modules\jest-cli\bin\jest.js
    - C:\code\EDM_DIG_UI\node_modules\jest\bin\jest.js
        at Function.Module._resolveFilename (internal/modules/cjs/loader.js:902:15)
        at Function.Module._load (internal/modules/cjs/loader.js:746:27)
        at Module.require (internal/modules/cjs/loader.js:974:19)
        at require (internal/modules/cjs/helpers.js:92:18)
        at new NgJestTransformer (C:\code\EDM_DIG_UI\node_modules\jest-preset-angular\build\ng-jest-transformer.js:37:102)
        at Object.createTransformer (C:\code\EDM_DIG_UI\node_modules\jest-preset-angular\build\index.js:5:30)
        at C:\code\EDM_DIG_UI\node_modules\@jest\transform\build\ScriptTransformer.js:395:39
        at processTicksAndRejections (internal/process/task_queues.js:95:5)
        at async Promise.all (index 0)
        at async ScriptTransformer.loadTransformers (C:\code\EDM_DIG_UI\node_modules\@jest\transform\build\ScriptTransformer.js:379:5)

Then I updated jest-preset-angular to latest and it didn't fix it.