Cold build takes upwards of 50 minutes, high transformTime

[JakeFDev]

TypeScript Version: 3.8.3, however I have tried 3.3.x as well as the latest 3.9.x and even updating the the rc version of 4.0

Search Terms:
Fresh/cold build takes upwards of 50 minutes, both watch and regular build mode. Very high transform time.

Code
This is a private repo I started contributing too so I can't share it, but this is our tsconfig.

{
    "compilerOptions": {
      "declaration": true,
      "importHelpers": true,
      "module": "commonjs",
      "esModuleInterop": true,
      "outDir": "dist",
      "rootDir": "src",
      "strict": false,
      "target": "esnext",
      "forceConsistentCasingInFileNames": true,
      "allowJs": true,
      "checkJs": false,
      "lib": ["esnext"],
      "moduleResolution": "node",
      "incremental": true,
      "noImplicitAny": false,
      "assumeChangesOnlyAffectDirectDependencies": true,
      "listEmittedFiles": true,
      "extendedDiagnostics": true
    },
    "include": [
      "src/**/*"
    ],
    "exclude": [
      "node_modules/**/*",
      "**/node_modules",
      "dist/**/*",
      "**/.*/",
      "*.test.ts"
    ]
}

Expected behavior:
Running a fresh build (without a .buildinfofile and the current dist folder if one exists) should compile in less time than it takes to do my laundry

Actual behavior:
Running a fresh build takes upwards of 1 hour, although I have seen a range of 35-50 minutes that is most common.

Related Issues:
#35729

[JakeFDev]

Attached is the result when running with the extended diagnostics option.

[orta]

Hi, this is a little hard to diagnose without access to the code - have you looked through the emitted JS/DTS files to see if anything is spectacularly large like in the issue you linked?

[DanielRosenwasser]

Here's a few other questions:

1. Do you see the same behavior if you turn --declaration off? This might help us understand whether it's an issue in the declaration emitter.
2. Do you see the same behavior if you run with --isolatedModules? This further helps us understand the use of semantic information at emit-time.
3. To add to what @orta asked, what if you try compiling individual files? Try running tsc --noResolve file.ts on the 5-10 largest files to see if we can isolate the issues.

@amcasey might also have some stuff in the works to help us understand where time is being spent in the compiler.

[DanielRosenwasser]

Also, I've heard that increasing Node's max heap size has helped others with transform-time speed issues so that garbage collection doesn't get triggered as often, which is at least useful as a workaround for you.

node --max-old-space-size=8192 node_modules/typescript/lib/tsc.js # increase to 8gb

[amcasey]

@DanielRosenwasser No harm in trying, but the memory usage doesn't look excessive and the check time is reasonable, so I don't guess --max-old-space-size will help.

Regarding tooling, I've been focused on check time, but it should be easy enough to put together a private build that can say something useful about where this time is going.

@JakeFDev This is a new one for me - most users report slowdowns in module resolution or check time. It also seems strange to me that we could spend so much time on emit and so little on file I/O.

[JakeFDev]

@orta I don't see anything big sticking out. The entire output folder of the build is ~1.7MB which doesn't seem out of question. Looking through the files everything is 500bytes up to 22KB.

Going to try running the suggestions, i'll report back.

[JakeFDev]

@amcasey using your build I discovered some weird things in some of our legacy code (thats in js)

The file in question exports its module like so

export const Visit = mongoose.model('Visit', VisitSchema);
module.exports = exports = Visit;

Only the legacy, old code exports this way.

That takes 37 seconds according to the transform debug build.

I changed it to

export const Visit = mongoose.model('Visit', VisitSchema);
export default Visit

And now the same file took 22ms. I will need to basically update every file that is using this approach and will update as I go if this is the problem.

[JakeFDev]

That was it. Cleaned up all those instances, down to a 15 second fresh build time.

Thanks everyone for the help!

[amcasey]

@JakeFDev That's great! Any chance you can anonymize one of your old files and share it as an example? I'd like to dig in and figure out why the transformation takes so long.

[JakeFDev]

@amcasey this file is pretty simple.

As is, this is the one that took 37 seconds before the change

const mongoose = require('mongoose');
const timestamps = require('mongoose-timestamp');
const beautifyUnique = require('mongoose-beautiful-unique-validation');

const VisitSchema = new mongoose.Schema({

        trackingId: {
            type: String,
            trim: true,
            required: true
        },
        date: {
            type: String,
            trim: true,
        },
        count: {
            type: Number,
            required: true,
            trim: true
        },
        group_id: {
            type: String,
            trim: true
        }
},{
    collection: 'visit',
    timestamps: { createdAt: 'created_at', updatedAt: 'updated_at' }
});

//BatchSchema.plugin(bcrypt);
// VisitSchema.plugin(timestamps);

// Enable beautifying on this schema
VisitSchema.plugin(beautifyUnique);

export const Visit = mongoose.model('Visit', VisitSchema);
module.exports = exports = Visit;

[amcasey]

@JakeFDev Thanks! Could you possibly share your tsconfig as well? I used some safe defaults, but mine only takes 2 seconds (in nightly).

Edit: I'm an idiot - it's at the top.

Edit 2: Still nothing.

transformTime time:         0.01s
commentTime time:           0.00s
printTime time:             0.04s
Emit time:                  0.04s
I/O Write time:             0.00s
Total time:                 1.28s

[amcasey]

@JakeFDev Are you using tsc directly or going through another tool?

[JakeFDev]

@amcasey we are running it as part of a script in our package.json

"build": "yarn run build-ts",
"build-ts": "tsc",
...

[amcasey]

Still nothing. 😢 Given that you're unblocked, we might have to let this go until we hear other reports. Thanks again!