Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[utils] [perf] Performance of fullResolve
While looking at a larger code base https://gitlab.com/gitlab-org/gitlab, I've came to realize that `fullResolve` takes a lot of CPU cycles, particularly the `hashObject` calls inside it. I applied the following patch locally to see how often this is called and how many unique hashes were produced: ```diff diff --git a/utils/resolve.js b/utils/resolve.js index 4a35c6a..3c28324 100644 --- a/utils/resolve.js +++ b/utils/resolve.js @@ -83,13 +83,28 @@ function relative(modulePath, sourceFile, settings) { return fullResolve(modulePath, sourceFile, settings).path; } +let prevSettings = null; +let nonEqualSettings = 0; +let totalCalls = 0; +let uniqueHashes = new Set(); function fullResolve(modulePath, sourceFile, settings) { // check if this is a bonus core module const coreSet = new Set(settings['import/core-modules']); if (coreSet.has(modulePath)) return { found: true, path: null }; const sourceDir = path.dirname(sourceFile); - const cacheKey = sourceDir + hashObject(settings).digest('hex') + modulePath; + + totalCalls+=1; + const hash = hashObject(settings).digest('hex'); + + if(prevSettings !== settings){ + uniqueHashes.add(hash); + prevSettings = settings; + nonEqualSettings+=1; + console.log(`fullResolve | Total calls:${totalCalls} | Non-Equal settings:${nonEqualSettings} | Unique hashes:${uniqueHashes.size} | dir:${sourceDir}`) + } + + const cacheKey = sourceDir + hash + modulePath; const cacheSettings = ModuleCache.getSettings(settings); ``` For our code base, `fullResolve` is called more than 570 thousand times. The simple in-equality `!==` code path is taken 1090 times. Actually only _four_ unique hashes are produced, meaning we only have four unique settings across our code base. I assume that a full object equality comparison might not be needed, and a simple object comparison with `!==` already would reduce the amount of `hashObject` calls by 570x. This is what is implemented in this commit. Time spend in `fullResolve` was reduced by ~38%: - Before: 17% (19.10s) of our total execution time - After: 11% (11.86s) of our total execution time The effect might even be more pronounced on machines that are slower when calculating `sha256` hashes or that have less memory, as the `hashObject` method tends to create loads of small strings which need to be garbage collected.
- Loading branch information