Add support for md4 in Node >=18.

[iclanton]

Summary

🤖 Generated by Copilot at 6f6ae98

This pull request adds WebAssembly-based md4 hashing for Node.js 18 or higher, and implements a BatchedHash class that improves the performance of hashing large data. It also refactors the createHash function and removes unused code from the createHash.js module. The changes involve adding new modules lib/util/hash/BatchedHash.js, lib/util/hash/md4.js, and lib/util/hash/wasm-hash.js, and modifying lib/util/createHash.js.

Details

🤖 Generated by Copilot at 6f6ae98

• Remove Hash class and HashConstructor type exports from createHash.js module (link)
• Add variables to lazily require crypto, md4, and BatchedHash modules in createHash.js module (link)
• Modify createHash function to handle md4 and native-md4 algorithms differently depending on Node.js version and use BatchedHash class for md4 WebAssembly hash in createHash.js module (link)
• Add BatchedHash class that extends Hash class and batches data chunks for underlying hash implementation in BatchedHash.js module (link)
• Add md4 function that creates WasmHash instance using md4 WebAssembly module and md4 instances pool in md4.js module (link)
• Add WasmHash class that implements hash interface using WebAssembly module, memory buffer, chunk size, and digest size in wasm-hash.js module (link)
• Add create function that creates WasmHash instance using WebAssembly module, instances pool, chunk size, and digest size in wasm-hash.js module (link)
• Export MAX_SHORT_STRING constant from wasm-hash.js module and use it in BatchedHash class (link, link)

[linux-foundation-easycla]

The committers listed above are authorized under a signed CLA.

• ✅ login: iclanton / name: Ian Clanton-Thuon (6f6ae98, 9b50972)

[webpack-bot]

For maintainers only:

• This needs to be documented (issue in webpack/webpack.js.org will be filed when merged)
• This needs to be backported to webpack 4 (issue will be created when merged)

[snitin315]

Why not upgrade the webpack version to v5?

[iclanton]

@snitin315 - There are a number of breaking changes between Webpack 4 and 5, making upgrading large and complex projects time-consuming.

Webpack 4 currently has many more weekly downloads on npmjs.org than Webpack 5 does, but Webpack 4 only works with Node 16 and earlier because of the issue being addressed in this PR. Node 16 is EOL on 2023-09-11 (next Monday).

[snitin315]

@iclanton Shouldn't webpack upgrade be considered as the migration effort for your app to Node v18? You can anyway use NODE_OPTIONS=--openssl-legacy-provider when using webpack v4 with Node.js v17+, and put a TODO to upgrade webpack v5 later. CMIIW, here we are using this function just to create a hash in filenames, so there's no such security threat.

[TheLarkInn]

@alexander-akait @snitin315 and myself discussed this further. Because Node 16 EOL is next week, we do not want our ~12 million webpack 4 consumers to be stuck with what will be flagged as a security vulnerability not updating Node. It would be ideal not to block them here.

[TheLarkInn]

Thank you for this contribution.

[chadlwilson]

Thanks so much for this @iclanton . I see you've been doing some work around the ecosystem to help with this. Is terser-webpack-plugin on your radar? I opened webpack-contrib/terser-webpack-plugin#578 to see if the maintainers are interested in addressing a similar issue there. (edit: but this may be more to do with the transitive dependency to terser ^1.4.3 within webpack 4 itself.)

webpack/package.json

Line 28 in dfffd6a

"terser-webpack-plugin": "^1.4.3",

[Shubhanshu88]

@chadlwilson I guess this PR by iclanton : #17659 is for the terser-webpack-plugin bump

[chadlwilson]

I totally missed that, thanks @Shubhanshu88 - I feel stupid now!