Hash does not change when only imported modules IDs change

[Poky85]

Webpack references modules with number IDs. Seems that when only module IDs change then bundle's MD5 hash does not change. Look at example. Two builds of the same Webpack chunk. The only difference are IDs of imported modules. Despite this the hashes are same.

https://gist.github.com/Poky85/d8fe2d4711d532fbb3ea25a5efbbf992
https://gist.github.com/Poky85/e441799fdf314684d3eb41744a2ba4c6

Steps to reproduce:

• There are 2 Webpack chunks. For example vendor.js and app.js.
• You can achieve this with Webpack CommonsChunkPlugin (https://webpack.github.io/docs/list-of-plugins.html#commonschunkplugin)
• Modules imported in vendor.js are required in app.js
• App.js changes. As result of this module IDs in vendor.js will also change (IDs are determined by occurence count if you use OccurrenceOrderPlugin (https://webpack.github.io/docs/list-of-plugins.html#occurrenceorderplugin)
• But vendor.js chunk keeps the same hash.

webpack.config:

`
var webpack = require("webpack");
var WebpackMd5HashPlugin = require('webpack-md5-hash');
var config = {

entry: {
    "bundle/app": ["./src/app.js"],
    "bundle/vendor": ["./src/vendor.js"]
},

output: {
    path: "./www/assets",
    publicPath: "/assets/",
    filename: "[name].[chunkhash].js",
    chunkFilename: "[name].[chunkhash].js"
},

plugins: [
    new WebpackMd5HashPlugin(),
    new webpack.optimize.OccurenceOrderPlugin(),
    new webpack.optimize.CommonsChunkPlugin({
        name: "bundle/vendor"
    })
],
bail: true

};

module.exports = config;
`

[Jyrno42]

I think this will solve your issue: facebook/create-react-app#210 (comment)

E.g. Use HashedModuleIdsPlugin to keep module ids consistent across builds and remove OccurenceOrderPlugin.

[jampy]

Where to get HashedModuleIdsPlugin from? Currently published Webpack version 1.13.3 doesn't seem to include it.

[alexindigo]

@jampy you can find examples here: https://github.com/webpack/webpack/blob/54aa3cd0d6167943713491fd5e1110b777336be6/test/configCases/hash-length/hashed-module-ids/webpack.config.js

[alexindigo]

But looks like it doesn't exist in 1.0 :(

[jeromecovington]

@Poky85 I'm seeing this issue in my own codebase as well, and want to verify what is causing the behavior, so that I can demonstrate that @alexindigo's solution solves this for me. I am attempting to follow the "steps to reproduce" in the issue description, but this phrase is a little tricky to me: "Modules imported in vendor.js are required in app.js". import and require seem to be nearly synonymous actions, so I'm not seeing how to make a change to my bundle code to reproduce a changed module id. Perhaps a sample of changed code that results in the changed module ids would help?

[alexindigo]

@jeromecovington As I understood it, you just have "relations" between chunks. (import gets transpiled into require in the end).

Try following, you have your main chunk (and using CommonChunk and OccurenceOrder plugins), so your get async chunks as well, like 1 or 2. Then you add another file to the whole setup, which will be included in the main (app) chunk. (you need to experiment where you require this new file) so it would push up occurence order for files in your async chunks.

I hope I made it clear, rather than more confusing :)

[jeromecovington]

I am also using the CommonsChunkPlugin, so any shared import-ed modules end up getting bundled as require statements in the common bundle. And any resulting additional require statements in common do seem to cause a chunkhash update there. However, I have seen changes in an entry point bundle (not common) related only to updated module ids not resulting in a chunkhash change for said entry point bundle. I'm having a hard time reproducing this behavior, but I know it has been a problem for me.

[alexindigo]

For me entry point chunk changes the most, but other chunks this is what getting ids update. What you can try, is to find require (import) statement for the modules that end up in async chunk and insert another require before that, so it would change occurrence order for modules in async chunk.

Or another way would be to introduce another library in the vendor chunk, that only referenced from within vendor chunk, it won't change content of the main (entry point) chunk, but will change the ids.

[jeromecovington]

Great, thanks @alexindigo I have been able to reproduce the behavior and will now be on track to verify the solution you published on npm.

[alexindigo]

Cool, thanks. Let me know how it's working out for you.

[jeromecovington]

Yep, @alexindigo seems to be working out well. Thanks for taking the time to publish on npm and answer my questions!

[Poky85]

I have created simple demo to test different hashing algorithms. See webpack/webpack#1856 (comment)

[jeromecovington]

Hm. I seem to be running into this issue again, using webpack-chunk-hash where it seems that the chunkhash digest is being produced prior to the module id insertion. In this case, I have an entry point where the ids are less by one (due to the removal of a module), yet its chunkhash is identical to the one produced for the entry point prior to the removal of the module. The change is not evident across multiple entry points, or in the common file, as the module removal affects only the single entry point in question. @alexindigo I was going to add this as an issue to https://github.com/alexindigo/webpack-chunk-hash but I don't see any place to add issues there?

[jeromecovington]

Ah, I see you added ids back into the hashing mechanism. I will take the update to your plugin and let you know how it goes, @alexindigo.

[jeromecovington]

Yep, taking the update seems to have sorted the ids changing, I am getting new chunkhash values now. Thanks.

[holyxiaoxin]

Try this:

const spawn = require( 'child_process' ).spawnSync
const ls = spawn( 'git', [ 'rev-parse', 'HEAD' ] );
const hashFromGitCommit = ls.stdout.toString().trim();

filename: `[name].${hashFromGitCommit}.js`,

[Munter]

@holyxiaoxin sadly that is not enough. When you change a file name you also need to change any reference to the file. The correct order to do this in to avoid wrong file caching is to traverse the dependency graph in a depth first post order traversal, where you move the file and update any reference to the file. This ensures that when a file changes its content, any file pointing to it will also be cache-busted with new content, because it contains the new name of the dependency

Any other consistent hashing algorithm is flawed, which is also why all other projects that try to solve this without employing a dependency graph have so many bugs, most of which they can't fix because of their architecture

[holyxiaoxin]

@Munter I might argue that the current hashing algorithm is too strict because of hash base on file path too. If you have 2 similar content in different locations, and would like them to have the same hash, you would have a problem. I would probably just write a simple plugin to do a system hash on the file content itself, maybe the name too(but it's not important for me).

[matpaul]

@alexindigo Good day,sorry but i can't find how to post issue in your repo -
So l use your plugin webpack-chunk-hash, also use HashNameslds and everything is ok but today i found issue when chunk ids changed hash not change(
l mean webpackJsonp([10,9]... change to webpackJsonp([11,8] and hash was the same(
Do you know how to fix this issue?

[alexindigo]

@matpaul Thanks for point it out. Looks like issues are off by default for forked projects. I tuned it on.
Please feel free to create issues here: https://github.com/alexindigo/webpack-chunk-hash/issues

As for the question. Do you mind to provide a bit of code to experiment with it.

Without looking at the code/config, I can assume that there are some parts that aren't exposed to the plugins. But I'd like to see more, to understand the problem.

Thanks.

[Thinking80s]

https://sebastianblade.com/using-webpack-to-achieve-long-term-cache/

[whq731]

To generate stable module ids For webpack 1.x version, we can use this plugin https://github.com/webpack/webpack/blob/master/lib/HashedModuleIdsPlugin.js
which has included in webpack 2