-
Notifications
You must be signed in to change notification settings - Fork 2.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Webpack incremental builds are slow #616
Comments
I'm able to drop the rebuild time significantly without bootstrap/font-awesome loaders in the |
Sounds great! |
I'm getting slow build time as well. Looking forward to your split set up. |
I am the same. it happened looks like webpack-hot-middleware didn't work. |
@quicksnap if you set your |
If I'm not mistaken, PR #614 upgraded |
Dupes #594 |
Weird.. I thought I commented a reply to @bdefore: Modifying or even removing the devtool doesn't do much for me. In my case, it's nearly all time due to the It would be nice to understand exactly what is going on. I tried digging into the plugin itself, but any modifications I made would not yield positive results, other than causing it not to generate the SASS file which I have a feeling it's not properly caching due to its usage of a Personally, I may just remove bootstrap loader and use a static build of bootstrap. I'm not sure the best route for this project. Would love it if someone could see why |
Perhaps it's simply that bootstrap generates a ton of CSS, and I will try bypassing |
I don't know much on bootstrap-loader, but saw this since i got tagged on it. Its possible what's below is irrelevant, but I'll pitch it in anyway. Though it sounds like its just the loader not caching, as as mentioned. I dont know what boostrap loader does, but do keep in mind that SASS doesn't get incrementally compiled the same way JS does. If you require(...) SASS, it creates a css "entry point" so to speak, but if inside that SASS, you @import other stuff, that's all part of the same "sass build". Sass builds are not incremental, and all SASS within a single "entry point" will get recompiled every time. node-sass <3.4 is slow, and css-loader after 0.14.5 is BRUTALLY slow (to the point of being unusable). Since all of your CSS may get recompiled if its all in the same CSS entry point, you can quickly end up in CSS builds that take longer than rebuilding your entire JS from scratch, even though you only added 1 character to 1 CSS file. Note that splitting your CSS by requiring it from JS isn't always a good answer in SASS because of the lack of @import (reference) like LESS does...you may end up duplicating a lot of stuff if you're not careful. tl;dr: CSS builds using the @import mechanic are not incremental and cause a full rebuild of that part. You probably want a static bootstrap build, and @import only variables and mixins as need be (but nothing else!) |
@Phoenixmatrix thanks. The problem with this setup is that whenever we change any of our JS files, it triggers an entire rebuild, including the CSS. I could not figure a way to tell webpack not to rebuild I tried splitting out into a different chunk, but no luck. My webpack chops are mediocre. I'm tooling around with |
Yeah, i realized that afterward rereading your comment. Whoops! For sure upgrading to node 3.4 should speed that up since compiling bootstrap shouldn't take very long at all, but obviously that only fixes symptoms (did you know node-sass is like 2-3 times faster on Linux than MacOSX? fun stuff). Will take a look at the bootstrap sass loader when I get the chance. |
Thanks for helping out! Appreciate it. Be sure to change the |
Ok! I just changed This significantly speeds things up for me. |
I think it still would be an even better improvement if we could get webpack to avoid rebuilding bootstrap unless the bootstrap config file changes. |
Yeah, so obviously that part is still broken. But also, yet another reason In our codebase its unusable (css build takes minutes, so we have to use |
I hope it is improved by the time our css grows. I really like CSS Modules... |
@quicksnap seems like you're suggesting to avoid bootstrap-sass-loader? That seems to be the bottleneck? |
i had a chat with the shakacode folks this morning and discovered that bootstrap-sass-loader is being rewritten and will soon be deprecated. the rewritten version is here, though it hasn't been touched in a couple weeks: https://github.com/shakacode/bootstrap-loader/tree/alex/bootstrap-4 |
@mmahalwy I'm not recommending avoiding it--I'll try to issue a PR to bootstrap-sass-loader soon. The change is minor. I don't see any problems using raw-loader over css-loader in bootstrap-sass-loader. |
Raw loader will not rewrite image and font URLs if you hash them. I don't On Tue, Dec 1, 2015, 1:24 PM Dan Schuman [email protected] wrote:
|
@Phoenixmatrix good point. I'll check on that, but I have a feeling it could cause issues now. |
I'll also look into passing params to css-loader to disable some things, or to downgrade css-loader in that repository. |
Or, having css-loader be a peer dependency. |
What was the reason you gave up on this route? Having bootstrap outside the main chunk might circumvent the recompile issue. |
@trueter yes--I couldn't figure how to prevent bootstra-sass-loader from being triggered every rebuild. I did get it into a separate chunk successfully, but it would always rebuild all chunks AFAIK (it would say 'emitted' next to the vendor chunk I created). |
@quicksnap any luck on this so far? |
@mmahalwy I haven't messed with adjusting bootstrap-sass-loader's properties yet. I'm also not sure if bootstrap-sass has any image URLs in it, so it may actually be good with raw-loader, which would be hella faster. I'm working on stuff right now that doesn't involve dev mode, but next time I get hit with a 4sec rebuild I'll probably dig in. |
@quicksnap what's your build time now? Did you figure out how to speed it up? |
@quicksnap could you create a pr with your code split work in progress? stumbled across this: http://tech.trivago.com/2015/12/15/parallel-webpack/ |
@delvarworld Well, I don't know what's going on there but I don't think you can cache assets with |
By the way, posting here since it's relevant to this discussion: there's a pending PR for DLL support (#1201) but it bases off the happypack PR #1090. With some effort, you could define an asset DLL (some README exists for defining more DLLs). There might be issues with Perhaps you could try those patches out? |
@halt-hammerzeit Thanks for the help so far, I've discovered when I run my dll initial build the
This is my only change to my + new webpack.DllReferencePlugin({
+ context: process.cwd(),
+ manifest: require(path.join(assetsPath, 'main-manifest.json'))
+ }),
+ new webpack.DllReferencePlugin({
+ context: process.cwd(),
+ manifest: require(path.join(assetsPath, 'vendor-manifest.json'))
+ }),
+ new webpack.DllReferencePlugin({
+ context: process.cwd(),
+ manifest: require(path.join(assetsPath, 'assets-manifest.json'))
+ }), For my dev.config.js (not the DLL one), my entry point is simply:
|
Oh, maybe that's the issue - it overwrites the file. |
Ok cool removing webpack-isomorphic-tools from the main webpack config file preserves the manifest. Something else that's confusing is that now all of my assets can only be served from :3000 instead of :3001. I see this line in a linked config, where 1 is subtracted:
But I'm not sure why this is needed? When my code starts it logs:
But all files at :3001 error (they didn't error before the DllPlugin), and those paths only exist at :3000 now. Do you know why this is happening, and if it's an error or expected behavior? |
The files should be available through |
@trueter did you run into this issue and do you have any insight as to why you did |
Ok, finally got somewhere with the Warning: It's easy to do this incorrectly. You can actually double your build size by accidentally bundling your dependencies into the Dll bundle and your main bundle. Several people in this thread have said the DllPlugin gave them no speed up, but I wonder if it was just a configuration issue. Warning: These steps are currently specific to this project, and more specifically to Warning I haven't set up the final script tag to work in a production build (yet), but it should be straightforward. TL;DR dll.config.js and dev.config.js and hard code Reducing My Hot Reload Time From 15s to 4s With The DllPluginI still have more work to do on optimizing (next will try HappyPack), but this is a serious improvement already. My Specific ProblemI have many large asset files that are being built into my bundle. I'm building a 3d game using React and the model files are in JSON, so I have many MB of 3d model data to load. Even though they're going through the file loader, they still get built into the main bundle. I'm also including large Javascript libraries, like Three.js. Webpack Is SlowOut of the box, Webpack isn't efficient. In fact, it's surprising it works the way it does. Webpack treats everything, both your source and your dependencies, as one big bundle. Even in development mode. What this means in practice is that if you change any source file, ALL Javascript files, including dependencies, are re-processed and re-bundled. You can see why this would be slow. I believe this happens even if you use the CommonChunks plugin. If you change your source file, even with CommonChunks, the common chunk will get re-written (I could be wrong but that's what the last section of this post suggests). What Is The
|
Good post
If your file is processed by webpack dev server (in any way) then it resides in RAM and is recompiled only on changes to it. I would suppose the DDLRefPlugin is intelligent enough to exclude files from the DLL from webpack dev server's list of files.
It is supposed to always overwrite
You do it in the main dev server if you're not gonna put your assets into the DLL. Otherwise, I guess, you do it in the DLL compilation run (for whatever reasons you have).
Yes, (you might also consider trying |
The solution @delvarworld outlined in great detail is close to the one proposed in PR (#1201), perhaps these should converge? The only differences as far as I can tell is that it doesn't disable CommonsChunk or isomorphic-tools (although it probably should) and it tracks babel-runtime modules in the vendor DLL. Edit: sound less obnoxious - was just trying to help. 😁 Have a great week everyone! |
It's not all that elegant but what I've done to make hot updates snappy is to run a separate, concurrent Webpack process to bundle my Bootstrap, with a configuration that doesn't have hot updates enabled. In simple terms, the basic syntax is: module.exports = [
{ config for regular bundle with hot updates enabled, css loader ignores bootstrap },
{ config for Bootstrap using raw loader without hot updates }
]; This way, it gets passed through the loaders on initial build, but isn't touched during incremental builds. It's also running concurrently on the initial build with my regular bundle which shaves a few seconds off at that point. Still pretty long build time initially, but it seems to help for incremental builds which are more aggravating when slow. I'd still like to speed up that initial build but that seems a lot more doable using something like this. I'm thinking of setting up a I'm no pro so if this approach would bite me in the ass please let me know before I dig too deep. |
Thanks @trueter, |
Try Now 1.5s to build 0.6s to rebuild !! module.exports = {
cache: true,
devtool: 'cheap-module-eval-source-map',
entry: './src/scripts/index.js',
externals: {
},
module: {
loaders: [
{
loader: 'babel-loader',
exclude: /node_modules/,
query: {
presets: ['es2015'],
cacheDirectory: true
}
}
]
},
output: {
},
plugins: [
new webpack.optimize.UglifyJsPlugin({ output: {comments: false}})
],
resolve: {
modulesDirectories: [
'node_modules'
]
}
}` |
@seeliang |
if you are using babel with lodash |
I am using Happypack & DLL plugin together. Currently, the performance is not so bad. It reduced from more than 15s to 4-5s when it rebuild |
has anyone tried webpack unsafe caching (another link)? This is also supposed to make incremental builds faster. |
hard-source-webpack-plugin might come in handy here. |
Also you can easily improve performance by moving modules that you have no intention of changing into new chunks by specific/extra conditions (e.g. module path or module size) 🔥 🚀 Check the webpack-split-chunks Example: plugins: [
new ChunksPlugin({
to: 'vendor',
test: /node_modules/ // can be function | RegExp | Array[RegExp]
})
] |
@bdefore Many thanks. You made my day. |
… for presumably faster builds erikras/react-redux-universal-hot-example#616
my rebuild time is 12s,what's wrong with my project... |
after used dll plugin,my project build time is less than 30s,but the rebuild time is still 13s... WTF |
@Pines-Cheng To optimise rebuild, i think you can start with separating dev build and prod build, try move vendor js files out of build. that is what i can think of for now |
Ran into the slow performance today when I cannot afford the snail speed of rebuilding. My case was 8s first build (with DLL) and another 8-10s for each rebuild (WTH???). It turned out that Well, in the end, it is not actually babel but a babel plugin So my lesson is that, in general, if you get some performance issue with the first build, |
how to fix this? |
On my machine, incremental builds in dev mode take anywhere from 1.5s - 5s.
I'm going to spend some time toying with webpack to see what's up, but I'd appreciate anyone else's ideas on what's going wrong.
The text was updated successfully, but these errors were encountered: