Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improvements to webpack tracing, including hot-reload #21652

Merged
merged 14 commits into from
Jan 29, 2021
Merged

Improvements to webpack tracing, including hot-reload #21652

merged 14 commits into from
Jan 29, 2021

Conversation

divmain
Copy link
Contributor

@divmain divmain commented Jan 29, 2021

@timneutkens I think this is ready for a review.

I've made some changes to the original design that seem to have paid off. The parenting relationships for traces of normal builds are applied more uniformly, resulting in more intelligible traces:

Screen Shot 2021-01-29 at 12 53 47 AM

Hot-reloading is surfaced now, too. I will note, however, that we will want to dig in deeper and find out where the large portion of time at the beginning of hot-reload is spent. Example:

Screen Shot 2021-01-29 at 12 53 28 AM

Where did those 180 ms go? At the least, we can now track how long a hot-reload takes, and have a place to start with further investigation.

@divmain divmain requested a review from timneutkens January 29, 2021 08:59
@divmain divmain requested review from ijjk, lfades and Timer as code owners January 29, 2021 08:59
divmain
divmain commented Jan 29, 2021
View reviewed changes
packages/next/build/tracer.ts
@@ -2,6 +2,49 @@ import api, { Span } from '@opentelemetry/api'

export const tracer = api.trace.getTracer('next', process.env.__NEXT_VERSION)

const compilerStacks = new WeakMap()

export function stackPush(compiler: any, spanName: string, attrs?: any): any {
Copy link
Contributor Author

@divmain divmain Jan 29, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using this approach allowed for easier association between parent and child. The error handling in stackPop down below also helped to identify when the start/stop tracing events were placed incorrectly.

divmain
divmain commented Jan 29, 2021
View reviewed changes
packages/next/build/webpack/plugins/profiling-plugin.ts
@@ -14,26 +19,85 @@ function getNormalModuleLoaderHook(compilation: any) {
return compilation.hooks.normalModuleLoader
}

function tracingIsEnabled() {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found something that works! No need for a top-level span to be present for the plugin to activate.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be moved to tracer.ts so that it can be reused in other places if needed?

@divmain divmain changed the title Webpack tracing Improvements to webpack tracing, including hot-reload Jan 29, 2021
@ijjk
Copy link
Member

ijjk commented Jan 29, 2021

Stats from current PR

Default Server Mode (Increase detected ⚠️)
General Overall increase ⚠️
vercel/next.js canary vercel/next.js webpack-tracing Change
buildDuration 10.8s 10.8s -1ms
nodeModulesSize 75 MB 75 MB ⚠️ +10.9 kB
Page Load Tests Overall increase ✓
vercel/next.js canary vercel/next.js webpack-tracing Change
/ failed reqs 0 0 ✓
/ total time (seconds) 2.146 2.045 -0.1
/ avg req/sec 1164.98 1222.49 +57.51
/error-in-render failed reqs 0 0 ✓
/error-in-render total time (seconds) 1.265 1.288 ⚠️ +0.02
/error-in-render avg req/sec 1976.98 1940.43 ⚠️ -36.55
Client Bundles (main, webpack, commons)
vercel/next.js canary vercel/next.js webpack-tracing Change
677f882d2ed8..38ed.js gzip 13.1 kB 13.1 kB ✓
framework.HASH.js gzip 39 kB 39 kB ✓
main-HASH.js gzip 6.63 kB 6.63 kB ✓
webpack-HASH.js gzip 751 B 751 B ✓
Overall change 59.4 kB 59.4 kB ✓
Legacy Client Bundles (polyfills)
vercel/next.js canary vercel/next.js webpack-tracing Change
polyfills-HASH.js gzip 31.3 kB 31.3 kB ✓
Overall change 31.3 kB 31.3 kB ✓
Client Pages
vercel/next.js canary vercel/next.js webpack-tracing Change
_app-2a09aa2..4a98.js gzip 1.28 kB 1.28 kB ✓
_error-8b758..aef6.js gzip 3.46 kB 3.46 kB ✓
hooks-c71ae4..70cd.js gzip 887 B 887 B ✓
index-bbee2f..528b.js gzip 227 B 227 B ✓
link-7faf09b..eba4.js gzip 1.64 kB 1.64 kB ✓
routerDirect..bf84.js gzip 303 B 303 B ✓
withRouter-a..5826.js gzip 302 B 302 B ✓
Overall change 8.09 kB 8.09 kB ✓
Client Build Manifests
vercel/next.js canary vercel/next.js webpack-tracing Change
_buildManifest.js gzip 321 B 321 B ✓
Overall change 321 B 321 B ✓
Rendered Page Sizes
vercel/next.js canary vercel/next.js webpack-tracing Change
index.html gzip 614 B 614 B ✓
link.html gzip 620 B 620 B ✓
withRouter.html gzip 607 B 607 B ✓
Overall change 1.84 kB 1.84 kB ✓
Serverless Mode
General Overall increase ⚠️
vercel/next.js canary vercel/next.js webpack-tracing Change
buildDuration 12.7s 12.8s ⚠️ +14ms
nodeModulesSize 75 MB 75 MB ⚠️ +10.9 kB
Client Bundles (main, webpack, commons)
vercel/next.js canary vercel/next.js webpack-tracing Change
677f882d2ed8..38ed.js gzip 13.1 kB 13.1 kB ✓
framework.HASH.js gzip 39 kB 39 kB ✓
main-HASH.js gzip 6.63 kB 6.63 kB ✓
webpack-HASH.js gzip 751 B 751 B ✓
Overall change 59.4 kB 59.4 kB ✓
Legacy Client Bundles (polyfills)
vercel/next.js canary vercel/next.js webpack-tracing Change
polyfills-HASH.js gzip 31.3 kB 31.3 kB ✓
Overall change 31.3 kB 31.3 kB ✓
Client Pages
vercel/next.js canary vercel/next.js webpack-tracing Change
_app-2a09aa2..4a98.js gzip 1.28 kB 1.28 kB ✓
_error-8b758..aef6.js gzip 3.46 kB 3.46 kB ✓
hooks-c71ae4..70cd.js gzip 887 B 887 B ✓
index-bbee2f..528b.js gzip 227 B 227 B ✓
link-7faf09b..eba4.js gzip 1.64 kB 1.64 kB ✓
routerDirect..bf84.js gzip 303 B 303 B ✓
withRouter-a..5826.js gzip 302 B 302 B ✓
Overall change 8.09 kB 8.09 kB ✓
Client Build Manifests
vercel/next.js canary vercel/next.js webpack-tracing Change
_buildManifest.js gzip 321 B 321 B ✓
Overall change 321 B 321 B ✓
Serverless bundles
vercel/next.js canary vercel/next.js webpack-tracing Change
_error.js 1 MB 1 MB ✓
404.html 2.67 kB 2.67 kB ✓
hooks.html 1.92 kB 1.92 kB ✓
index.js 1 MB 1 MB ✓
link.js 1.06 MB 1.06 MB ✓
routerDirect.js 1.05 MB 1.05 MB ✓
withRouter.js 1.05 MB 1.05 MB ✓
Overall change 5.19 MB 5.19 MB ✓
Webpack 5 Mode (Increase detected ⚠️)
General Overall increase ⚠️
vercel/next.js canary vercel/next.js webpack-tracing Change
buildDuration 13s 13.1s ⚠️ +99ms
nodeModulesSize 75 MB 75 MB ⚠️ +10.9 kB
Page Load Tests Overall increase ✓
vercel/next.js canary vercel/next.js webpack-tracing Change
/ failed reqs 0 0 ✓
/ total time (seconds) 2.095 2.012 -0.08
/ avg req/sec 1193.37 1242.4 +49.03
/error-in-render failed reqs 0 0 ✓
/error-in-render total time (seconds) 1.351 1.363 ⚠️ +0.01
/error-in-render avg req/sec 1851.12 1833.76 ⚠️ -17.36
Client Bundles (main, webpack, commons)
vercel/next.js canary vercel/next.js webpack-tracing Change
597-7c719119..27b6.js gzip 13 kB 13 kB ✓
framework.HASH.js gzip 39.3 kB 39.3 kB ✓
main-HASH.js gzip 6.58 kB 6.58 kB ✓
webpack-HASH.js gzip 954 B 954 B ✓
Overall change 59.8 kB 59.8 kB ✓
Legacy Client Bundles (polyfills)
vercel/next.js canary vercel/next.js webpack-tracing Change
polyfills-HASH.js gzip 31.1 kB 31.1 kB ✓
Overall change 31.1 kB 31.1 kB ✓
Client Pages
vercel/next.js canary vercel/next.js webpack-tracing Change
_app-6e0a903..b885.js gzip 1.26 kB 1.26 kB ✓
_error-7a100..a392.js gzip 3.38 kB 3.38 kB ✓
hooks-f7e199..11b9.js gzip 904 B 904 B ✓
index-3dc22c..ffbb.js gzip 232 B 232 B ✓
link-c010d28..b60a.js gzip 1.63 kB 1.63 kB ✓
routerDirect..323a.js gzip 308 B 308 B ✓
withRouter-a..2ef8.js gzip 304 B 304 B ✓
Overall change 8.02 kB 8.02 kB ✓
Client Build Manifests
vercel/next.js canary vercel/next.js webpack-tracing Change
_buildManifest.js gzip 299 B 299 B ✓
Overall change 299 B 299 B ✓
Rendered Page Sizes
vercel/next.js canary vercel/next.js webpack-tracing Change
index.html gzip 585 B 585 B ✓
link.html gzip 592 B 592 B ✓
withRouter.html gzip 579 B 579 B ✓
Overall change 1.76 kB 1.76 kB ✓

Diffs

Diff for index.html 
@@ -43,7 +43,7 @@
         "props": { "pageProps": {} },
         "page": "/",
         "query": {},
-        "buildId": "pTe5shWFjyK2oB9lopKxS",
+        "buildId": "DCNosug1KeykeS-5VlE4b",
         "isFallback": false,
         "gip": true
       }
@@ -77,11 +77,11 @@
       async=""
     ></script>
     <script
-      src="/_next/static/pTe5shWFjyK2oB9lopKxS/_buildManifest.js"
+      src="/_next/static/DCNosug1KeykeS-5VlE4b/_buildManifest.js"
       async=""
     ></script>
     <script
-      src="/_next/static/pTe5shWFjyK2oB9lopKxS/_ssgManifest.js"
+      src="/_next/static/DCNosug1KeykeS-5VlE4b/_ssgManifest.js"
       async=""
     ></script>
   </body>
Diff for link.html 
@@ -48,7 +48,7 @@
         "props": { "pageProps": {} },
         "page": "/link",
         "query": {},
-        "buildId": "pTe5shWFjyK2oB9lopKxS",
+        "buildId": "DCNosug1KeykeS-5VlE4b",
         "isFallback": false,
         "gip": true
       }
@@ -82,11 +82,11 @@
       async=""
     ></script>
     <script
-      src="/_next/static/pTe5shWFjyK2oB9lopKxS/_buildManifest.js"
+      src="/_next/static/DCNosug1KeykeS-5VlE4b/_buildManifest.js"
       async=""
     ></script>
     <script
-      src="/_next/static/pTe5shWFjyK2oB9lopKxS/_ssgManifest.js"
+      src="/_next/static/DCNosug1KeykeS-5VlE4b/_ssgManifest.js"
       async=""
     ></script>
   </body>
Diff for withRouter.html 
@@ -43,7 +43,7 @@
         "props": { "pageProps": {} },
         "page": "/withRouter",
         "query": {},
-        "buildId": "pTe5shWFjyK2oB9lopKxS",
+        "buildId": "DCNosug1KeykeS-5VlE4b",
         "isFallback": false,
         "gip": true
       }
@@ -77,11 +77,11 @@
       async=""
     ></script>
     <script
-      src="/_next/static/pTe5shWFjyK2oB9lopKxS/_buildManifest.js"
+      src="/_next/static/DCNosug1KeykeS-5VlE4b/_buildManifest.js"
       async=""
     ></script>
     <script
-      src="/_next/static/pTe5shWFjyK2oB9lopKxS/_ssgManifest.js"
+      src="/_next/static/DCNosug1KeykeS-5VlE4b/_ssgManifest.js"
       async=""
     ></script>
   </body>
Commit: 24850bb

@timneutkens
Copy link
Member

Looks great 🚀

@ijjk

This comment has been minimized.

Sign in to view
@ijjk
Copy link
Member

ijjk commented Jan 29, 2021

Stats from current PR

Default Server Mode (Increase detected ⚠️)
General Overall increase ⚠️
vercel/next.js canary vercel/next.js webpack-tracing Change
buildDuration 11.8s 11.9s ⚠️ +36ms
nodeModulesSize 75 MB 75 MB ⚠️ +11.2 kB
Page Load Tests Overall increase ✓
vercel/next.js canary vercel/next.js webpack-tracing Change
/ failed reqs 0 0 ✓
/ total time (seconds) 2.476 2.484 ⚠️ +0.01
/ avg req/sec 1009.69 1006.59 ⚠️ -3.1
/error-in-render failed reqs 0 0 ✓
/error-in-render total time (seconds) 1.704 1.683 -0.02
/error-in-render avg req/sec 1467.2 1485.7 +18.5
Client Bundles (main, webpack, commons)
vercel/next.js canary vercel/next.js webpack-tracing Change
677f882d2ed8..38ed.js gzip 13.1 kB 13.1 kB ✓
framework.HASH.js gzip 39 kB 39 kB ✓
main-HASH.js gzip 6.63 kB 6.63 kB ✓
webpack-HASH.js gzip 751 B 751 B ✓
Overall change 59.4 kB 59.4 kB ✓
Legacy Client Bundles (polyfills)
vercel/next.js canary vercel/next.js webpack-tracing Change
polyfills-HASH.js gzip 31.3 kB 31.3 kB ✓
Overall change 31.3 kB 31.3 kB ✓
Client Pages
vercel/next.js canary vercel/next.js webpack-tracing Change
_app-2a09aa2..4a98.js gzip 1.28 kB 1.28 kB ✓
_error-8b758..aef6.js gzip 3.46 kB 3.46 kB ✓
hooks-c71ae4..70cd.js gzip 887 B 887 B ✓
index-bbee2f..528b.js gzip 227 B 227 B ✓
link-7faf09b..eba4.js gzip 1.64 kB 1.64 kB ✓
routerDirect..bf84.js gzip 303 B 303 B ✓
withRouter-a..5826.js gzip 302 B 302 B ✓
Overall change 8.09 kB 8.09 kB ✓
Client Build Manifests
vercel/next.js canary vercel/next.js webpack-tracing Change
_buildManifest.js gzip 321 B 321 B ✓
Overall change 321 B 321 B ✓
Rendered Page Sizes
vercel/next.js canary vercel/next.js webpack-tracing Change
index.html gzip 614 B 614 B ✓
link.html gzip 620 B 620 B ✓
withRouter.html gzip 607 B 607 B ✓
Overall change 1.84 kB 1.84 kB ✓
Serverless Mode
General Overall increase ⚠️
vercel/next.js canary vercel/next.js webpack-tracing Change
buildDuration 14.2s 14.3s ⚠️ +42ms
nodeModulesSize 75 MB 75 MB ⚠️ +11.2 kB
Client Bundles (main, webpack, commons)
vercel/next.js canary vercel/next.js webpack-tracing Change
677f882d2ed8..38ed.js gzip 13.1 kB 13.1 kB ✓
framework.HASH.js gzip 39 kB 39 kB ✓
main-HASH.js gzip 6.63 kB 6.63 kB ✓
webpack-HASH.js gzip 751 B 751 B ✓
Overall change 59.4 kB 59.4 kB ✓
Legacy Client Bundles (polyfills)
vercel/next.js canary vercel/next.js webpack-tracing Change
polyfills-HASH.js gzip 31.3 kB 31.3 kB ✓
Overall change 31.3 kB 31.3 kB ✓
Client Pages
vercel/next.js canary vercel/next.js webpack-tracing Change
_app-2a09aa2..4a98.js gzip 1.28 kB 1.28 kB ✓
_error-8b758..aef6.js gzip 3.46 kB 3.46 kB ✓
hooks-c71ae4..70cd.js gzip 887 B 887 B ✓
index-bbee2f..528b.js gzip 227 B 227 B ✓
link-7faf09b..eba4.js gzip 1.64 kB 1.64 kB ✓
routerDirect..bf84.js gzip 303 B 303 B ✓
withRouter-a..5826.js gzip 302 B 302 B ✓
Overall change 8.09 kB 8.09 kB ✓
Client Build Manifests
vercel/next.js canary vercel/next.js webpack-tracing Change
_buildManifest.js gzip 321 B 321 B ✓
Overall change 321 B 321 B ✓
Serverless bundles
vercel/next.js canary vercel/next.js webpack-tracing Change
_error.js 1 MB 1 MB ✓
404.html 2.67 kB 2.67 kB ✓
hooks.html 1.92 kB 1.92 kB ✓
index.js 1 MB 1 MB ✓
link.js 1.06 MB 1.06 MB ✓
routerDirect.js 1.05 MB 1.05 MB ✓
withRouter.js 1.05 MB 1.05 MB ✓
Overall change 5.19 MB 5.19 MB ✓
Webpack 5 Mode (Increase detected ⚠️)
General Overall increase ⚠️
vercel/next.js canary vercel/next.js webpack-tracing Change
buildDuration 14.6s 14.5s -93ms
nodeModulesSize 75 MB 75 MB ⚠️ +11.2 kB
Page Load Tests Overall increase ✓
vercel/next.js canary vercel/next.js webpack-tracing Change
/ failed reqs 0 0 ✓
/ total time (seconds) 2.426 2.368 -0.06
/ avg req/sec 1030.41 1055.7 +25.29
/error-in-render failed reqs 0 0 ✓
/error-in-render total time (seconds) 1.693 1.673 -0.02
/error-in-render avg req/sec 1476.44 1494.45 +18.01
Client Bundles (main, webpack, commons)
vercel/next.js canary vercel/next.js webpack-tracing Change
597-7c719119..27b6.js gzip 13 kB 13 kB ✓
framework.HASH.js gzip 39.3 kB 39.3 kB ✓
main-HASH.js gzip 6.58 kB 6.58 kB ✓
webpack-HASH.js gzip 954 B 954 B ✓
Overall change 59.8 kB 59.8 kB ✓
Legacy Client Bundles (polyfills)
vercel/next.js canary vercel/next.js webpack-tracing Change
polyfills-HASH.js gzip 31.1 kB 31.1 kB ✓
Overall change 31.1 kB 31.1 kB ✓
Client Pages
vercel/next.js canary vercel/next.js webpack-tracing Change
_app-6e0a903..b885.js gzip 1.26 kB 1.26 kB ✓
_error-7a100..a392.js gzip 3.38 kB 3.38 kB ✓
hooks-f7e199..11b9.js gzip 904 B 904 B ✓
index-3dc22c..ffbb.js gzip 232 B 232 B ✓
link-c010d28..b60a.js gzip 1.63 kB 1.63 kB ✓
routerDirect..323a.js gzip 308 B 308 B ✓
withRouter-a..2ef8.js gzip 304 B 304 B ✓
Overall change 8.02 kB 8.02 kB ✓
Client Build Manifests
vercel/next.js canary vercel/next.js webpack-tracing Change
_buildManifest.js gzip 299 B 299 B ✓
Overall change 299 B 299 B ✓
Rendered Page Sizes
vercel/next.js canary vercel/next.js webpack-tracing Change
index.html gzip 585 B 585 B ✓
link.html gzip 592 B 592 B ✓
withRouter.html gzip 579 B 579 B ✓
Overall change 1.76 kB 1.76 kB ✓

Diffs

Diff for index.html 
@@ -43,7 +43,7 @@
         "props": { "pageProps": {} },
         "page": "/",
         "query": {},
-        "buildId": "GMfRGO87PZvxCwuByD58D",
+        "buildId": "h2ysPGZ9PVJ6deiBh0C8U",
         "isFallback": false,
         "gip": true
       }
@@ -77,11 +77,11 @@
       async=""
     ></script>
     <script
-      src="/_next/static/GMfRGO87PZvxCwuByD58D/_buildManifest.js"
+      src="/_next/static/h2ysPGZ9PVJ6deiBh0C8U/_buildManifest.js"
       async=""
     ></script>
     <script
-      src="/_next/static/GMfRGO87PZvxCwuByD58D/_ssgManifest.js"
+      src="/_next/static/h2ysPGZ9PVJ6deiBh0C8U/_ssgManifest.js"
       async=""
     ></script>
   </body>
Diff for link.html 
@@ -48,7 +48,7 @@
         "props": { "pageProps": {} },
         "page": "/link",
         "query": {},
-        "buildId": "GMfRGO87PZvxCwuByD58D",
+        "buildId": "h2ysPGZ9PVJ6deiBh0C8U",
         "isFallback": false,
         "gip": true
       }
@@ -82,11 +82,11 @@
       async=""
     ></script>
     <script
-      src="/_next/static/GMfRGO87PZvxCwuByD58D/_buildManifest.js"
+      src="/_next/static/h2ysPGZ9PVJ6deiBh0C8U/_buildManifest.js"
       async=""
     ></script>
     <script
-      src="/_next/static/GMfRGO87PZvxCwuByD58D/_ssgManifest.js"
+      src="/_next/static/h2ysPGZ9PVJ6deiBh0C8U/_ssgManifest.js"
       async=""
     ></script>
   </body>
Diff for withRouter.html 
@@ -43,7 +43,7 @@
         "props": { "pageProps": {} },
         "page": "/withRouter",
         "query": {},
-        "buildId": "GMfRGO87PZvxCwuByD58D",
+        "buildId": "h2ysPGZ9PVJ6deiBh0C8U",
         "isFallback": false,
         "gip": true
       }
@@ -77,11 +77,11 @@
       async=""
     ></script>
     <script
-      src="/_next/static/GMfRGO87PZvxCwuByD58D/_buildManifest.js"
+      src="/_next/static/h2ysPGZ9PVJ6deiBh0C8U/_buildManifest.js"
       async=""
     ></script>
     <script
-      src="/_next/static/GMfRGO87PZvxCwuByD58D/_ssgManifest.js"
+      src="/_next/static/h2ysPGZ9PVJ6deiBh0C8U/_ssgManifest.js"
       async=""
     ></script>
   </body>
Commit: dc54680

@kodiakhq kodiakhq bot merged commit 652a1e8 into canary Jan 29, 2021
@kodiakhq kodiakhq bot deleted the webpack-tracing branch January 29, 2021 10:32
@weyert
Copy link

weyert commented Jan 29, 2021

Would this allow as a consumer of next.js to get visibility when having a SSR application? I would delighted to be able to get traces to Google Trace via its OpenTracing exporter

This was referenced Aug 3, 2021
Open
Open
Open
Open
Open
Open
Open
This was referenced Aug 31, 2021
Open
Open
Open
Open
Open
Open
Open
This was referenced Sep 15, 2021
Open
Open
Open
Open
Open
Open
Open
Open
Open
Open
This was referenced Nov 9, 2021
Open
Open
Open
Open
@snyk-bot snyk-bot mentioned this pull request Dec 16, 2021
Open
@vercel vercel locked as resolved and limited conversation to collaborators Jan 29, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@timneutkens timneutkens timneutkens approved these changes

@ijjk ijjk Awaiting requested review from ijjk

@lfades lfades Awaiting requested review from lfades

@Timer Timer Awaiting requested review from Timer

Assignees
No one assigned
Labels
type: next
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@divmain @ijjk @timneutkens @weyert