core(byte-efficiency): compute FCP & LCP savings #15104

adamraine · 2023-05-23T00:30:32Z

First pass at computing FCP / LCP graph savings for byte efficiency audits. To gauge these estimates, I did a DT throttling run on cnn.com:

https://googlechrome.github.io/lighthouse/viewer/?gist=3101939dec6e6aa3c54ef73af500b86a
https://trace.cafe/t/QxCTNKUbin

offscreen-images { FCP: 0, LCP: 0 }
unminified-css { FCP: 0, LCP: 0 }
unminified-javascript { FCP: 0, LCP: 0 }
unused-css-rules { FCP: 0, LCP: 0 }
unused-javascript { FCP: 480, LCP: 2470 }
modern-image-formats { FCP: 0, LCP: 0 }
uses-optimized-images { FCP: 0, LCP: 0 }
uses-text-compression { FCP: 0, LCP: 0 }
uses-responsive-images { FCP: 0, LCP: 490 }
efficient-animated-content { FCP: 0, LCP: 0 }
duplicated-javascript { FCP: 0, LCP: 0 }
legacy-javascript { FCP: 0, LCP: 0 }

My initial read:

unused-javascript FCP/LCP estimates seem appropriate. All of the pre-FCP downloads and many of the pre-LCP downloads are JS.
uses-responsive-images and modern-image-formats are underestimating the impact on LCP
- The LCP image resource is listed with potential savings in both audits
- The LCP load time is 46% of LCP (5,180 ms)
- Potential solution: When the LCP resource is listed in a BE audit, use the estimated savings of the LCP resource network node as a minimum value for the overall LCP savings.

adamraine · 2023-05-23T21:13:16Z

Run on cnn.com w/ simulated throttling. CNN changes content a lot so this isn't a mirror image of the analysis above.

https://googlechrome.github.io/lighthouse/viewer/?gist=7e431eebd16bc37944e9b1ceaff80c3b
https://trace.cafe/t/GKby7xbdFx

Using pessemistic graph

offscreen-images { FCP: 0, LCP: 1340 }
unminified-css { FCP: 0, LCP: 0 }
unminified-javascript { FCP: 0, LCP: 300 }
unused-css-rules { FCP: 0, LCP: 150 }
unused-javascript { FCP: 1200, LCP: 11250 }
modern-image-formats { FCP: 0, LCP: 1550 }
uses-optimized-images { FCP: 0, LCP: 0 }
uses-text-compression { FCP: 0, LCP: 150 }
uses-responsive-images { FCP: 0, LCP: 2000 }
efficient-animated-content { FCP: 0, LCP: 0 }
duplicated-javascript { FCP: 0, LCP: 160 }
legacy-javascript { FCP: 150, LCP: 150 }

Using optimistic graph:

offscreen-images { FCP: 0, LCP: 150 }
unminified-css { FCP: 0, LCP: 0 }
unminified-javascript { FCP: 0, LCP: 300 }
unused-css-rules { FCP: 0, LCP: 0 }
unused-javascript { FCP: 1350, LCP: 10650 }
modern-image-formats { FCP: 0, LCP: 150 }
uses-optimized-images { FCP: 0, LCP: 0 }
uses-text-compression { FCP: 0, LCP: 0 }
uses-responsive-images { FCP: 0, LCP: 150 }
efficient-animated-content { FCP: 0, LCP: 0 }
duplicated-javascript { FCP: 0, LCP: 130 }
legacy-javascript { FCP: 150, LCP: 0 }

Notes:

Most outrageous part is our estimated 50s LCP. I don't think the LCP would realistically be this big because it happened to be an image in an add that was displayed well into the page load.
LCP resource is listed in modern-image-formats, but the LCP load time was only 590ms. Unsure if this detail makes a difference for these estimates.
Using the pessimistic graph seems fine so far, but we do use the optimistic graph when doing a similar calculation in render blocking resources.
- It does have a valid reason to use the optimistic FCP graph
- For the general case byte efficiency audits, It might make sense to use the correlation of optimistic/pessimistic estimates. This would have the largest affect on image audit savings since we exclude low priority image nodes in the optimistic LCP graph.

paulirish · 2023-06-09T22:48:10Z

core/audits/byte-efficiency/byte-efficiency-audit.js

+      if (lcpRecord && lcpBreakdown.loadStart && lcpBreakdown.loadEnd) {
+        const lcpResult = results.find(result => result.url === lcpRecord.url);
+        if (lcpResult) {
+          const lcpLoadTime = lcpBreakdown.loadEnd - lcpBreakdown.loadStart;


the benefit is only to download time.. but this loadTime duration includes ttfb time

tbh looking at your original results I'd say that the image audits were not underestimating LCP savings

What do you mean "includes ttfb time"? lcpLoadTime is supposed to represent the "Resource load time" phase of LCP so it's just the download time.

tbh looking at your original results I'd say that the image audits were not underestimating LCP savings

In the first set of results, modern-image-formats reports 0 savings on LCP when calculated using the LCP graph even though the LCP image is listed in that audit. This is definitely an underestimate IMO.

connorjclark · 2023-06-15T20:16:03Z

core/audits/byte-efficiency/byte-efficiency-audit.js

+        {providedWastedBytesByUrl: result.wastedBytesByUrl, label: 'lcp'}
+      );
+
+      // The LCP graph can underestimate the LCP savings if there is potential savings on the LCP record itself.


I'm having trouble understanding this. How is this the case? If the LCP node download time is reduced, why isn't the simulation taking that into account when returning LCP savings?

Graph timings are computed using the end time of the entire graph, the LCP request node is not necessarily the final node in the graph. Presumably, it's possible for the node that marks the end of the entire graph to have it's timing unaffected by changes to the LCP request node.

Can you provide a breakdown for a specific example where this is the case? I'd expect there to be some dependency between the final node (is it the CPU node that has the layout?) and the network request node whose bytes are being reduced.

The case in the PR comment is an example of this happening. LCP image is listed modern-image-formats but the LCP graph savings are 0. I did not save any artifacts other than the trace, so I'll try and get another example of such a case.

Here is an example I captured with DT throttling. modern-image-formats shows the LCP record savings as larger than the LCP graph savings:

artifacts.json.zip
traceAndDtLog.zip
(had to split it up because it was too big for GH)

core/audits/byte-efficiency/byte-efficiency-audit.js

adamraine added 4 commits May 22, 2023 16:19

WIP: compute FCP & LCP savings for byte efficiency audits

086cd00

ope

049f130

graphs

c23c193

console

e452b93

Alanesa mentioned this pull request May 23, 2023

Upgrade logger #15106

Closed

adamraine added 4 commits June 8, 2023 13:10

Merge branch 'main' into byte-efficiency-savings

f73e001

reorg

f1fdca3

include lcp record savings

369df9b

comment

f489a86

vercel bot deployed to Preview June 8, 2023 21:44 View deployment

adamraine added 3 commits June 9, 2023 11:32

Merge branch 'main' into byte-efficiency-savings

32d8dd3

comment

caa183f

unit test fix

7556484

vercel bot deployed to Preview June 9, 2023 19:40 View deployment

Merge branch 'main' into byte-efficiency-savings

92213ac

vercel bot deployed to Preview June 9, 2023 20:38 View deployment

unit tests

7be2067

adamraine changed the title ~~WIP: compute FCP & LCP savings for byte efficiency audits~~ core(byte-efficiency): compute FCP & LCP savings Jun 9, 2023

adamraine marked this pull request as ready for review June 9, 2023 22:44

adamraine requested a review from a team as a code owner June 9, 2023 22:44

adamraine requested review from connorjclark and removed request for a team June 9, 2023 22:44

vercel bot deployed to Preview June 9, 2023 22:45 View deployment

paulirish reviewed Jun 9, 2023

View reviewed changes

devtools-bot assigned connorjclark Jun 9, 2023

devtools-bot added the waiting4reviewer label Jun 9, 2023

smokes

e058cf1

vercel bot deployed to Preview June 12, 2023 15:57 View deployment

no smoke expectations

cc55816

vercel bot deployed to Preview June 12, 2023 17:25 View deployment

connorjclark reviewed Jun 15, 2023

View reviewed changes

core/audits/byte-efficiency/byte-efficiency-audit.js Show resolved Hide resolved

only review LCP download time

f894073

vercel bot deployed to Preview June 15, 2023 23:36 View deployment

adamraine added 2 commits June 15, 2023 16:43

Merge branch 'main' into byte-efficiency-savings

5cdc409

sample

08e1ba7

vercel bot deployed to Preview June 15, 2023 23:45 View deployment

precision

c2bd4fe

vercel bot deployed to Preview June 16, 2023 18:54 View deployment

test

f97bf07

vercel bot deployed to Preview June 16, 2023 19:04 View deployment

Merge branch 'main' into byte-efficiency-savings

2551675

vercel bot deployed to Preview July 5, 2023 22:33 View deployment

connorjclark approved these changes Jul 5, 2023

View reviewed changes

adamraine merged commit 3b0950c into main Jul 5, 2023

adamraine deleted the byte-efficiency-savings branch July 5, 2023 23:06

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

core(byte-efficiency): compute FCP & LCP savings #15104

core(byte-efficiency): compute FCP & LCP savings #15104

adamraine commented May 23, 2023 •

edited

Loading

adamraine commented May 23, 2023 •

edited

Loading

paulirish Jun 9, 2023

adamraine Jun 9, 2023

adamraine Jun 9, 2023

connorjclark Jun 15, 2023

adamraine Jun 15, 2023

connorjclark Jun 15, 2023 •

edited

Loading

adamraine Jun 15, 2023

adamraine Jun 21, 2023

core(byte-efficiency): compute FCP & LCP savings #15104

core(byte-efficiency): compute FCP & LCP savings #15104

Conversation

adamraine commented May 23, 2023 • edited Loading

adamraine commented May 23, 2023 • edited Loading

paulirish Jun 9, 2023

Choose a reason for hiding this comment

adamraine Jun 9, 2023

Choose a reason for hiding this comment

adamraine Jun 9, 2023

Choose a reason for hiding this comment

connorjclark Jun 15, 2023

Choose a reason for hiding this comment

adamraine Jun 15, 2023

Choose a reason for hiding this comment

connorjclark Jun 15, 2023 • edited Loading

Choose a reason for hiding this comment

adamraine Jun 15, 2023

Choose a reason for hiding this comment

adamraine Jun 21, 2023

Choose a reason for hiding this comment

adamraine commented May 23, 2023 •

edited

Loading

adamraine commented May 23, 2023 •

edited

Loading

connorjclark Jun 15, 2023 •

edited

Loading