core: update FCP score curve #12556
core: update FCP score curve #12556
Conversation
| p10: 2700, | ||
| median: 4500, |
There was a problem hiding this comment.
these are updated based on the 1.5x multiplier mentioned above applied to the updated control points
| const result = await FcpAudit.audit(artifacts, context); | ||
| assert.equal(result.score, 0.06); | ||
| assert.equal(result.numericValue, 5668.275); | ||
| }); |
There was a problem hiding this comment.
we didn't have any unit test that needed updating from the score change (aside from sample_v2 snapshot ones), which is always suspicious, so added one here
| @@ -29,12 +29,11 @@ class FirstContentfulPaint3G extends Audit { | |||
| */ | |||
| static get defaultOptions() { | |||
| return { | |||
| // 25th and 5th percentiles HTTPArchive on Fast 3G -> multiply by 1.5 for RTT differential -> median and PODR | |||
| // p10 is then derived from them. | |||
| // 25th and 8th percentiles HTTPArchive on Slow 4G -> multiply by 1.5 for RTT differential -> median and p10. | |||
| // https://bigquery.cloud.google.com/table/httparchive:lighthouse.2018_04_01_mobile?pli=1 | |||
There was a problem hiding this comment.
isnt the dataset different?
There was a problem hiding this comment.
ah yeah, good catch
|
Hey there! Considering these changes, the web.dev articles should be updated as well, right? For example, here it says a good score is less than 2s but that's not what you see on Lighthouse if you run a scan: https://web.dev/first-contentful-paint/#how-lighthouse-determines-your-fcp-score. Thanks! |
|
Yes, definitely. The documentation changes unfortunately didn't land at the same time but the PR is pending for them: GoogleChrome/web.dev#5544 |
Part of #11866
Updates the FCP score-curve control points to match the latest Lighthouse performance data and to align with other web performance tools.
Our FCP control points were last updated more than three years ago. It's not clear why FCP in HTTP Archive has trended lower over the time since (maybe changes in the sampled URLs?), but they clearly don't match the thresholds we would pick today.
Other tools are also trying to align on a set of FCP thresholds. At one point 1000ms/3000ms was recommended for FCP good/average thresholds (I believe that's what CrUX uses today), but more recent analysis has found the 1000ms to be difficult to be widely achievable, and so the current recommendation for tooling is 1800ms/3000ms.
While in some cases a more easily achievable threshold might be too easy for Lighthouse, in this case it's such an improvement over the current speed curve that this seems like an excellent move.
In addition, if we were picking new control points without outside guidance, the May 2021 HTTP Archive run gives
p08 - 1787ms
p25 - 2595ms
(3000ms is closer to p35, but 2595ms is close enough that it seems ok)
With the change in HTTP Archive throttling in mind, we can also look at the July 2020 run, the month before the throttling change, and we find
p08 - 1662ms
P25 - 2484ms
(1800ms is closer to p10, 3000ms is closer to p40)
The old throttling is closer to what we see in CLI Lighthouse, so we could move to these even tougher numbers, but the proposed numbers are still a big improvement over the current control points and alignment would be ideal, so it seems worth moving to 1800/3000.
Desktop
There's no web vitals guidance for desktop specifically, but we documented using
SELECT QUANTILES(renderStart, 21) FROM [httparchive:summary_pages.2018_12_15_desktop] LIMIT 1000to derive the desktop control points. Running that today:May 2021
p08 - 1000ms
p25 - 1500ms
July 2020
P08 - 1000ms
P25 - 1600ms
These match our current values (934/1600) very closely, so I suggest we leave them unchanged.