Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

core: update FCP score curve #12556

Merged
merged 3 commits into from
May 25, 2021
Merged

core: update FCP score curve #12556

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
e9a33e8
core: update FCP score curve
brendankenny May 25, 2021
ed1161c
everyone loves csv
brendankenny May 25, 2021
fe29068
update bigquery table references
brendankenny May 25, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 4 additions & 5 deletions lighthouse-core/audits/metrics/first-contentful-paint-3g.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,12 +29,11 @@ class FirstContentfulPaint3G extends Audit {
*/
static get defaultOptions() {
return {
// 25th and 5th percentiles HTTPArchive on Fast 3G -> multiply by 1.5 for RTT differential -> median and PODR
// p10 is then derived from them.
// 25th and 8th percentiles HTTPArchive on Slow 4G -> multiply by 1.5 for RTT differential -> median and p10.
// https://bigquery.cloud.google.com/table/httparchive:lighthouse.2018_04_01_mobile?pli=1
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isnt the dataset different?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah yeah, good catch

// https://www.desmos.com/calculator/fflcrsn9sj
p10: 3504,
median: 6000,
// https://www.desmos.com/calculator/xi5oympawp
p10: 2700,
median: 4500,
Comment on lines +35 to +36
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these are updated based on the 1.5x multiplier mentioned above applied to the updated control points

};
}

Expand Down
10 changes: 5 additions & 5 deletions lighthouse-core/audits/metrics/first-contentful-paint.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,12 +37,12 @@ class FirstContentfulPaint extends Audit {
static get defaultOptions() {
return {
mobile: {
// 25th and 5th percentiles HTTPArchive -> median and PODR, then p10 is derived from them.
// https://bigquery.cloud.google.com/table/httparchive:lighthouse.2018_04_01_mobile?pli=1
// see https://www.desmos.com/calculator/oqlvmezbze
// 25th and 8th percentiles HTTPArchive -> median and p10.
// https://bigquery.cloud.google.com/table/httparchive:lighthouse.2021_05_01_mobile
// see https://www.desmos.com/calculator/6wi8rhipve
scoring: {
p10: 2336,
median: 4000,
p10: 1800,
median: 3000,
},
},
desktop: {
Expand Down
2 changes: 1 addition & 1 deletion lighthouse-core/test/audits/metrics/first-contentful-paint-3g-test.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ describe('Performance: first-contentful-paint-3g audit', () => {
// Use InlineSnapshot here so changes to Lantern coefficients can be easily updated en masse
expect({score: result.score, value: Math.round(result.numericValue)}).toMatchInlineSnapshot(`
Object {
"score": 0.99,
"score": 0.97,
"value": 2087,
}
`);
Expand Down
29 changes: 24 additions & 5 deletions lighthouse-core/test/audits/metrics/first-contentful-paint-test.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,17 @@
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
*/
'use strict';
const Audit = require('../../../audits/metrics/first-contentful-paint.js');
const FcpAudit = require('../../../audits/metrics/first-contentful-paint.js');
const assert = require('assert').strict;
const options = Audit.defaultOptions;
const options = FcpAudit.defaultOptions;
const constants = require('../../../config/constants.js');

const pwaTrace = require('../../fixtures/traces/progressive-app-m60.json');
const pwaDevtoolsLog = require('../../fixtures/traces/progressive-app-m60.devtools.log.json');

const frameTrace = require('../../fixtures/traces/frame-metrics-m90.json');
const frameDevtoolsLog = require('../../fixtures/traces/frame-metrics-m90.devtools.log.json');

/**
* @param {{
* {LH.SharedFlagsSettings['formFactor']} formFactor
Expand All @@ -35,16 +38,32 @@ describe('Performance: first-contentful-paint audit', () => {
it('evaluates valid input correctly', async () => {
const artifacts = {
traces: {
[Audit.DEFAULT_PASS]: pwaTrace,
[FcpAudit.DEFAULT_PASS]: pwaTrace,
},
devtoolsLogs: {
[Audit.DEFAULT_PASS]: pwaDevtoolsLog,
[FcpAudit.DEFAULT_PASS]: pwaDevtoolsLog,
},
};

const context = getFakeContext({formFactor: 'mobile', throttlingMethod: 'provided'});
const result = await Audit.audit(artifacts, context);
const result = await FcpAudit.audit(artifacts, context);
assert.equal(result.score, 1);
assert.equal(result.numericValue, 498.87);
});

it('evaluates a modern trace correctly', async () => {
const artifacts = {
traces: {
[FcpAudit.DEFAULT_PASS]: frameTrace,
},
devtoolsLogs: {
[FcpAudit.DEFAULT_PASS]: frameDevtoolsLog,
},
};

const context = getFakeContext({formFactor: 'mobile', throttlingMethod: 'provided'});
const result = await FcpAudit.audit(artifacts, context);
assert.equal(result.score, 0.06);
assert.equal(result.numericValue, 5668.275);
});
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we didn't have any unit test that needed updating from the score change (aside from sample_v2 snapshot ones), which is always suspicious, so added one here

});
4 changes: 2 additions & 2 deletions lighthouse-core/test/report/report-generator-test.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -104,8 +104,8 @@ describe('ReportGenerator', () => {
expect(lines.length).toBeGreaterThan(100);
expect(lines.slice(0, 3).join('\n')).toMatchInlineSnapshot(`
"requestedUrl,finalUrl,category,name,title,type,score
\\"http://localhost:10200/dobetterweb/dbw_tester.html\\",\\"http://localhost:10200/dobetterweb/dbw_tester.html\\",\\"Performance\\",\\"performance-score\\",\\"Overall Performance Category Score\\",\\"numeric\\",\\"0.64\\"
\\"http://localhost:10200/dobetterweb/dbw_tester.html\\",\\"http://localhost:10200/dobetterweb/dbw_tester.html\\",\\"Performance\\",\\"first-contentful-paint\\",\\"First Contentful Paint\\",\\"numeric\\",\\"0.51\\"
\\"http://localhost:10200/dobetterweb/dbw_tester.html\\",\\"http://localhost:10200/dobetterweb/dbw_tester.html\\",\\"Performance\\",\\"performance-score\\",\\"Overall Performance Category Score\\",\\"numeric\\",\\"0.6\\"
\\"http://localhost:10200/dobetterweb/dbw_tester.html\\",\\"http://localhost:10200/dobetterweb/dbw_tester.html\\",\\"Performance\\",\\"first-contentful-paint\\",\\"First Contentful Paint\\",\\"numeric\\",\\"0.24\\"
"
`);

Expand Down
4 changes: 2 additions & 2 deletions lighthouse-core/test/results/sample_v2.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@
"id": "first-contentful-paint",
"title": "First Contentful Paint",
"description": "First Contentful Paint marks the time at which the first text or image is painted. [Learn more](https://web.dev/first-contentful-paint/).",
"score": 0.51,
"score": 0.24,
"scoreDisplayMode": "numeric",
"numericValue": 3969.135,
"numericUnit": "millisecond",
Expand Down Expand Up @@ -5277,7 +5277,7 @@
}
],
"id": "performance",
"score": 0.64
"score": 0.6
},
"accessibility": {
"title": "Accessibility",
Expand Down