Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: performance benchmarking #647

Draft
wants to merge 30 commits into
base: develop
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 20 commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
6c9d049
fix repo after force push
grablack Oct 15, 2021
0f5d3f9
Merge branch 'paypal:develop' into benchmarking
grablack Oct 15, 2021
3f5a4f3
restore package version
grablack Oct 15, 2021
05aa205
Merge branch 'benchmarking' of github.com:grablack/paypal-messaging-c…
grablack Oct 15, 2021
5eb8f66
delay start for dev server if url not provided
grablack Oct 18, 2021
6398cb0
space in file name
grablack Oct 18, 2021
cf70b07
update output dir
grablack Oct 18, 2021
26a3ffb
add timeout
grablack Oct 18, 2021
c7a9d17
output lightscores in root
grablack Oct 18, 2021
d2ceb09
lighthouse folder
grablack Oct 18, 2021
eed2994
remove extra zeros
grablack Oct 19, 2021
183be8f
refactor
grablack Oct 25, 2021
9269951
refactor
grablack Oct 25, 2021
d0c2a3e
update readme
grablack Oct 26, 2021
84d2ce7
update readme
grablack Oct 26, 2021
5060126
update labels
grablack Nov 9, 2021
d870dd5
update lighthouse gathering to async
grablack Dec 13, 2021
efe8e1c
added components file
grablack Dec 14, 2021
7d323e4
linting comments
grablack Dec 15, 2021
0ea6a51
comments
grablack Dec 16, 2021
3f35341
use entries to consolidate
grablack Dec 17, 2021
2ed9e18
changes
grablack Dec 17, 2021
40a77ab
need settimeout for jenkins
grablack Dec 17, 2021
07f6bad
make settimout a function
grablack Dec 17, 2021
3c1d786
move to modules
grablack Jan 5, 2022
e4e9747
update variable from puppeteer
grablack Jan 5, 2022
9c68b19
update comments
grablack Jan 5, 2022
7fdd014
combine desktop mobile score gathering
grablack Jan 10, 2022
e4e17b9
comment changes
grablack Jan 10, 2022
99c89a9
Merge branch 'develop' into benchmarking
grablack Mar 2, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@ coverage
oom_1
**/__diff_output__/**
tests/__reports__/*.html
lighthouse
.lighthouseci

# OS generated files #
######################
Expand Down
4 changes: 4 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,10 @@ npm start

To run functional tests, first run `npm run dev:standalone` in one command line instance and `npm run test:func` in a second command line instance. The `dev:standalone` command creates static pages that the functional tests are run on.

## Performance Benchmark

To run performance benchmark, first run `npm run dev:standalone` in one command line instance and `npm run benchmark` in a second command line instance. The `dev:standalone` command creates static pages that the functional tests are run on. To use lighthouse benchmarking install the lighthouse cli via `npm install -g @lhci/cli`. Prior to running benchmark run `LIGHTHOUSE_URL={URL_WITH_NO_BRACKETS} lhci autorun --config=./tools/performance/config/lighthouserc-desktop.js` and `LIGHTHOUSE_URL={URL_WITH_NO_BRACKETS} lhci autorun --config=./tools/performance/config/lighthouserc-mobile.js`

## Releasing

This package is published weekly, **Every Wednesday**. Please [view our Changelog](CHANGELOG.md) to stay updated with bug fixes and new features.
2 changes: 2 additions & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@
"build:sandbox": "webpack --env.NODE_ENV=sandbox",
"build:production": "webpack --env.NODE_ENV=production",
"build:analyze": "webpack --env.analyze",
"build:analyzeComponents": "webpack --env.analyzeComponents",
"build:analyzeMComponents": "webpack --env.analyzeMessageComponents",
"build:demo": "webpack --env.demo",
"dev": "webpack-dev-server --config webpack.config.dev.js --env.NODE_ENV=local --env.TARGET=sdk --env.STAGE_TAG=local",
"dev:standalone": "webpack-dev-server --config webpack.config.dev.js --env.TARGET=standalone --env.NODE_ENV=local --env.STAGE_TAG=local",
Expand Down
40 changes: 40 additions & 0 deletions tools/performance/compile.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
const fs = require('fs');

const basePath = process.cwd();

let html = `<html>
<head>
<title>Performance Benchmark</title>
<style>
td:first-of-type {
min-width: 250px;
max-width:500px;
overflow: auto;
padding-right: 1rem;
}
td {
padding: 0.25rem 0.45rem;
}
</style>
</head>
<body>
<div>${new Date().toDateString()}</div>`;
try {
// get html from json files
html += JSON.parse(fs.readFileSync(`${basePath}/dist/lighthouseScores.json`)).html;
paypal-are marked this conversation as resolved.
Show resolved Hide resolved
html += JSON.parse(fs.readFileSync(`${basePath}/dist/components.json`)).html;
html += JSON.parse(fs.readFileSync(`${basePath}/dist/metrics.json`)).html;

html += `</body></html>`;

// output report html document
fs.writeFile(`${basePath}/dist/performanceData${new Date().toISOString()}.html`, html, err => {
if (err) {
console.log('Error occured when writting performanceData.html');
} else {
console.log('performanceData.html created');
}
});
} catch (err) {
throw new Error('Make sure lighthouse.json, components.json, and metrics.json exist first');
}
100 changes: 100 additions & 0 deletions tools/performance/components.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
const fs = require('fs');

const basePath = process.cwd();

/**
* Get the json data for the analyzed bundles
* @param {string} file - file name
* @returns {promise} - array of objects
*/
const getComponentFileData = file => {
paypal-are marked this conversation as resolved.
Show resolved Hide resolved
return new Promise(resolve => {
fs.readFile(`${basePath}/dist/${file}.json`, { encoding: 'utf-8' }, (err, data) => {
if (err) {
resolve([]);
}

resolve(
JSON.parse(data).map(row => ({
label: row.label,
parsedSize: row.parsedSize,
gzipSize: row.gzipSize,
groups: row.groups
}))
);
});
});
};

/**
* Create an array of promises for the bundle data
* @returns {array} - array of promises
*/
const getComponentJson = () => {
return [getComponentFileData('messagesReport'), getComponentFileData('componentsReport')];
};

/**
* Create HTML table row
* @param {array} jsonDataArray - each row is a file object
* @returns {string} - HTML table rows
*/
const getFileSizeTableRowHtml = jsonDataArray => {
return jsonDataArray
.map(row => `<tr><td>${row.label}</td><td>${row.parsedSize} bytes</td><td>${row.gzipSize} bytes</td></tr>`)
.join('');
};

/**
* Create HTML for all components
* @param {array} messagesReport - bundle data for messages
* @param {array} componentsReport - bundle data for components
* @returns {string} - html for all components
*/
const getComponentHtml = (messagesReport, componentsReport) => {
const headings = `<tr><td>Name</td><td>Unzipped</td><td>Gzipped</td></tr>`;
let html = `<h2>NPM Modules</h2>`;
html += `<table><tr><td>Largest</td><td>${messagesReport[0].groups[0].groups[0].label}</td><td>${messagesReport[0].groups[0].groups[0].parsedSize} bytes (unzipped)</td></tr></table>`;

// Messaging Size
const messaging = getFileSizeTableRowHtml(messagesReport);

// Modals Sizes
const modals = getFileSizeTableRowHtml(componentsReport);

// Largest file sizes
const largestFiles = getFileSizeTableRowHtml(
[...componentsReport, ...messagesReport].sort((a, b) => b.gzipSize - a.gzipSize).splice(0, 3)
);

html += `<h2>File Sizes</h2>`;
paypal-are marked this conversation as resolved.
Show resolved Hide resolved
html += `<table>`;
html += headings;
html += messaging;
html += modals;
html += `</table>`;

html += `<h2>Largest Files</h2>`;

html += `<table>`;
html += headings;
html += largestFiles;
html += `</table>`;

return html;
};

Promise.all(getComponentJson()).then(reports => {
const [messagesReport, componentsReport] = reports;
const html = getComponentHtml(messagesReport, componentsReport);

// save html to json file to be used in compile.js
fs.writeFile('dist/components.json', JSON.stringify({ html: `${html}` }), err => {
if (err) {
console.log('components.json failed to save');
console.log(err);
} else {
console.log('components.json saved');
}
});
});
38 changes: 38 additions & 0 deletions tools/performance/config/lighthouserc-desktop.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
const process = require('process');

if (!process.env.ARG) process.env.ARG = process.argv.pop().replace('--', '');

module.exports = {
ci: {
collect: {
numberOfRuns: 2,
url: [
`${process.env.LIGHTHOUSE_URL}/${process.env.STAGE_TAG ? `?stage_tag=${process.env.STAGE_TAG}` : ''}`,
`${process.env.LIGHTHOUSE_URL}/category/jewelry${
process.env.STAGE_TAG ? `?stage_tag=${process.env.STAGE_TAG}` : ''
}`,
`${process.env.LIGHTHOUSE_URL}/product/7${
process.env.STAGE_TAG ? `?stage_tag=${process.env.STAGE_TAG}` : ''
}`,
`${process.env.LIGHTHOUSE_URL}/cart${
process.env.STAGE_TAG ? `?stage_tag=${process.env.STAGE_TAG}` : ''
}`,
`${process.env.LIGHTHOUSE_URL}/checkout${
process.env.STAGE_TAG ? `?stage_tag=${process.env.STAGE_TAG}` : ''
}`
],
settings: {
preset: 'desktop',
output: 'json',
maxWaitForLoad: 10000,
chromeFlags: '--no-sandbox --disable-storage-reset --disable-dev-shm-usage --in-process-gpu'
},
psiStrategy: 'desktop'
},
upload: {
target: 'filesystem',
outputDir: 'lighthouse',
reportFilenamePattern: 'desktop-report-%%PATHNAME%%-%%DATETIME%%.%%EXTENSION%%'
}
}
};
37 changes: 37 additions & 0 deletions tools/performance/config/lighthouserc-mobile.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
const process = require('process');

if (!process.env.ARG) process.env.ARG = process.argv.pop().replace('--', '');

module.exports = {
ci: {
collect: {
numberOfRuns: 2,
url: [
`${process.env.LIGHTHOUSE_URL}/${process.env.STAGE_TAG ? `?stage_tag=${process.env.STAGE_TAG}` : ''}`,
`${process.env.LIGHTHOUSE_URL}/category/jewelry${
process.env.STAGE_TAG ? `?stage_tag=${process.env.STAGE_TAG}` : ''
}`,
`${process.env.LIGHTHOUSE_URL}/product/7${
process.env.STAGE_TAG ? `?stage_tag=${process.env.STAGE_TAG}` : ''
}`,
`${process.env.LIGHTHOUSE_URL}/cart${
process.env.STAGE_TAG ? `?stage_tag=${process.env.STAGE_TAG}` : ''
}`,
`${process.env.LIGHTHOUSE_URL}/checkout${
process.env.STAGE_TAG ? `?stage_tag=${process.env.STAGE_TAG}` : ''
}`
],
settings: {
output: 'json',
maxWaitForLoad: 10000,
chromeFlags: '--no-sandbox --disable-storage-reset --disable-dev-shm-usage --in-process-gpu'
},
psiStrategy: 'mobile'
},
upload: {
target: 'filesystem',
outputDir: 'lighthouse',
reportFilenamePattern: 'mobile-report-%%PATHNAME%%-%%DATETIME%%.%%EXTENSION%%'
}
}
};
Loading