Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: performance benchmarking #647

Draft
wants to merge 30 commits into
base: develop
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
6c9d049
fix repo after force push
grablack Oct 15, 2021
0f5d3f9
Merge branch 'paypal:develop' into benchmarking
grablack Oct 15, 2021
3f5a4f3
restore package version
grablack Oct 15, 2021
05aa205
Merge branch 'benchmarking' of github.com:grablack/paypal-messaging-c…
grablack Oct 15, 2021
5eb8f66
delay start for dev server if url not provided
grablack Oct 18, 2021
6398cb0
space in file name
grablack Oct 18, 2021
cf70b07
update output dir
grablack Oct 18, 2021
26a3ffb
add timeout
grablack Oct 18, 2021
c7a9d17
output lightscores in root
grablack Oct 18, 2021
d2ceb09
lighthouse folder
grablack Oct 18, 2021
eed2994
remove extra zeros
grablack Oct 19, 2021
183be8f
refactor
grablack Oct 25, 2021
9269951
refactor
grablack Oct 25, 2021
d0c2a3e
update readme
grablack Oct 26, 2021
84d2ce7
update readme
grablack Oct 26, 2021
5060126
update labels
grablack Nov 9, 2021
d870dd5
update lighthouse gathering to async
grablack Dec 13, 2021
efe8e1c
added components file
grablack Dec 14, 2021
7d323e4
linting comments
grablack Dec 15, 2021
0ea6a51
comments
grablack Dec 16, 2021
3f35341
use entries to consolidate
grablack Dec 17, 2021
2ed9e18
changes
grablack Dec 17, 2021
40a77ab
need settimeout for jenkins
grablack Dec 17, 2021
07f6bad
make settimout a function
grablack Dec 17, 2021
3c1d786
move to modules
grablack Jan 5, 2022
e4e9747
update variable from puppeteer
grablack Jan 5, 2022
9c68b19
update comments
grablack Jan 5, 2022
7fdd014
combine desktop mobile score gathering
grablack Jan 10, 2022
e4e17b9
comment changes
grablack Jan 10, 2022
99c89a9
Merge branch 'develop' into benchmarking
grablack Mar 2, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@ coverage
oom_1
**/__diff_output__/**
tests/__reports__/*.html
lighthouse
.lighthouseci

# OS generated files #
######################
Expand Down
71 changes: 43 additions & 28 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,44 +38,55 @@ npm start

## Scripts

- `build`
- runs [semantic-release/assets.sh](./scripts/semantic-release/assets.sh) to build assets for all environments in `dist/`
- `build`

- `build:<env>` where `<env>` is `stage`, `sandbox`, or `production`
- runs webpack with `NODE_ENV=<env>`
- runs [semantic-release/assets.sh](./scripts/semantic-release/assets.sh) to build assets for all environments in `dist/`

- `build:analyze`
- runs [webpack-bundle-analyzer](https://github.com/webpack-contrib/webpack-bundle-analyzer) in static gzip mode
- `build:<env>` where `<env>` is `stage`, `sandbox`, or `production`

- `build:demo`
- runs webpack with `env.demo` set
- runs webpack with `NODE_ENV=<env>`

- `dev`
- runs webpack-dev-server with `TARGET=sdk`, `NODE_ENV=local`, `STAGE_TAG=local`
- `build:analyze`

- `dev:<target>` where `<target>` is `standalone`, `modal`, or `lander`
- runs webpack-dev-server with `TARGET=<target>`, `NODE_ENV=local`, `STAGE_TAG=local`
- note: `modal` uses `TARGET=standalone-modal`
- runs [webpack-bundle-analyzer](https://github.com/webpack-contrib/webpack-bundle-analyzer) in static gzip mode

- `dev:<env>` where `<env>` is `stage`, `sandbox`, or `production`
- runs webpack-dev-server with `TARGET=standalone` and `NODE_ENV=<env>`
- `build:demo`

- `lint`
- checks our codebase for style issues
- runs webpack with `env.demo` set

- `preinstall`
- runs automatically on `npm install` and removes `node_modules/`
- `dev`

- `start`
- runs `npm run dev`
- runs webpack-dev-server with `TARGET=sdk`, `NODE_ENV=local`, `STAGE_TAG=local`

- `test`
- runs all unit tests
- `dev:<target>` where `<target>` is `standalone`, `modal`, or `lander`

- `test:<type>` where `<type>` is `func`, `func:nosnaps` or `func:ciupdate`
- `func` runs all snapshot functional tests
- `func:nosnaps` runs all non-snapshot functional tests
- `func:ciupdate` updates all snapshots generated by functional tests
- runs webpack-dev-server with `TARGET=<target>`, `NODE_ENV=local`, `STAGE_TAG=local`
- note: `modal` uses `TARGET=standalone-modal`

- `dev:<env>` where `<env>` is `stage`, `sandbox`, or `production`

- runs webpack-dev-server with `TARGET=standalone` and `NODE_ENV=<env>`

- `lint`

- checks our codebase for style issues

- `preinstall`

- runs automatically on `npm install` and removes `node_modules/`

- `start`

- runs `npm run dev`

- `test`

- runs all unit tests

- `test:<type>` where `<type>` is `func`, `func:nosnaps` or `func:ciupdate`
- `func` runs all snapshot functional tests
- `func:nosnaps` runs all non-snapshot functional tests
- `func:ciupdate` updates all snapshots generated by functional tests

## Testing

Expand All @@ -95,12 +106,16 @@ Example
CONFIG_PATH=US/DEV_US_MULTI npm run test:func:snapshots -- --testPathPattern sdk
```

Alternatively, you can remove ` -- --testPathPattern {integrationType}` and just run the following to run tests on an account for all integration types.
Alternatively, you can remove `-- --testPathPattern {integrationType}` and just run the following to run tests on an account for all integration types.

```
CONFIG_PATH={locale}/{account} npm run test:func:snapshots
```

## Performance Benchmark

To run performance benchmark, first run `npm run dev:standalone` in one command line instance and `npm run benchmark` in a second command line instance. The `dev:standalone` command creates static pages that the functional tests are run on. To use lighthouse benchmarking install the lighthouse cli via `npm install -g @lhci/cli`. Prior to running benchmark run `LIGHTHOUSE_URL={URL_WITH_NO_BRACKETS} lhci autorun --config=./tools/performance/config/lighthouserc-desktop.js` and `LIGHTHOUSE_URL={URL_WITH_NO_BRACKETS} lhci autorun --config=./tools/performance/config/lighthouserc-mobile.js`

## Releasing

This package is published weekly, **Every Wednesday**. Please [view our Changelog](CHANGELOG.md) to stay updated with bug fixes and new features.
2 changes: 2 additions & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@
"build:sandbox": "webpack --env.NODE_ENV=sandbox",
"build:production": "webpack --env.NODE_ENV=production",
"build:analyze": "webpack --env.analyze",
"build:analyzeComponents": "webpack --env.analyzeComponents",
"build:analyzeMComponents": "webpack --env.analyzeMessageComponents",
"build:demo": "webpack --env.demo",
"dev": "webpack-dev-server --config webpack.config.dev.js --env.TARGET=sdk --env.NODE_ENV=local --env.STAGE_TAG=local",
"dev:ci": "webpack-dev-server --config webpack.config.dev.js --env.TARGET=ci --env.NODE_ENV=local --env.STAGE_TAG=local",
Expand Down
60 changes: 60 additions & 0 deletions tools/performance/compile.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
const fs = require('fs');

const { getComponentHtml } = require(`./components.js`);
const { createLighthouseHtml } = require(`./lighthouse.js`);
const { createMetricsHtml } = require(`./metrics.js`);
const basePath = process.cwd();

/**
* Write html to file
* @param {string} html - html
*/
const writeHtmlToFile = html =>
fs.writeFile(`${basePath}/dist/performanceData${new Date().toISOString()}.html`, html, err => {
if (err) {
console.log('Error occured when writting performanceData.html');
} else {
console.log('performanceData.html created');
}
});

/**
* Take data and create html string
* @param {object} data - contains all variables needed for final document
* @returns {string} - html
*/
const html = data => `<html>
<head>
<title>Performance Benchmark</title>
<style>
td:first-of-type {
min-width: 250px;
max-width:500px;
overflow: auto;
padding-right: 1rem;
}
td {
padding: 0.25rem 0.45rem;
word-wrap: break-word;
}
</style>
</head>
<body>
<div>${new Date().toDateString()}</div>
${createLighthouseHtml(data)}
${getComponentHtml(data)}
${createMetricsHtml(data)}
</body>
</html>`;

try {
const htmlJsonData = {
...JSON.parse(fs.readFileSync(`${basePath}/dist/lighthouseScores.json`)),
...JSON.parse(fs.readFileSync(`${basePath}/dist/components.json`)),
...JSON.parse(fs.readFileSync(`${basePath}/dist/metrics.json`))
};
// create html and write to file
writeHtmlToFile(html(htmlJsonData));
} catch (err) {
console.log(err);
}
99 changes: 99 additions & 0 deletions tools/performance/components.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
const fs = require('fs');

const basePath = process.cwd();

/**
* Get the json data for the analyzed bundles
* @param {string} file - file name
* @returns {promise} - array of objects
*/
const getComponentFileData = file =>
new Promise(resolve => {
fs.readFile(`${basePath}/dist/${file}.json`, { encoding: 'utf-8' }, (err, data) => {
if (err) {
console.log(err);
throw err;
}

resolve(
JSON.parse(data).map(row => ({
label: row.label,
parsedSize: row.parsedSize,
gzipSize: row.gzipSize,
groups: row.groups
}))
);
});
});

/**
* Create an array of promises for the bundle data
* @returns {array} - array of promises
*/
const getComponentJson = () => {
return [getComponentFileData('messagesReport'), getComponentFileData('componentsReport')];
};

/**
* Create HTML table row
* @param {array} jsonDataArray - each row is a file object
* @returns {string} - HTML table rows
*/
const getFileSizeTableRowHtml = jsonDataArray => {
return jsonDataArray
.map(row => `<tr><td>${row.label}</td><td>${row.parsedSize} bytes</td><td>${row.gzipSize} bytes</td></tr>`)
.join('');
};

/**
* Create HTML for all components
* @param {array} messagesReport - bundle data for messages
* @param {array} componentsReport - bundle data for components
* @returns {string} - html for all components
*/
const getComponentHtml = ({ messagesReport, componentsReport }) => {
// Messaging Size
const messaging = getFileSizeTableRowHtml(messagesReport);

// Modals Sizes
const modals = getFileSizeTableRowHtml(componentsReport);

// Largest file sizes
const largestFiles = getFileSizeTableRowHtml(
[...componentsReport, ...messagesReport].sort((a, b) => b.gzipSize - a.gzipSize).splice(0, 3)
);

const headings = `<tr><td>Name</td><td>Unzipped</td><td>Gzipped</td></tr>`;

return `<h2>NPM Modules</h2>
<table><tr><td>Largest</td><td>${messagesReport[0].groups[0].groups[0].label}</td><td>${messagesReport[0].groups[0].groups[0].parsedSize} bytes (unzipped)</td></tr></table>
<h2>File Sizes</h2>
<table>${headings}${messaging}${modals}</table>
<h2>Largest Files</h2>
<table>${headings}${largestFiles}</table>`;
};

/**
* Save the html to a json file
* @param {*} json
*/
const outputLighthouseJson = json => {
// save html to json file to be used in compile.js
fs.writeFile('dist/components.json', JSON.stringify({ ...json }), err => {
if (err) {
console.log('components.json failed to save');
console.log(err);
} else {
console.log('components.json saved');
}
});
};

if (process.env.BENCHMARK === 'true') {
Promise.all(getComponentJson()).then(reports => {
const [messagesReport, componentsReport] = reports;
outputLighthouseJson({ messagesReport, componentsReport });
});
}

module.exports = { getComponentHtml };
38 changes: 38 additions & 0 deletions tools/performance/config/lighthouserc-desktop.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
const process = require('process');

if (!process.env.ARG) process.env.ARG = process.argv.pop().replace('--', '');

module.exports = {
ci: {
collect: {
numberOfRuns: 2,
url: [
`${process.env.LIGHTHOUSE_URL}/${process.env.STAGE_TAG ? `?stage_tag=${process.env.STAGE_TAG}` : ''}`,
`${process.env.LIGHTHOUSE_URL}/category/jewelry${
process.env.STAGE_TAG ? `?stage_tag=${process.env.STAGE_TAG}` : ''
}`,
`${process.env.LIGHTHOUSE_URL}/product/7${
process.env.STAGE_TAG ? `?stage_tag=${process.env.STAGE_TAG}` : ''
}`,
`${process.env.LIGHTHOUSE_URL}/cart${
process.env.STAGE_TAG ? `?stage_tag=${process.env.STAGE_TAG}` : ''
}`,
`${process.env.LIGHTHOUSE_URL}/checkout${
process.env.STAGE_TAG ? `?stage_tag=${process.env.STAGE_TAG}` : ''
}`
],
settings: {
preset: 'desktop',
output: 'json',
maxWaitForLoad: 10000,
chromeFlags: '--no-sandbox --disable-storage-reset --disable-dev-shm-usage --in-process-gpu'
},
psiStrategy: 'desktop'
},
upload: {
target: 'filesystem',
outputDir: 'lighthouse',
reportFilenamePattern: 'desktop-report-%%PATHNAME%%-%%DATETIME%%.%%EXTENSION%%'
}
}
};
37 changes: 37 additions & 0 deletions tools/performance/config/lighthouserc-mobile.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
const process = require('process');

if (!process.env.ARG) process.env.ARG = process.argv.pop().replace('--', '');

module.exports = {
ci: {
collect: {
numberOfRuns: 2,
url: [
`${process.env.LIGHTHOUSE_URL}/${process.env.STAGE_TAG ? `?stage_tag=${process.env.STAGE_TAG}` : ''}`,
`${process.env.LIGHTHOUSE_URL}/category/jewelry${
process.env.STAGE_TAG ? `?stage_tag=${process.env.STAGE_TAG}` : ''
}`,
`${process.env.LIGHTHOUSE_URL}/product/7${
process.env.STAGE_TAG ? `?stage_tag=${process.env.STAGE_TAG}` : ''
}`,
`${process.env.LIGHTHOUSE_URL}/cart${
process.env.STAGE_TAG ? `?stage_tag=${process.env.STAGE_TAG}` : ''
}`,
`${process.env.LIGHTHOUSE_URL}/checkout${
process.env.STAGE_TAG ? `?stage_tag=${process.env.STAGE_TAG}` : ''
}`
],
settings: {
output: 'json',
maxWaitForLoad: 10000,
chromeFlags: '--no-sandbox --disable-storage-reset --disable-dev-shm-usage --in-process-gpu'
},
psiStrategy: 'mobile'
},
upload: {
target: 'filesystem',
outputDir: 'lighthouse',
reportFilenamePattern: 'mobile-report-%%PATHNAME%%-%%DATETIME%%.%%EXTENSION%%'
}
}
};
Loading