Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor Phantomas to run using headless Chrome #726

Merged
merged 158 commits into from
Feb 2, 2019
Merged
Changes from 1 commit
Commits
Show all changes
158 commits
Select commit Hold shift + click to select a range
c265c4b
[ImgBot] Optimize images
ImgBotApp Dec 2, 2018
c9634ff
Dependencies cleanup
macbre Dec 27, 2018
ddf2101
Remove lib/engines
macbre Dec 27, 2018
3527430
Require Node.js 8+
macbre Dec 27, 2018
7db6109
Code cleanup and Promise-based async handling
macbre Dec 27, 2018
eda8602
Remove q module
macbre Dec 27, 2018
cb94c16
browser: emit more events
macbre Dec 27, 2018
34bcbc4
Resolve a promise with a results wrapper
macbre Dec 27, 2018
d26844a
Linting fixes
macbre Dec 27, 2018
ba0c1f5
Create a small scope object and pass it to a module
macbre Dec 27, 2018
ee48d67
Simplify modules export structure
macbre Dec 27, 2018
b8920f4
phantomas.emit - log events
macbre Dec 27, 2018
827e15f
Browser: log a path to Chromium binary
macbre Dec 28, 2018
f9e5431
Remove old code required by PhantomJS layer
macbre Dec 28, 2018
923b5e1
Loader: extensions and core modules
macbre Dec 28, 2018
1b8c39c
Extensions loaded
macbre Dec 28, 2018
3e7136b
MaxListenersExceededWarning: Possible EventEmitter memory leak detected
macbre Dec 28, 2018
ffb5db3
Use low-level Chrome Devtools Protocol when handling network traffic …
macbre Dec 28, 2018
41a4589
More work on network traffic handling
macbre Dec 28, 2018
c65d5db
Improved handling of HTTP responses + types detection
macbre Dec 29, 2018
6054f68
modules/assetsTypes: bodySize is now reliable, yay!
macbre Dec 29, 2018
92e08be
Add js-yaml (required by tests) and get rid of simple-queue
macbre Dec 29, 2018
da3b1fb
test/modules/mock.js: use new, simpler module structure
macbre Dec 29, 2018
d66b726
Some work on tests
macbre Dec 29, 2018
772699a
Travis: install Chrome
macbre Dec 30, 2018
f7dd1f5
Set up Travis for Chrome stable
macbre Dec 30, 2018
ab1de07
Improve tests code
macbre Dec 30, 2018
3e5b9e6
Browser: handle PHANTOMAS_CHROMIUM_EXECUTABLE env variable
macbre Dec 30, 2018
84b4473
loader: be compatible with Node.js v8
macbre Dec 30, 2018
81bc6c9
requestsMonitor: all tests now pass
macbre Dec 30, 2018
4a2fdc0
phantomas object will probably go away
macbre Dec 30, 2018
afbab88
vows: v0.8.2
macbre Dec 30, 2018
4da2f27
loader: linting + JSdoc
macbre Dec 30, 2018
004c28f
examples: show offenders
macbre Dec 30, 2018
a5eba26
Fix test/modules/requestsStats-test.js
macbre Dec 30, 2018
82168ef
test/results-test.js: offenders can be objects too
macbre Dec 30, 2018
0931e88
requestsMonitor: handle base64-encoded assets
macbre Dec 30, 2018
c3fb168
Use http-serve (with gzip compression)
macbre Dec 31, 2018
048ea8d
assetsTypes: more tests do pass now
macbre Dec 31, 2018
7b23a46
Inject core/scope.js into the browser
macbre Jan 1, 2019
971cbe5
Browser: handle connection errors within a promise
macbre Jan 1, 2019
081e8be
requestsMonitor: lowercase response headers
macbre Jan 1, 2019
23f889f
Resolve "Protocol error (Runtime.callFunctionOn): Target closed."
macbre Jan 2, 2019
3bd0460
Inject phantomas page scope JS code via page.evaluateOnNewDocument wh…
macbre Jan 6, 2019
79e29fb
domQueries: move page scope code into a separate file and run it via …
macbre Jan 6, 2019
1466493
index.js: expose the function that will pass events from page scope c…
macbre Jan 6, 2019
aa31f8c
Use __phantomas_emit to send messages from page scope code to Node.js…
macbre Jan 6, 2019
a140879
setMetric: pass calls from page scope code + default to zero when no …
macbre Jan 6, 2019
34b4953
browser: pass pageerror event from Puppeteer to phantomas + test case…
macbre Jan 6, 2019
c4a8298
Remove old core/ files that were running in PhantomJS scope
macbre Jan 8, 2019
9baa092
examples: add a short README
macbre Jan 8, 2019
f3443f7
scopeMessage: handle log and addOffender called from page scope code
macbre Jan 8, 2019
d42d891
A separate log label for phantomas.log calls from page scope code
macbre Jan 8, 2019
3163a6f
domMutations: module works again :)
macbre Jan 8, 2019
6ea4cc3
domComplexity: start a refactoring
macbre Jan 8, 2019
a236813
domComplexity: set nodesWithInlineCSS metric
macbre Jan 8, 2019
c6ff68c
documentHeight: refactor the module
macbre Jan 9, 2019
6bf839e
console: add a test case
macbre Jan 9, 2019
43abdfa
localStorageEntries: refactor a module
macbre Jan 9, 2019
bd6ae87
Browser: set phantomas-specific user agent
macbre Jan 11, 2019
dc4231b
jQuery: port the module
macbre Jan 11, 2019
a04766f
events: module ported
macbre Jan 11, 2019
e705c09
ajaxRequest: port the module
macbre Jan 12, 2019
65cefc0
globalVariables: port the module
macbre Jan 12, 2019
3f9b6eb
Scope: expose getParam function + autoload scope.js for modules
macbre Jan 12, 2019
1cec8e0
javaScriptBottlenecks: module ported
macbre Jan 12, 2019
225228d
modules: scope.js is now autoloaded if exists
macbre Jan 12, 2019
4014a97
javaScriptBottlenecks: remove debug console.log() calls
macbre Jan 12, 2019
40c221f
modules: call spyEnabled(false) to avoid incorrect reporting of event…
macbre Jan 12, 2019
b34e084
jQuery: check globalVariables offenders
macbre Jan 12, 2019
fa05081
test: comment out wait-* test - vows hangs because of them
macbre Jan 12, 2019
d6126ce
jquery-multiple: make tests more deterministic
macbre Jan 12, 2019
2bdc644
imagesScaledDown: metric works again
macbre Jan 12, 2019
e96e130
hiddenImages: set metric properly
macbre Jan 12, 2019
67b26ae
lazyLoadableImagesBelowTheFold: works again
macbre Jan 12, 2019
5b9eeb0
Emit "milestone" event + port requestsTo module
macbre Jan 13, 2019
7b8a47e
Cookie: use page.setCookie
macbre Jan 13, 2019
e6c66ba
Bring back --no-externals mode
macbre Jan 14, 2019
26edeaf
Support --allow-domain switch
macbre Jan 14, 2019
14d34c7
Travis: run on Xenial
macbre Jan 14, 2019
fa64733
requestsMonitor: properly set entry.transferedSize when requests inte…
macbre Jan 14, 2019
fe224b1
Add handling of --wait-for-network-idle
macbre Jan 19, 2019
df4b22c
Make js-redirect test pass
macbre Jan 19, 2019
f80363b
timing.html test passed as well
macbre Jan 19, 2019
9178969
Add test case for "headersBiggerThanContent" metric
macbre Jan 19, 2019
af42456
domains: structured offenders
macbre Jan 19, 2019
8215220
Structured offenders for domQueries metrics
macbre Jan 19, 2019
438f37f
nodesWithInlineCSS: structured offender
macbre Jan 19, 2019
500ce11
gzipRequests: structured offender
macbre Jan 19, 2019
ecdfa63
base64Size: add a test case
macbre Jan 19, 2019
154679d
cachingTooShort: add a test case
macbre Jan 19, 2019
4915aa7
Allow modules tests to assert offenders
macbre Jan 19, 2019
a2654a6
browser: add response.getContent() helper
macbre Jan 20, 2019
be4691a
browser.js: log how many bytes where fetched via netrwork
macbre Jan 20, 2019
a1ac85f
Install analyze-css v0.12.7
macbre Jan 20, 2019
8973a2a
Add addToAvgMetric function to Results object
macbre Jan 20, 2019
f9d5b2d
scope: expose addToAvgMetric and querySelectorAll helpers
macbre Jan 20, 2019
0712b40
analyze-css: use a npm module
macbre Jan 20, 2019
118ce1b
analyzeCss: handle inline CSS + all tests now pass
macbre Jan 20, 2019
feb243f
Use eslint instead of jshint
macbre Jan 20, 2019
2d0ec04
Remove scripts/phantomas.js
macbre Jan 20, 2019
5ac5357
eslint: work in progress (55 -> 38 problems)
macbre Jan 20, 2019
2799fea
eslint: work in progress (38 -> 18 problems)
macbre Jan 20, 2019
966ae00
Code linting completed
macbre Jan 21, 2019
452ba02
repaints: remove as there is no window.mozPaintCount in Chrome
macbre Jan 21, 2019
430e3d9
A small test case for promise API
macbre Jan 21, 2019
7cf9cd2
requestsStats: add test cases for smallestResponse and biggestRespons…
macbre Jan 22, 2019
ea85985
Add offenders for "requests" metric - url, response type and size
macbre Jan 22, 2019
11dcf0c
headers: use entry.headersSize
macbre Jan 22, 2019
8d2685f
Promise: reject when not given a string as URL
macbre Jan 22, 2019
2b37372
Update metrics metadata file
macbre Jan 22, 2019
2e5519d
Support timeout option
macbre Jan 22, 2019
175536d
Merge pull request #723 from macbre/imgbot
macbre Jan 22, 2019
837f44a
Add asserts for /jquery-ajax.html
macbre Jan 26, 2019
0a6e6bb
Travis: install Chrome version that is suggested by Puppeteer
macbre Jan 26, 2019
4dcbee8
package.json: add keywords and update js-yaml
macbre Jan 26, 2019
4d2cd71
metadata.json: add testsCovered entry
macbre Jan 26, 2019
4ae4873
Travis: run tests on Node.js LTS version only
macbre Jan 26, 2019
e6e67e2
Cover bodySize and contentLength metrics with tests
macbre Jan 26, 2019
cc19af3
Add test case for "iframesCount" metric
macbre Jan 26, 2019
480239a
/jquery-ajax.html test is flaky
macbre Jan 26, 2019
b833dbe
Revert "Travis: install Chrome version that is suggested by Puppeteer"
macbre Jan 26, 2019
d41e397
Add a test case for "notFound" metric
macbre Jan 26, 2019
1361126
browser.js: improved handling of blocked and "Not found" responses
macbre Jan 26, 2019
8647568
alerts: add a test case
macbre Jan 26, 2019
c5db797
assetsTypes: report size the same way as Chrome's dev tools do (heade…
macbre Jan 26, 2019
b4a0177
Add test case for jsonSize and jsonCount
macbre Jan 26, 2019
e8ac060
/jquery-ajax.html: set "wait-for-network-idle" option
macbre Jan 26, 2019
46b7a46
domMutations: increase test coverage
macbre Jan 26, 2019
7b8aba0
Fix /foo.json asset
macbre Jan 26, 2019
33382a8
Test case for consoleMessages
macbre Jan 26, 2019
2917364
Introduce phantomas.awaitBeforeClose()
macbre Jan 27, 2019
e99ac19
ajax-json.html: improve test stability by using "wait-for-selector" o…
macbre Jan 27, 2019
c9852e2
Remove /wait-for-selector.html tests asset
macbre Jan 27, 2019
35a41ec
Timeout should be provided in seconds, Puppeteer will then get it in …
macbre Jan 27, 2019
2f03bcf
waitForEvent: handle wait-for-event option + test cases
macbre Jan 27, 2019
b925b70
phantomas.awaitBeforeClose: pass page object to the closure
macbre Jan 27, 2019
513d803
Handle post-load-delay option
macbre Jan 27, 2019
0d0f207
Update README files
macbre Jan 27, 2019
d44cb90
Handle page-source option
macbre Jan 27, 2019
ed7a186
Introduce test/integration-test-extra.js + test "page-source" option …
macbre Jan 27, 2019
f1ead3e
Ignore HTML files
macbre Jan 27, 2019
cd186ec
Check the coverage of options tests
macbre Jan 27, 2019
7ef7877
Handle "screenshot" option
macbre Jan 27, 2019
dc2ac35
Add test cases for "screenshot" option
macbre Jan 27, 2019
3b25349
analyzeCss: use phantomas.awaitBeforeClose() to avoid race condition …
macbre Jan 27, 2019
405b53d
Handle "scroll" option + test cases
macbre Jan 28, 2019
c1f8a27
devices: read descriptors from puppeteer/DeviceDescriptors
macbre Feb 1, 2019
e5ea316
lib/index.js: awaitEmit - first rought implementation
macbre Feb 1, 2019
f16bdeb
AwaitEventEmitter: drop-in replacement for EventEmitter
macbre Feb 1, 2019
171d118
Use beforeClose event instead of a custom phantomas.awaitBeforeClose(…
macbre Feb 1, 2019
217425a
devices: add integration test cases
macbre Feb 2, 2019
7bdf51d
jshint: remove code annotations and config from package.json
macbre Feb 2, 2019
89eacbf
har: not all fields are set, but hey - now it works :)
macbre Feb 2, 2019
46db610
/https-fonts.html: add a test case
macbre Feb 2, 2019
f17b04a
Detect brotli compression
macbre Feb 2, 2019
d5494ed
staticAssets: add an integration test
macbre Feb 2, 2019
96554bc
httpAuth: handle HTTP authentication
macbre Feb 2, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
alerts: add a test case
Metrics found: 172 (tests coverage: 50.00%)

  /alerts-prompts.html
    ✓ should be generated
    ✓ should have "windowAlerts" metric properly set
    ✓ should have "windowConfirms" metric properly set
    ✓ should have "windowPrompts" metric properly set
    ✓ should have "windowAlerts" offender(s) properly set
    ✓ should have "windowConfirms" offender(s) properly set
    ✓ should have "windowPrompts" offender(s) properly set
macbre committed Jan 26, 2019

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
commit 86475688c231c0790889c01c21770a532d068896
18 changes: 18 additions & 0 deletions lib/browser.js
Original file line number Diff line number Diff line change
@@ -60,6 +60,24 @@ Browser.prototype.init = async () => {
this.events.emit('consoleLog', msg);
});

// https://github.com/GoogleChrome/puppeteer/blob/v1.11.0/docs/api.md#event-dialog
// Emitted when a JavaScript dialog appears, such as alert, prompt, confirm or beforeunload
this.page.on('dialog', async dialog => {
// https://github.com/GoogleChrome/puppeteer/blob/v1.11.0/docs/api.md#class-dialog
const message = dialog.message();
debug('dialog: %s [%s]', dialog._type, message);

switch(dialog._type) {
case 'alert':
case 'confirm':
case 'prompt':
this.events.emit(dialog._type, message);
break;
}

await dialog.dismiss();
});

// @see https://github.com/GoogleChrome/puppeteer/blob/v1.11.0/docs/api.md#event-pageerror
this.page.on('pageerror', x => {
const lines = x.message.split('\n');
9 changes: 6 additions & 3 deletions lib/metadata/metadata.json
Original file line number Diff line number Diff line change
@@ -75,21 +75,24 @@
},
"windowAlerts": {
"desc": "number of calls to window.alert",
"offenders": true,
"unit": "number",
"module": "alerts",
"testsCovered": false
"testsCovered": true
},
"windowConfirms": {
"desc": "number of calls to window.confirm",
"offenders": true,
"unit": "number",
"module": "alerts",
"testsCovered": false
"testsCovered": true
},
"windowPrompts": {
"desc": "number of calls to window.prompt",
"offenders": true,
"unit": "number",
"module": "alerts",
"testsCovered": false
"testsCovered": true
},
"cssBase64Length": {
"desc": "total length of base64-encoded data in CSS source (will warn about base64-encoded data bigger than 4 kB)",
29 changes: 13 additions & 16 deletions modules/alerts/alerts.js
Original file line number Diff line number Diff line change
@@ -4,26 +4,23 @@
*/
'use strict';

module.exports = function(phantomas) {
var alerts = [],
confirms = [],
prompts = [];
module.exports = phantomas => {
phantomas.setMetric('windowAlerts'); // @desc number of calls to window.alert
phantomas.setMetric('windowConfirms'); // @desc number of calls to window.confirm
phantomas.setMetric('windowPrompts'); // @desc number of calls to window.prompt

phantomas.on('alert', function(msg) {
alerts.push(msg);
phantomas.on('alert', msg => {
phantomas.incrMetric('windowAlerts');
phantomas.addOffender('windowAlerts', msg);
});

phantomas.on('confirm', function(msg) {
confirms.push(msg);
phantomas.on('confirm', msg => {
phantomas.incrMetric('windowConfirms');
phantomas.addOffender('windowConfirms', msg);
});

phantomas.on('prompt', function(msg) {
prompts.push(msg);
});

phantomas.on('report', function() {
phantomas.setMetric('windowAlerts', alerts.length); // @desc number of calls to window.alert
phantomas.setMetric('windowConfirms', confirms.length); // @desc number of calls to window.confirm
phantomas.setMetric('windowPrompts', prompts.length); // @desc number of calls to window.prompt
phantomas.on('prompt', msg => {
phantomas.incrMetric('windowPrompts');
phantomas.addOffender('windowPrompts', msg);
});
};
12 changes: 12 additions & 0 deletions test/integration-spec.yaml
Original file line number Diff line number Diff line change
@@ -461,3 +461,15 @@
- { url: 'http://127.0.0.1:8888/not_found/foo.js', type: 'other', size: 130 }
notFound:
- 'http://127.0.0.1:8888/not_found/foo.js'


# alerts, prompts, confirms
- url: "/alerts-prompts.html"
metrics:
windowAlerts: 1
windowConfirms: 1
windowPrompts: 1
offenders:
windowAlerts: [ 'Oh dear!' ]
windowConfirms: [ 'Oh dear?' ]
windowPrompts: [ "What's your name, dear?" ]
9 changes: 9 additions & 0 deletions test/webroot/alerts-prompts.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
<body>
<script>
alert('Oh dear!');
confirm('Oh dear?');
prompt('What\'s your name, dear?');

console.log('Done');
</script>
</body>