Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tests: move webtest expectations to platform/generic #13997

Merged
merged 3 commits into from
May 12, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion lighthouse-core/scripts/roll-to-devtools.sh
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,9 @@ echo -e "$check Locale JSON files copied."
lh_webtests_dir="third-party/chromium-webtests/webtests/http/tests/devtools/lighthouse/"
fe_webtests_dir="$dt_dir/test/webtests/http/tests/devtools/lighthouse"
rsync -avh "$lh_webtests_dir" "$fe_webtests_dir" --exclude="OWNERS" --delete

lh_webtests_exp_dir="third-party/chromium-webtests/webtests/platform/generic/http/tests/devtools/lighthouse/"
fe_webtests_exp_dir="$dt_dir/test/webtests/platform/generic/http/tests/devtools/lighthouse"
rsync -avh "$lh_webtests_exp_dir" "$fe_webtests_exp_dir" --exclude="OWNERS" --delete
echo ""
echo "Done. To run the webtests: "
echo " DEVTOOLS_PATH=\"$dt_dir\" yarn test-devtools"
3 changes: 2 additions & 1 deletion lighthouse-core/test/chromium-web-tests/web-test-server.sh
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@ set -x
python3 \
"$BLINK_TOOLS_PATH/latest/third_party/blink/tools/run_web_tests.py" \
--layout-tests-directory="$DEVTOOLS_PATH/test/webtests" \
--additional-platform-directory="$DEVTOOLS_PATH/test/webtests/platform/generic" \
--build-directory="$latest_content_shell/out" \
$*
status=$?
Expand All @@ -66,6 +67,6 @@ set -e

rm -rf "$LH_ROOT/.tmp/layout-test-results"
cp -r "$latest_content_shell/out/Release/layout-test-results" "$LH_ROOT/.tmp/layout-test-results"
cp "$DEVTOOLS_PATH/test/webtests/http/tests/devtools/lighthouse/"*-expected.txt "$LH_ROOT/third-party/chromium-webtests/webtests/http/tests/devtools/lighthouse"
cp "$DEVTOOLS_PATH/test/webtests/platform/generic/http/tests/devtools/lighthouse/"*-expected.txt "$LH_ROOT/third-party/chromium-webtests/webtests/platform/generic/http/tests/devtools/lighthouse"

exit $status
Original file line number Diff line number Diff line change
Expand Up @@ -10,14 +10,17 @@
await TestRunner.showPanel('lighthouse');

const containerElement = LighthouseTestRunner.getContainerElement();
const checkboxes = containerElement.querySelectorAll('.checkbox');
for (const checkbox of checkboxes) {
if (checkbox.textElement.textContent === 'Performance' || checkbox.textElement.textContent === 'Clear storage') {
continue;
}

if (checkbox.checkboxElement.checked) {
checkbox.checkboxElement.click();
const ensureDisabledNames = ['Accessibility', 'Best practices', 'SEO', 'Progressive Web App'];
for (const checkboxName of ensureDisabledNames) {
const checkboxes = Array.from(containerElement.querySelectorAll('.checkbox'));
for (const checkbox of checkboxes) {
if (checkbox.textElement.textContent !== checkboxName) {
continue;
}

if (checkbox.checkboxElement.checked) {
checkbox.checkboxElement.click();
}
}
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,24 +14,27 @@
});

const containerElement = LighthouseTestRunner.getContainerElement();
const checkboxes = containerElement.querySelectorAll('.checkbox');
for (const checkbox of checkboxes) {
if (checkbox.textElement.textContent === 'Performance' || checkbox.textElement.textContent === 'Clear storage') {
continue;
}
const ensureDisabledNames = ['Accessibility', 'Best practices', 'SEO', 'Progressive Web App'];
for (const checkboxName of ensureDisabledNames) {
const checkboxes = Array.from(containerElement.querySelectorAll('.checkbox'));
for (const checkbox of checkboxes) {
if (checkbox.textElement.textContent !== checkboxName) {
continue;
}

if (checkbox.checkboxElement.checked) {
checkbox.checkboxElement.click();
if (checkbox.checkboxElement.checked) {
checkbox.checkboxElement.click();
}
}
}

LighthouseTestRunner.dumpStartAuditState();

const Events = Lighthouse.LighthousePanel.getEvents();
const warningText = containerElement.querySelector('.lighthouse-warning-text');

// Wait for warning event to be handled
LighthouseTestRunner._panel().controller.addEventListener(Events.PageWarningsChanged, () => {
const warningText = containerElement.querySelector('.lighthouse-warning-text');
TestRunner.addResult(`Warning Text: ${warningText.textContent}`);
TestRunner.completeTest();
});
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,14 +10,17 @@
await TestRunner.showPanel('lighthouse');

const containerElement = LighthouseTestRunner.getContainerElement();
const checkboxes = containerElement.querySelectorAll('.checkbox');
for (const checkbox of checkboxes) {
if (checkbox.textElement.textContent === 'Performance' || checkbox.textElement.textContent === 'Clear storage') {
continue;
}

if (checkbox.checkboxElement.checked) {
checkbox.checkboxElement.click();
const ensureDisabledNames = ['Accessibility', 'Best practices', 'SEO', 'Progressive Web App'];
for (const checkboxName of ensureDisabledNames) {
const checkboxes = Array.from(containerElement.querySelectorAll('.checkbox'));
for (const checkbox of checkboxes) {
if (checkbox.textElement.textContent !== checkboxName) {
continue;
}

if (checkbox.checkboxElement.checked) {
checkbox.checkboxElement.click();
}
}
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,15 +10,17 @@
await TestRunner.showPanel('lighthouse');

const containerElement = LighthouseTestRunner.getContainerElement();
const checkboxes = containerElement.querySelectorAll('.checkbox');
for (const checkbox of checkboxes) {
if (checkbox.textElement.textContent === 'Progressive Web App' ||
checkbox.textElement.textContent === 'Clear storage') {
continue;
}
const ensureDisabledNames = ['Performance', 'Accessibility', 'Best practices', 'SEO'];
for (const checkboxName of ensureDisabledNames) {
const checkboxes = Array.from(containerElement.querySelectorAll('.checkbox'));
for (const checkbox of checkboxes) {
if (checkbox.textElement.textContent !== checkboxName) {
continue;
}

if (checkbox.checkboxElement.checked) {
checkbox.checkboxElement.click();
if (checkbox.checkboxElement.checked) {
checkbox.checkboxElement.click();
}
}
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,12 +22,30 @@
TestRunner.addResult('\n\n**Prevents audit with no categories**');
LighthouseTestRunner.openStartAudit();
const containerElement = LighthouseTestRunner.getContainerElement();
const checkboxes = containerElement.querySelectorAll('.checkbox');
checkboxes.forEach(checkbox => checkbox.checkboxElement.checked && checkbox.checkboxElement.click());
const ensureDisabledNames = ['Performance', 'Accessibility', 'Best practices', 'SEO', 'Progressive Web App'];
for (const checkboxName of ensureDisabledNames) {
const checkboxes = Array.from(containerElement.querySelectorAll('.checkbox'));
for (const checkbox of checkboxes) {
if (checkbox.textElement.textContent !== checkboxName) {
continue;
}

if (checkbox.checkboxElement.checked) {
checkbox.checkboxElement.click();
}
}
}
LighthouseTestRunner.dumpStartAuditState();

TestRunner.addResult('\n\n**Allows audit with a single category**');
checkboxes[0].checkboxElement.click();
const checkboxes = Array.from(containerElement.querySelectorAll('.checkbox'));
for (const checkbox of checkboxes) {
if (checkbox.textElement.textContent !== 'Performance') {
continue;
}

checkbox.checkboxElement.click();
}
LighthouseTestRunner.dumpStartAuditState();

TestRunner.addResult('\n\n**Allows audit on undockable page**');
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,14 +10,17 @@
await TestRunner.showPanel('lighthouse');

const containerElement = LighthouseTestRunner.getContainerElement();
const checkboxes = containerElement.querySelectorAll('.checkbox');
for (const checkbox of checkboxes) {
if (checkbox.textElement.textContent === 'Performance' || checkbox.textElement.textContent === 'Clear storage') {
continue;
}

if (checkbox.checkboxElement.checked) {
checkbox.checkboxElement.click();
const ensureDisabledNames = ['Accessibility', 'Best practices', 'SEO', 'Progressive Web App'];
for (const checkboxName of ensureDisabledNames) {
const checkboxes = Array.from(containerElement.querySelectorAll('.checkbox'));
for (const checkbox of checkboxes) {
if (checkbox.textElement.textContent !== checkboxName) {
continue;
}

if (checkbox.checkboxElement.checked) {
checkbox.checkboxElement.click();
}
}
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Tests that Lighthouse block requests specified in DevTools BlockedURLsPane.
[x] Legacy navigation
[x] Clear storage
[x] Simulated throttling
Generate report: enabled visible
Analyze page load: enabled visible

Requests made: http://127.0.0.1:8000/devtools/lighthouse/resources/lighthouse-basic.html
Requests blocked: http://127.0.0.1:8000/devtools/lighthouse/resources/lighthouse-basic.css
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,6 @@ Tests that Lighthouse panel displays a warning when important data may affect pe
[x] Legacy navigation
[x] Clear storage
[x] Simulated throttling
Generate report: enabled visible
Analyze page load: enabled visible
Warning Text: There may be stored data affecting loading performance in these locations: Web SQL, IndexedDB. Audit this page in an incognito window to prevent those resources from affecting your scores.

Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Tests that mobile emulation works.
[x] Legacy navigation
[x] Clear storage
[x] Simulated throttling
Generate report: enabled visible
Analyze page load: enabled visible

=============== Lighthouse Results ===============
URL: http://127.0.0.1:8000/devtools/lighthouse/resources/lighthouse-emulate-pass.html
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Tests that lighthouse panel passes flags.
[x] Legacy navigation
[x] Clear storage
[ ] Simulated throttling
Generate report: enabled visible
Analyze page load: enabled visible

=============== Lighthouse Results ===============
formFactor: desktop
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Tests that Lighthouse report is translated.
[x] Legacy navigation
[x] Clear storage
[x] Simulated throttling
Generate report: enabled visible
Analyze page load: enabled visible
resolved to locale es

i18n footerIssue: "Notificar un problema"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Tests that audits panel works when only the pwa category is selected.
[x] Legacy navigation
[x] Clear storage
[x] Simulated throttling
Generate report: enabled visible
Analyze page load: enabled visible

=============== Audits run ===============
apple-touch-icon
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ Tests that audits panel prevents run of unauditable pages.
[x] Clear storage
[x] Simulated throttling
Help text: At least one category must be selected.
Generate report: disabled visible
Analyze page load: disabled visible


**Allows audit with a single category**
Expand All @@ -30,7 +30,7 @@ Generate report: disabled visible
[x] Legacy navigation
[x] Clear storage
[x] Simulated throttling
Generate report: enabled visible
Analyze page load: enabled visible


**Allows audit on undockable page**
Expand All @@ -45,7 +45,7 @@ Generate report: enabled visible
[x] Legacy navigation
[x] Clear storage
[x] Simulated throttling
Generate report: enabled visible
Analyze page load: enabled visible


**Prevents audit on internal page**
Expand All @@ -62,5 +62,5 @@ URL: about:blank
[x] Clear storage
[x] Simulated throttling
Help text: Can only audit HTTP/HTTPS pages and Chrome extensions. Navigate to a different page to start an audit.
Generate report: disabled visible
Analyze page load: disabled visible

Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Tests that navigation mode works correctly.
[ ] Legacy navigation
[x] Clear storage
[x] Simulated throttling
Generate report: enabled visible
Analyze page load: enabled visible

=============== Lighthouse Status Updates ===============
Loading…
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Tests that audits panel works.
[x] Legacy navigation
[x] Clear storage
[x] Simulated throttling
Generate report: enabled visible
Analyze page load: enabled visible

=============== Lighthouse Status Updates ===============
Loading…
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Tests that audits panel renders View Trace button.
[x] Legacy navigation
[x] Clear storage
[x] Simulated throttling
Generate report: enabled visible
Analyze page load: enabled visible

=============== Audits run ===============
bootup-time
Expand Down