Skip to content

Commit

Permalink
Bug 1827236 - Disable benchmark tests on beta and release due to perm…
Browse files Browse the repository at this point in the history 
…a. r=sparky,jmaher, a=test-only

There is a perma failure for browsertime benchmark tests affecting mozilla-beta branch. It likely will affect mozilla-release as well.
This patch temporarily disables the tests from running on these branches.

Differential Revision: https://phabricator.services.mozilla.com/D175058
  • Loading branch information
@92kns
92kns committed Apr 10, 2023
1 parent 228b4a8 commit e31e632
Show file tree
Hide file tree
Showing 2 changed files with 34 additions and 31 deletions.
9 changes: 6 additions & 3 deletions taskcluster/ci/test/browsertime-desktop.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -317,23 +317,26 @@ browsertime-benchmark:
firefox:
by-test-platform:
windows.*-32.*: []
.*64.*shippable-qr/.*: [trunk, mozilla-release]
linux.*64.*shippable-qr/.*: [trunk]
(windows|macosx1015).*64.*shippable-qr/.*: [trunk, mozilla-release]
default: []
default: []
unity-webgl:
by-app:
firefox:
by-test-platform:
windows.*-32.*: []
.*64.*shippable-qr/.*: [trunk, mozilla-release]
linux.*64.*shippable-qr/.*: [trunk]
(windows|macosx1015).*64.*shippable-qr/.*: [trunk, mozilla-release]
default: []
default: []
default:
by-app:
firefox:
by-test-platform:
windows.*-32.*: []
.*64.*shippable-qr/.*: [trunk, mozilla-beta, mozilla-release]
linux.*64.*shippable-qr/.*: [trunk]
(windows|macosx1015).*64.*shippable-qr/.*: [trunk, mozilla-beta, mozilla-release]
default: []
default: []
max-run-time:
Expand Down
56 changes: 28 additions & 28 deletions testing/perfdocs/generated/raptor.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -116,8 +116,8 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr
* - **browsertime-benchmark-firefox-ares6**
- ✅
- ✅
- ✅
- ✅
- ❌
- ❌


.. list-table:: **test-macosx1015-64-shippable-qr/opt**
Expand Down Expand Up @@ -312,8 +312,8 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr
* - **browsertime-benchmark-firefox-assorted-dom**
- ✅
- ✅
- ✅
- ✅
- ❌
- ❌


.. list-table:: **test-macosx1015-64-shippable-qr/opt**
Expand Down Expand Up @@ -509,8 +509,8 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr
* - **browsertime-benchmark-firefox-jetstream2**
- ✅
- ✅
- ✅
- ✅
- ❌
- ❌


.. list-table:: **test-macosx1015-64-shippable-qr/opt**
Expand Down Expand Up @@ -706,8 +706,8 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr
* - **browsertime-benchmark-firefox-matrix-react-bench**
- ✅
- ✅
- ✅
- ✅
- ❌
- ❌


.. list-table:: **test-macosx1015-64-shippable-qr/opt**
Expand Down Expand Up @@ -893,8 +893,8 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr
* - **browsertime-benchmark-firefox-motionmark-animometer**
- ✅
- ✅
- ✅
- ✅
- ❌
- ❌


.. list-table:: **test-macosx1015-64-shippable-qr/opt**
Expand Down Expand Up @@ -1085,8 +1085,8 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr
* - **browsertime-benchmark-firefox-motionmark-htmlsuite**
- ✅
- ✅
- ✅
- ✅
- ❌
- ❌


.. list-table:: **test-macosx1015-64-shippable-qr/opt**
Expand Down Expand Up @@ -1497,8 +1497,8 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr
* - **browsertime-benchmark-firefox-speedometer**
- ✅
- ✅
- ✅
- ✅
- ❌
- ❌


.. list-table:: **test-macosx1015-64-shippable-qr/opt**
Expand Down Expand Up @@ -1734,8 +1734,8 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr
* - **browsertime-benchmark-firefox-speedometer**
- ✅
- ✅
- ✅
- ✅
- ❌
- ❌


.. list-table:: **test-macosx1015-64-shippable-qr/opt**
Expand Down Expand Up @@ -1983,7 +1983,7 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr
* - **browsertime-benchmark-firefox-speedometer3**
- ✅
- ✅
- ✅
- ❌
- ❌


Expand Down Expand Up @@ -2226,10 +2226,10 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr
- ❌
- ❌
* - **browsertime-benchmark-firefox-speedometer3**
- ✅
- ✅
- ✅
- ❌
- ❌


.. list-table:: **test-macosx1015-64-shippable-qr/opt**
Expand Down Expand Up @@ -2422,8 +2422,8 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr
* - **browsertime-benchmark-firefox-stylebench**
- ✅
- ✅
- ✅
- ✅
- ❌
- ❌


.. list-table:: **test-macosx1015-64-shippable-qr/opt**
Expand Down Expand Up @@ -2614,8 +2614,8 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr
* - **browsertime-benchmark-firefox-sunspider**
- ✅
- ✅
- ✅
- ✅
- ❌
- ❌


.. list-table:: **test-macosx1015-64-shippable-qr/opt**
Expand Down Expand Up @@ -2802,8 +2802,8 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr
* - **browsertime-benchmark-firefox-twitch-animation**
- ✅
- ✅
- ✅
- ✅
- ❌
- ❌


.. list-table:: **test-macosx1015-64-shippable-qr/opt**
Expand Down Expand Up @@ -3007,10 +3007,10 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr
- ❌
- ❌
* - **browsertime-benchmark-firefox-unity-webgl**
- ✅
- ✅
- ✅
- ❌
- ❌


.. list-table:: **test-macosx1015-64-shippable-qr/opt**
Expand Down Expand Up @@ -3245,10 +3245,10 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr
- ❌
- ❌
* - **browsertime-benchmark-firefox-unity-webgl**
- ✅
- ✅
- ✅
- ❌
- ❌


.. list-table:: **test-macosx1015-64-shippable-qr/opt**
Expand Down Expand Up @@ -4418,8 +4418,8 @@ Standard benchmarks are third-party tests (i.e. Speedometer) that we have integr
* - **browsertime-benchmark-firefox-webaudio**
- ✅
- ✅
- ✅
- ✅
- ❌
- ❌


.. list-table:: **test-macosx1015-64-shippable-qr/opt**
Expand Down

0 comments on commit e31e632

Please sign in to comment.