Enhanced Tracking Protection needs Page Load Testing

[colintheshots]

User Story

• As a member of the Fenix team, I would like performance measurements on the relative page load speeds of Tracking Protection versus the new Enhanced Tracking Protection feature.

Dependencies

• This will require some assistance from the Performance Team to learn how to run comparative Raptor testing using Try with live loads and custom parameters and/or APKs.

Acceptance Criteria

• We have accurate page load performance measurements of ETP vs. TP on the latest Fenix with enough runs to have reasonable certainty in the results.

┆Issue is synchronized with this Jira Task

[vesta0]

The goal of this experiment is to determine performance impact of switching from Strict TP (current) to Basic TP. Based on the results of this experiment, we will make a decision on whether Fenix default/recommended setting should be Basic or Strict (see here)

Cohort A:

• Block Cross-site Tracking Cookies from Disconnect's Advertising, Analytics, and Social list

Cohort B (Current implementation):

• Blocking tracking scripts (not just the tracking cookies) from Disconnect's Advertising, Analytics, and Social list

@Amejia481 and @ekager would you please add the name of lists/prefs to be used for each cohort? Thank you!

[Amejia481]

Cohort a (ETP):

• TrackingProtectionPolicy.strict()

Cohort B (Current TP implementation):

• TrackingProtectionPolicy.recommended()

[liuche]

Remaining work E2.

[vesta0]

Cohort a (ETP):

* `TrackingProtectionPolicy.strict()`

I don't believe this is correct. ETP (#1749) is a new category and refers to blocking Cross-site Tracking Cookies from Disconnect's Advertising, Analytics, and Social list. It is less strict than what we currently block in Fenix which is blocking tracking scripts/sub-resources, not just cookies.

Cohort B (Current TP implementation):

* `TrackingProtectionPolicy.recommended()`

[Amejia481]

The default policies on ac have been updated on 12.0.0 to be aligned with the mockups and what Desktop is using mozilla-mobile/android-components#4349.

Now the cohorts should match as follow:

• Cohort A: (Standard on the mockups)

TrackingProtectionPolicy.recommended()

• Cohort B: (Strict on the mockups)

TrackingProtectionPolicy.strict()

[mcomella]

Triage: waiting on any form of page load testing with visual metrics.

[mcomella]

From Eric on this issue:

Yeah, please close. Testing has happened already at least twice that I'm aware of.