Issue #4901: Update metrics for data review on initial metrics

[jonalmeida]

@travis79 I tried to separate out the data review into two requests to make it a bit easier: one for browser metrics, the other for activation and legacy IDs for deletion.

[jonalmeida]

Request for data collection review form

All questions are mandatory. You must receive review from a data steward peer on your responses to these questions before shipping new data collection.

1. What questions will you answer with this data?

• These probes will be used to understand what the Focus user base looks based on the core preferences that drive our search revennue: user's default search engine choice, whether Focus is their default browser and the locale that is used.

2. Why does Mozilla need to answer these questions? Are there benefits for users? Do we need this information to address product or business requirements?

• The benefits to users is that we will be able to know who the power users of Focus are in the future, what needs we can provide.

3. What alternative methods did you consider to answer these questions? Why were they not sufficient?

• No current alternative methods.

4. Can current instrumentation answer these questions?

• Currently, we have telemetry from a deprecated and outdated system which we can no longer rely on. These metrics will help us verify whether we are correctly collecting data from Glean before turning off the old system.

5. List all proposed measurements and indicate the category of data collection for each measurement, using the Firefox data collection categories found on the Mozilla wiki.

Measurement Description	Data Collection Category	Tracking Bug #
The default browser is set as Focus.	category 2 - interaction data	#4901
The search engine the user has set as the default.	category 2 - interaction data	#4901
The locale the user is currently in.	category 2 - interaction data	#4901

6. How long will this data be collected?

• Until 2022-08-01 for browser.is_default, and never for the other two.

7. What populations will you measure?

• All release, beta, and nightly users with telemetry enabled.

8. Please provide a general description of how you will analyze this data.

• Glean

9. Where do you intend to share the results of your analysis?

• Only on Glean and with mobile teams.

[jonalmeida]

Request for data collection review form

All questions are mandatory. You must receive review from a data steward peer on your responses to these questions before shipping new data collection.

1. What questions will you answer with this data?

• Similar to Fenix, we plan to count the exact number of activation, per distribution, per manufacturer.
• We also send our legacy telemetry ID as part of the deletion request for users who wish to have their reported Telemetry data deleted.

2. Why does Mozilla need to answer these questions? Are there benefits for users? Do we need this information to address product or business requirements?

• The activation data will also help us evaluate the difference in activations from the other data sources which are currently being used for activation counts (e.g. service-telemetry)

3. What alternative methods did you consider to answer these questions? Why were they not sufficient?

• Citing the learnings from Fenix again, no current alternative methods.

4. Can current instrumentation answer these questions?

• Currently, we have telemetry from a deprecated and outdated system which we can no longer rely on.

5. List all proposed measurements and indicate the category of data collection for each measurement, using the Firefox data collection categories found on the Mozilla wiki.

Measurement Description	Data Collection Category	Tracking Bug #
activation.activation_id	category 4	#4901
legacy_ids.client_id	category 4	#4901

6. How long will this data be collected?

• Always.

7. What populations will you measure?

• All release, beta, and nightly users with telemetry enabled.

8. Please provide a general description of how you will analyze this data.

• Glean

9. Where do you intend to share the results of your analysis?

• Only on Glean, mobile teams and BD have internal access.

[jonalmeida]

@travis79 Thanks for review! I've made the appropriate changes.

Glean integration will be going out in our next release, so I'll need to cherry-pick this patch to that release branch. Would I need another data review for that?

[travis79]

@travis79 Thanks for review! I've made the appropriate changes.

Glean integration will be going out in our next release, so I'll need to cherry-pick this patch to that release branch. Would I need another data review for that?

So long as the metrics.yaml points to the review I'm getting ready to add to this i
ssue, it should be fine. The only reason you would need to get another review is if you were adding more metrics.

[travis79]

Request for data collection review form

All questions are mandatory. You must receive review from a data steward peer on your responses to these questions before shipping new data collection.

1. **What questions will you answer with this data?**


* These probes will be used to understand what the Focus user base looks based on the core preferences that drive our search revennue: user's default search engine choice, whether Focus is their default browser and the locale that is used.


1. **Why does Mozilla need to answer these questions?  Are there benefits for users? Do we need this information to address product or business requirements?**


* The benefits to users is that we will be able to know who the power users of Focus are in the future, what needs we can provide.


1. **What alternative methods did you consider to answer these questions? Why were they not sufficient?**


* No current alternative methods.


1. **Can current instrumentation answer these questions?**


* Currently, we have telemetry from a deprecated and outdated system which we can no longer rely on. These metrics will help us verify whether we are correctly collecting data from Glean before turning off the old system.


1. **List all proposed measurements and indicate the category of data collection for each measurement, using the Firefox [data c](https://wiki.mozilla.org/Firefox/Data_Collection)[ollection ](https://wiki.mozilla.org/Firefox/Data_Collection)[categories](https://wiki.mozilla.org/Firefox/Data_Collection) found on the Mozilla wiki.**

Measurement Description Data Collection Category Tracking Bug #
The default browser is set as Focus. category 2 - interaction data #4901
The search engine the user has set as the default. category 2 - interaction data #4901
The locale the user is currently in. category 2 - interaction data #4901

1. **How long will this data be collected?**


* Until 2022-08-01 for `browser.is_default`, and never for the other two.


1. **What populations will you measure?**


* All release, beta, and nightly users with telemetry enabled.


1. **Please provide a general description of how you will analyze this data.**


* Glean


1. **Where do you intend to share the results of your analysis?**


* Only on Glean and with mobile teams.

Data Review

1. Is there or will there be documentation that describes the schema for the ultimate data set in a public, complete, and accurate way?

Yes, through the metrics.yaml file and the Glean Dictionary

2. Is there a control mechanism that allows the user to turn the data collection on and off?

Yes, through the "Send Usage Data" toggle in the app preferences

3. If the request is for permanent data collection, is there someone who will monitor the data over time?

Yes, for the two metrics listed as permanent, @jonalmeida will monitor them

4. Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 2, User Interaction

5. Is the data collection request for default-on or default-off?

Default-on

6. Does the instrumentation include the addition of any new identifiers (whether anonymous or otherwise; e.g., username, random IDs, etc. See the appendix for more details)?

No

7. Is the data collection covered by the existing Firefox privacy notice?

Yes

8. Does the data collection use a third-party collection tool?

No

Result

data-review+

[travis79]

I need to escalate the review for category 4 identifiers just to ensure legal is aware. It shouldn't be a problem since we are only duplicating what we have already done in Fenix but let's follow the procedures on this. I'll review the second request in this issue as soon as possible, after we get a response on this.

[jonalmeida]

I need to escalate the review for category 4 identifiers just to ensure legal is aware. It shouldn't be a problem since we are only duplicating what we have already done in Fenix but let's follow the procedures on this. I'll review the second request in this issue as soon as possible, after we get a response on this.

Relman would like to get a build for the next Focus release out and I didn't anticipate how the timeline needed for this (my error!).

For that reason, I'm going to disable the activation_id telemetry and remove it from the metrics file with a separate issue that can land at a comfortable timeline for everyone.

[jonalmeida]

@Mergifyio backport releases_v91.0

[mergify]

Command backport releases_v91.0: success

Backports have been created

• #5087 Issue #4901: Update metrics for data review on initial metrics (backport #5065) has been created for branch releases_v91.0