For #4967 - Record search / ads providers appearing in search results #4968

Mugurell · 2021-07-02T14:35:10Z

This brings the ads/search telemetry up to par with Fenix.

Mugurell · 2021-07-02T15:06:16Z

Request for data collection review form

All questions are mandatory. You must receive review from a data steward peer on your responses to these questions before shipping new data collection.

What questions will you answer with this data?

These probes will be used to better model high-level trends in revenue prediction. They will also be used going forward as a standard metric (the way search volume is now) to understand the revenue impact of new prospective features.

Why does Mozilla need to answer these questions? Are there benefits for users? Do we need this information to address product or business requirements?

This measure helps Mozilla better understand revenue trends and (potentially) aids future forecasting. It also helps us have confidence on the revenue impact of features. This does not mean Mozilla would withold a feature that reduces search revenue, it merely helps us quantify it. On the contrary: right now it is difficult to really understand the revenue impact of new features, and in the absence of evidence it is very difficult to justify some needed changes.

What alternative methods did you consider to answer these questions? Why were they not sufficient?

No current alternative methods.

Can current instrumentation answer these questions?

Currently there are no events to track this.

List all proposed measurements and indicate the category of data collection for each measurement, using the Firefox data c ollection categories found on the Mozilla wiki.

Measurement Description	Data Collection Category	Tracking Bug #
A specific Search Engine Results Page was displayed	category 2 - interaction data	#4967
Ads are shown on a specific Search Engine Results Page	category 2 - interaction data	#4967
User clicked on an ad from a specific Search Engine Results Page	category 2 - interaction data	#4967

How long will this data be collected?

Until 2022-07-01

What populations will you measure?

All release, beta, and nightly users with telemetry enabled.

Please provide a general description of how you will analyze this data.

Glean

Where do you intend to share the results of your analysis?

Only on Glean and with mobile teams.

Mugurell · 2021-07-02T15:07:33Z

Setting do-not-land to update the metrics documentation with the telemetry renewal review first.

Mugurell · 2021-07-02T16:20:34Z

app/src/main/java/org/mozilla/focus/telemetry/GleanMetricsService.kt

@@ -79,7 +80,9 @@ class GleanMetricsService(context: Context) : MetricsService {
    private fun collectPrefMetrics(
        components: Components,
        settings: Settings
-    ) = CoroutineScope(IO).async {
+    ) = CoroutineScope(Main).async {


Otherwise I'd get a java.lang.IllegalThreadStateException: Must have a Handler the same as was seen in mozilla-mobile/fenix#17045

travis79 · 2021-07-13T17:08:19Z

1. **List all proposed measurements and indicate the category of data collection for each measurement, using the Firefox [data c](https://wiki.mozilla.org/Firefox/Data_Collection)[ollection ](https://wiki.mozilla.org/Firefox/Data_Collection)[categories](https://wiki.mozilla.org/Firefox/Data_Collection) found on the Mozilla wiki.**


* All data is Category 2, maybe Category 3 if we consider ads as `categories of webpages visited over time`.

@Mugurell

A couple of quick requests, neither of which block me being able to perform the review at this time:

Can you please edit your comment to list out the metrics being added here? The review form asks for this in tabular form but just a simple list that includes the "measurement description", the "data collection category" should be sufficent (and any links to other issues or bugs). From a stewarding standpoint, I happened to know how to find this in the metrics.yaml file but it is preferred that the newly added metrics are listed explicitly in the review request.

Also, I would say that this is not Category 3 data since they are only the counts being recorded and not the specific ad or url. I think this should be Category 2, User Interaction data.

And finally:

Data Review

Is there or will there be documentation that describes the schema for the ultimate data set in a public, complete, and accurate way?

Yes, through the metrics.yaml file and the Glean Dictionary

Is there a control mechanism that allows the user to turn the data collection on and off?

Yes, through the Send Usage Data preference in the app settings.

If the request is for permanent data collection, is there someone who will monitor the data over time?

N/A, collection set to expire 2022-07-01

Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 2, User Interaction or lower

Is the data collection request for default-on or default-off?

Default-on

Does the instrumentation include the addition of any new identifiers?

No

Is the data collection covered by the existing Firefox privacy notice?

Yes

Does the data collection use a third-party collection tool?

No

Result

data-review+

app/src/main/java/org/mozilla/focus/telemetry/TelemetryWrapper.kt

pocmo · 2021-07-19T14:34:37Z

app/src/main/java/org/mozilla/focus/telemetry/GleanMetricsService.kt

-    ) = CoroutineScope(IO).async {
+    ) = CoroutineScope(Main).async {
+        // Doing this on Main because this might be the first time BrowserStore is accessed and initialized
+        // and this needs to be done on the Main thread.


Hm, the comment is a bit misleading, the Store can be used on any thread, but the new install() calls can't.

The whole pattern here is confusing to me, we launch a separate coroutine with the same dispatcher (and an unbound scope) and then await it immediately. Why? 🤔

I wonder if we could move the install() calls into initialize() here, before launching the coroutine, making this more explicit here? 🤔

Thank you. Will look into this.

Looked to be a great solution! Thank you!

Mugurell · 2021-07-20T16:35:11Z

app/src/main/java/org/mozilla/focus/FocusApplication.kt

@@ -47,6 +48,7 @@ open class FocusApplication : LocaleAwareApplication(), CoroutineScope {

            TelemetryWrapper.init(this)


Wouldn't it be nice to only initialize all these only if isTelemetryEnabled() and then observe changes and if the users enable telemetry do the initialization or immediately stop the functionality?
Currently the first two will do their own isTelemetryEnabled() check and abort the initialization but the FactsProcessor won't.

Mugurell · 2021-07-22T13:41:55Z

Understood from Amedyne that this will have to also be uplifted to 91. (after testing)

jonalmeida · 2021-07-24T04:52:25Z

org.mozilla.focus.activity.ThreeDotMainMenuTest > browserMenuItemsTest[test(AVD) - 9] FAILED
androidx.test.espresso.NoMatchingViewException: No views in hierarchy found matching: with id: org.mozilla.focus.debug:id/settings

Task :app:connectedFocusDebugAndroidTest FAILED

UI test failure; restarting in case it's intermittent.

This brings the ads/search telemetry up to par with Fenix.

Mugurell · 2021-07-26T09:06:14Z

Rebased on the current main.

jonalmeida · 2021-08-16T18:03:02Z

@Mergifyio backport releases_v91.0

mergify · 2021-08-16T18:03:13Z

Command backport releases_v91.0: success

Backports have been created

#5146 For #4967 - Record search / ads providers appearing in search results (backport #4968) has been created for branch releases_v91.0

jonalmeida · 2021-08-16T19:57:57Z

@Mergifyio backport releases_v91.0

Oops, this was supposed to be 92.

jonalmeida · 2021-08-16T19:58:04Z

@Mergifyio backport releases_v92.0

mergify · 2021-08-16T19:58:34Z

Command backport releases_v91.0: success

Backports have been created

#5146 For #4967 - Record search / ads providers appearing in search results (backport #4968) has been created for branch releases_v91.0

mergify · 2021-08-16T19:58:42Z

Command backport releases_v92.0: success

Backports have been created

#5147 For #4967 - Record search / ads providers appearing in search results (backport #4968) has been created for branch releases_v92.0

Mugurell added the do not land PRs that requires coordination before landing label Jul 2, 2021

Mugurell commented Jul 2, 2021

View reviewed changes

Mugurell added 🕵️‍♀️ needs review PRs that need to be reviewed and removed do not land PRs that requires coordination before landing labels Jul 16, 2021

pocmo reviewed Jul 19, 2021

View reviewed changes

Mugurell commented Jul 20, 2021

View reviewed changes

pocmo approved these changes Jul 21, 2021

View reviewed changes

Mugurell added pr:needs-ac-bump PR that needs a AC bump and removed 🕵️‍♀️ needs review PRs that need to be reviewed labels Jul 23, 2021

jonalmeida closed this Jul 24, 2021

jonalmeida reopened this Jul 24, 2021

For #4967 - Record search / ads providers appearing in search results

0e0b852

This brings the ads/search telemetry up to par with Fenix.

Mugurell added the 🛬 needs landing PRs that are ready to land label Jul 26, 2021

mergify bot merged commit 3ce7235 into mozilla-mobile:main Jul 26, 2021

Mugurell deleted the 4967Telemetry branch July 27, 2021 07:15

mergify bot mentioned this pull request Aug 16, 2021

For #4967 - Record search / ads providers appearing in search results (backport #4968) #5146

Closed

mergify bot mentioned this pull request Aug 16, 2021

For #4967 - Record search / ads providers appearing in search results (backport #4968) #5147

Merged

Mugurell restored the 4967Telemetry branch June 14, 2022 17:22

iorgamgabriel mentioned this pull request Jul 26, 2022

Renew telemetry expiring in 106 #7418

Merged

4 tasks

mcarare mentioned this pull request Jan 10, 2023

Bug 1809447 - Set ads telemetry in Focus to never expire. mozilla-mobile/firefox-android#523

Merged

4 tasks

This was referenced Jan 31, 2024

Bug 1875093 - AC Action to Fetch and Set Page Preferences for Translations mozilla-mobile/firefox-android#5348

Merged

Bug 1877588 - TranslationsMiddleware Dispatcher Scope Issue mozilla-mobile/firefox-android#5409

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

For #4967 - Record search / ads providers appearing in search results #4968

For #4967 - Record search / ads providers appearing in search results #4968

Mugurell commented Jul 2, 2021 •

edited

Loading

Mugurell commented Jul 2, 2021 •

edited

Loading

Mugurell commented Jul 2, 2021

Mugurell Jul 2, 2021

travis79 commented Jul 13, 2021

pocmo Jul 19, 2021

Mugurell Jul 19, 2021

Mugurell Jul 20, 2021

Mugurell Jul 20, 2021

Mugurell commented Jul 22, 2021

jonalmeida commented Jul 24, 2021

Mugurell commented Jul 26, 2021

jonalmeida commented Aug 16, 2021

mergify bot commented Aug 16, 2021

jonalmeida commented Aug 16, 2021

jonalmeida commented Aug 16, 2021

mergify bot commented Aug 16, 2021

mergify bot commented Aug 16, 2021

		@@ -47,6 +48,7 @@ open class FocusApplication : LocaleAwareApplication(), CoroutineScope {

		TelemetryWrapper.init(this)

For #4967 - Record search / ads providers appearing in search results #4968

For #4967 - Record search / ads providers appearing in search results #4968

Conversation

Mugurell commented Jul 2, 2021 • edited Loading

Mugurell commented Jul 2, 2021 • edited Loading

Request for data collection review form

Mugurell commented Jul 2, 2021

Mugurell Jul 2, 2021

Choose a reason for hiding this comment

travis79 commented Jul 13, 2021

Data Review

Result

pocmo Jul 19, 2021

Choose a reason for hiding this comment

Mugurell Jul 19, 2021

Choose a reason for hiding this comment

Mugurell Jul 20, 2021

Choose a reason for hiding this comment

Mugurell Jul 20, 2021

Choose a reason for hiding this comment

Mugurell commented Jul 22, 2021

jonalmeida commented Jul 24, 2021

Mugurell commented Jul 26, 2021

jonalmeida commented Aug 16, 2021

mergify bot commented Aug 16, 2021

jonalmeida commented Aug 16, 2021

jonalmeida commented Aug 16, 2021

mergify bot commented Aug 16, 2021

mergify bot commented Aug 16, 2021

Mugurell commented Jul 2, 2021 •

edited

Loading

Mugurell commented Jul 2, 2021 •

edited

Loading