Performance telemetry for awesomebar suggestion provider durations

[grigoryk]

See mozilla-mobile/android-components#6802 for details; requires that PR.

Pull Request checklist

• Tests: This PR includes thorough tests or an explanation of why it does not
• Screenshots: This PR includes screenshots or GIFs of the changes made or an explanation of why it does not
• Accessibility: The code in this PR follows accessibility best practices or does not include any user facing features. In addition, it includes a screenshot of a successful accessibility scan to ensure no new defects are added to the product.

After merge

• Milestone: Make sure issues finished by this pull request are added to the milestone of the version currently in development.

To download an APK when reviewing a PR:

1. click on Show All Checks,
2. click Details next to "Taskcluster (pull_request)" after it appears and then finishes with a green checkmark,
3. click on the "Fenix - assemble" task, then click "Run Artifacts".
4. the APK links should be on the left side of the screen, named for each CPU architecture

[grigoryk]

TODO: data review form & update PR w/ links.

[mcomella]

I looked briefly and the general pattern seems to fit what we're doing for other performance telemetry. I'll come back for a more thorough look tomorrow.

[mcomella]

Took another look and the general pattern of durations for performance purposes looks good to me: I'll leave the full review up to someone more familiar with what's actually being measured though.

[grigoryk]

Request for data collection review form

All questions are mandatory. You must receive review from a data steward peer on your responses to these questions before shipping new data collection.

1. What questions will you answer with this data?

This data will allow us to monitor real-life performance of all of the awesomebar suggestion providers. How fast/slow they are, what does distribution of duration looks like, etc.

2. Why does Mozilla need to answer these questions? Are there benefits for users? Do we need this information to address product or business requirements? Some example responses:

Awesomebar is one of the main points of interaction with the browser for our users. It needs to be fast and responsive. Measuring real-life performance will allow us to understand better how users are experiencing this part of the product.

• Establish baselines and measure changes (across releases) in awesomebar performance.
• Track what effect product changes have on the performance of the awesomebar.

3. What alternative methods did you consider to answer these questions? Why were they not sufficient?

There are qualitative alternatives to understanding raw performance of suggestion providers (and of the awesomebar as a whole) - they mainly take form of monitoring user feedback from various channels. The quality of the data is hard to rely on, and data collection/processing effort is high.

Automatic collection of raw performance metrics is both much more accurate, and doesn't involve a lot of effort to monitor over time once dashboards are established.

4. Can current instrumentation answer these questions?

No.

5. List all proposed measurements and indicate the category of data collection for each measurement, using the Firefox data collection categories found on the Mozilla wiki.

Note that the data steward reviewing your request will characterize your data collection based on the highest (and most sensitive) category.

Measurement Description	Data Collection Category	Tracking Bug #
Duration distribution of history suggestion queries	Category 1	https://github.com/mozilla-mobile/android-components/issues/4992
Duration distribution of bookmark suggestion queries	Category 1	https://github.com/mozilla-mobile/android-components/issues/4992
Duration distribution of search engine suggestion queries	Category 1	https://github.com/mozilla-mobile/android-components/issues/4992
Duration distribution of session (tabs) suggestion queries	Category 1	https://github.com/mozilla-mobile/android-components/issues/4992
Duration distribution of synced tabs suggestion queries	Category 1	https://github.com/mozilla-mobile/android-components/issues/4992
Duration distribution of clipboard suggestion queries	Category 1	https://github.com/mozilla-mobile/android-components/issues/4992
Duration distribution of shortcuts suggestion queries	Category 1	https://github.com/mozilla-mobile/android-components/issues/4992

6. How long will this data be collected? Choose one of the following:

Data currently expires on 2020-09-15.

7. What populations will you measure?

All channels, all user population.

8. If this data collection is default on, what is the opt-out mechanism for users?

Standard Glean SDK opt-out.

9. Please provide a general description of how you will analyze this data.

Through redash dashboards.

10. Where do you intend to share the results of your analysis?

Fenix, a-c and a-s teams.

11. Is there a third-party tool (i.e. not Telemetry) that you are proposing to use for this data collection? If so:

N/A

[grigoryk]

A-C PR landed, so this one can land once it gets a data review; I'll update the metrics.yaml file w/ a link to a data review response before landing.

[grigoryk]

Ah, I also need to bump the pinned nightly a-c version once the new release is out. Will do that in the morning.

[liuche]

Data-review+ only

Data Review Form (to be filled by Data Stewards)

1. Is there or will there be documentation that describes the schema for the ultimate data set in a public, complete, and accurate way?

Yes, these are documented in the metrics.md file, in this PR

2. Is there a control mechanism that allows the user to turn the data collection on and off?

Yes, metrics ping controlled by Fenix in-app data collection settings

3. If the request is for permanent data collection, is there someone who will monitor the data over time?

6mo, has expiry

4. Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Type 1, duration of technical calls (Type 2 for inferred usage of these features)

5. Is the data collection request for default-on or default-off?

Default on

6. Does the instrumentation include the addition of any new identifiers (whether anonymous or otherwise; e.g., username, random IDs, etc. See the appendix for more details)?

No, timing/technical data only

7. Is the data collection covered by the existing Firefox privacy notice?

Yes

8. Does there need to be a check-in in the future to determine whether to renew the data? (Yes/No) (If yes, set a todo reminder or file a bug if appropriate)**

Has expiry

9. Does the data collection use a third-party collection tool? If yes, escalate to legal.
   No

[grigoryk]

Bumped A-C version, fixed failing glean test; should get a green CI run now.

[ekager]

Looks good. One spelling nit + pls regenerate metrics.md then merge 👽

[codecov-io]

Codecov Report

Merging #10276 into master will decrease coverage by 0.01%.
The diff coverage is 0.00%.

@@             Coverage Diff              @@
##             master   #10276      +/-   ##
============================================
- Coverage     19.19%   19.17%   -0.02%     
  Complexity      599      599              
============================================
  Files           360      360              
  Lines         14807    14821      +14     
  Branches       2001     2011      +10     
============================================
  Hits           2842     2842              
- Misses        11705    11719      +14     
  Partials        260      260              

Impacted Files	Coverage Δ	Complexity Δ
...va/org/mozilla/fenix/components/metrics/Metrics.kt	19.50% <0.00%> (-0.89%)	0.00 <0.00> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cb30706...472604d. Read the comment docs.

[mcarare]

@grigoryk Does this telemetry need an extension for the expiry date? It has already expired on 2020-11-15