Set up ways to measure success from the Beta launch

[rhiannareechaye]

How do we measure the Beta?

• Will be done in Grafana
• This can be done in parallel with the filtering ticket
• One way of doing it is ‘chase AR down to 0’ rather than what DCR is supporting (note re galleries is still supported by AR)
• Worth asking David Lawes around the metrics

Tasks

Preview Give feedback

1. Create basic dashboard to render number of requests alone

[rhiannareechaye]

Here are some notes from James on a similar topic:

We'd like to track the metrics of articles served via dotcom-rendering vs apps-rendering.

It would be good if we could also enrich this data for reasons we aren't serving from DCR based on our release plan: #8750

The work will probably need to be done in MAPI based on the RenderingPlatformPicker.

TL;DR; on the release plan, we could filter on:

We will need to support these for beta release:

Offline reading
We will need to support these for prod release:

Dark more
Then we can filter on these aspects in prod:

Content is commentable
Content is of a specific layout
Content has video
The last list is what we should report on.

[cemms1]

We can't know for any article whether it is definitely rendered by a particular platform via MAPI alone.

The native apps each request:

• /rendered-items in MAPI for AR https://github.com/guardian/mobile-apps-api/blob/main/mobile-item/conf/routes#L11
• /items in MAPI for templates https://github.com/guardian/mobile-apps-api/blob/main/mobile-item/conf/routes#L7

There is no equivalent route for DCR because there is no proxy request, and instead the request for the DCR rendered page is sent directly to frontend, including the query parameter ?dcr=apps.

We are able to say whether articles could be rendered by DCR or AR via the RenderingPlatformPicker.

The RenderingPlatformPicker is only used for collection requests, so will be called many times (at least once per item in the collection) so we possibly don't want to log at this level since it will produce a huge number of logs per request.

As a initial solution, we will:

• log each request to /rendered-items and /items in MAPI to understand the support level for AR and templates respectively
• log each request to /<article-id>?dcr=apps in frontend to understand the support level for DCR
• combine the result of the two log streams in Grafana to provide a wider picture on actual support levels for DCR vs AR vs templates

In the above, we will want to monitor the number of requests so that we can understand a percentage of requests

As a follow-up, we could:

• log how many articles could be rendered by AR and DCR based on the RenderingPlatformPicker along with any additional contextual data such as conditions for rendering in each platform

[cemms1]

Dashboard link: https://metrics.gutools.co.uk/d/4vmjMRMIz/dcar-migration-progress