[DEPR]: Remove Learner View in Insights, Data API, and Pipeline

[ashultz0]

Editorial note: This deprecated a particular view in Insights ("Learner View"), plus that view's supporting code in other analytics repos. It didn't entirely deprecate Insights and all associated analytics repos. -Kyle

---

Proposal Date

3 March 2022

Ticket Acceptance Date

31 March 2022

Technology Removal Date

May 1 2022

First Open edX Named Release Without This Functionality

Olive

Rationale

The learner view in insights and associated data API and pipeline are expensive to operate and provide data out of alignment with the rest of analytics. It is so detailed that it provides too much individual data to be viewable for a course with more than a few students, and that data creates PII risk.

Most of analytics is aggregated data. The learner view contains individual information including usernames and emails. This means that the learner view includes sensitive PII. This creates unnecessary security concerns Insights as well as the entire pipeline and API.

To support this data requires an expensive pipeline job and an elasticsearch cluster ( see #16 ). On edx.org, data supporting this view is roughly as as the rest of Insights combined.

Removal

Insights code supporting the learner view such as: https://github.com/openedx/edx-analytics-dashboard/tree/master/analytics_dashboard/learner_analytics_api

Learner view data API code such as https://github.com/openedx/edx-analytics-data-api/blob/master/analytics_data_api/v0/views/learners.py as well as elasticsearch support code and the corresponding calls in https://github.com/openedx/edx-analytics-data-api-client

Learner view pipeline support such as https://github.com/edx/edx-analytics-pipeline/blob/master/edx/analytics/tasks/insights/module_engagement.py and elasticsearch support.

Replacement

There is no replacement currently planned.

Deprecation

No response

Migration

No response

Additional Info

No response

[kdmccormick]

Forum post: https://discuss.openedx.org/t/deprecation-removal-learner-view-in-insights-data-api-and-analytics-pipeline/6788

[dianakhuang]

This is still under discussion after the accepted date. See the Discourse post for more details.

[jmbowman]

The Discourse post resolved at it being ok for us to remove this, just need to actually rip it out now.

[pdpinch]

When I was looking through the commit history to prepare the olive release notes, I found the following PRs, which suggest that this feature has been removed.

• feat: deprecate learner view code openedx-unsupported/edx-analytics-data-api#571
• feat: remove engagement timeline from data client openedx-unsupported/edx-analytics-data-api-client#89
• chore: deprecate learner view openedx-unsupported/edx-analytics-dashboard#1332

Can this issue be closed, or at least updated?

[ashultz0]

It has been removed from the frontend parts of the system but there are some reports in enterprise code tied to the back end structures that have to be rooted out as well to close this and then the back end itself can be removed.

As far as Olive is concerned this feature is not available, but the code is still being mined out.

[bmtcril]

Is this still being worked on, or can this ticket be closed out?

[ashultz0]

still in the yak shaving stage of trying to get rid of the code that lets us get rid of the code that lets us get rid of the code :(

[jmbowman]

Can we ticket up and link that chain of code removal, or at least the task that's currently blocking this?

[ashultz0]

I'm a bit detached from where this is now because I've changed groups so I summon @alangsto who is another person who knows something :)

[alangsto]

Cosmonauts has tickets to remove the learner view, but the work has not been prioritized. If this work should be prioritized, I can bring it up during our next grooming meeting.

[kdmccormick]

Could you link the tickets here Ali? I understand that they're probably 2U-internal, but I'd like to tie all threads together here, whether private or public.

[alangsto]

Here is the link to the epic: https://2u-internal.atlassian.net/browse/MST-1428

[dianakhuang]

This is being closed in favor #221 , since that other ticket will cover these views.