App learning progress reset when app moved from alpha to beta

[partha-basu]

Describe the bug
The learning progress - Story/Lesson completion - was lost when the app moved from alpha to beta channel.
I received a notification that I've been upgraded and then taken to the home/landing screen with profile selection options.
Once the profile is chosen it shows the Learning Home Screen.
Generally when I go to a topic/Lesson I have a "continue" option but it went back to the starter screen.

question: I would expect the app not to lose progress whenever there is an update to the app.

To Reproduce
Can't reproduce the behavior anymore since it's a one-time thing.
Expected behavior
I would expect the app not to lose progress whenever there is an update to the app.

Demonstration
N.A.

Environment

• Device/emulator being used: Pixel 5
• Android or SDK version (e.g. Android 5 or SDK 21): Android 13
• App version (you can get this through system app settings or via the admin controls menu in-app): 0.9-beta-3f935261e0

Additional context
Add any other context about the problem here.

[BenHenning]

Thanks for filing @partha-basu. For clarity on a couple of points: do you still see a 'Continue' button for the lesson that you tried to continue playing? If so, does it successfully resume the lesson for you?

Also, are any complete lessons no longer marked as completed?

[partha-basu]

@BenHenning - I restarted the journey. Now the behavior is same as before - able to leave lesson midway and then continue from drop-off point. The progress reset was a one-time thing post update. Flagging as wanted to make sure it doesn't happen with each update.

[BenHenning]

Thanks for clarifying @partha-basu. This is actually expected for partially completed lessons (fully completed lessons should never reset). It's something that we definitely want to fix by introducing manual downloading support in the app, but perhaps we should investigate stop-gaps in the meantime.

Sending this over to @seanlip for product weigh-in, and since he's also looking through the original checkpointing PRD to consider different approaches. We've also talked about some potential mitigation factors here, but they're not entirely trivial.

[partha-basu]

@BenHenning - thanks for the confirmation. If that is the expected behavior for all updates - having some kind of call-out/warning (something like - partially completed lesson progress would be lost!) in the playstore page or before updating would help (I did get a pop-up that I'm being upgraded to beta channel but that did not call out what would specifically change for me).

[BenHenning]

Well noted @partha-basu. I'm not sure if Play Store description is correct place for this, or if we need to consider other options (such as bundling multiple lesson versions within the app).

[seanlip]

Had an offline discussion with @BenHenning and noting that this is important to fix (but let's try to fix with as minimal contortions as possible). We don't want users to be scared of updating the app due to possible loss of progress.

[BenHenning]

So thinking about this a bit more, I suspect the most correct and reliable way to attempt this would be to, upon attempting to load a checkpoint, "replay" that checkpoint against the newest version of the exploration (only if the version doesn't match). There are some properties of the current checkpoint that we would unconditionally overwrite:

• Exploration title & version would be updated to the latest
• The hint state would be dropped (rather than trying to figure out if it's compatible)

Otherwise, all previous answers (for both completed and the top pending state) will be "replayed" against their corresponding states in the new version of the exploration. In order to "pass", the following must be met:

1. The state needs to exist
2. The state needs to be reachable via the previous card's state using the correctly submitted answer for that state
3. All answers must be type-compatible with their corresponding states

The feedbacks will then be replaced with the latest versions given each answer (as if the user replayed the whole lesson).

This is the most robust way to try and avoid needing to throw away the checkpoint, and I think works against all possible exploration transformations in a fairly organic way (since it simulates the learner re-playing everything).