Migrate AssessmentItem data to proper JSON fields

[bjester]

Builds upon #897 and #3901

Desired behavior

Proper resolution of this will involve several steps:

• Converting answers, and hints fields into a jsonfield JSONField (not a proper postgres JSONField to avoid a complex migration for now).
• To ensure proper interaction with ValuesViewsets, we should copy the extension of the jsonfield JSONField that is done in Kolibri here: https://github.com/learningequality/kolibri/blob/release-v0.14.x/kolibri/core/fields.py#L102
• This will then require updating of the frontend to expect JSON, as opposed to a string for these fields Update API to not return answers and hints as strings #1855

By itself, these steps would at least be sufficient to ensure that the incoming data is validated as JSON on save.

However, as the data will now be stored in IndexedDB in the frontend as JSON, any edits to them will be recorded by keypath, rather than the entire text field changing. This has the advantage of giving more granular edits, but means we need to update our serializers to properly handle this.

• Some original work will be required to be done, probably using the DotPathValueMixin, and in a similar way to the TagField used in the ContentNode serializer.
• The way that the data is serialized to the frontend should also be updated in order to be sent as a map (see e.g. tags for contentnodes), however, as there is additional data rather than just a tag name, a more complex method would be required.

Current behavior

Data is stored as text but is validated as JSON in #3901

[manavagr1108]

Hey @bjester, I am willing to work on this issue. Can I take up this issue?

[bjester]

Hi @manavagr1108, thank you for your interest in working on Studio. My apologies for the late response.

This issue isn't the best for an external contributor to address. It is rather convoluted as we can't simply change the field types, since we have millions of existing records in the database. We have a strict time limit on Django migrations, which requires us to handle this situation with care. Not only that, but there could be invalid JSON in the existing fields too.