Question/quiz validation error

[marcellamaki]

Originally posted by @radinamatic in #12111 (comment)

That quiz with 107 questions kept throwing error when the learner inputs the answer on the last questions and clicks the Next button (or presses Enter when navigating by keyboard).

This seems like it may be either a validation problem or an error handling problem at some step along the quiz or assessment process. This needs some further exploration to determine if the quiz had an invalid question (and how to prevent such edge cases) or, if there is something wrong with advancing through the quiz on the learner side. Or possibly something else.

[nucleogenesis]

That quiz with 107 questions can be considered "corrupted data". When saving the quiz during creation, the total number of questions for each section was added together.

The first section was the default value, a number 10 -- but because you'd changed the number of questions by typing in the number 7, the value on the second section was the text "7".

The base Exam data structure has a "num_questions" property so the quiz creator adds that same value for each section together to get the Exam's total sum of questions from all sections.

So when that quiz was saved, I added the number 10 to the text "7" which resulted in Javascript saving the text "107" in that exam's num_questions field.

I've fixed this problem by forcing the values to be numbers when adding them together.

---

That all said, this is a problem that could be avoided by summing the questions when the user takes the quiz rather than relying on the vestigal total quiz questions count. That means -- when the Learner begins a quiz, code on their computer will count the questions. Basically, there isn't really any need for the num_questions value to be saved on the base exam.

Or, perhaps, the value of num_questions could be dynamically set in the relevant serializer -- basically the API can choose to ignore the saved num_questions property and count the sections' questions themselves. Or perhaps a method on Exam would be helpful.

---

In any case -- there are definitely some improvements that we can apply here, but I think this is relatively low priority overall since it is only a bug in the case of corrupted data.

[rtibbles]

I think dynamically summing the questions from all sections seems like the most sensible resolution here.

[AlexVelezLl]

This is a problem when we have a section but we dont have questions resources on that section.

Compartir.pantalla.-.2024-05-15.13_33_36.mp4

It keeps adding the 10 questions in the section even if there are not resources in it, and thats why learners cant complete the quiz.

What should we do with the sections that do not have resources? I would think we should filter them out and not show them to students.

[AlexVelezLl]

I think dynamically summing the questions from all sections seems like the most sensible resolution here.

Should we do it when saving the quiz? Or when reading it? I think it would be better to do it on save, and have exam.question_count have the actual value of the actual questions (with exercise resources) that exist (i.e. accumulate section.questions.length instead of section.question_count)

[nucleogenesis]

We currently are doing this when we save the quiz.

The idea is that in Learn, when the user goes to take the exam, that is when we just count all of the questions in Exam.question_sources (aka the quiz's sections) rather than relying on the Exam.question_count property to be accurate.

I think this is fine - but I wonder if perhaps it'd be better to implement an on-save validator on the Exam class that basically serves as the "One validation to rule them all" which can return an error when the Exam.question_count value doesn't match the number of questions in the sections total.

Overall I think it'd be better to utilize the Exam model and handle this logic in one place. It feels to me that if we just say "we'll code it client-side to derive the value of Exam.question_count" is kind of resigning the Exam.question_count property to being functionally pointless... like if we can't trust that number then why is it even on the model anymore - why not just drop the column?