Skip to content

Commit

Permalink
Fix #114: Implement answer classification controller (#211)
Browse files Browse the repository at this point in the history
* Introduce first pass interface for ExplorationProgressController.

* Fill in the stubbed logic for ExplorationProgressController. Still no
tests to verify correctness.

Also, added a method to facilitate notifying of DataProvider changes on
the UI thread.

* Fix lateinit issue in ExplorationProgressController due to wrongly ordered
initialization.

* Fix a variaty of issues in the exp progress controller, properly hook it up to
the data controller, and start adding tests.

* Created a separate ExplorationRetriever, hooked up
AnswerClassificationController, and attempted to make
ExplorationProgressController thread-safe.

The thread-safety led to significant interface changes in the progress
controller, and led to discovering some issues with the mediator live data
approach to interop coroutines and LiveData. This locking mechanism will
need to change since the optimal solution requires resolving #90.

* Change the locking mechanism for ExplorationProgressController to work
with the current MediatorLiveData implementation (see #90 for more
context). Fix existing progress controller tests and add a few more. All
current progress controller tests are passing.

* Finish tests for ExplorationProgressController and add test classification
support for the second test exploration (about_oppia).

* First iteration at implementing real answer classification for the Oppia
prototype. This uses a Dagger-powered solution to make it straightforward
to add new rule types and interactions in a way that automatically hooks
into the classifier.

This only adds TextInput support, so existing tests do not pass. Tests and
the app do build.

* Bind all text input rules, add numeric input rules, and add support for
two bound input values.

* Add numbers with units classification support.

* Add multiple choice input classification support. Fix some of the
exploration progress controller tests for numeric input.

* Add item selection classification support. Clean up all rule classifier
comments to point to their corresponding Oppia web versions.

* Add fractions input classification support.

* Add Continue module. Resolve some TODOs, add TODOs to test classifiers,
and fix ExplorationProgressController tests (which also included fixing
one bug in the TextInput FuzzyEquals and the progress controller's
fallback routing logic).

* Add thorough tests for AnswerClassificationController.

* Consolidate generic rule classifiers into a single class.
  • Loading branch information
BenHenning authored Oct 9, 2019
1 parent 7af4c1b commit a501b23
Show file tree
Hide file tree
Showing 56 changed files with 2,169 additions and 141 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -5,14 +5,26 @@ import dagger.BindsInstance
import dagger.Component
import org.oppia.app.activity.ActivityComponent
import org.oppia.data.backends.gae.NetworkModule
import org.oppia.domain.classify.InteractionsModule
import org.oppia.domain.classify.rules.continueinteraction.ContinueModule
import org.oppia.domain.classify.rules.fractioninput.FractionInputModule
import org.oppia.domain.classify.rules.itemselectioninput.ItemSelectionInputModule
import org.oppia.domain.classify.rules.multiplechoiceinput.MultipleChoiceInputModule
import org.oppia.domain.classify.rules.numberwithunits.NumberWithUnitsRuleModule
import org.oppia.domain.classify.rules.numericinput.NumericInputRuleModule
import org.oppia.domain.classify.rules.textinput.TextInputRuleModule
import org.oppia.util.logging.LoggerModule
import org.oppia.util.threading.DispatcherModule
import javax.inject.Provider
import javax.inject.Singleton

/** Root Dagger component for the application. All application-scoped modules should be included in this component. */
@Singleton
@Component(modules = [ApplicationModule::class, DispatcherModule::class, NetworkModule::class, LoggerModule::class])
@Component(modules = [
ApplicationModule::class, DispatcherModule::class, NetworkModule::class, LoggerModule::class,
ContinueModule::class, FractionInputModule::class, ItemSelectionInputModule::class, MultipleChoiceInputModule::class,
NumberWithUnitsRuleModule::class, NumericInputRuleModule::class, TextInputRuleModule::class, InteractionsModule::class
])
interface ApplicationComponent {
@Component.Builder
interface Builder {
Expand Down
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
package org.oppia.domain.classify

import org.oppia.app.model.AnswerGroup
import org.oppia.app.model.Interaction
import org.oppia.app.model.InteractionObject
import org.oppia.app.model.Outcome
import org.oppia.app.model.State
import java.lang.IllegalStateException
import javax.inject.Inject

// TODO(#59): Restrict the visibility of this class to only other controllers.
Expand All @@ -14,66 +15,46 @@ import javax.inject.Inject
*
* This controller should only be interacted with via background threads.
*/
class AnswerClassificationController @Inject constructor() {
// TODO(#114): Add support for classifying answers based on an actual exploration. Also, classify() should take an
// Interaction, not a State.

class AnswerClassificationController @Inject constructor(
private val interactionClassifiers: Map<String, @JvmSuppressWildcards InteractionClassifier>
) {
/**
* Classifies the specified answer in the context of the specified [Interaction] and returns the [Outcome] that best
* matches the learner's answer.
*/
internal fun classify(currentState: State, answer: InteractionObject): Outcome {
return when (currentState.name) {
// Exp 5
"Welcome!" -> simulateMultipleChoiceForWelcomeStateExp5(currentState, answer)
"What language" -> simulateTextInputForWhatLanguageStateExp5(currentState, answer)
"Numeric input" -> simulateNumericInputForNumericInputStateExp5(currentState, answer)
"Things you can do" -> currentState.interaction.defaultOutcome
// Exp 6
"First State" -> currentState.interaction.defaultOutcome
"So what can I tell you" -> simulateMultipleChoiceForWelcomeStateExp6(currentState, answer)
"Example1" -> currentState.interaction.defaultOutcome // TextInput with ignored answer.
"Example3" -> currentState.interaction.defaultOutcome
"End Card" -> currentState.interaction.defaultOutcome
else -> throw Exception("Cannot submit answer to unexpected state: ${currentState.name}.")
internal fun classify(interaction: Interaction, answer: InteractionObject): Outcome {
val interactionClassifier = checkNotNull(interactionClassifiers[interaction.id]) {
"Encountered unknown interaction type: ${interaction.id}, expected one of: ${interactionClassifiers.keys}"
}
// TODO(#207): Add support for additional classification types.
return classifyAnswer(
answer, interaction.answerGroupsList, interaction.defaultOutcome, interactionClassifier, interaction.id)
}

private fun simulateMultipleChoiceForWelcomeStateExp5(currentState: State, answer: InteractionObject): Outcome {
return when {
answer.objectTypeCase != InteractionObject.ObjectTypeCase.NON_NEGATIVE_INT ->
throw Exception("Expected non-negative int answer, not $answer.")
answer.nonNegativeInt == 0 -> currentState.interaction.answerGroupsList[0].outcome
answer.nonNegativeInt == 2 -> currentState.interaction.answerGroupsList[1].outcome
else -> currentState.interaction.defaultOutcome
// Based on the Oppia web version:
// https://github.com/oppia/oppia/blob/edb62f/core/templates/dev/head/pages/exploration-player-page/services/answer-classification.service.ts#L57.
private fun classifyAnswer(
answer: InteractionObject, answerGroups: List<AnswerGroup>, defaultOutcome: Outcome,
interactionClassifier: InteractionClassifier, interactionId: String
): Outcome {
for (answerGroup in answerGroups) {
for (ruleSpec in answerGroup.ruleSpecsList) {
val ruleClassifier = checkNotNull(interactionClassifier.getRuleClassifier(ruleSpec.ruleType)) {
"Expected interaction $interactionId to have classifier for rule type: ${ruleSpec.ruleType}, but only" +
" has: ${interactionClassifier.getRuleTypes()}"
}
try {
if (ruleClassifier.matches(answer, ruleSpec.inputMap)) {
// Explicit classification matched.
return answerGroup.outcome
}
} catch (e: Exception) {
throw IllegalStateException("Failed when classifying answer $answer for interaction $interactionId", e)
}
}
}
}

private fun simulateTextInputForWhatLanguageStateExp5(currentState: State, answer: InteractionObject): Outcome {
return when {
answer.objectTypeCase != InteractionObject.ObjectTypeCase.NORMALIZED_STRING ->
throw Exception("Expected string answer, not $answer.")
answer.normalizedString.toLowerCase() == "finnish" -> currentState.interaction.getAnswerGroups(6).outcome
else -> currentState.interaction.defaultOutcome
}
}

private fun simulateNumericInputForNumericInputStateExp5(currentState: State, answer: InteractionObject): Outcome {
return when {
answer.objectTypeCase != InteractionObject.ObjectTypeCase.SIGNED_INT ->
throw Exception("Expected signed int answer, not $answer.")
answer.signedInt == 121 -> currentState.interaction.answerGroupsList.first().outcome
else -> currentState.interaction.defaultOutcome
}
}

private fun simulateMultipleChoiceForWelcomeStateExp6(currentState: State, answer: InteractionObject): Outcome {
return when {
answer.objectTypeCase != InteractionObject.ObjectTypeCase.NON_NEGATIVE_INT ->
throw Exception("Expected non-negative int answer, not $answer.")
answer.nonNegativeInt == 3 -> currentState.interaction.answerGroupsList[1].outcome
answer.nonNegativeInt == 0 -> currentState.interaction.answerGroupsList[2].outcome
else -> currentState.interaction.defaultOutcome
}
// Default outcome classification.
return defaultOutcome
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
package org.oppia.domain.classify

/** A general-purpose [InteractionClassifier] that utilizes a Dagger-bound [RuleClassifier] map. */
internal class GenericInteractionClassifier(
private val ruleClassifiers: Map<String, RuleClassifier>
): InteractionClassifier {
override fun getRuleTypes(): Set<String> {
return ruleClassifiers.keys
}

override fun getRuleClassifier(ruleType: String): RuleClassifier? {
return ruleClassifiers[ruleType]
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
package org.oppia.domain.classify

/**
* An answer classifier for a specific interaction type. Instances of this classifier should be bound to a map of
* interaction IDs to classifier instances so that they can be used by the [AnswerClassificationController].
*/
interface InteractionClassifier {
/** Returns a set of rule types that this interaction supports classifying. */
fun getRuleTypes(): Set<String>

/** Returns the [RuleClassifier] corresponding to the specified rule type. */
fun getRuleClassifier(ruleType: String): RuleClassifier?
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
package org.oppia.domain.classify

import dagger.Module
import dagger.Provides
import dagger.multibindings.IntoMap
import dagger.multibindings.StringKey
import org.oppia.domain.classify.rules.ContinueRules
import org.oppia.domain.classify.rules.FractionInputRules
import org.oppia.domain.classify.rules.ItemSelectionInputRules
import org.oppia.domain.classify.rules.MultipleChoiceInputRules
import org.oppia.domain.classify.rules.NumberWithUnitsRules
import org.oppia.domain.classify.rules.NumericInputRules
import org.oppia.domain.classify.rules.TextInputRules

/** Module that provides a map of [InteractionClassifier]s. */
@Module
class InteractionsModule {
@Provides
@IntoMap
@StringKey("Continue")
fun provideContinueInteractionClassifier(
@ContinueRules ruleClassifiers: Map<String, @JvmSuppressWildcards RuleClassifier>
): InteractionClassifier {
return GenericInteractionClassifier(ruleClassifiers)
}

@Provides
@IntoMap
@StringKey("FractionInput")
fun provideFractionInputInteractionClassifier(
@FractionInputRules ruleClassifiers: Map<String, @JvmSuppressWildcards RuleClassifier>
): InteractionClassifier {
return GenericInteractionClassifier(ruleClassifiers)
}

@Provides
@IntoMap
@StringKey("ItemSelectionInput")
fun provideItemSelectionInputInteractionClassifier(
@ItemSelectionInputRules ruleClassifiers: Map<String, @JvmSuppressWildcards RuleClassifier>
): InteractionClassifier {
return GenericInteractionClassifier(ruleClassifiers)
}

@Provides
@IntoMap
@StringKey("MultipleChoiceInput")
fun provideMultipleChoiceInputInteractionClassifier(
@MultipleChoiceInputRules ruleClassifiers: Map<String, @JvmSuppressWildcards RuleClassifier>
): InteractionClassifier {
return GenericInteractionClassifier(ruleClassifiers)
}

@Provides
@IntoMap
@StringKey("NumberWithUnits")
fun provideNumberWithUnitsInteractionClassifier(
@NumberWithUnitsRules ruleClassifiers: Map<String, @JvmSuppressWildcards RuleClassifier>
): InteractionClassifier {
return GenericInteractionClassifier(ruleClassifiers)
}

@Provides
@IntoMap
@StringKey("NumericInput")
fun provideNumericInputInteractionClassifier(
@NumericInputRules ruleClassifiers: Map<String, @JvmSuppressWildcards RuleClassifier>
): InteractionClassifier {
return GenericInteractionClassifier(ruleClassifiers)
}

@Provides
@IntoMap
@StringKey("TextInput")
fun provideTextInputInteractionClassifier(
@TextInputRules ruleClassifiers: Map<String, @JvmSuppressWildcards RuleClassifier>
): InteractionClassifier {
return GenericInteractionClassifier(ruleClassifiers)
}
}
11 changes: 11 additions & 0 deletions domain/src/main/java/org/oppia/domain/classify/RuleClassifier.kt
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
package org.oppia.domain.classify

import org.oppia.app.model.InteractionObject

/** An answer classifier for a specific interaction rule. */
interface RuleClassifier {
/**
* Returns whether the specified answer matches the rule's parameter inputs per this rule's classification strategy.
*/
fun matches(answer: InteractionObject, inputs: Map<String, InteractionObject>): Boolean
}
Loading

0 comments on commit a501b23

Please sign in to comment.