Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix #114: Implement answer classification controller #211

Merged
merged 21 commits into from
Oct 9, 2019
Merged
Show file tree
Hide file tree
Changes from 20 commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
e19af2b
Introduce first pass interface for ExplorationProgressController.
BenHenning Sep 25, 2019
43766b0
Fill in the stubbed logic for ExplorationProgressController. Still no
BenHenning Sep 26, 2019
c8eaa66
Fix lateinit issue in ExplorationProgressController due to wrongly or…
BenHenning Sep 27, 2019
a0eb3ce
Fix a variaty of issues in the exp progress controller, properly hook…
BenHenning Sep 29, 2019
1ea9d01
Merge branch 'develop' into introduce-exploration-progress-controller
BenHenning Sep 29, 2019
41141b6
Created a separate ExplorationRetriever, hooked up
BenHenning Oct 1, 2019
ccbac0e
Merge branch 'develop' into introduce-exploration-progress-controller
BenHenning Oct 1, 2019
5f326fc
Change the locking mechanism for ExplorationProgressController to work
BenHenning Oct 3, 2019
b5b7485
Finish tests for ExplorationProgressController and add test classific…
BenHenning Oct 4, 2019
a3c4667
Merge branch 'develop' into introduce-exploration-progress-controller
BenHenning Oct 4, 2019
e2dfd5a
Merge branch 'introduce-exploration-progress-controller' into impleme…
BenHenning Oct 6, 2019
340b22b
First iteration at implementing real answer classification for the Oppia
BenHenning Oct 6, 2019
f0bfc44
Bind all text input rules, add numeric input rules, and add support for
BenHenning Oct 6, 2019
d5c313e
Add numbers with units classification support.
BenHenning Oct 7, 2019
54ef1c2
Add multiple choice input classification support. Fix some of the
BenHenning Oct 7, 2019
e8eae1b
Add item selection classification support. Clean up all rule classifier
BenHenning Oct 7, 2019
ffb324c
Add fractions input classification support.
BenHenning Oct 7, 2019
24935cc
Add Continue module. Resolve some TODOs, add TODOs to test classifiers,
BenHenning Oct 7, 2019
2d7a58a
Add thorough tests for AnswerClassificationController.
BenHenning Oct 7, 2019
4bac447
Consolidate generic rule classifiers into a single class.
BenHenning Oct 7, 2019
2e95ce7
Merge branch 'develop' into implement-answer-classification-controller
BenHenning Oct 7, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -5,14 +5,26 @@ import dagger.BindsInstance
import dagger.Component
import org.oppia.app.activity.ActivityComponent
import org.oppia.data.backends.gae.NetworkModule
import org.oppia.domain.classify.InteractionsModule
import org.oppia.domain.classify.rules.continueinteraction.ContinueModule
import org.oppia.domain.classify.rules.fractioninput.FractionInputModule
import org.oppia.domain.classify.rules.itemselectioninput.ItemSelectionInputModule
import org.oppia.domain.classify.rules.multiplechoiceinput.MultipleChoiceInputModule
import org.oppia.domain.classify.rules.numberwithunits.NumberWithUnitsRuleModule
import org.oppia.domain.classify.rules.numericinput.NumericInputRuleModule
import org.oppia.domain.classify.rules.textinput.TextInputRuleModule
import org.oppia.util.logging.LoggerModule
import org.oppia.util.threading.DispatcherModule
import javax.inject.Provider
import javax.inject.Singleton

/** Root Dagger component for the application. All application-scoped modules should be included in this component. */
@Singleton
@Component(modules = [ApplicationModule::class, DispatcherModule::class, NetworkModule::class, LoggerModule::class])
@Component(modules = [
ApplicationModule::class, DispatcherModule::class, NetworkModule::class, LoggerModule::class,
ContinueModule::class, FractionInputModule::class, ItemSelectionInputModule::class, MultipleChoiceInputModule::class,
NumberWithUnitsRuleModule::class, NumericInputRuleModule::class, TextInputRuleModule::class, InteractionsModule::class
])
interface ApplicationComponent {
@Component.Builder
interface Builder {
Expand Down
Original file line number Diff line number Diff line change
@@ -1,26 +1,60 @@
package org.oppia.domain.classify

import org.oppia.app.model.AnswerGroup
import org.oppia.app.model.Interaction
import org.oppia.app.model.InteractionObject
import org.oppia.app.model.Outcome
import java.lang.IllegalStateException
import javax.inject.Inject

// TODO(#59): Restrict the visibility of this class to only other controllers.
/**
* Controller responsible for classifying user answers to a specific outcome based on Oppia's interaction rule engine.
* This controller is not meant to be interacted with directly by the UI. Instead, UIs wanting to submit answers should
* do so via various progress controllers, like [StoryProgressController].
* do so via various progress controllers, like [org.oppia.domain.topic.StoryProgressController].
*
* This controller should only be interacted with via background threads.
*/
class AnswerClassificationController @Inject constructor() {
class AnswerClassificationController @Inject constructor(
private val interactionClassifiers: Map<String, @JvmSuppressWildcards InteractionClassifier>
) {
/**
* Classifies the specified answer in the context of the specified [Interaction] and returns the [Outcome] that best
* matches the learner's answer.
*/
internal fun classify(interaction: Interaction, answer: InteractionObject): Outcome {
// Assume only the default outcome is returned currently since this stubbed implementation is not actually used by
// downstream stubbed progress controllers.
return interaction.defaultOutcome
val interactionClassifier = checkNotNull(interactionClassifiers[interaction.id]) {
"Encountered unknown interaction type: ${interaction.id}, expected one of: ${interactionClassifiers.keys}"
}
// TODO(#207): Add support for additional classification types.
return classifyAnswer(
answer, interaction.answerGroupsList, interaction.defaultOutcome, interactionClassifier, interaction.id)
}

// Based on the Oppia web version:
// https://github.com/oppia/oppia/blob/edb62f/core/templates/dev/head/pages/exploration-player-page/services/answer-classification.service.ts#L57.
private fun classifyAnswer(
answer: InteractionObject, answerGroups: List<AnswerGroup>, defaultOutcome: Outcome,
interactionClassifier: InteractionClassifier, interactionId: String
): Outcome {
for (answerGroup in answerGroups) {
for (ruleSpec in answerGroup.ruleSpecsList) {
val ruleClassifier = checkNotNull(interactionClassifier.getRuleClassifier(ruleSpec.ruleType)) {
"Expected interaction $interactionId to have classifier for rule type: ${ruleSpec.ruleType}, but only" +
" has: ${interactionClassifier.getRuleTypes()}"
}
try {
if (ruleClassifier.matches(answer, ruleSpec.inputMap)) {
// Explicit classification matched.
return answerGroup.outcome
}
} catch (e: Exception) {
throw IllegalStateException("Failed when classifying answer $answer for interaction $interactionId", e)
}
}
}

// Default outcome classification.
return defaultOutcome
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
package org.oppia.domain.classify

/** A general-purpose [InteractionClassifier] that utilizes a Dagger-bound [RuleClassifier] map. */
internal class GenericInteractionClassifier(
private val ruleClassifiers: Map<String, RuleClassifier>
): InteractionClassifier {
override fun getRuleTypes(): Set<String> {
return ruleClassifiers.keys
}

override fun getRuleClassifier(ruleType: String): RuleClassifier? {
return ruleClassifiers[ruleType]
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
package org.oppia.domain.classify

/**
* An answer classifier for a specific interaction type. Instances of this classifier should be bound to a map of
* interaction IDs to classifier instances so that they can be used by the [AnswerClassificationController].
*/
interface InteractionClassifier {
/** Returns a set of rule types that this interaction supports classifying. */
fun getRuleTypes(): Set<String>

/** Returns the [RuleClassifier] corresponding to the specified rule type. */
fun getRuleClassifier(ruleType: String): RuleClassifier?
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
package org.oppia.domain.classify

import dagger.Module
import dagger.Provides
import dagger.multibindings.IntoMap
import dagger.multibindings.StringKey
import org.oppia.domain.classify.rules.ContinueRules
import org.oppia.domain.classify.rules.FractionInputRules
import org.oppia.domain.classify.rules.ItemSelectionInputRules
import org.oppia.domain.classify.rules.MultipleChoiceInputRules
import org.oppia.domain.classify.rules.NumberWithUnitsRules
import org.oppia.domain.classify.rules.NumericInputRules
import org.oppia.domain.classify.rules.TextInputRules

/** Module that provides a map of [InteractionClassifier]s. */
@Module
class InteractionsModule {
@Provides
@IntoMap
@StringKey("Continue")
fun provideContinueInteractionClassifier(
@ContinueRules ruleClassifiers: Map<String, @JvmSuppressWildcards RuleClassifier>
): InteractionClassifier {
return GenericInteractionClassifier(ruleClassifiers)
}

@Provides
@IntoMap
@StringKey("FractionInput")
fun provideFractionInputInteractionClassifier(
@FractionInputRules ruleClassifiers: Map<String, @JvmSuppressWildcards RuleClassifier>
): InteractionClassifier {
return GenericInteractionClassifier(ruleClassifiers)
}

@Provides
@IntoMap
@StringKey("ItemSelectionInput")
fun provideItemSelectionInputInteractionClassifier(
@ItemSelectionInputRules ruleClassifiers: Map<String, @JvmSuppressWildcards RuleClassifier>
): InteractionClassifier {
return GenericInteractionClassifier(ruleClassifiers)
}

@Provides
@IntoMap
@StringKey("MultipleChoiceInput")
fun provideMultipleChoiceInputInteractionClassifier(
@MultipleChoiceInputRules ruleClassifiers: Map<String, @JvmSuppressWildcards RuleClassifier>
): InteractionClassifier {
return GenericInteractionClassifier(ruleClassifiers)
}

@Provides
@IntoMap
@StringKey("NumberWithUnits")
fun provideNumberWithUnitsInteractionClassifier(
@NumberWithUnitsRules ruleClassifiers: Map<String, @JvmSuppressWildcards RuleClassifier>
): InteractionClassifier {
return GenericInteractionClassifier(ruleClassifiers)
}

@Provides
@IntoMap
@StringKey("NumericInput")
fun provideNumericInputInteractionClassifier(
@NumericInputRules ruleClassifiers: Map<String, @JvmSuppressWildcards RuleClassifier>
): InteractionClassifier {
return GenericInteractionClassifier(ruleClassifiers)
}

@Provides
@IntoMap
@StringKey("TextInput")
fun provideTextInputInteractionClassifier(
@TextInputRules ruleClassifiers: Map<String, @JvmSuppressWildcards RuleClassifier>
): InteractionClassifier {
return GenericInteractionClassifier(ruleClassifiers)
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
package org.oppia.domain.classify

import org.oppia.app.model.InteractionObject

/** An answer classifier for a specific interaction rule. */
interface RuleClassifier {
/**
* Returns whether the specified answer matches the rule's parameter inputs per this rule's classification strategy.
*/
fun matches(answer: InteractionObject, inputs: Map<String, InteractionObject>): Boolean
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,175 @@
package org.oppia.domain.classify.rules

import org.oppia.app.model.InteractionObject
import org.oppia.domain.classify.RuleClassifier
import javax.inject.Inject

/**
* A convenience [RuleClassifier] which performs parameter extraction and sanitation to simplify classifiers, with the
* possible configurations for zero, one, or two parameters with the same or differing types compared to the answer
* being matched.
*
* Child classes must ensure all specified types properly correspond to the type to which the parameter's specified
* [InteractionObject.ObjectTypeCase] also corresponds.
*/
internal class GenericRuleClassifier private constructor(
private val expectedAnswerObjectType: InteractionObject.ObjectTypeCase,
private val orderedExpectedParameterTypes: LinkedHashMap<String, InteractionObject.ObjectTypeCase>,
private val matcherDelegate: MatcherDelegate
): RuleClassifier {
override fun matches(answer: InteractionObject, inputs: Map<String, InteractionObject>): Boolean {
check(answer.objectTypeCase == expectedAnswerObjectType) {
"Expected answer to be of type ${expectedAnswerObjectType.name} not ${answer.objectTypeCase.name}"
}
val parameterInputs = orderedExpectedParameterTypes.toList().map { (parameterName, expectedObjectType) ->
retrieveInputObject(parameterName, expectedObjectType, inputs)
}
return matcherDelegate.matches(answer, parameterInputs)
}

private fun retrieveInputObject(
parameterName: String, expectedObjectType: InteractionObject.ObjectTypeCase, inputs: Map<String, InteractionObject>
): InteractionObject {
val input = checkNotNull(inputs[parameterName]) {
"Expected classifier inputs to contain parameter with name '$parameterName' but had: ${inputs.keys}"
}
check(input.objectTypeCase == expectedObjectType) {
"Expected input value to be of type ${expectedObjectType.name} not ${input.objectTypeCase.name}"
}
return input
}

internal interface NoInputInputMatcher<T> {
/**
* Returns whether the validated and extracted answer matches the expectations per the specification of this
* classifier.
*/
fun matches(answer: T): Boolean
}

internal interface SingleInputMatcher<T> {
/**
* Returns whether the validated and extracted answer matches the single validated and extracted input parameter per
* the specification of this classifier.
*/
fun matches(answer: T, input: T): Boolean
}

internal interface MultiTypeSingleInputMatcher<AT, IT> {
/**
* Returns whether the validated and extracted answer matches the single validated and extracted input parameter per
* the specification of this classifier.
*/
fun matches(answer: AT, input: IT): Boolean
}

internal interface DoubleInputMatcher<T> {
/**
* Returns whether the validated and extracted answer matches the two validated and extracted input parameters per
* the specification of this classifier.
*/
fun matches(answer: T, firstInput: T, secondInput: T): Boolean
}

internal sealed class MatcherDelegate {

abstract fun matches(answer: InteractionObject, inputs: List<InteractionObject>): Boolean

internal class NoInputMatcherDelegate<T: Any>(
private val matcher: NoInputInputMatcher<T>,
private val extractObject: (InteractionObject) -> T
): MatcherDelegate() {
override fun matches(answer: InteractionObject, inputs: List<InteractionObject>): Boolean {
check(inputs.isEmpty())
return matcher.matches(extractObject(answer))
}
}

internal class SingleInputMatcherDelegate<T: Any>(
private val matcher: SingleInputMatcher<T>,
private val extractObject: (InteractionObject) -> T
): MatcherDelegate() {
override fun matches(answer: InteractionObject, inputs: List<InteractionObject>): Boolean {
check(inputs.size == 1)
return matcher.matches(extractObject(answer), extractObject(inputs.first()))
}
}

internal class MultiTypeSingleInputMatcherDelegate<AT: Any, IT: Any>(
private val matcher: MultiTypeSingleInputMatcher<AT, IT>,
private val extractAnswerObject: (InteractionObject) -> AT,
private val extractInputObject: (InteractionObject) -> IT
): MatcherDelegate() {
override fun matches(answer: InteractionObject, inputs: List<InteractionObject>): Boolean {
check(inputs.size == 1)
return matcher.matches(extractAnswerObject(answer), extractInputObject(inputs.first()))
}
}

internal class DoubleInputMatcherDelegate<T: Any>(
private val matcher: DoubleInputMatcher<T>,
private val extractObject: (InteractionObject) -> T
): MatcherDelegate() {
override fun matches(answer: InteractionObject, inputs: List<InteractionObject>): Boolean {
check(inputs.size == 2)
return matcher.matches(extractObject(answer), extractObject(inputs[0]), extractObject(inputs[1]))
}
}
}

/** Factory to create new [GenericRuleClassifier]s. */
internal class Factory @Inject constructor(
private val interactionObjectTypeExtractorRepository: InteractionObjectTypeExtractorRepository
) {
/** Returns a new [GenericRuleClassifier] for an answer that is not matched to any input values. */
inline fun <reified T: Any> createNoInputClassifier(
expectedObjectType: InteractionObject.ObjectTypeCase, matcher: NoInputInputMatcher<T>
): GenericRuleClassifier {
val objectExtractor = interactionObjectTypeExtractorRepository.getExtractor<T>(expectedObjectType)
return GenericRuleClassifier(
expectedObjectType, LinkedHashMap(), MatcherDelegate.NoInputMatcherDelegate(matcher, objectExtractor))
}

/**
* Returns a new [GenericRuleClassifier] for a single input value with the same type as the answer being classified.
*/
inline fun <reified T: Any> createSingleInputClassifier(
expectedObjectType: InteractionObject.ObjectTypeCase, inputParameterName: String, matcher: SingleInputMatcher<T>
): GenericRuleClassifier {
val objectExtractor = interactionObjectTypeExtractorRepository.getExtractor<T>(expectedObjectType)
return GenericRuleClassifier(
expectedObjectType, linkedMapOf(inputParameterName to expectedObjectType),
MatcherDelegate.SingleInputMatcherDelegate(matcher, objectExtractor))
}

/**
* Returns a new [GenericRuleClassifier] for a single input value that has a different type than the answer being
* classified.
*/
inline fun <reified AT: Any, reified IT: Any> createMultiTypeSingleInputClassifier(
expectedAnswerObjectType: InteractionObject.ObjectTypeCase,
expectedInputObjectType: InteractionObject.ObjectTypeCase, inputParameterName: String,
matcher: MultiTypeSingleInputMatcher<AT, IT>
): GenericRuleClassifier {
val answerObjectExtractor = interactionObjectTypeExtractorRepository.getExtractor<AT>(expectedAnswerObjectType)
val inputObjectExtractor = interactionObjectTypeExtractorRepository.getExtractor<IT>(expectedInputObjectType)
return GenericRuleClassifier(
expectedAnswerObjectType, linkedMapOf(inputParameterName to expectedInputObjectType),
MatcherDelegate.MultiTypeSingleInputMatcherDelegate(matcher, answerObjectExtractor, inputObjectExtractor))
}

/** Returns a new [GenericRuleClassifier] for two input values of the same type as the answer it classifies. */
inline fun <reified T: Any> createDoubleInputClassifier(
expectedObjectType: InteractionObject.ObjectTypeCase, firstInputParameterName: String,
secondInputParameterName: String, matcher: DoubleInputMatcher<T>
): GenericRuleClassifier {
val objectExtractor = interactionObjectTypeExtractorRepository.getExtractor<T>(expectedObjectType)
val parameters = linkedMapOf(
firstInputParameterName to expectedObjectType,
secondInputParameterName to expectedObjectType
)
return GenericRuleClassifier(
expectedObjectType, parameters, MatcherDelegate.DoubleInputMatcherDelegate(matcher, objectExtractor))
}
}
}
Loading