change FallbackClassifier confidence to fallback threshold

[wochinge]

Context

Problem
The FallbackClassifier currently sets its confidence as 1 - top intent confidence when trigger the nlu_fallback intent. This looks weird in the Rasa X NLU inbox as the messages might have the intent nlu_fallback but when users look at the confidence scores nlu_fallback doesn't have the highest confidence score.

Solution
We need to revisit this in Rasa X. For now we've decided to change the FallbackClassifier score to the fallback threshold.

Definition of Done

• When FallbackClassifier triggers the nlu_fallback it does that with confidence self.component_config[THRESHOLD_KEY]

[ancalita]

@wochinge to double-check, the change needs to be made in this line? Do I need to check that message.data[INTENT] is nlu_fallback and set confidence as instructed in DoD, otherwise keep existing confidence assignment?

[wochinge]

the change needs to be made in this line

Exactly.

I need to check that message.data[INTENT] is nlu_fallback and set confidence as instructed in DoD, otherwise keep existing confidence assignment?

That check should already be done by this.

The way the FallbackClassifier works is that it replaces the predicted intent with the nlu_fallback. should_fallback checks if the prediction confidence was below the threshold which the user configured for FallbackClassifier.