Standardize testing output: Response Selection

[tabergma]

Proposed changes:

• Plot histogram with confidence distribution for response selection
• Write correct and incorrect predictions for response selection

related to #5748

Status (please check what you already did):

• added some tests for the functionality
• updated the documentation
• updated the changelog (please check changelog for instructions)
• reformat files using black (please check Readme for instructions)

[dakshvar22]

A bit unlikely that it happens but it could break someone's code relying on this function in some glue code of their own. Should we add a deprecation warning or keep a dummy function in rasa.nlu.test which just calls the one in rasa.utils.plotting?

[tabergma]

Not sure if that is needed as it will be released with 2.0. I also removed two cli options for naming the histogram and confusion matrix file which are not useful anymore. If you think it is needed, I'll keep those and add a deprecation warning.

[tabergma]

@tmbo Just wanted to double check if this is fine.

[dakshvar22]

@tabergma I pushed a small fix to add the response_key to the intent part of response selection results so that when viewing the error file, the dict looks like -

{
    "text": "How do I know someone is actually doing something for my offsets?",
    "intent_target": "faq/is-this-legit",
    "response": "All offset projects have to pass a rigorous verification process set by https://goldstandard.org .",
    "response_prediction": {
      "name": "The cost per ton depends on the project you donate to. Some projects can remove a ton of CO2 for less than $1!",
      "confidence": 0.15156520903110504
    }

instead of

{
    "text": "How do I know someone is actually doing something for my offsets?",
    "intent_target": "faq",
    "response": "All offset projects have to pass a rigorous verification process set by https://goldstandard.org .",
    "response_prediction": {
      "name": "The cost per ton depends on the project you donate to. Some projects can remove a ton of CO2 for less than $1!",
      "confidence": 0.15156520903110504
    }

Otherwise all intents would just be faq which is a bit unhelpful.

[dakshvar22]

@tabergma In the confusion matrix should we use the intent/response_key as the labels of the rows and columns instead of the actual responses? The mapping is deterministic since there is only one response per intent/response_key but it would make the confusion matrix fit well when you have large number of unique responses. What do you think?

[tabergma]

@dakshvar22 Good idea. Will try it now.

[dakshvar22]

Ahh, that would need a bigger refactor. Because getting the intent/response_key of the predicted response is not trivial right now. Let's keep it as a separate issue.

[tabergma]

😄 Ok. As an alternative, we could also maybe use a legend that maps the actual response to A, B, etc. What do you think?

[dakshvar22]

I would say let us directly replace them with the intent/response_key separately because that's the most ideal.

[dakshvar22]

Just a small key change for avoiding the ambiguity. Looks great! 💯