New Response Selector training data format

[dakshvar22]

Proposed changes:

• Fixes Proposal to change training data format for ResponseSelector #6480

The new training data format for using retrieval intents looks like this -

nlu.yaml

- intent: chitchat/ask_name
  examples: |
    - What do people call you?
    - Do you have a name for yourself?

- intent: chitchat/ask_weather
  examples: |
    - Oh, do you mind checking the weather for me please?
    - I like sunny days in Berlin.

responses.yaml -

responses:
  utter_chitchat/ask_name:
  - image: "https://i.imgur.com/zTvA58i.jpeg"
    text: hello, my name is retrieval bot.
  - text: Oh yeah, I am called the retrieval bot.

  utter_chitchat/ask_weather:
  - text: Oh, it does look sunny right now in Berlin.
    image: "https://i.imgur.com/vwv7aHN.png"
  - text: I am not sure of the whole week but I can see the sun is out today.

domain.yaml

intents:
    - chitchat

rules.yml

- rule: Response with a chitchat utterance whenever user indulges in some chitchat
  steps:
  - intent: chitchat
  - action: utter_chitchat

Two major changes -

1. The bot builder doesn't need to know any difference between an utterance action and a retrieval action. Both of these actions now have the same utter_ prefix
2. Response templates for response selector also now start with utter_ prefix and hence are more consistent with other utterance templates.

Status (please check what you already did):

• added some tests for the functionality
• updated the documentation
• updated the changelog (please check changelog for instructions)
• reformat files using black (please check Readme for instructions)

[dakshvar22]

@tmbo Can you please review the implementation logic? If the way in which the information exchange between NLU training data and Domain looks fine, I'll clean up the code, add tests and documentation.

[tmbo]

the approach looks good 👍 - I think this is a great improvement to the usability of the response selector ⭐

[tmbo]

really excited about this 💯

[github-actions]

Hey @dakshvar22! 👋 To run model regression tests, comment with the /modeltest command and a configuration.

Tips 💡: The model regression test will be run on push events. You can re-run the tests by re-add status:model-regression-tests label or use a Re-run jobs button in Github Actions workflow.

Tips 💡: Every time when you want to change a configuration you should edit the comment with the previous configuration.

You can copy this in your comment and customize:

/modeltest

```yml
##########
## Available datasets
##########
# - "Carbon Bot"
# - "Hermit"
# - "Private 1"
# - "Private 2"
# - "Private 3"
# - "Sara"

##########
## Available configurations
##########
# - "BERT + DIET(bow) + ResponseSelector(bow)"
# - "BERT + DIET(seq) + ResponseSelector(t2t)"
# - "ConveRT + DIET(bow) + ResponseSelector(bow)"
# - "ConveRT + DIET(seq) + ResponseSelector(t2t)"
# - "Spacy + DIET(bow) + ResponseSelector(bow)"
# - "Spacy + DIET(seq) + ResponseSelector(t2t)"
# - "Sparse + ConveRT + DIET(bow) + ResponseSelector(bow)"
# - "Sparse + ConveRT + DIET(seq) + ResponseSelector(t2t)"
# - "Sparse + DIET(bow) + ResponseSelector(bow)"
# - "Sparse + DIET(seq) + ResponseSelector(t2t)"
# - "Sparse + Spacy + DIET(bow) + ResponseSelector(bow)"
# - "Sparse + Spacy + DIET(seq) + ResponseSelector(t2t)"

## Example configuration
#################### syntax #################
## include:
##   - dataset: ["<dataset_name>"]
##     config: ["<configuration_name>"]
#
## Example:
## include:
##  - dataset: ["Carbon Bot"]
##    config: ["Sparse + DIET(bow) + ResponseSelector(bow)"]
#
## Shortcut:
## You can use the "all" shortcut to include all available configurations or datasets
#
## Example: Use the "Sparse + EmbeddingIntent + ResponseSelector(bow)" configuration
## for all available datasets
## include:
##  - dataset: ["all"]
##    config: ["Sparse + DIET(bow) + ResponseSelector(bow)"]
#
## Example: Use all available configurations for the "Carbon Bot" and "Sara" datasets
## and for the "Hermit" dataset use the "Sparse + DIET + ResponseSelector(T2T)" and
## "Sparse + ConveRT + DIET + ResponseSelector(T2T)" configurations:
## include:
##  - dataset: ["Carbon Bot", "Sara"]
##    config: ["all"]
##  - dataset: ["Hermit"]
##    config: ["Sparse + DIET(seq) + ResponseSelector(t2t)", "Sparse + ConveRT + DIET(seq) + ResponseSelector(t2t)"]

include:
 - dataset: ["Carbon Bot"]
   config: ["Sparse + DIET(bow) + ResponseSelector(bow)"]

```

[github-actions]

/modeltest

include:
 - dataset: ["Carbon Bot", "Sara"]
   config: ["all"]

[github-actions]

The model regression tests have started. It might take a while, please be patient.
As soon as results are ready you'll see a new comment with the results.

Used configuration can be found in the comment.

[github-actions]

Commit: b7606ca, The full report is available as an artifact.

Dataset: Carbon Bot

Configuration	Intent Classification Micro F1	Entity Recognition Micro F1	Response Selection Micro F1
BERT + DIET(bow) + ResponseSelector(bow) test: 1m1s, train: 5m59s, total: 6m59s	0.7709	0.6260	0.5133
BERT + DIET(seq) + ResponseSelector(t2t) test: 1m10s, train: 4m42s, total: 5m51s	0.7748	0.8591	0.5316
ConveRT + DIET(bow) + ResponseSelector(bow) test: 38s, train: 2m32s, total: 3m10s	0.8427	0.6260	0.6291
ConveRT + DIET(seq) + ResponseSelector(t2t) test: 48s, train: 3m29s, total: 4m16s	0.8136	0.8609	0.6026
Sparse + ConveRT + DIET(bow) + ResponseSelector(bow) test: 43s, train: 3m29s, total: 4m11s	0.8311	0.6260	0.6026
Sparse + ConveRT + DIET(seq) + ResponseSelector(t2t) test: 51s, train: 3m55s, total: 4m46s	0.8408	0.8831	0.6424
Sparse + DIET(bow) + ResponseSelector(bow) test: 21s, train: 2m49s, total: 3m9s	0.7359	0.6260	0.3377
Sparse + DIET(seq) + ResponseSelector(t2t) test: 30s, train: 3m12s, total: 3m42s	0.7301	0.6618	0.4702

Dataset: Sara

Configuration	Intent Classification Micro F1	Entity Recognition Micro F1	Response Selection Micro F1
BERT + DIET(bow) + ResponseSelector(bow) test: 1m49s, train: 9m12s, total: 11m1s	0.7777	0.8683	0.8196
BERT + DIET(seq) + ResponseSelector(t2t) test: 2m1s, train: 6m44s, total: 8m45s	0.8413	0.8861	0.8152
ConveRT + DIET(bow) + ResponseSelector(bow) test: 57s, train: 5m34s, total: 6m30s	0.8981	0.8683	0.9326
ConveRT + DIET(seq) + ResponseSelector(t2t) test: 1m11s, train: 4m26s, total: 5m37s	0.8883	0.9095	0.9326
Sparse + ConveRT + DIET(bow) + ResponseSelector(bow) test: 1m3s, train: 7m5s, total: 8m8s	0.9001	0.8683	0.9261
Sparse + ConveRT + DIET(seq) + ResponseSelector(t2t) test: 1m17s, train: 5m17s, total: 6m33s	0.8962	0.9087	0.9304
Sparse + DIET(bow) + ResponseSelector(bow) test: 30s, train: 5m51s, total: 6m21s	0.8384	0.8683	0.0196
Sparse + DIET(seq) + ResponseSelector(t2t) test: 43s, train: 4m10s, total: 4m52s	0.8384	0.8428	0.8783

[wochinge]

I only reviewed the rasa.shared changes. They are good to go apart from the comments.

I didn't check the rest, but shouldn't we still support old stories which use respond_ instead of utter_?

[wochinge]

Let's move it to rasa.shared then instead of re-adding this constant 👍

[dakshvar22]

@wochinge Since this was an experimental feature we had the liberty to deprecate the old stuff. It's not just respond_ actions that we need to then still support. Even response templates need a utter_ prefix just like usual utterance template. Both these changes are in the migration guide to help the user migrate.