Last utterance is filling slot in new form

[desmarchris]

Rasa version: 2.1.2

Rasa SDK version (if used & relevant): 2.1.2

Python version: 3.8.6

Operating system (windows, osx, ...): osx

Issue:
I have a form that finishes by asking for a size of shoe, filled with a number entity extracted using Duckling. After I submit this form, I kick off another form that asks a survey. The survey asks for a 1-5 rating and for open text feedback. What happens is that when the survey form is kicked off after the user gave their shoe size, the number extracted for shoe size also fills the rating slot, skipping this question. Example:

Your input ->  check inventory
I can help you find some shoes!
What color are you looking for?
Your input ->  blue
What size shoe would you like?
Your input ->  9
You're in luck! We have those in stock.
Is there any other feedback you can provide?

The last question is the second question in my form, showing that the first question (rating question) was skipped.

Bot is located here: https://github.com/RasaHQ/retail-demo

Enable squad definition of done

• Determine if this is expected behaviour given configuration
• If not, then fix the bug
• Else, open an enhancement issue

[wochinge]

@ArjaanBuijk didn't you fix this in #7771 ?

[ArjaanBuijk]

@desmarchris ,
Can you post the full debug log?
It sounds similar to #7771, but not exactly.

[ancalita]

Hi @desmarchris , I've just had a go at a bot conversation using rasa shell --debug after going through all the steps in your README. I was not able to replicate the issue in my first conversation: after the questions about colour and shoe size I got this bot reply which seems to be expected behaviour:

You're in luck! We have those in stock.
? On a scale of 1-5, how would you rate your conversation today?

Could you please let me know what else I can try to replicate this issue?

[desmarchris]

@ancalita yes sorry that's because I changed the order of the questions to get around it :) I have it on my to do list to rewrite how to replicate this.

[desmarchris]

@ancalita try using this commit: https://github.com/RasaHQ/retail-demo/tree/05aeb3486dd995262799e3995570da2cefb27398

I just changed the order of the slot filling after this. Same steps apply

[ancalita]

@desmarchris Thanks for sending this - I was able to replicate using this commit.

I noticed one thing in the domain.yml of this commit that I have a hunch might be causing this behaviour: there is only one number slot that gets used by both forms: product_stock_form and survey_form.
When I introduced a new slot rating and a new entity feedback_rating, the behaviour you flagged disappears.

Instead I have a new issue: although the survey_form gets activated after the product_stock_form, it reverts to action_listen because There was no action found to ask for slot 'rating' name to be filled.
Here's an extract of the debug log:

2021-04-29 11:24:50 DEBUG    rasa.core.policies.rule_policy  - There is a rule for the next action 'action_survey_start'.
2021-04-29 11:24:50 DEBUG    rasa.core.policies.ensemble  - Predicted next action using policy_2_RulePolicy.
2021-04-29 11:24:50 DEBUG    rasa.core.processor  - Predicted next action 'action_survey_start' with confidence 1.00.
2021-04-29 11:24:50 DEBUG    rasa.core.actions.action  - Calling action endpoint to run action 'action_survey_start'.
2021-04-29 11:24:50 DEBUG    rasa.core.processor  - Policy prediction ended with events '[]'.
2021-04-29 11:24:50 DEBUG    rasa.core.processor  - Action 'action_survey_start' ended with events '[<rasa.shared.core.events.FollowupAction object at 0x14b744c40>, <rasa.shared.core.events.SlotSet object at 0x14f841430>]'.
2021-04-29 11:24:50 DEBUG    rasa.core.processor  - Current slot values: 
	color: None
	email: None
	number: None
	rating: None
	open_feedback: None
	requested_slot: None
	verified_email: None
	survey_complete: None
	session_started_metadata: None
2021-04-29 11:24:50 DEBUG    rasa.core.processor  - Predicted next action 'survey_form' with confidence 1.00.
2021-04-29 11:24:50 DEBUG    rasa.core.actions.forms  - Activated the form 'survey_form'.
2021-04-29 11:24:50 DEBUG    rasa.core.actions.forms  - No pre-filled required slots to validate.
2021-04-29 11:24:50 DEBUG    rasa.core.actions.forms  - Validating user input 'UserUttered(text: 9, intent: inform, entities: 9 (Type: number, Role: None, Group: None), use_text_for_featurization: False)'.
2021-04-29 11:24:50 DEBUG    rasa.core.actions.forms  - Validating extracted slots: {}
2021-04-29 11:24:50 DEBUG    rasa.core.actions.forms  - Request next slot 'rating'
2021-04-29 11:24:50 DEBUG    rasa.core.actions.forms  - There was no action found to ask for slot 'rating' name to be filled.
2021-04-29 11:24:50 DEBUG    rasa.core.processor  - Policy prediction ended with events '[]'.
2021-04-29 11:24:50 DEBUG    rasa.core.processor  - Action 'survey_form' ended with events '[<rasa.shared.core.events.ActiveLoop object at 0x14b683be0>, <rasa.shared.core.events.SlotSet object at 0x14f841730>]'.
2021-04-29 11:24:50 DEBUG    rasa.core.processor  - Current slot values: 
	color: None
	email: None
	number: None
	rating: None
	open_feedback: None
	requested_slot: rating
	verified_email: None
	survey_complete: None
	session_started_metadata: None
2021-04-29 11:24:50 DEBUG    rasa.core.policies.memoization  - Current tracker state:
2021-04-29 11:24:50 DEBUG    rasa.core.policies.memoization  - There is a memorised next action 'action_listen'
2021-04-29 11:24:50 DEBUG    rasa.core.policies.ted_policy  - TED predicted 'action_listen' based on user intent.
2021-04-29 11:24:50 DEBUG    rasa.core.policies.rule_policy  - Predicted 'action_listen' after loop 'survey_form'.
2021-04-29 11:24:50 DEBUG    rasa.core.policies.ensemble  - Predicted next action using policy_2_RulePolicy.
2021-04-29 11:24:50 DEBUG    rasa.core.processor  - Predicted next action 'action_listen' with confidence 1.00.
2021-04-29 11:24:50 DEBUG    rasa.core.processor  - Policy prediction ended with events '[]'.
2021-04-29 11:24:50 DEBUG    rasa.core.processor  - Action 'action_listen' ended with events '[]'.
2021-04-29 11:24:50 DEBUG    rasa.core.lock_store  - Deleted lock for conversation 'd67271b0207e463e9b8bb62534243b22'.

I've also edited the custom actions SurveyStart and SurveySubmit to return SlotSet("rating", None) instead of SlotSet("number", None). Is there anything I'm missing?

[ancalita]

I've managed to trigger the survey form immediately after the product_stock_form by modifying the action code:

class SurveyStart(Action):

    def name(self) -> Text:
        return "action_survey_start"

    async def run(self, dispatcher: CollectingDispatcher,
            tracker: Tracker,
            domain: Dict[Text, Any]) -> List[Dict[Text, Any]]:

        if tracker.get_slot("survey_complete")==True:
            return[]
        else:
            dispatcher.utter_message(template="utter_ask_survey_form_number")
            return [FollowupAction("survey_form"), ActiveLoop("survey_form")]

However it identifies the entity number instead of feedback_rating 😒 and then it predicts utter_start_over - still investigating. Here's a snippet of debug log:

2021-04-29 12:20:08 DEBUG    rasa.core.processor  - Predicted next action 'survey_form' with confidence 1.00.
2021-04-29 12:20:08 DEBUG    rasa.core.actions.forms  - Validating user input 'UserUttered(text: 2, intent: inform, entities: 2 (Type: number, Role: None, Group: None), use_text_for_featurization: False)'.
2021-04-29 12:20:08 DEBUG    rasa.core.actions.forms  - Trying to extract requested slot 'rating' ...
2021-04-29 12:20:08 DEBUG    rasa.core.actions.forms  - Got mapping '{'entity': 'feedback_rating', 'type': 'from_entity'}'
2021-04-29 12:20:08 DEBUG    rasa.core.actions.forms  - Failed to extract requested slot 'rating'
2021-04-29 12:20:08 DEBUG    rasa.core.actions.forms  - Validating extracted slots: {}

[ancalita]

Hi @desmarchris sorry for the multiple posts; I've managed to sort the issue you were having which leads me to think this was a usage error.

Here were the steps I took:

• deduplicate number slot used in both forms & redefine the slot in survey_form as rating
• rating slot is defined by entity number like so:

  survey_form:
    rating:
    - entity: number
      type: from_entity
    open_feedback:
    - type: from_text

• add the code in bold in SurveyStart Action in the else branch:

        if tracker.get_slot("survey_complete")==True:
            return[]
        else:
            **dispatcher.utter_message(template="utter_ask_survey_form_number")**
            return [FollowupAction("survey_form"), **ActiveLoop("survey_form")**]

@wochinge do you agree with my conclusion?

[desmarchris]

oh sweet! thanks for exploring @ancalita!

[ancalita]

@desmarchris Ok, great! If all works on your end as well, I will close this ticket.
If you have any issues with this going forward, please re-open the issue and we'll look into it further.