Colang v2 Interaction Loop with Query Rewriting

[niels-garve]

Hi all,

I am creating an AI solution based on the interaction loop of Colang v2. To improve the intent detection for user messages like confirmations ("yes", "no", etc.) I'd like to add Query Rewriting; basically an LLM call prompting to rephrase the user message with context from the conversation.

Example

User: Show me some bikes
Bot: Sure! Are you looking for some mountain bikes
User: yes

Now, query rewriting would transform the "yes" into "Show me some mountain bikes".

I don't find a solution; I tried with a custom action, but from where do I get the conversation. I could use the default prompt for intent detection, but I also would like to reduce cognitive load on LLM calls. Intent detection is one thing, rephrasing another.

Here's what I have so far:

main.co

import core
import llm

flow main
    activate llm continuation

    global $last_user_message
    while True
        await user said something as $something
        await RewriteQueryAction(user_message=$something.transcript)
        bot say "{$last_user_message}"
        loop

flow loop
    when user asked about bikes
        answering bike questions
    or when unhandled user intent # For any user utterance that does not match
        bot say "My apologies, I can't answer your question"

flow user asked about bikes
    user said "Show me bikes"
        or user said "I want to see bikes"
        or user said "Do you have any bicycles?"
        or user said "Can you show me some bikes?"
        or user said "I'm looking for bikes"
        or user said "Bicycles available?"
        or user said "What bicycles do you have?"
        or user said "Can you display bikes for me?"
        or user said "I'd like to see your bikes"
        or user said "Do you sell bicycles?"
        or user said "Bike options?"
        or user said "What kinds of bikes do you have?"
        or user said "Show me your bicyles collection"
        or user said "Can I browse your bikes?"
        or user said "What bikes do you offer?"
        or user said "I'm interested in buying a bike"
        or user said "Any bicycles for sale?"
        or user said "What's in your bicycle inventory?"
        or user said "Do you have a bike catalog?"
        or user said "I'd like to look at bikes"

flow answering bike questions
    """You are an assistant that should talk to the user about bikes.
    Politely decline to talk about anything else.

    Last user question is: "{{ last_user_message }}"
    Generate the output in the following format:

    bot say '''<<the response>>'''
    """
    ...

config.py

@action()
async def RewriteQueryAction(user_message: str, context: dict, llm: BaseLLM) -> ActionResult:
    context_updates = {}

    # FIXME: no LLM prompt yet, just a static rewrite
    rewrote_query = "Test"
    context_updates["last_user_message"] = rewrote_query

    return ActionResult(context_updates=context_updates)

def init(app: LLMRails):
    app.register_action(RewriteQueryAction, "RewriteQueryAction")

I use a default config.yml

I'd appreciate your help!

Thanks and cheers,
Niels

[niels-garve]

Update: I'm currently trying input rails for user message rephrasing, with no luck. It feels like the correct place to do that, though:

import core
import llm
import guardrails

flow input rails $input_text
    rephrase user message

flow rephrase user message
    global $user_message
    $user_message = await RephraseUserMessageAction()

flow main
    activate llm continuation

    while True
        when user asked about bikes
            answering bike questions
        or when unhandled user intent # For any user utterance that does not match
            bot say "My apologies, I can't answer your question"

flow user asked about bikes
    user said "Show me bikes"
        or user said "I want to see bikes"
        or user said "Do you have any bicycles?"
        or user said "Can you show me some bikes?"
        or user said "I'm looking for bikes"
        or user said "Bicycles available?"
        or user said "What bicycles do you have?"
        or user said "Can you display bikes for me?"
        or user said "I'd like to see your bikes"
        or user said "Do you sell bicycles?"
        or user said "Bike options?"
        or user said "What kinds of bikes do you have?"
        or user said "Show me your bicyles collection"
        or user said "Can I browse your bikes?"
        or user said "What bikes do you offer?"
        or user said "I'm interested in buying a bike"
        or user said "Any bicycles for sale?"
        or user said "What's in your bicycle inventory?"
        or user said "Do you have a bike catalog?"
        or user said "I'd like to look at bikes"

flow answering bike questions
    """You are an assistant that should talk to the user about bikes.
    Politely decline to talk about anything else.

    Last user question is: "{{ last_user_message }}"
    Generate the output in the following format:

    bot say '''<<the response>>'''
    """
    ...

[niels-garve]

Opened a PR: #869