[Bug]: multi-line answer is empty from ReAct agent

[vitalyford]

Bug Description

Based on extra_final_response() from llama_index/agent/react/output_parser.py, the answer will be extracted until a new line character is encountered or the end of the string. In cases when the final answer spans across multiple lines, it will return an empty response.

A possible fix is to remove \n| and just keep $. But then what's the usability of $ there? Need to put more thinking into it though if anything else is affected by it; hence, no PR yet. Has someone encountered this before?

Version

0.8.27

Steps to Reproduce

Query a ReAct agent to respond in an enumerated or bullet-list style.

Relevant Logs/Tracbacks

No response

[dosubot]

🤖

Hello,

Thank you for bringing this issue to our attention. You're correct in your observation that the extract_final_response() function in llama_index/agent/react/output_parser.py could potentially return an empty response when the final answer spans across multiple lines. This is due to the regular expression pattern used in the function, which stops matching at the first newline character.

Your proposed solution of removing \n| and just keeping $ in the regular expression pattern could potentially fix this issue. This modification would allow the regular expression to match until the end of the string, regardless of newline characters. Here's how the modified function might look:

def extract_final_response(input_text: str) -> Tuple[str, str]:
    pattern = r"\s*Thought:(.*?)Answer:(.*?)(?:$)"

    match = re.search(pattern, input_text, re.DOTALL)
    if not match:
        raise ValueError(
            "Could not extract final answer from input text: {}".format(input_text)
        )

    thought = match.group(1).strip()
    answer = match.group(2).strip()
    return thought, answer

However, please note that this change could have other side effects depending on the structure of your input text. It's important to thoroughly test this modification with different scenarios to ensure it doesn't introduce new issues.

As for the exact structure of the input text that the extract_final_response() function is expected to handle, I wasn't able to find specific documentation within the LlamaIndex codebase. If you could provide more details about the structure of the input text you're working with, it would be helpful in further investigating this issue and proposing a more robust solution.

Best,
Dosu

Sources

• llama_index/agent/react/output_parser.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.