React Agent Workflow

[AryanSakhala]

Getting trouble in implementing this with vLLms for all llm functions.

Getting raise ValueError("LLM must be a function calling model!") ValueError: LLM must be a function calling model! specific error.

For all the llm callingfunctions like

llm.get_tool_calls_from_response(
            response, error_on_no_tool_call=False
        )

[logan-markewich]

Correct, vllm does not implement FunctionCallingLLM

Although if I'm reading this right, vllm does support function calling
vllm-project/vllm#7912

But only for certain llms (like llama3.1)

I'd suggest using llama3.2 + the openai server + the OpenAILike class

There's a small example here that you can follow for setup
https://colab.research.google.com/drive/1h-eHhoKUnkJQ2TTsw4HxOd0f1zVd97Ol?usp=sharing

(note, I haven't actually tried this yet with vllm + openai like, but in theory, should work)

The other option is re-implementing the workflow to implement a react loop instead of a function calling loop, but it will be less reliable imo
https://docs.llamaindex.ai/en/stable/examples/workflow/react_agent/

[AryanSakhala]

Thanks for your help.
React AGet workflow worked but,
React Agent workflow is not yielding the best result

[AryanSakhala]

Vllm yet is giving bit of a hard time
React agent workflow works but is not the best

Tried with Meta 3.1 70 B Instruct, and variants of Phi, Mistral

I will close the issue.