EPIC: Auto mode

[trufae]

• support custom hints to improve automation
• change prompt from :auto to '
• implement errors from r2lang.cmd
• support chatml-function-calling via llama-cpp support chatml-function-calling via llama-cpp #4
• support functionary v2 via llama-cpp
• evaluate different models and compile a list of supported ones and figure out what prompts would work best for them

[trufae]

i've added doc/auto/*.txt and integrated the indexer context message to make the local functionary know which commands to run and how to achieve certain things with r2. this way we can improve the local model auto interaction quite a lot. pleae give it a try because i was able to do several more things properly without gpt thanks to this :)

also, ive updated some models, and mistral2 supports 32K contexts.. so maybe we can use that to compress large disassemblies or decompilation outputs to get an improved output processing. Because the maxtokens and context window is another issue we face with local models. until llama gets improved to handle larger ones in a proper way we must find ways to workaround those limitations.

unfornately i cant use some 3rd party apis because im in europe and some AI vendors dont give access to us :__ so once again another reason to use local models

[dnakov]

That's awesome, just tried functionary and it's actually usable with these simple instructions! I thought it'd be a bigger effort :)

[trufae]

To take into consideration for a another approach on auto mode without function calling https://github.com/user1342/Monocle

[trufae]

If everything looks good to u i would like to release 0.6 and publish it in pip. So we can start messing with other stuff from this stable point once again :) sgty?

[dnakov]

sounds great!

[nitanmarcel]

In this topic, what do you guys think for this?

https://docs.phidata.com/introduction

I'm more interested in this:

https://docs.phidata.com/knowledge/introduction

The thought of having a way we can help the assistant know how to use radare excites me 😂

[dnakov]

@nitanmarcel there's already a RAG in r2ai that can be used for that I think. I remember pancake got one working a few months ago.