I agree. It would also be great if we could choose a different model to generate those titles. GPT-3.5 or Haiku are good enough to create titles, and that would save users a little money.

@Barafu Do you know what model it uses? Agreed it should use a cheap model for the title

@endolith It uses the same model that was used for chat. It is very visible in the logs of token usage, since it reads ~500 tokens and outputs ~10.

oh yeah that's wasteful

It also looks like it puts the entire conversation in the system message instead of the "generate a title" prompt in the system message and the conversation in a user message? Is that why it needs to duplicate the instructions?

import { Message } from '../stores/types'

export function nameConversation(msgs: Message[]): Message[] {
    const format = (msgs: string[]) => msgs.map((msg) => msg).join('\n\n---------\n\n')
    return [
        {
            id: '1',
            role: 'system',
            content: `Name the conversation based on the chat records.
Please provide a concise name, within 10 characters and without quotation marks.
Please use the speak language in the conversation.
You only need to answer with the name.
The following is the conversation:

\`\`\`
${format(msgs.map((msg) => msg.content))}
\`\`\`

Please provide a concise name, within 10 characters and without quotation marks.
Please use the speak language in the conversation.
You only need to answer with the name.
The conversation is named:`,
        },
    ]
}

and what format is the msg.content? If it contains code sections with three backticks, then the prompt needs to be one level higher and have 4 backticks.

@endolith Just for reference: There is a web based chat UI that I used before, and it creates the most useful titles for chats of all I tested. I asked the developer, and he shared the prompt template.

Generate a title in less than 6 words for the following message (language: ${lang}):\n"""\nUser: ${user_message}\nAssistant: ${assistant_message}\n"""

As usual with LLMs, less is more. Note the absence of the system prompt ("You are a helpful assasin assistant bla bla")

@Barafu Well I tried to improve the prompt. I did two tests of it with gpt-3.5 and it worked better than the current prompt. If you want, you can try both prompts and see which produce better titles and I can change #1397 if yours works better