Support system prompt natively for Gemini 1.5

[MichaelDoyle]

Gemini 1.5 supports system instructions natively, but we do not utilize it currently. This proposal would fix that.

The main gotcha here, is that Gemini implemented systemInstructions as a new input parameter, rather than leveraging messages with the role=system. Since our GenerateRequest abstraction is currently predicated on this paradigm (role=system) I had to do a bit of legwork. This proposal will:

• If system instructions are supported, look to see if the first message has role=system
• Transform it into a message with role=user and pass to the model as systemInstructions
• Splice the system message off of the history
• Throw an Error any time a message with role=system appears anywhere else in the request

Alternatively, we could change GenerateRequest to accept systemInstructions directly.

[mbleigh]

Food for thought: I intended ModelInfo to be user-facing indicators of what's supported. So a model that has a synthetic system prompt would still support a system prompt as far as ModelInfo is concerned. I'm not sure how to rationalize that with the need to change model factory behavior based on underlying capability tho...

[mbleigh]

I'd recommend just detecting and splicing off the first system message. Something like:

const systemMessage = messages.find(m => m.role === 'system');
if (systemMessage) messages.splice(messages.indexOf(systemMessage), 1);
const systemInstruction = toGeminiSystemInstruction(systemMessage);

[MichaelDoyle]

I had this :) but it seemed to negatively impact the trace, as the system message was no longer part of the "input".

[mbleigh]

How about above you do const messages = [...request.messages] to copy the array, then it won't be modified in place and should still show up in the trace.

[MichaelDoyle]

Food for thought: I intended ModelInfo to be user-facing indicators of what's supported. So a model that has a synthetic system prompt would still support a system prompt as far as ModelInfo is concerned. I'm not sure how to rationalize that with the need to change model factory behavior based on underlying capability tho...

Yes, this is a great point. I'll noodle on this some more.

[MichaelDoyle]

Updated supports: systemRole to be a user-facing indicator.