Python: ADR: Support for chat history and scratchpads in YAML prompts

[matthewbolanos]

Note

The following code samples below are for communication purposes only. As part of the ADR, the correct names, syntax, and more should be defined.

In LangChain, there is a concept of placeholders for chat history and agent scratchpads within prompts. This issue is to capture the work necessary to provide similar functionality.

Chat History placeholder

With the chat history placeholder, prompt engineers to easily inject chat history into a prompt without needing to iterate through the chat history and generating it themselves while also being able to decorate the beginning and end with system messages. For example...

<system>You are a helpful assistant</system>
<placeholder messages="{{$chathistory}}" />
<system>Predict what the user will say next using the following JSON schema...</system>

This placeholder can also be used for examples that are expressed within a ChatHistory object.

<system>Use the following examples to help the user the correct way</system>
<placeholder messages="{{$examples}}" />
<user>I need help with...</user>

In the YAML prompt the variables should be expressed as inputs with type ChatHistory

inputs:
  - name: examples
     type: ChatHistory
     required: true

Agent scratchpad placeholder

In addition to providing read only context within a prompt, there also needs to be a placeholder for any tool calls / results that are
automatically added to the chat history during automatic function calling. In LangChain, these are expressed as the agent's scratchpad. In LangChain, the developer needs to manually wire up the message placeholder with the scratchpad of the agent using LCEL, but in Semantic Kernel, we have an opportunity to simplify this.

<system>You are a helpful assistant</system>
<placeholder messages="{{$chathistory}}" />
<system>You will now help the user</system>
<placeholder scratchpad="{{$scratchpad}}" />
<system>Only call another function if it is valuable to the user</system>

The scratchpad should then be made available as an output variable and/or metadata of the semantic function.

Note

I'm not the biggest fan of this (multiple outputs seems challenging to handle), so I'm open for additional proposals on how to handle the scratchpad.

output:
  - name: scratchpad
     type: ChatHistory
  - name: output
     type: string

Once the developer has the output, they can then append it to the chat history object (if they want).

Filtering placeholders

Because the placeholders take in a ChatHistory object, prompt engineers should be able to manipulate it with KernelFunctions. For example, there may be a simple filter that takes the top N messages...

<system>You are a helpful assistant</system>
<placeholder messages="{{historyFilter.topN $chathistory 25}}" />
<system>Predict what the user will say next using the following JSON schema...</system>

[github-actions]

This issue is stale because it has been open for 90 days with no activity.

[github-actions]

This issue was closed because it has been inactive for 14 days since being marked as stale.