Access 'configurable' field values in a prompt within a conversation QA chain

[Chengdyc]

Hi, I'm new to langserve and I've set up a chain using the chat with persistence example. https://github.com/langchain-ai/langserve/blob/main/examples/chat_with_persistence/server.py. I have the per_request_config_modifier set up to extract the authenticated user id and set it into the configurable object. i.e.
config.setdefault("configurable", {})["user_name"] = authed_user.user_name
I want to pass this user_name to the LLM for generating a greeting addressed to the user (this is just a toy example for now, we'll need to pass other user-specific context to LLM or other runnables in the chain later).
How do I go about accessing configurable fields? I tried calling configurable_fields() on the chain but got errors complaining that configuration key is not found. Any pointers is appreciated. thanks!

[pmosconi]

Hi @Chengdyc ,
after a few hours of head scratching and trial and error, I've been able to make it.
I'm not using a configurable field, but a prompt variable that is assigned in the chain's first step reading from config.

Here's some code:

import os
from typing import Any, Dict
from fastapi import Request
from langchain_openai import AzureChatOpenAI
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain_core.runnables import RunnablePassthrough

# Pass the chat history keeping the list of messages structure
def history_pass(x: list) -> list:
    lst = []
    for i in x:
        lst.append(i)
    return lst

# Get the user ID from the runnable config
def get_user(input: Any, config: Dict[str, Any]) -> str:
    configurable = config.get("configurable", {})
    return configurable["user_id"]


model = AzureChatOpenAI(
    openai_api_version=os.getenv("OPENAI_API_VERSION"),
    azure_deployment=os.getenv("AZURE_MODEL_DEPLOYMENT"),
    temperature=0,
    streaming=True,
)
prompt = ChatPromptTemplate.from_messages(
    [
        (
            "system",
            "You're a very knowledgeable historian who provides accurate and eloquent answers to historical questions. \
            Always start your answer by saying the User ID, and then answer the question.",
        ),
        MessagesPlaceholder(variable_name="chat_history"),
        ("human", "I am {user_id}, {question}"),
    ]
)

# Define the chain
# Need first step to retrieve the user ID
chain = ({
  "user_id": get_user,
  "chat_history": history_pass,
  "question": RunnablePassthrough(),
}) | prompt | model


# Use to setup in langserve
# Get user ID from request, where it was set by the VerifyToken dependency
def _per_request_config_modifier(
    config: Dict[str, Any], request: Request
) -> Dict[str, Any]:
    """Update the config"""
    config = config.copy()
    configurable = config.get("configurable", {})
    user_id = request.state.user
    configurable["user_id"] = user_id
    config["configurable"] = configurable
    return config

I hope it can be useful

[Chengdyc]

Thanks! looks like the trick is to add a step in the LCEL chain to extract the user_id from configurable into the chain payload.