Change OpenAI API key at Runtime, uses placeholder key instead of configurable field

[rarchitgupta]

I've tried to implement a ChatOpenAI runnable which receives the openai_api_key from the request headers

async def per_req_config_modifier(config: Dict, request: Request) -> Dict:
    """
    Modify the config for each request
    """
    user = await get_current_active_user_from_request(request)
    openai_api_key = ""
    if "x-api-key" in request.headers:
        openai_api_key = request.headers["x-api-key"]
    else:
        raise HTTPException(401, "No OpenAI API key provided")
    config["configurable"] = {}
    # Attention: Make sure that the user ID is over-ridden for each request.
    # We should not be accepting a user ID from the user in this case!
    config["configurable"]["user_id"] = user["uid"]
    config["configurable"]["openai_api_key"] = openai_api_key

    return config

I then use it to extend my chain, initializing the llm like so:

llm = ChatOpenAI(openai_api_key="placeholder_key").configurable_fields(
    openai_api_key=ConfigurableField(
        id="openai_api_key",
        name="OpenAI API Key",
        description=("API Key for OpenAI interactions"),
    ),
)

When I run the chain, I get the error:

openai.AuthenticationError: Error code: 401 - {'error': {'message': 'Incorrect API key provided: placehol***_key. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_api_key'}}

Is there a reason why the placeholder key is being picked up instead of the configurable key even though I have debugged the per_req_config_modifier to make sure that the config object indeed contains the correct API key?

Things I've referred to so far for this:

https://stackoverflow.com/questions/77857466/with-langchain-how-can-i-change-the-openai-api-key-at-runtime

https://github.com/langchain-ai/langserve/blob/main/examples/configurable_chain/server.py#L95-#L112

[FlorianOttenwaelder]

I’m encountering the same issue where the placeholder key is being used instead of the configurable key, even though the correct API key is present in the config object after debugging. Has anyone found a solution or workaround for this?