I wish langserve could do: RemoteRunnable usable as a seamless drop in replacement for the hosted runnable

[nat-n]

I'd like to be able to use a LangServe instance as a gateway for accessing different models, as part of a strategy to improve governance and convenience within our organization. The goal would be for consumers to be able to use a RemoteRunnable as a drop in replacement for a ChatOpenAI or similar, so all access to external dependencies goes through one microservice.

Essentially you can take part of a chain and put it in a separate process, and everything should continue to work the same.

This is sort of possible today but only in a very limited sense because the RemoteRunnable doesn't proxy config in way one would expect.

[eyurtsev]

Could you elaborate on what you mean by proxying config?

[nat-n]

I meant that whatever configurable_fields are defined on the Runnable passed to add_routes can be set via with_config on the client RemoteRunnable. I thought this wasn't possible, but after a quick test now it seems that it does work 😳

Nevermind, I'll close this discussion.