Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NIM mode requires NVIDIA_API_KEY #7

Closed
mattf opened this issue Mar 22, 2024 · 1 comment
Closed

NIM mode requires NVIDIA_API_KEY #7

mattf opened this issue Mar 22, 2024 · 1 comment

Comments

@mattf
Copy link
Collaborator

mattf commented Mar 22, 2024

environment

➜ pip freeze | grep langchain-nvidia-ai-endpoints
langchain-nvidia-ai-endpoints==0.0.4

issue

➜ python
Python 3.12.1 (main, Dec 18 2023, 00:00:00) [GCC 13.2.1 20231205 (Red Hat 13.2.1-6)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from langchain_nvidia_ai_endpoits import ChatNVIDIA
>>> nim = ChatNVIDIA().mode("nim", base_url="http://localhost:9999")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/tmp/issue/.venv/lib/python3.12/site-packages/langchain_core/load/serializable.py", line 120, in __init__
    super().__init__(**kwargs)
  File "/tmp/issue/.venv/lib/python3.12/site-packages/pydantic/v1/main.py", line 341, in __init__
    raise validation_error
pydantic.v1.error_wrappers.ValidationError: 1 validation error for ChatNVIDIA
__root__ -> __root__
  argument of type 'NoneType' is not iterable (type=type_error)

expectation

an endpoint setup to work with a local NIM at http://localhost:9999

workaround

>>> import os
>>> os.environ.update({"NVIDIA_API_KEY": "bogus"})
>>> nim = ChatNVIDIA().mode("nim", base_url="http://localhost:9999")
>>> nim
ChatNVIDIA(client=NVEModel(base_url='http://localhost:9999', get_session_fn=<class 'requests.sessions.Session'>, get_asession_fn=<class 'aiohttp.client.ClientSession'>, endpoints={'infer': '{base_url}/chat/completions', 'status': '{base_url}/pexec/status/{request_id}', 'models': '{base_url}/models'}, api_key=SecretStr('**********'), is_staging=False, timeout=60, interval=0.02, last_inputs={}, last_response=None, payload_fn=<function NVEModel.<lambda> at 0x7f4b9b411300>, headers_tmpl={'call': {'Accept': 'application/json', 'Authorization': 'Bearer {api_key}', 'User-Agent': 'langchain-nvidia-ai-endpoints'}, 'stream': {'Accept': 'text/event-stream', 'content-type': 'application/json', 'Authorization': 'Bearer {api_key}', 'User-Agent': 'langchain-nvidia-ai-endpoints'}}, stagify=functools.partial(<function NVEModel._stagify at 0x7f4b9b4120c0>, is_staging=False)), curr_mode='open')
@mattf
Copy link
Collaborator Author

mattf commented Apr 1, 2024

fixed by #13

@mattf mattf closed this as completed Apr 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant