More informative exceptions

[sduttanv]

The error log could be more informative for nemo embeddings.

In the code calling the nemo embeddings, I get the following error:

File "/home/anudeep/swastika/oran_chatbot/env/lib/python3.10/site-packages/langchain_core/vectorstores.py", line 550, in from_documents
return cls.from_texts(texts, embedding, metadatas=metadatas, **kwargs)
File "/home/anudeep/swastika/oran_chatbot/env/lib/python3.10/site-packages/langchain_community/vectorstores/faiss.py", line 930, in from_texts
embeddings = embedding.embed_documents(texts)
File "/home/anudeep/swastika/oran_chatbot/env/lib/python3.10/site-packages/langchain_community/embeddings/nemo.py", line 122, in embed_documents
return [self._embedding_func(text, input_type="passage") for text in documents]
File "/home/anudeep/swastika/oran_chatbot/env/lib/python3.10/site-packages/langchain_community/embeddings/nemo.py", line 122, in
return [self._embedding_func(text, input_type="passage") for text in documents]
File "/home/anudeep/swastika/oran_chatbot/env/lib/python3.10/site-packages/langchain_community/embeddings/nemo.py", line 109, in _embedding_func
embedding = response_json["data"][0]["embedding"]
KeyError: 'data'

The error was actually due to input_length>512, and only shows in the NREM docker logs as:

Failed to process the request(s) for model instance 'NV-Embed-QA_tokenizer_0_3', message: RequestValidationError: Input length 548 exceeds maximum allowed token size 512

So, it is very difficult to debug such cases. It'd be super helpful if the error messages are more informative. Thank you!!

[mattf]

this reproduces against the rest endpoint.

short input -

➜ curl --request POST \
  --url "https://ai.api.nvidia.com/v1/retrieval/nvidia/embeddings" \
  --header "Authorization: Bearer $API_KEY_REQUIRED_IF_EXECUTING_OUTSIDE_NGC" \
  --header "Accept: application/json" \
  --header "Content-Type: application/json" \
  --data "{\"input\": \"$(printf 'nvidia %.s' {1..1})\", \"input_type\": \"query\", \"model\": \"NV-Embed-QA\"}"
{"object":"list","data":[{"index":0,"embedding":[0.00847625732421875,-0.028900146484375,0.00458526611328125,
...
-0.0574951171875,0.01514434814453125,0.0201416015625],"object":"embedding"}],"model":"NV-Embed-QA","usage":{"prompt_tokens":0,"total_tokens":0}}

long input -

➜ curl --request POST \
  --url "https://ai.api.nvidia.com/v1/retrieval/nvidia/embeddings" \
  --header "Authorization: Bearer $API_KEY_REQUIRED_IF_EXECUTING_OUTSIDE_NGC" \
  --header "Accept: application/json" \
  --header "Content-Type: application/json" \
  --data "{\"input\": \"$(printf 'nvidia %.s' {1..256})\", \"input_type\": \"query\", \"model\": \"NV-Embed-QA\"}"
{"type":"urn:nvcf-worker-service:problem-details:bad-request","title":"Bad Request","status":400,"detail":"Inference error","instance":"/v2/nvcf/pexec/functions/09c64e32-2b65-4892-a285-2f585408d118","requestId":"5fa22ed3-2ac0-48cf-aeeb-155a7c4bac82"}

no differentiation from other input issues -

➜ curl --request POST \
  --url "https://ai.api.nvidia.com/v1/retrieval/nvidia/embeddings" \
  --header "Authorization: Bearer $API_KEY_REQUIRED_IF_EXECUTING_OUTSIDE_NGC" \
  --header "Accept: application/json" \
  --header "Content-Type: application/json" \
  --data "{\"input\": \"$(printf 'nvidia %.s' {1..1})\", \"input_type\": \"querry\", \"model\": \"NV-Embed-QA\"}"   
{"type":"urn:nvcf-worker-service:problem-details:bad-request","title":"Bad Request","status":400,"detail":"Inference error","instance":"/v2/nvcf/pexec/functions/09c64e32-2b65-4892-a285-2f585408d118","requestId":"255698da-ab7f-4c5f-955d-277a6d280889"}

➜ curl --request POST \
  --url "https://ai.api.nvidia.com/v1/retrieval/nvidia/embeddings" \
  --header "Authorization: Bearer $API_KEY_REQUIRED_IF_EXECUTING_OUTSIDE_NGC" \
  --header "Accept: application/json" \
  --header "Content-Type: application/json" \
  --data "{\"input\": \"$(printf 'nvidia %.s' {1..1})\", \"input_type\": \"query\", \"model\": \"NV-Embed-QA.\"}"   
{"type":"urn:nvcf-worker-service:problem-details:not-found","title":"Not Found","status":404,"detail":"Inference error","instance":"/v2/nvcf/pexec/functions/09c64e32-2b65-4892-a285-2f585408d118","requestId":"7a6aba3b-7bd1-46d4-945e-3c8e2f26706f"

➜ curl --request POST \
  --url "https://ai.api.nvidia.com/v1/retrieval/nvidia/embeddings" \
  --header "Authorization: Bearer $API_KEY_REQUIRED_IF_EXECUTING_OUTSIDE_NGC" \
  --header "Accept: application/json" \
  --header "Content-Type: application/json" \
  --data "{\"input\": \"$(printf 'nvidia %.s' {1..1})\", \"inputtype\": \"query\", \"model\": \"NV-Embed-QA\"}" 
{"type":"urn:nvcf-worker-service:problem-details:bad-request","title":"Bad Request","status":400,"detail":"Inference error","instance":"/v2/nvcf/pexec/functions/09c64e32-2b65-4892-a285-2f585408d118","requestId":"43a257da-9f44-4d8a-b3fe-c62e5488932f"}

➜ curl --request POST \
  --url "https://ai.api.nvidia.com/v1/retrieval/nvidia/embeddings" \
  --header "Authorization: Bearer $API_KEY_REQUIRED_IF_EXECUTING_OUTSIDE_NGC" \
  --header "Accept: application/json" \
  --header "Content-Type: application/json" \
  --data "{\"inputs\": \"$(printf 'nvidia %.s' {1..1})\", \"input_type\": \"query\", \"model\": \"NV-Embed-QA\"}"
{"type":"urn:nvcf-worker-service:problem-details:bad-request","title":"Bad Request","status":400,"detail":"Inference error","instance":"/v2/nvcf/pexec/functions/09c64e32-2b65-4892-a285-2f585408d118","requestId":"a0dc133c-8f03-473f-95ed-29a05e55cba9"}