RAG configs

[githubdebugger]

Here is my setup:

1. Docker based local installation.
2. Added OpenAI keys in .env, both for OPENAI_API_KEY and RAG_OPENAI_API_KEY (redundant, but, still added, as I am using main model as DeepSeek)
   uncommented
   EMBEDDINGS_PROVIDER=openai
   EMBEDDINGS_MODEL=text-embedding-3-small
3. Added Deepseek details in librechat.yaml and checked for volume binding in docker-compose.yml so it would pick up librechat.yaml file
4. Also, updated binding in docker-compose.override.yml file as well

This is the setup and all the containers come up fine without any issues and I can get response from all OpenAI models and DeepSeek models.

Now, coming to file upload and embeddings.

Lobechat and embeddings:

I have been testing lobechat in parallel with Librechat for file uploads/embeddings with similar setup, OpenAI/DeepSeek.

When it comes to lobechat, I tried with a 5MB PDF file and upload was quick, chunking + embedding everything finished in less than 15-20 seconds. My system CPU barely went up in this case.

Librechat and embeddings (and issues and question):

I tried the exact same PDF file with librechat:

1. While the upload was quick, I think chunking and emedding took more than 10-12 mins, with a python process constantly at 20% for that entire period. I am not sure why this was the case with librechat.
2. When I continued in the same chat (where I uploaded the file), librechat could give proper context to the LLM to response correctly, everything was great (except 1).
3. Then I tried to delete the file from manage files, it would say that it deleted the file, but, on refresh I would see the files back under manage files
4. As it would show the files are still in the repo/library, if I add them to the chat context, it would show that it got added, but except the file name LLM's had no other data to talk about.
5. I tried bringing down the containers and bringing them up, but 3 and 4 repeatedly happened.

I tried a few more PDF's with both lobechat and librechat and steps 1-5 happened repeatedly. Lobechat would take few seconds, whereas librechat would take 10-15 mins per file. And the chat context of the PDF was available only in the chat when it was initially added. Anytime I referenced the files in another chat, LLM's had no clue about the details of the files.

I have used librechat previously (until the first time RAG was introduced, I remember file uploads was not this slow and I could add files to any number of chat contexts). Ya, IIRC we had to add config in rag.yml file back then? I am not sure what I am missing now?

Could someone please help me with the configs. Am I doing it correctly? Seems like I am missing something.

Thanks!

[githubdebugger]

Today I tried this on another PC, the outcome has been the same.

Also, the file once added if I create a new chat or if I bring down the container and back up again, if I attach the same file again, the LLM do not get the context attached at all, even though you can see the file has been attached in the conversation.

Could someone please help me in this, if I am missing anything?

Thanks!

[ScarFX]

Starting a new chat and attaching the file again (through the right hand menu NOT re uploading it on attach file button on text bar) should add context without embedding a 2nd time.

[ScarFX]

Do you have env var PDF_EXTRACT_IMAGES=True in your rag_api container? It may explain the large processing time. It can make a big difference even with much smaller pdfs. Try setting it to False @githubdebugger

[githubdebugger]

Do you have env var PDF_EXTRACT_IMAGES=True in your rag_api container? It may explain the large processing time. It can make a big difference even with much smaller pdfs. Try setting it to False @githubdebugger

I do not have any settings other than the default for RAG (just the settings which I mentioned in the OP), I even gave a search for PDF_EXTRACT_IMAGES in the repo where I start the docker containers, I got no results for this. Am I missing something here?

This behavior - #4621 (comment)
Also, if you could tell me if this is the behaviour in Librechat for files added already, but if the same file is brought in a new chat context, it does not have any clue? Do I need to add the same file again (chunk and embed the file again? does not make sense actually, seems like there is something that I am missing).

[ScarFX]

Can you send an example of a pdf that is taking 10+ min long? Trying to rule out if it is the pdf or something else.

[githubdebugger]

Can you send an example of a pdf that is taking 10+ min long? Trying to rule out if it is the pdf or something else.

This was a book that I purchased. Let me try with some research papers from arXiv and see how it goes and if I still face it, I will share it.

Appreciate your help and response. Thanks!

[ScarFX]

What you are experiencing with deleted files still showing up in file manager in name only is that they are not "successfully" deleted. I would check your logs for rag_api delete http response should be 200 but if not then the file names and info don't get removed even if removed from database.

[githubdebugger]

What you are experiencing with deleted files still showing up in file manager in name only is that they are not "successfully" deleted. I would check your logs for rag_api delete http response should be 200 but if not then the file names and info don't get removed even if removed from database.

From the config wise seems like I am all okay, as its mostly default and just adding the API keys. I'll share the logs when I spin this up again.