Privacy

[DutchPete]

How's the privacy aspect? OpenAI gives you an API key that belongs to OpenAI, and Smart Connections processes your notes. Can OpenAI read your notes, do they get stored?

What does processing by Smart Connections involve?

[brianpetro]

@DutchPete, thanks for reaching out about this. Let's see if I can help clarify this relatively common question for you and others that may see this.

To begin, there is only one external API provider that Smart Connections uses: OpenAI's API. This contrasts with similar plugins that rely on other third-party servers for processing.

While I wish that we could all run AI locally on our computers, which would be ideal from a privacy perspective, instead, relying on only one external API greatly reduces the potential for possible privacy issues. 

Additionally, your embeddings/vectors are stored locally in a file on your computer. Once again, this contrasts with some other plugins that use third-party services for embedding/vector storage.

But, since the plugin does use OpenAI for the necessary AI components, which at this time there are no comparable alternatives, you will have to review their Terms of Service and other documentation to understand better what to expect when using their services. 

Lastly, to help with occasions where there are only some sensitive files/folders, Smart Connections provides exclusion settings that you can use to prevent certain files/folders from ever being processed by OpenAI.

I hope that clarifies how Smart Connections works and its scope.

Thanks for reaching out about this,
Brian🌴

[pinuke]

Well, to be fair, mac books don't really have an impressive amount of VRAM, and most of the time the limiting factor on AI performance is going to be VRAM.

If this was going to be implemented, it would definitely need to rely on consumer-end mining rigs and definitely not traditional consumer-end PCs lol.

[spolisar]

Hugging Face does provide access to models for download locally. However, I haven't seen any evidence that this is worthwhile. Any attempts I've seen at running such models locally have been woefully underwhelming in their abilities and speed.

I think you're a bit off in dismissing locally hosted options. Sure, nothing else is on the same level as ChatGPT for cases like conversation, but there is genuine competition in embedding performance. If you want to see how different models stack up against each other in embedding, take a look at MTEB. There are models that outperform the GPT embedding model (text-embedding-ada-002) that can be run on consumer-end PCs. For some of them, a dedicated GPU might be necessary, but there are others that can come close on average and even beat GPT embedding on some tasks.

[brianpetro]

Hey @spolisar , welcome to the discussion!

And thank you for the helpful resource with the latest benchmarks.

I'm in no way trying to dismiss the importance of locally hosted options. Undoubtedly, they are the future.

I'd be happy to integrate as soon as:

1. the benchmarks check out on reasonable consumer hardware, and
2. the model can be installed relatively simply

It would be helpful if you could keep us updated on the potential local models as they begin to meet the above criteria.

Thanks for the insights!
Brian🌴

[spolisar]

Thanks for the welcome.

What do you think is reasonable consumer hardware? Are dedicated GPUs on the table, or just CPUs? It's obvious that an A100 isn't reasonable for a consumer, but where do you draw the line?

Would you count something like a docker image as a relatively simple installation?

[pinuke]

@spolisar reasonable would be anything the average Obsidian user has. You have to remember, Obsidian is equal parts developer community to writing and academics. Unless there is an interest in gaming/dev, the average college student is probably not going to have more than a consumer-grade laptop (maybe not even more than a chromebook)

[DutchPete]

Thanks for this detailed answer, Brian. I'll check out OpenAI's ToS.

[pinuke]

Thanks for the contribution and its good that you show that you want to learn more about the plugin before using it!

Just make sure to put replies to an answer as a reply and not answer. Github discussions work slightly differently than issues!