Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inconsistent behavior when changing models #913

Open
izosozi opened this issue Dec 5, 2024 · 2 comments
Open

Inconsistent behavior when changing models #913

izosozi opened this issue Dec 5, 2024 · 2 comments
Labels
enhancement New feature or request

Comments

@izosozi
Copy link

izosozi commented Dec 5, 2024

System:

Smart Connections 2.3.40

Obsidian 1.7.7

MacOS 15.1.1 & Windows 11 (reproduced on both, possibly iOS 18 also but had trouble confirming)

Steps to reproduce:

  1. Uninstalled Smart Connections and deleted .smart-env
  2. Installed Smart Connections
  3. After sources are imported, the "Making Smart Connections" notification shows that bge-micro-v2 (the default model) is being used for embedding
  4. In Smart Connections settings > Smart Sources > Embedding Model, switched to BGE-small

Expected:
The previous notification box showing embedding progress with bge-micro-v2 should disappear and a new box showing embedding progress with BGE-small should appear OR there should be an error message to indicate that BGE-small was not loaded

Actual:
Nothing visibly changes. The bge-micro-v2 box also never updates, and continues showing a progress indicator 1/### indefinitely. CPU/GPU usage indicate no embedding is happening at all.

  1. Relaunched Obsidian

Expected:
The new embedding progress notification should show BGE-small and begin embedding OR should revert to bge-micro-v2 if there was an error with BGE-small and provide an error message here or earlier

Actual:
The notification still shows bge-micro-v2. The progress indicator begins from some number greater than 1, indicating some embedding was done prior to relaunching Obsidian, probably in the few seconds before switching the model in the settings. This time, the progress indicator does change, and CPU/GPU shows higher usage as expected.

  1. Checked the Embedding Model setting again

Expected:
Unclear, since I'm not certain which model is actually being used

Actual:
"BGE-small" is displayed, but I'm not sure if this means that model has been downloaded or is being actively used for embeddings, or if it simply means that the setting selection has been stored.

Additional notes

  • I had a similar/the same issue when switching to and from Nomic.
  • It seemed that sometimes the embedding did continue in the background based on CPU/GPU usage
@brianpetro
Copy link
Owner

Thanks for the detailed report.

The notifications can definitely be handled better, especially wrt clean-up.

Until then, after changing models, it might be helpful to restart Obsidian to clean things up. If the correct model isn't indicated after a restart then there might be a bug that needs addressing 🌴

@brianpetro brianpetro added the enhancement New feature or request label Dec 5, 2024
@3-ark
Copy link

3-ark commented Dec 6, 2024

I find this problem still exists in newer version (41) for 'transformer'. And if we can add any embedding models now? I didn't find the option in setting. I tried to change in the main.js directly, but it doesn't work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants