Journey to JARVIS: A Roadmap for Obsidian Smart Connections

[brianpetro]

Hello Smart Connections community!

I'm excited to share a high-level overview of my personal focus for this open-source project, which I call "Journey to JARVIS." My aim is to enhance Obsidian with a suite of powerful features.

Unfortunately, this means I won't have the bandwidth to personally build integrations for additional models, languages, and platforms. However, I wholeheartedly encourage the community to contribute to these aspects, as your involvement is crucial for the growth and success of this project.

Here's a quick glimpse into the core features I'm working on:

1. Smart Chat: Implement robust NLP for smooth conversational experiences.
2. Smart Context: Provide contextually relevant suggestions and adaptive learning.
3. Smart Commands: Intuitive text commands and keyboard shortcuts.
4. Smart Suggestions: Assist with scheduling and personalization.
5. Smart Notifications: Keep users organized with reminders and notifications.
6. Smart Agent: Automate tasks to simplify users' routines.
7. Smart Instruction: Offer accessible help and support resources.

I hope this roadmap sparks your enthusiasm and inspires you to join the journey!

Together, we can make Obsidian Smart Connections a game-changer in productivity and AI-driven tools.

Looking forward to your thoughts, contributions, and collaboration!

Brian 🌴

[RealRavens]

Awesome Brian! We're here with you on this whole journey. I hope the community really rallies behind this project. So much potential here.

[jackallard17]

This is very exciting! Had no idea that this plugin had such an ambitious vision.

[brianpetro]

@jackallard17 yes! It's a perfect storm. I started building the plugin to improve my own PKM system. But, the feedback from the community enabled me to see opportunities I would've missed otherwise. Plus, integration with Obsidian, my preferred "second brain," enables an unparalleled opportunity to integrate the context necessary for maximizing personal alignment with AI.

[neoromantic]

Very exciting and very much what we should expects from PKM in modern world.

Can I help anyhow (being a CTO for 20 years & developer)?

[brianpetro]

Hey @neoromantic , thanks for the question!

Ways that many people can help include enabling integration with external APIs, additional languages, local models and models from other cloud providers. Also documentation, instructions, and sharing useful workflows for those who aren't as savvy with the code.

General feedback has also been really helpful.

I'm sure I'm missing some. So don't hesitate to share your CTO insights.

Thanks,
Brian 🌴

[tardigrde]

I was looking for something like this, thus I am also willing to help. The points you listed look good to me. What would be VERY important to me is: local models, so I do not have to share my mixed (personal and professional) vault with OpenAI or Google.
See: e.g. https://github.com/imartinez/privateGPT.

Is Journey to JARVIS currently living in this dicussion?

[KuecukMimarBey]

I would love to have a local first, privat ai model, i am not a familiar with code or programming at all, but i will spread the message

[brianpetro]

Version 2 has a local embedding model. It does require a lot of resources, making it quite slow, but does significantly reduce the number of notes that are sent to OpenAI to only the segments of notes which are used as context during a Smart Chat.

[KuecukMimarBey]

so what kind of resources does a local ai will need ? i would be nice to know, i will purchase a laptop soon ~ 2 months

[brianpetro]

The local embedding model works on a 2018 windows laptop at ~600 tokens/sec and at ~1100 tokens/sec on a 2023 M2 Mac.

I don't have any data on local chat models.

And I don't have any recommendations for computers for local models, as a lot can change based on new software architecture developments. Ideally, a sufficiently capable model will be able to run on current hardware that would be considered average, but only time will tell.

🌴

[jwhco]

The local embedding model works on a 2018 windows laptop at ~600 tokens/sec and at ~1100 tokens/sec on a 2023 M2 Mac.

I'm shocked at how well the local models work on older computers. This is impressive. I'm on an end-of-life HP Omen with an out-of-date GPU. ~3,000 tokens/sec. Chat responses as fast as SAAS platforms.

[rodrigueswilson]

Can you adapt your app to use as a copilot inside Zotero? The user could chat with one article at a time, all articles inside a folder, or all articles in the library. The app could identify Smart Connections between articles. The app inside Obsidian and the app inside Zotero could communicate.

[brianpetro]

I'm not familiar with Zotero. Does it use markdown?

[rodrigueswilson]

Zotero is a top-rated app used by academics for reference purposes, and it stores papers, especially in PDF format. Many people use it to read their papers, underline, and write annotations. Then, they export notes in markdown to Obsidian. As a non-coder, your app has features that I am trying in Obsidian, and I would like to replicate it in Zotero to chat with the papers I have gathered.

[brianpetro]

@rodrigueswilson thanks for clarifying that for me!

While I'm not sure about what the Zotero plugin would require, I do think PDF support for Smart Connections would go a long way to achieving it.

There are some PDF-to-markdown options I've been planning to explore further, so I'll keep this in mind when I do.

Thanks for your feedback and advice on how I can improve Smart Connections!

Brian 🌴

[rodrigueswilson]

You are doing wonderful work; Smart Connections is already an excellent tool, congratulations!