🌟 Join the Conversation: Share Your Ideas and Shape the Future 🌟

[atxcowboy]

Hey there, amazing GitHub community!

We are thrilled to invite you to join us in an exciting journey of collaboration and innovation. Your voice matters, and we want to hear your brilliant ideas, feature requests, and thoughts on our megasearch project. This Discussions page is where the magic happens, and we can't wait to see what you bring to the table!

Why should you participate? Let us tell you why:

1️⃣ Make an Impact: Your ideas have the power to shape the future of our project. We believe in the strength of collective intelligence, and by sharing your insights, you can directly influence the direction we take.

2️⃣ Foster Creativity: GitHub Discussions is an open playground for creativity and brainstorming. Explore new possibilities, suggest innovative features, and discuss enhancements that could revolutionize the project. Your imagination is the limit!

3️⃣ Connect with Like-minded Individuals: Engaging in discussions on GitHub brings together a diverse group of talented individuals who are passionate about the same goals. Connect with fellow developers, designers, and enthusiasts who share your interests and can offer fresh perspectives.

4️⃣ Learn and Grow: Discussions aren't just about sharing ideas; they're also a fantastic opportunity to learn from others. Discover different viewpoints, gain insights into new techniques, and expand your knowledge. It's a win-win situation!

5️⃣ Transparent and Open Environment: GitHub Discussions provides a transparent platform for collaboration. Everyone can see and contribute to the conversations, fostering an environment of openness and inclusivity. We value your input and want to ensure that your voice is heard.

Remember, this is a safe space for sharing and collaboration. We encourage you to be respectful, constructive, and supportive in all interactions. Let's foster an environment where everyone feels empowered to contribute and be heard.

Your ideas matter to us, and we can't wait to see what incredible insights you bring to the table. We're starting with a small idea, but in the vast ocean of ideas on Github, this will certainly become something bigger.

[atxcowboy]

I guess it's on me to start.

Here are a few ideas I have for the project:

1. Due to the current limitations of prompt tokens, injecting outside knowledge from search engines is rather limited. Most LLM only allow for 2K tokens at this time, so I think it would be nice to store new knowledge gained from search engines in a short term memory, i.e. a vector database. I am not sure if Megasearch really should have its own database.
   As there is typically no need to reinvent the wheel and we live in an interoperable world, we could hook into the vector database of another extension?

2. It takes time to connect to the search engine, fetch the data and then summarize it before the LLM actually starts its work. Right now the user is not informed that something is happening at all until the "... is typing" finally shows up on the screen.
   This needs to improve, i.e. by having a status message "... is searching" or maybe something even better.
   What's your thought on this?

3. The Megasearch could reach out further beyond just walking through some search results by opening the linked page and doing its own analysis of what it finds. I guess this would need to happen in the background (and in a parallel way) to not delay the user experience further?

4. The parameters need to be stored in a config file.