Friend app should be able to chat with and have context from your gmail ($1000)

[kodjima33]

Is your feature request related to a problem? Please describe.
Users want to be able to chat with their emails. Gmail doesn't have native AI features and is not smart and doesn't do it well. We need to let people to connect to Gmail through a plugin and upload their emails into Friend app and that data should be used same way we currently use memories.

Describe the solution you'd like
You need to make a "Connect Gmail" plugin for Friend app, in which the user will be able to authorize their gmail app and export all the data and then import it into Friend app.

Mails should be imported properly and vectorized in the backend, when chatting the user should be able to ask questions about their email.

A new section of the app with your Gmails indexed, should be in there.

Question

• A simple Gmail Chat API search works? instead of indexing all this stuff? or why not?

Learn how to create plugins here

Additional context
This is a paid task. Reward is $1000 in cash. Simply link your PR with this task and we will check it asap. Check how Contributions and bounties work here
If no one is assigned to this task, then no one is working on it. We don't lock bounty tasks unless you've created PRs before.

[ologunB]

Can you kindly assign me to this challenge @kodjima33

[heidy00]

Hello @kodjima33,
I am working on it.
I have experience with it and flutter.
can you assign for me?
Thank you

[josancamon19]

I suggest #533 and this work together.

[josancamon19]

This is 90% not a flutter task.

@ologunB @heidy00 let me know if you are still interested.

[heidy00]

Hello,
Thank you for your suggestion.

[heidy00]

Hello @josancamon19 ,
please contact with me via telegram.
https://t.me/vsheidy
This is my Telegram.

[josancamon19]

let's chat on discord https://discord.gg/9puRjM2g
@josancamon19

[heidy00]

let's chat on discord https://discord.gg/9puRjM2g @josancamon19

I can't add your discord.
vsheidy000
it is my username

[ologunB]

@heidy00 u can work on it.

[Washyl]

Can you kindly assign me to this challenge @kodjima33

[samarsheikh001]

Please assign it to me, I am close to finish it.

[kodjima33]

@Washyl @samarsheikh001 @ologunB @heidy00 haven't seen any code so far - can someone please do a PR, we will check and if it's good, will assign the issue?

For now, the issue is still unassigned due to lack of code here

[beastoin]

🤔 should I...

[beastoin]

Question: A simple Gmail Chat API search works? instead of indexing all this stuff? or why not?

Answer: Simple Gmail Chat API search should works, cause Gmail is so generous on rate-limiting[1]. ver1 of this plugin should be use Gmail API to retrieve relevant emails ( messages in term of Gmail API ) whenever users ask, we don't need to index all of emails.

[1]: https://developers.google.com/gmail/api/reference/quota

[kodjima33]

@beastoin yeah i think for initial implementation we can just use chat API

But eventually we definitely will need to index emails in app's database to give more context for ai

[beastoin]

yes got it @kodjima33 , will draft a proposal soon.

[decipher2k]

@beastoin yeah i think for initial implementation we can just use chat API

But eventually we definitely will need to index emails in app's database to give more context for ai

keep in mind that some users do have thousands of mails in their inbox.

i'd suggest doing the mail handling serverside at the plugin server and searching mails with a non-ai based approach, fetching them regularily using IMAP. maybe it could be useful to fetch only the last 100 mails or so.

thus, the google quota won't be a problem, and there won't be many resources required.

[invarrow]

@kodjima33 @josancamon19 completed the plugin to chat with mail

On the implementation side of it,
🟢 supports gmail auth similar to notion-crm and webhooks to handle triggers
🟢 to support the chat with mail, found it best to have it trigger on chat messages instead of memory, so added chat-query as a capability for external integrations.
🟢 The webhook is hooked up with langchain's mail toolkit for querying for mails. It also supports sending mails which can also be added if required.

wanted to get feedback and info on how to submit and have this plugin added since it requires the webhook to be hosted aswell..
will be cleaning and making a pr shortly

[josancamon19]

Closing, not planned for now, will reopen later probably.