Excalidraw For AI Workflows

[VentureTactics]

One of the big challenges when using LLMs effectively is chaining prompt responses together in a pre-defined workflow, to get the response you need.

Excalidraw can bridge the gap by (a) making JSON webhooks easy to send and receive data, and (b) by creating a visual structure around AI workflows tied together with your Obsidian data, dataview queries, and frontmatter properties.

You could, for example, implement something like the Chat Stream plugin which attempts to solve some of this with JSON Canvas, albeit with a lot of limitations Excalidraw could overcome with the right effort. See my Chat Stream example below:

To speed things up, you may consider utilizing some open-source code from something like n8n, and baking it into the back end:
https://github.com/n8n-io/n8n

Key features might include the ability to:

• Use a node/file/text box as a prompt, including any ancestor nodes connected to it.
• This should include a node that's specifically for the System Prompt.
• Assign a role to a file as the User or Assistant
• Connect these together in a Thread with a UID or group
• Somehow use this information with Templater (or similar) to, when triggered, take outputs from previous steps (or some of their other context) and input them into a new file that gets created in the workflow.
• Send hooks, await the response, and return the response for the next step in the workflow.
• Utilize the OpenAI assistants API (or similar) for function calling workflows (https://platform.openai.com/docs/assistants/overview?context=with-streaming).