Feature: StarSearch thread history

[jpmcb]

StarSearch threads

Soon, we'll be bringing "threads" to StarSearch.

A thread is a mechanism for StarSearch to understand the chat history, have abit of "memory", and have many message threads that a user can go back to. In this way, users can have many persistent chats that are not simple "one-shots". They can go back, continue a conversation, and save their history.

This is still very early, in progress work in the API, but we want to get a head start on implementing this now so we can quickly integrate it and get it in front of real users. In short, the mental model you can think of is sort of how ChatGPT works: it has individual message threads, a single user has many threads, and you can always go back to individual messages for their history or to continue the conversation.

Here's a very rough idea of what this might look like implemented in the app:

but it's dealers choice at this point since we'll want @open-sauced/design to come in and give this a final, pristine design.

What will the routes look like?

⚠️ ⚠️ ⚠️ Note - all these routes and the data they carry are subject to change: work closely with @jpmcb for any relevant updates / information!!!!

GET /v2/star-search

Gets paginated rows for the authenticated user of all their threads:

{
  "data": [
    {
      // unique ID of the thread
      "id": "abc-123",

      // timestamp metadata
      "created_at": "2024-02-20T20:08:20.209Z",
      "updated_at": "2024-05-22T13:44:08.452Z",
      "deleted_at": null,
      
      // AI generated title of the thread
      "title": "People who know rust and tailwind"
    },
    {
      "id": "xyz-987",
      "created_at": "2024-02-20T20:08:20.209Z",
      "updated_at": "2024-05-22T13:44:08.452Z",
      "deleted_at": null,
      "title": "New releases of kubernetes/kubernetes"
    }
  ],
  "meta": {
    "page": 1,
    "limit": 10,
    "itemCount": 2,
    "pageCount": 1,
    "hasPreviousPage": false,
    "hasNextPage": false
  }
}

GET /v2/star-search/<uuid>

Gets an individual StarSearch thread by its unique ID and gathers all its message history for the authenticated user.

{
  "thread": {
    "id": "abc-123",
    "created_at": "2024-02-20T20:08:20.209Z",
    "updated_at": "2024-05-22T13:44:08.452Z",
    "deleted_at": null,
    "title": "People who know rust and tailwind"
  },
  "history": [
    {
      "id": "tre-678",
      "type": "user_input",
      "message": "Who are people that know tailwind and rust?",
      "is_error": false,
      "error": null,
      "actor": "user"
    },
    {
      "id": "def-456",
      "type": "function_call",

      // the raw SSE payload that would have previously been sent for the function call

      "message": "{ ... }",
      "is_error": false,
      "error": null,
      "actor": "starsearch"
    },
    {
      "id": "zyx-987",
      "type": "content",

      // the raw FINAL sse payload that would have previously been sent for the resulting message.
      // this would have the entire history (nested within the parsable json) like we get currently
      // within the star search server sent events.

      "message": "{ ... }",
      "is_error": false,
      "error": null,
      "actor": "starsearch"
    }
  ]
}

POST /v2/star-search/new

This authenticated route would return a UUID for the new chat that the client can then display.

{
  "thread": {
    "id": "abc-123",
    "created_at": "2024-02-20T20:08:20.209Z",
    "updated_at": "2024-05-22T13:44:08.452Z",
    "deleted_at": null,
    "title": ""
  },

  // note: this looks very similar to the "GET" for a star search thread with a UUID
  // but returns no history and no title
  "history: []
}

POST /v2/star-search/<uuid>/stream

This authenticated route would return the SSE json payloads as they exist today for StarSearch.

DELETE /v2/star-search/<uuid>

Deletes a star search thread for the authenticated user by the unique ID of the thread.

---

(for the OpenSauced core team, please refer to my full, internal proposal for reference to how this will actually work in the backend and further details).

Tasks:

• Feature: Update API endpoints in current StarSearch to prepare for Thread History #3551

[github-actions]

Thanks for the issue, our team will look into it as soon as possible! If you would like to work on this issue, please wait for us to decide if it's ready. The issue will be ready to work on once we remove the "needs triage" label.

To claim an issue that does not have the "needs triage" label, please leave a comment that says ".take". If you have any questions, please reach out to us on Discord or follow up on the issue itself.

For full info on how to contribute, please check out our contributors guide.

[jpmcb]

Threading work is ready to go into the API - once client changes are ready to go, we can coordinate getting that into beta.

The biggest changes, as laid out above, is the new POST v2/star-search/{id}/stream flow where the client will need the ID of a thread either through creating a new thread in POST v2/star-search or from an existing thread in GET v2/star-search