Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: aila categoriser feature with chat ID and user ID #12

Merged
merged 3 commits into from
Aug 28, 2024

Conversation

stefl
Copy link
Contributor

@stefl stefl commented Aug 28, 2024

Description

  • Removes a pending TODO about moving the initial lesson categorisation step into a feature
  • When the user starts a chat, and there is no lesson plan yet set, we scan the initial set of messages until we are able to categorise the input as describing what lesson the user wants to create
  • In this step we want to make sure we have the chat ID and the user ID so that we can track LLM token usage
  • We also want to be able to mock this step out so that we don't hit OpenAI in tests
  • This introduces an AilaCategorisation feature which is mockable with an example mock service and an example test

Copy link

vercel bot commented Aug 28, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
oak-ai-lesson-assistant ✅ Ready (Inspect) Visit Preview 💬 Add feedback Aug 28, 2024 9:36am

@stefl stefl changed the title Aila Categoriser feature with Chat ID and User ID feat: Aila Categoriser feature with Chat ID and User ID Aug 28, 2024
Copy link

github-actions bot commented Aug 28, 2024

Playwright e2e tests

Job summary

Download report

To view traces locally, unzip the report and run:

npx playwright show-report ~/Downloads/playwright-report

@simonrose121 simonrose121 changed the title feat: Aila Categoriser feature with Chat ID and User ID feat: aila categoriser feature with chat ID and user ID Aug 28, 2024
@@ -65,7 +65,7 @@ export class LessonPlans {
private _prisma: PrismaClientWithAccelerate;
constructor(private readonly prisma: PrismaClientWithAccelerate) {
this._prisma = prisma;
this._rag = new RAG(this._prisma);
this._rag = new RAG(this._prisma, { chatId: "none" });
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Models are mainly used for ingest, so we don't have a chatId or a userId

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wondered about having a special string for this. We might want to do same for userId at some point? Perhaps not

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes we might want to also store something for the user ID in non-user calls. I'm expecting we look through the database at some point to see where we are missing what we need, and that "none" should mean we can see those rows. I had to pass a string here because the underlying method now throws if chatId is not supplied.

Copy link

@@ -285,4 +286,37 @@ describe("Aila", () => {
expect(ailaInstance.lesson.plan.title).toBe(newTitle);
}, 20000);
});

describe("categorisation", () => {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just adding one example test here to show how we can pass in the mock categoriser during Aila initialisation

@@ -23,6 +23,7 @@ export interface AilaLessonService {
readonly hasSetInitialState: boolean;
applyPatches(patches: string): void;
initialise(plan: LooseLessonPlan): void;
setUpInitialLessonPlan(messages: Message[]): Promise<void>;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moves the initial lesson plan setup to the Aila Lesson which makes more sense - resolves the TODO in the Aila class

Copy link
Collaborator

@mantagen mantagen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, couple queries

this._prisma = options.prisma ?? globalPrisma;

this._lesson = new AilaLesson({
aila: this,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have seen some fiddly bugs arise from passing aila: this as a prop, as the types aren't necessarily accurate (due to a quasi race condition on instantiation)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not saying don't do this, just highlighting because I think you were away when it was discovered

Copy link
Contributor Author

@stefl stefl Aug 28, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay that's interesting! I see. So if you've not yet set some property on the instance before you pass it, then it could cause a problem.

@@ -65,7 +65,7 @@ export class LessonPlans {
private _prisma: PrismaClientWithAccelerate;
constructor(private readonly prisma: PrismaClientWithAccelerate) {
this._prisma = prisma;
this._rag = new RAG(this._prisma);
this._rag = new RAG(this._prisma, { chatId: "none" });
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wondered about having a special string for this. We might want to do same for userId at some point? Perhaps not

@stefl stefl merged commit 6b0839c into main Aug 28, 2024
10 checks passed
@stefl stefl deleted the aila_categoriser branch August 28, 2024 19:19
@oak-machine-user
Copy link
Collaborator

🎉 This PR is included in version 1.3.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants