feat: rag new schema, standalone package #448
Conversation
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
Playwright test results
Details
Flaky testsNo persona › tests/modifiy-lesson.test.ts › Modify a lesson plan › Modify a lesson resource Skipped testsNo persona › tests/auth.test.ts › authenticate through Clerk UI |
packages/db/prisma/schema.prisma
Outdated
|
|
||
| id String @id @default(cuid()) | ||
| oakLessonId Int? @map("oak_lesson_id") | ||
| oakLessonSlug String @map("oak_lesson_slug") |
There was a problem hiding this comment.
Adds slug as a mandatory field
packages/db/prisma/schema.prisma
Outdated
| lessonPlan Json @map("lesson_plan") @db.JsonB | ||
| subjectSlug String @map("subject_slug") | ||
| keyStageSlug String @map("key_stage_slug") | ||
| isPublished Boolean @default(false) @map("is_published") |
There was a problem hiding this comment.
Adds isPublished field
| eyfs: "early-years-foundation-stage", | ||
| }; | ||
|
|
||
| export function parseKeyStage(maybeKeyStage: string): string { |
There was a problem hiding this comment.
Note to self - use parseKeyStage function from posthog event helpers
There was a problem hiding this comment.
Same for rag/parseKeyStage I imagine
There was a problem hiding this comment.
Looking good on the whole. There are some parts which could benefit from being extracted to functions, but it's not a big deal. For example when transforming data or making raw queries. This could:
- Make conditional / loop logic to be simpler to follow and simpler for typescript
- Make high level process clearer, as the lower level detail is tucked away and replaced with useful function names
- Make unit testing simpler, and make it simpler for future devs to update a single part
This relates to code style and isn't a blocker
| }; | ||
|
|
||
| export function parseKeyStage(maybeKeyStage: string): string { | ||
| maybeKeyStage = maybeKeyStage.toLowerCase().replace(/[^a-z0-9]/g, ""); |
There was a problem hiding this comment.
Sonar might complain here about reassigning an argument
| lessonPlan = CompletedLessonPlanSchema.parse( | ||
| JSON.parse(maybeLessonPlanString), | ||
| ); | ||
|
|
||
| lessonPlan.keyStage = parseKeyStage(lessonPlan.keyStage); |
There was a problem hiding this comment.
Nitpick: I personally have a preference for this style, but it's not important!
| lessonPlan = CompletedLessonPlanSchema.parse( | |
| JSON.parse(maybeLessonPlanString), | |
| ); | |
| lessonPlan.keyStage = parseKeyStage(lessonPlan.keyStage); | |
| const parsedLessonPlan = CompletedLessonPlanSchema.parse( | |
| JSON.parse(maybeLessonPlanString), | |
| ); | |
| lessonPlan = { | |
| ...parsedLessonPlan, | |
| keyStage: parseKeyStage(lessonPlan.keyStage) | |
| }; |
There was a problem hiding this comment.
Also, shouldn't let lessonPlan have a type?
| eyfs: "early-years-foundation-stage", | ||
| }; | ||
|
|
||
| export function parseKeyStage(maybeKeyStage: string): string { |
There was a problem hiding this comment.
Same for rag/parseKeyStage I imagine
| const ragLessonPlans: { | ||
| oakLessonId?: number; | ||
| oakLessonSlug: string; | ||
| ingestLessonId?: string; | ||
| subjectSlug: string; | ||
| keyStageSlug: string; | ||
| lessonPlan: LooseLessonPlan; | ||
| }[] = []; | ||
|
|
||
| for (const lesson of lessons) { | ||
| if (!lesson.lessonPlan) { | ||
| throw new IngestError("Lesson is missing lesson plan", { | ||
| ingestId, | ||
| lessonId: lesson.id, | ||
| }); | ||
| } | ||
|
|
||
| const lessonPlan = LessonPlanSchema.parse(lesson.lessonPlan.data); | ||
|
|
||
| ragLessonPlans.push({ | ||
| oakLessonId: lesson.oakLessonId, | ||
| oakLessonSlug: lesson.data.lessonSlug, | ||
| ingestLessonId: lesson.id, | ||
| subjectSlug: lesson.data.subjectSlug, | ||
| keyStageSlug: lesson.data.keyStageSlug, | ||
| lessonPlan, | ||
| }); | ||
| } |
There was a problem hiding this comment.
A functional approach for this (may or may not be better in your eyes!)
| const ragLessonPlans: { | |
| oakLessonId?: number; | |
| oakLessonSlug: string; | |
| ingestLessonId?: string; | |
| subjectSlug: string; | |
| keyStageSlug: string; | |
| lessonPlan: LooseLessonPlan; | |
| }[] = []; | |
| for (const lesson of lessons) { | |
| if (!lesson.lessonPlan) { | |
| throw new IngestError("Lesson is missing lesson plan", { | |
| ingestId, | |
| lessonId: lesson.id, | |
| }); | |
| } | |
| const lessonPlan = LessonPlanSchema.parse(lesson.lessonPlan.data); | |
| ragLessonPlans.push({ | |
| oakLessonId: lesson.oakLessonId, | |
| oakLessonSlug: lesson.data.lessonSlug, | |
| ingestLessonId: lesson.id, | |
| subjectSlug: lesson.data.subjectSlug, | |
| keyStageSlug: lesson.data.keyStageSlug, | |
| lessonPlan, | |
| }); | |
| } | |
| const ragLessonPlans = lessons.map(lesson => | |
| if (!lesson.lessonPlan) { | |
| throw new IngestError("Lesson is missing lesson plan", { | |
| ingestId, | |
| lessonId: lesson.id, | |
| }); | |
| } | |
| const lessonPlan = LessonPlanSchema.parse(lesson.lessonPlan.data); | |
| return ({ | |
| oakLessonId: lesson.oakLessonId, | |
| oakLessonSlug: lesson.data.lessonSlug, | |
| ingestLessonId: lesson.id, | |
| subjectSlug: lesson.data.subjectSlug, | |
| keyStageSlug: lesson.data.keyStageSlug, | |
| lessonPlan, | |
| }); | |
| }) |
| @@ -0,0 +1,97 @@ | |||
| /** | |||
There was a problem hiding this comment.
Should this file be index.test.ts?
|
Description
BEHIND FEATURE FLAG // not ready for review