From dd536639cc511701f56964ffb72e87bc4c5e0154 Mon Sep 17 00:00:00 2001 From: Karthik Kalyanaraman <105607645+karthikscale3@users.noreply.github.com> Date: Fri, 28 Jun 2024 13:15:44 -0700 Subject: [PATCH] Release 1.5.7 (#159) * Pagination bug * Bug fix * chore: add docker cmd * Compatibility fixes for SDK version 2.0.0 (#69) * Pagination bug * Bug fix * Fix for schema changes * Render tool calling * Support for Langgraph, Qdrant & Groq (#73) * Pagination bug * Bug fix * Add langgraph support * QDrant support * Add Groq support * update README * update README * feat: optimise docker image for self host setup * adding api access to traces endpoint * clean up * refactor * feat: add clickhouse db create on app start (#79) * docs: add railway deploy, fix sdk badges (#81) * untrack .env * Revert "untrack .env" This reverts commit 4551d7eb33a22b4ba94be8eb1e20d620c6e0a567. * Playground and Prompt Management (#83) * Pagination bug * Bug fix * Playground - basic implementation * Playground - streaming and nonstreaming * Move playground inside project * API key flow * Api key * Playground refactor * Add chat hookup * anthropic streaming support * Bug fixes to openai playground * Anthropic bugfixes * Anthropic bugfix * Cohere first iteration * Cohere role fixes * Cohere api fix * Parallel running * Playground cost calculation non streaming * playground - streaming token calculation * latency and cost * Support for Groq * Add model name * Prompt management views * Remove current promptset flow * Prompt management API hooks * Prompt registry final * Playground bugfixes * Bug fix playground * Rearrange project nav * Fix playground * Fix prompts * Bugfixes * Minor fix * Prompt versioning bugfix * Bugfix * fix: clickhouse table find queries (#82) * Fix to surface multiple LLM requests inside LLM View (#84) * Pagination bug * Bug fix * Fix for surfacing multiple LLM requests in LLMView * Minor bugfixes (#86) * Pagination bug * Bug fix * Bugfixes * api to fetch promptset with prompt filters * bug fixes * fix invalid redirect * fix invalid status code * Project Switcher (#90) * Pagination bug * Bug fix * Project switcher * Feat: dataset download (#60) * API: download dataset * API: Download dataset * updated download-dataset api * Updated: download_dataset api * Updated download dataset API * Updated Download API: changed Response to Next Response, add a condition to ensure max page size is 500 * updated the download-dataset API: fixed the format and removed redundant lines of code * Updated download_daatset API: file name and removed 'id' param * Added the Download dataset button. * Merged developemnt into my branch * Updated button size * Fixes --------- Co-authored-by: Karthik Kalyanaraman * Update prompt registry with instructions to fetch prompts (#91) * Pagination bug * Bug fix * Update prompt registry * Minor bugfix (#94) * Pagination bug * Bug fix * Minor bugfix * chore: update github repo badges * optimizing token count function * Add GPT4-O Pricing and Playground (#98) * Pagination bug * Bug fix * Add GPT4-O support * Add GPT4-O support * Update cost * Dylan/s3en 2234 add perplexity support to playground (#89) * adding perplexity to playground types * adding ui stuff:' * adding perplexity chat api * fixing perplexity model dropdown --------- Co-authored-by: Karthik Kalyanaraman * api changes * add api access to get api and fix all bugs * bug fix * bug fix * updating descriptions to optional * prio python * cleanup and fixes * more bug fixes * more fixes * remove console log * updating trace_service functions * add migration * add format function, updating from day to hour * adding dropwdown menu * updating query key * updating query keys v2 * clean up * fix bug * Minor bugfix (#102) * Pagination bug * Bug fix * Minor bugfix * fixes * Bug fixes to Evaluations (#111) * Pagination bug * Bug fix * Bug fixes to evals v2 * Minor fix * upgrading tiktoken, adding omni to pricing (#110) Co-authored-by: Karthik Kalyanaraman * Update package * Dataset bugfix (#113) * Pagination bug * Bug fix * Dataset bugfix * Minor linking fix (#114) * Pagination bug * Bug fix * Dataset bugfix * Minor fix * api bug fix (#116) * fixing dropdown color (#117) * Support for Image inputs (#119) * Pagination bug * Bug fix * Support image inputs * Bug fix Spans with status_code = "ERROR" (#120) * Pagination bug * Bug fix * Support image inputs * Bug fixes * Dylan/s3en 2265 save prompt and model settings directly from playground (#99) * bug fixes, adding save prompt button in playground * adding save prompt dialog * adding create registry dialog * working registry selection * Adding new prompt to exist registry * storing * button padding fix * refactor, adding combobox * increasing dropdwon width --------- Co-authored-by: Karthik Kalyanaraman * adding image display * adding property filtering (#123) * optimizing cost function * optimizing per cost per hour function * removing console statements * updating queryKey for datasets * init advanced filter dialog * adding operator combobox * updating ui * adding attribute type * bug fixes * cleanup * remove console * updating queryKeys * adding missing table checks * adding more checks * chore: update clickhouse docker for user management * style: remove blank space * updating save prompt button * Redesign of Manual Evaluations (#134) * Pagination bug * Bug fix * Schema changes for evals * Fix APIs for evals * Minor bugfixes * Bugfix * Evaluation changes * Manual eval simplification * Minor fix * Manual evaluations UX changes * Chart title fix * Disable experiments * fix * Dylan/s3en 2318 add the ability to import a conversation from the traces to (#133) * adding import conversation button * init import conversation dialog * adding trace rows * fixing scroll bug * adding import prompt to playground * fixes * Minor * Bugfixes --------- Co-authored-by: Karthik Kalyanaraman * storing * remove import for images * importing attribute stuff * fixing operations display * cleaning up advanced filters * cleanup * docs: update docker instructions * removing console, fixing button * add stars history (#139) * Dylan/s3en 2401 ability to filter by user for metrics and traces (#138) * adding user id combobox, api * fixing sql query * removing user filter on deselect{ * moving user combobox into filter dialog * adding filtering by user for token charts * bug fixes, more userId route, function updates * latency filtering * updating to alphanumeric * removing console, comment * updating default filter operation to and * adding clear all button, cleanup * Dylan/s3en 2350 aggregate metrics by models v2 (#141) * adding user id combobox, api * fixing sql query * removing user filter on deselect{ * moving user combobox into filter dialog * adding filtering by user for token charts * bug fixes, more userId route, function updates * latency filtering * updating to alphanumeric * removing console, comment * updating default filter operation to and * adding clear all button, cleanup * init commit model combobox * adding api, updating function, adding to metrics page * adding model to charts, apis, service * remove consoles * Automated Evaluations - Support for Inspect AI (#142) * Pagination bug * Bug fix * Experiments view * re-order * Comparison view * Experiment report * Experiments - hook up APIs * Experiments pagination * deduplicate paginated data * Fix loading and error states - experiments * Rename experiments and evaluations * Add loading spinner * Link evaluation * Link to evals * Fixes * Minor UI fixes * Bugfixes * DSPy support (#144) * Pagination bug * Bug fix * Bugfixes * DSPy support * Update README * Prompt registry and Playground integration (#146) * Pagination bug * Bug fix * Bugfixes * Playground enhancements * Fix * Playground improvment * Bug fix * Minor bug fixes (#148) * Pagination bug * Bug fix * Bugfixes * Bugfixes * feat: adding example code for starters * Bugfixes (#152) * Pagination bug * Bug fix * Bugfixes * crewai support * fix ollama bug * updating card link to traces * adding copy to clipboard for prompt id * Dylan/s3en 2415 expose project creation api for hivenetwork (#156) * adding api key auth for project, displaying team id * migrating, adding api key gen to teams page * removing comment * removing apiKeyHash from project return * Minorfixes --------- Co-authored-by: Karthik Kalyanaraman * Dylan/s3en 2393 update UI tracing tab to show the prompt version being used v2 (#154) * Adding prompt id to traces table * adding prompt combobox, api route, service function * adding prompt version to traces tab * Fix --------- Co-authored-by: Karthik Kalyanaraman --------- Co-authored-by: Darshit Suratwala Co-authored-by: darshit-s3 <119623510+darshit-s3@users.noreply.github.com> Co-authored-by: dylan Co-authored-by: dylanzuber-scale3 <116033320+dylanzuber-scale3@users.noreply.github.com> Co-authored-by: Rohit Kadhe Co-authored-by: Rohit Kadhe <113367036+rohit-kadhe@users.noreply.github.com> Co-authored-by: MayuriS24 <159064413+MayuriS24@users.noreply.github.com> Co-authored-by: yemiscale3 <111706151+yemiadej@users.noreply.github.com> --- .../[project_id]/prompts/[prompt_id]/page.tsx | 23 ++- app/(protected)/projects/page-client.tsx | 2 +- app/(protected)/settings/team/page-client.tsx | 89 +++++++++++- app/api/api-key/route.ts | 89 ++++++++++-- app/api/project/route.ts | 20 ++- app/api/prompt-ids/route.ts | 37 +++++ components/project/traces/trace-filter.tsx | 22 ++- components/project/traces/trace-row.tsx | 8 +- components/project/traces/traces.tsx | 4 +- components/shared/api-key.tsx | 25 +++- components/shared/prompt-combobox.tsx | 135 ++++++++++++++++++ lib/services/trace_service.ts | 23 +++ lib/utils.ts | 21 ++- .../migration.sql | 20 +++ prisma/schema.prisma | 1 + 15 files changed, 486 insertions(+), 33 deletions(-) create mode 100644 app/api/prompt-ids/route.ts create mode 100644 components/shared/prompt-combobox.tsx create mode 100644 prisma/migrations/20240626153855_add_api_key_hash_to_team/migration.sql diff --git a/app/(protected)/project/[project_id]/prompts/[prompt_id]/page.tsx b/app/(protected)/project/[project_id]/prompts/[prompt_id]/page.tsx index 1f70a944..a1d83bb9 100644 --- a/app/(protected)/project/[project_id]/prompts/[prompt_id]/page.tsx +++ b/app/(protected)/project/[project_id]/prompts/[prompt_id]/page.tsx @@ -10,7 +10,7 @@ import { Switch } from "@/components/ui/switch"; import { cn } from "@/lib/utils"; import { Prompt } from "@prisma/client"; import CodeEditor from "@uiw/react-textarea-code-editor"; -import { ChevronLeft } from "lucide-react"; +import { ChevronLeft, ClipboardIcon } from "lucide-react"; import { useParams, useRouter } from "next/navigation"; import { useState } from "react"; import { useQuery, useQueryClient } from "react-query"; @@ -196,9 +196,24 @@ export default function Page() {
-

- {promptsetId} -

+
+

{ + navigator.clipboard.writeText(promptsetId); + toast.success("Copied to clipboard"); + }} + className="flex-grow" + > + {promptsetId} +

+ { + navigator.clipboard.writeText(promptsetId); + toast.success("Copied to clipboard"); + }} + /> +
diff --git a/app/(protected)/projects/page-client.tsx b/app/(protected)/projects/page-client.tsx index 5c2afd8d..41d3e52b 100644 --- a/app/(protected)/projects/page-client.tsx +++ b/app/(protected)/projects/page-client.tsx @@ -178,7 +178,7 @@ function ProjectCard({ )}
- + diff --git a/app/(protected)/settings/team/page-client.tsx b/app/(protected)/settings/team/page-client.tsx index e788df6a..0e7618a9 100644 --- a/app/(protected)/settings/team/page-client.tsx +++ b/app/(protected)/settings/team/page-client.tsx @@ -11,6 +11,7 @@ import { import { Form, FormControl, + FormDescription, FormField, FormItem, FormLabel, @@ -18,14 +19,44 @@ import { } from "@/components/ui/form"; import { Input } from "@/components/ui/input"; import { zodResolver } from "@hookform/resolvers/zod"; +import { CodeIcon, RefreshCwIcon } from "lucide-react"; import { useState } from "react"; import { FieldValues, useForm } from "react-hook-form"; import { toast } from "sonner"; import { z } from "zod"; export default function TeamView({ user }: { user: any }) { - // const queryClient = useQueryClient(); + const [apiKey, setApiKey] = useState(); const [busy, setBusy] = useState(false); + + const handleApiKeyGenerated = (newApiKey: string) => { + setApiKey(newApiKey); + }; + + const generateApiKey = async () => { + try { + setBusy(true); + const response = await fetch(`/api/api-key?team_id=${user.Team.id}`, { + method: "POST", + headers: { + "Content-Type": "application/json", + }, + }); + const result = await response.json(); + setApiKey(result.data.apiKey); + handleApiKeyGenerated(result.data.apiKey); + toast("Copy your API Key!", { + description: "Please copy your API key. It will not be shown again.", + }); + } catch (error: any) { + toast("Error generating API Key!", { + description: `There was an error generating your API Key: ${error.message}`, + }); + } finally { + setBusy(false); + } + }; + const NameFormSchema = z.object({ name: z .string() @@ -89,7 +120,63 @@ export default function TeamView({ user }: { user: any }) { )} /> + Team ID +
+

{ + navigator.clipboard.writeText(user.Team.id); + toast.success("Copied to clipboard"); + }} + className="text-sm select-all selection:bg-blue-200" + > + {user.Team.id} +

+
+ API Key + + {user.Team.apiKeyHash && !apiKey + ? "Note: If you regenerate the API key, the old key will be replaced." + : "Note: Click to copy this API key as it will NOT be shown again."} + + {apiKey && ( +
+

{ + navigator.clipboard.writeText(apiKey); + toast.success("Copied to clipboard"); + }} + className="text-sm select-all selection:bg-blue-200" + > + {apiKey} +

+
+ )} + {!apiKey && ( + + )} {(selectedFilters.length > 0 || advancedFilters.length > 0 || - selectedUserId !== "") && ( + selectedUserId !== "" || + selectedPromptId !== "") && ( + + + + + No attribute found. + + {promptIds.map((id: string) => ( + { + setSelectedPromptIdState( + currentValue === selectedPromptId ? "" : currentValue + ); + setSelectedPrompt( + currentValue === selectedPromptId ? "" : currentValue + ); + handleSelectPrompt(currentValue); + setOpen(false); + }} + > + + {id} + + ))} + + + + + ); +} diff --git a/lib/services/trace_service.ts b/lib/services/trace_service.ts index e41fa6a0..25dde674 100644 --- a/lib/services/trace_service.ts +++ b/lib/services/trace_service.ts @@ -89,6 +89,7 @@ export interface ITraceService { ) => Promise; AddSpans: (spans: Span[], project_id: string) => Promise; GetUsersInProject: (project_id: string) => Promise; + GetPromptsInProject: (project_id: string) => Promise; GetModelsInProject: (project_id: string) => Promise; } @@ -122,6 +123,28 @@ export class TraceService implements ITraceService { } } + async GetPromptsInProject(project_id: string): Promise { + try { + const tableExists = await this.client.checkTableExists(project_id); + if (!tableExists) { + return []; + } + const query = sql + .select([ + `DISTINCT JSONExtractString(attributes, 'prompt_id') AS prompt_id`, + ]) + .from(project_id); + const result: any[] = await this.client.find(query); + return result + .map((row) => row.prompt_id) + .filter((prompt_id) => prompt_id !== ""); + } catch (error) { + throw new Error( + `An error occurred while trying to get the prompts ${error}` + ); + } + } + async GetModelsInProject(project_id: string): Promise { try { const tableExists = await this.client.checkTableExists(project_id); diff --git a/lib/utils.ts b/lib/utils.ts index 0135d712..7a3abb67 100644 --- a/lib/utils.ts +++ b/lib/utils.ts @@ -320,7 +320,10 @@ export function parseQueryString(url: string): Record { }); } -export async function authApiKey(api_key?: string): Promise { +export async function authApiKey( + api_key?: string, + team_auth?: boolean +): Promise { if (!api_key) { return NextResponse.json( { @@ -330,6 +333,22 @@ export async function authApiKey(api_key?: string): Promise { ); } + if (team_auth) { + const team = await prisma.team.findFirst({ + where: { + apiKeyHash: hashApiKey(api_key), + }, + }); + + if (!team) { + return NextResponse.json( + { error: "Unauthorized. Invalid API key" }, + { status: 401 } + ); + } else { + return NextResponse.json({ data: { team: team } }, { status: 200 }); + } + } const project = await prisma.project.findFirst({ where: { apiKeyHash: hashApiKey(api_key), diff --git a/prisma/migrations/20240626153855_add_api_key_hash_to_team/migration.sql b/prisma/migrations/20240626153855_add_api_key_hash_to_team/migration.sql new file mode 100644 index 00000000..ce9848e3 --- /dev/null +++ b/prisma/migrations/20240626153855_add_api_key_hash_to_team/migration.sql @@ -0,0 +1,20 @@ +/* + Warnings: + + - You are about to drop the column `expectedOutput` on the `Data` table. All the data in the column will be lost. + - You are about to drop the column `runId` on the `Data` table. All the data in the column will be lost. + - You are about to drop the column `name` on the `Run` table. All the data in the column will be lost. + - A unique constraint covering the columns `[apiKeyHash]` on the table `Team` will be added. If there are existing duplicate values, this will fail. + - Added the required column `projectId` to the `Run` table without a default value. This is not possible if the table is not empty. + - Added the required column `runId` to the `Run` table without a default value. This is not possible if the table is not empty. + - Added the required column `taskId` to the `Run` table without a default value. This is not possible if the table is not empty. + - Added the required column `updatedAt` to the `Run` table without a default value. This is not possible if the table is not empty. + +*/ + +-- AlterTable +ALTER TABLE "Team" ADD COLUMN "apiKeyHash" TEXT; + +-- CreateIndex +CREATE UNIQUE INDEX "Team_apiKeyHash_key" ON "Team"("apiKeyHash"); + diff --git a/prisma/schema.prisma b/prisma/schema.prisma index 89585f57..50b9d77c 100644 --- a/prisma/schema.prisma +++ b/prisma/schema.prisma @@ -172,6 +172,7 @@ model Team { users User[] status String? @default("active") TeamInvite TeamInvite[] + apiKeyHash String? @unique projects Project[] createdAt DateTime @default(now()) updatedAt DateTime @updatedAt