Release 1.4.5 (#121)

* Pagination bug * Bug fix * chore: add docker cmd * Compatibility fixes for SDK version 2.0.0 (#69) * Pagination bug * Bug fix * Fix for schema changes * Render tool calling * Support for Langgraph, Qdrant & Groq (#73) * Pagination bug * Bug fix * Add langgraph support * QDrant support * Add Groq support * update README * update README * feat: optimise docker image for self host setup * adding api access to traces endpoint * clean up * refactor * feat: add clickhouse db create on app start (#79) * docs: add railway deploy, fix sdk badges (#81) * untrack .env * Revert "untrack .env" This reverts commit 4551d7e. * Playground and Prompt Management (#83) * Pagination bug * Bug fix * Playground - basic implementation * Playground - streaming and nonstreaming * Move playground inside project * API key flow * Api key * Playground refactor * Add chat hookup * anthropic streaming support * Bug fixes to openai playground * Anthropic bugfixes * Anthropic bugfix * Cohere first iteration * Cohere role fixes * Cohere api fix * Parallel running * Playground cost calculation non streaming * playground - streaming token calculation * latency and cost * Support for Groq * Add model name * Prompt management views * Remove current promptset flow * Prompt management API hooks * Prompt registry final * Playground bugfixes * Bug fix playground * Rearrange project nav * Fix playground * Fix prompts * Bugfixes * Minor fix * Prompt versioning bugfix * Bugfix * fix: clickhouse table find queries (#82) * Fix to surface multiple LLM requests inside LLM View (#84) * Pagination bug * Bug fix * Fix for surfacing multiple LLM requests in LLMView * Minor bugfixes (#86) * Pagination bug * Bug fix * Bugfixes * api to fetch promptset with prompt filters * bug fixes * fix invalid redirect * fix invalid status code * Project Switcher (#90) * Pagination bug * Bug fix * Project switcher * Feat: dataset download (#60) * API: download dataset * API: Download dataset * updated download-dataset api * Updated: download_dataset api * Updated download dataset API * Updated Download API: changed Response to Next Response, add a condition to ensure max page size is 500 * updated the download-dataset API: fixed the format and removed redundant lines of code * Updated download_daatset API: file name and removed 'id' param * Added the Download dataset button. * Merged developemnt into my branch * Updated button size * Fixes --------- Co-authored-by: Karthik Kalyanaraman <[email protected]> * Update prompt registry with instructions to fetch prompts (#91) * Pagination bug * Bug fix * Update prompt registry * Minor bugfix (#94) * Pagination bug * Bug fix * Minor bugfix * chore: update github repo badges * optimizing token count function * Add GPT4-O Pricing and Playground (#98) * Pagination bug * Bug fix * Add GPT4-O support * Add GPT4-O support * Update cost * Dylan/s3en 2234 add perplexity support to playground (#89) * adding perplexity to playground types * adding ui stuff:' * adding perplexity chat api * fixing perplexity model dropdown --------- Co-authored-by: Karthik Kalyanaraman <[email protected]> * api changes * add api access to get api and fix all bugs * bug fix * bug fix * updating descriptions to optional * prio python * cleanup and fixes * more bug fixes * more fixes * remove console log * updating trace_service functions * add migration * add format function, updating from day to hour * adding dropwdown menu * updating query key * updating query keys v2 * clean up * fix bug * Minor bugfix (#102) * Pagination bug * Bug fix * Minor bugfix * fixes * Bug fixes to Evaluations (#111) * Pagination bug * Bug fix * Bug fixes to evals v2 * Minor fix * upgrading tiktoken, adding omni to pricing (#110) Co-authored-by: Karthik Kalyanaraman <[email protected]> * Update package * Dataset bugfix (#113) * Pagination bug * Bug fix * Dataset bugfix * Minor linking fix (#114) * Pagination bug * Bug fix * Dataset bugfix * Minor fix * api bug fix (#116) * fixing dropdown color (#117) * Support for Image inputs (#119) * Pagination bug * Bug fix * Support image inputs * Bug fix Spans with status_code = "ERROR" (#120) * Pagination bug * Bug fix * Support image inputs * Bug fixes * Dylan/s3en 2265 save prompt and model settings directly from playground (#99) * bug fixes, adding save prompt button in playground * adding save prompt dialog * adding create registry dialog * working registry selection * Adding new prompt to exist registry * storing * button padding fix * refactor, adding combobox * increasing dropdwon width --------- Co-authored-by: Karthik Kalyanaraman <[email protected]> --------- Co-authored-by: Darshit Suratwala <[email protected]> Co-authored-by: darshit-s3 <[email protected]> Co-authored-by: dylan <[email protected]> Co-authored-by: dylanzuber-scale3 <[email protected]> Co-authored-by: Rohit Kadhe <[email protected]> Co-authored-by: Rohit Kadhe <[email protected]> Co-authored-by: MayuriS24 <[email protected]>
Scale3-Labs · May 22, 2024 · 292c9ce · 292c9ce
1 parent 9d72d43
commit 292c9ce
Show file tree

Hide file tree

Showing 12 changed files with 683 additions and 289 deletions.
diff --git a/app/(protected)/project/[project_id]/prompts/[prompt_id]/page.tsx b/app/(protected)/project/[project_id]/prompts/[prompt_id]/page.tsx
@@ -20,6 +20,7 @@ export default function Page() {
   const router = useRouter();
   const [prompts, setPrompts] = useState<Prompt[]>([]);
   const [selectedPrompt, setSelectedPrompt] = useState<Prompt>();
+  const [createDialogOpen, setCreateDialogOpen] = useState<boolean>(false);
   const [live, setLive] = useState<boolean>(false);
   const queryClient = useQueryClient();
 
@@ -72,6 +73,8 @@ export default function Page() {
           <CreatePromptDialog
             promptsetId={promptsetId}
             version={prompts.length + 1}
+            open={createDialogOpen}
+            setOpen={setCreateDialogOpen}
           />
         </div>
       </div>
@@ -93,11 +96,15 @@ export default function Page() {
               currentPrompt={selectedPrompt}
               promptsetId={promptsetId}
               version={prompts.length + 1}
+              open={createDialogOpen}
+              setOpen={setCreateDialogOpen}
             />
           ) : (
             <CreatePromptDialog
               promptsetId={promptsetId}
               version={prompts.length + 1}
+              open={createDialogOpen}
+              setOpen={setCreateDialogOpen}
             />
           )}
         </div>
@@ -247,7 +254,6 @@ function PageLoading() {
           <ChevronLeft className="w-6 h-6 mr-2" />
           Back
         </Button>
-        <CreatePromptDialog promptsetId={""} disabled={true} />
       </div>
       <div className="flex gap-4 w-full h-screen">
         <Skeleton className="w-[340px] h-screen" />

diff --git a/components/evaluate/evaluation-table.tsx b/components/evaluate/evaluation-table.tsx
@@ -164,17 +164,21 @@ export default function EvaluationTable({
       !currentData ? (
         <EvaluationTableSkeleton />
       ) : (
-        currentData.map((span: any, i: number) => (
-          <EvaluationRow
-            key={i}
-            page={i + 1}
-            span={span}
-            projectId={projectId}
-            testId={test.id}
-            onCheckedChange={onCheckedChange}
-            selectedData={selectedData}
-          />
-        ))
+        currentData.map((span: any, i: number) => {
+          if (span.status_code !== "ERROR") {
+            return (
+              <EvaluationRow
+                key={i}
+                page={i + 1}
+                span={span}
+                projectId={projectId}
+                testId={test.id}
+                onCheckedChange={onCheckedChange}
+                selectedData={selectedData}
+              />
+            );
+          }
+        })
       )}
       {showLoader && (
         <div className="flex justify-center py-8">

diff --git a/components/playground/common.tsx b/components/playground/common.tsx
@@ -1,3 +1,6 @@
+"use client";
+
+import PromptRegistryDialog from "@/components/playground/prompt-registry-dialog";
 import LLMPicker from "@/components/shared/llm-picker";
 import { Button } from "@/components/ui/button";
 import { Separator } from "@/components/ui/separator";
@@ -9,6 +12,8 @@ import {
 import { cn } from "@/lib/utils";
 import { MinusCircleIcon, PlusIcon } from "lucide-react";
 import { useEffect, useRef, useState } from "react";
+import { useQuery } from "react-query";
+import { toast } from "sonner";
 
 export function RoleBadge({
   role,
@@ -34,15 +39,24 @@ export function ExpandingTextArea({
   value,
   onChange,
   setFocusing,
+  saveButtonRef,
+  handleSave,
 }: {
   value: string;
   onChange: any;
   setFocusing?: any;
+  saveButtonRef: React.RefObject<HTMLButtonElement>;
+  handleSave: (open: boolean) => void;
 }) {
   const textAreaRef = useRef<HTMLTextAreaElement>(null);
 
   const handleClickOutside = (event: any) => {
-    if (textAreaRef.current && !textAreaRef.current.contains(event.target)) {
+    if (
+      textAreaRef.current &&
+      !textAreaRef.current.contains(event.target) &&
+      saveButtonRef.current &&
+      !saveButtonRef.current.contains(event.target)
+    ) {
       setFocusing(false);
     }
   };
@@ -63,13 +77,25 @@ export function ExpandingTextArea({
   };
 
   return (
-    <textarea
-      className="rounded-md text-sm w-[290px] bg-background"
-      ref={textAreaRef}
-      defaultValue={value}
-      onChange={handleChange}
-      style={{ overflowY: "auto", resize: "none", height: "auto" }}
-    />
+    <div className="relative w-[290px]">
+      <textarea
+        className="rounded-md text-sm w-[290px] bg-background pr-10"
+        ref={textAreaRef}
+        defaultValue={value}
+        onChange={handleChange}
+        style={{ overflowY: "auto", resize: "none", height: "auto" }}
+      />
+      <div className="absolute right-2 bottom-2 py-2">
+        <Button
+          className="text-sm px-6"
+          size={"icon"}
+          onClick={() => handleSave(true)}
+          ref={saveButtonRef}
+        >
+          Save
+        </Button>
+      </div>
+    </div>
   );
 }
 
@@ -107,6 +133,34 @@ export function Message({
     }
   };
   const [editing, setEditing] = useState(false);
+  const [dialogOpen, setDialogOpen] = useState(false);
+  const [selectedPromptRegistry, setSelectedPromptRegistry] =
+    useState<any>(null);
+  const saveButtonRef = useRef<HTMLButtonElement>(null);
+  const [currentPrompt, setCurrentPrompt] = useState<any>(undefined);
+
+  useQuery({
+    queryKey: ["fetch-prompts-query", selectedPromptRegistry?.id],
+    queryFn: async () => {
+      const response = await fetch(
+        `/api/promptset?promptset_id=${selectedPromptRegistry?.id}`
+      );
+      if (!response.ok) {
+        const error = await response.json();
+        throw new Error(error?.message || "Failed to fetch tests");
+      }
+      const result = await response.json();
+      setCurrentPrompt(result?.promptsets?.prompts[0] || undefined);
+      return result;
+    },
+    enabled: !!selectedPromptRegistry,
+    onError: (error) => {
+      toast.error("Failed to fetch prompts", {
+        description: error instanceof Error ? error.message : String(error),
+      });
+    },
+  });
+
   return (
     <>
       <div className="flex items-center justify-between cursor-pointer group hover:bg-muted rounded-md p-4">
@@ -129,13 +183,17 @@ export function Message({
               </p>
             )}
             {editing && (
-              <ExpandingTextArea
-                onChange={(value: string) => {
-                  setMessage({ ...message, content: value });
-                }}
-                value={message.content}
-                setFocusing={setEditing}
-              />
+              <div>
+                <ExpandingTextArea
+                  onChange={(value: string) => {
+                    setMessage({ ...message, content: value });
+                  }}
+                  value={message.content}
+                  setFocusing={setEditing}
+                  saveButtonRef={saveButtonRef}
+                  handleSave={setDialogOpen}
+                />
+              </div>
             )}
           </div>
         </div>
@@ -151,6 +209,11 @@ export function Message({
         </Button>
       </div>
       <Separator />
+      <PromptRegistryDialog
+        openDialog={dialogOpen}
+        setOpenDialog={setDialogOpen}
+        passedPrompt={message.content}
+      />
     </>
   );
 }