From 54b3b774f22910d70b723b6fad76530958cb14ef Mon Sep 17 00:00:00 2001 From: "mergify[bot]" <37929162+mergify[bot]@users.noreply.github.com> Date: Mon, 3 Jun 2024 11:28:00 -0700 Subject: [PATCH] [8.14] Adds LLM performance matrix doc to ESS & serverless (backport #5286) (#5295) * Adds LLM performance matrix doc to ESS & serverless (#5286) * Adds LLM performance matrix doc to ESS & serverless * experimenting with matrix formatting * uses alternative matrix formatting * updates matrix format, adds alternative version * uses updated table format * updates ESS version of table * fixes table * removes outdated table formatting * Update docs/assistant/llm-performance-matrix.asciidoc Co-authored-by: Joe Peeples * Update docs/serverless/assistant/llm-performance-matrix.mdx Co-authored-by: Joe Peeples * Update docs/serverless/assistant/llm-performance-matrix.mdx Co-authored-by: Joe Peeples * Update docs/assistant/llm-performance-matrix.asciidoc Co-authored-by: Joe Peeples * Update docs/assistant/llm-performance-matrix.asciidoc Co-authored-by: natasha-moore-elastic <137783811+natasha-moore-elastic@users.noreply.github.com> * Update docs/serverless/assistant/llm-performance-matrix.mdx Co-authored-by: natasha-moore-elastic <137783811+natasha-moore-elastic@users.noreply.github.com> * Update docs/assistant/llm-performance-matrix.asciidoc --------- Co-authored-by: Joe Peeples Co-authored-by: natasha-moore-elastic <137783811+natasha-moore-elastic@users.noreply.github.com> (cherry picked from commit adb07fac599ebcfc1414d963d71caf427bad7c67) # Conflicts: # docs/serverless/serverless-security.docnav.json * remove serverless pages from backport branch * fixes build --------- Co-authored-by: Benjamin Ironside Goldstein <91905639+benironside@users.noreply.github.com> Co-authored-by: Benjamin Ironside Goldstein --- docs/assistant/llm-performance-matrix.asciidoc | 15 +++++++++++++++ docs/assistant/security-assistant.asciidoc | 1 + 2 files changed, 16 insertions(+) create mode 100644 docs/assistant/llm-performance-matrix.asciidoc diff --git a/docs/assistant/llm-performance-matrix.asciidoc b/docs/assistant/llm-performance-matrix.asciidoc new file mode 100644 index 0000000000..3303902c7c --- /dev/null +++ b/docs/assistant/llm-performance-matrix.asciidoc @@ -0,0 +1,15 @@ +[[llm-performance-matrix]] += Large language model performance matrix + +This table describes the performance of various large language models (LLMs) for different use cases in {elastic-sec}, based on our internal testing. To learn more about these use cases, refer to <> or <>. + +[cols="1,1,1,1,1,1,1", options="header"] +|=== +| *Feature* | *Model* | | | | | +| | *Claude 3: Opus* | *Claude 3: Sonnet* | *Claude 3: Haiku* | *GPT-4o* | *GPT-4 Turbo*| *GPT-4 32K* + +| *Assistant - General* | Excellent | Excellent | Excellent | Excellent | Excellent | Excellent +| *Assistant - {esql} Generation*| Great | Great | Poor | Excellent | Poor | Excellent +| *Assistant - Alert Questions* | Excellent | Excellent | Excellent | Excellent | Poor | Good (limited context) +| *Attack discovery* | Excellent | Great | Poor | Poor | Good | Good (limited context) +|=== \ No newline at end of file diff --git a/docs/assistant/security-assistant.asciidoc b/docs/assistant/security-assistant.asciidoc index 9f0066833c..7c94ff23db 100644 --- a/docs/assistant/security-assistant.asciidoc +++ b/docs/assistant/security-assistant.asciidoc @@ -194,6 +194,7 @@ In addition to practical advice, AI Assistant can offer conceptual advice, tips, include::ai-alert-triage.asciidoc[leveloffset=+1] +include::llm-performance-matrix.asciidoc[leveloffset=+1] include::azure-openai-setup.asciidoc[leveloffset=+1] include::connect-to-openai.asciidoc[leveloffset=+1] include::connect-to-bedrock.asciidoc[leveloffset=+1]