Skip to content

Commit

Permalink
Obs AI Assistant Fetch user instructions using user_id (elastic#200137)
Browse files Browse the repository at this point in the history 
## Summary

[Obs AI Assistant] Fetch user instructions using id instead of username
for knowledge base instructions elastic#192701

To avoid potential collisions when fetching data, we should query for
the user id instead of the user name when getting instructions.
  • Loading branch information
@arturoliduena
arturoliduena authored Nov 19, 2024
1 parent 674bf63 commit 518dc25
Show file tree
Hide file tree
Showing 2 changed files with 89 additions and 62 deletions.
4 changes: 3 additions & 1 deletion ...observability_solution/observability_ai_assistant/server/service/util/get_access_query.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,9 @@ export function getAccessQuery({
bool: {
should: [
{ term: { public: true } },
...(user ? [{ term: { 'user.name': user.name } }] : []),
...(user
? [{ term: user.id ? { 'user.id': user.id } : { 'user.name': user.name } }]
: []),
],
minimum_should_match: 1,
},
Expand Down
147 changes: 86 additions & 61 deletions x-pack/test/observability_ai_assistant_functional/tests/conversations/index.spec.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@ import expect from '@kbn/expect';
import { MessageRole } from '@kbn/observability-ai-assistant-plugin/common';
import { ChatFeedback } from '@kbn/observability-ai-assistant-plugin/public/analytics/schemas/chat_feedback';
import { pick } from 'lodash';
import { parse as parseCookie } from 'tough-cookie';
import { kbnTestConfig } from '@kbn/test';
import {
createLlmProxy,
isFunctionTitleRequest,
Expand All @@ -17,12 +19,15 @@ import {
import { interceptRequest } from '../../common/intercept_request';
import { FtrProviderContext } from '../../ftr_provider_context';

import { editor } from '../../../observability_ai_assistant_api_integration/common/users/users';

export default function ApiTest({ getService, getPageObjects }: FtrProviderContext) {
const observabilityAIAssistantAPIClient = getService('observabilityAIAssistantAPIClient');
const ui = getService('observabilityAIAssistantUI');
const testSubjects = getService('testSubjects');
const browser = getService('browser');
const supertest = getService('supertest');
const supertestWithoutAuth = getService('supertestWithoutAuth');
const retry = getService('retry');
const log = getService('log');
const telemetry = getService('kibana_ebt_ui');
Expand All @@ -35,6 +40,20 @@ export default function ApiTest({ getService, getPageObjects }: FtrProviderConte

const flyoutService = getService('flyout');

async function login(username: string, password: string | undefined) {
const response = await supertestWithoutAuth
.post('/internal/security/login')
.set('kbn-xsrf', 'xxx')
.send({
providerType: 'basic',
providerName: 'basic',
currentURL: '/',
params: { username, password },
})
.expect(200);
return parseCookie(response.headers['set-cookie'][0])!;
}

async function deleteConversations() {
const response = await observabilityAIAssistantAPIClient.editor({
endpoint: 'POST /internal/observability_ai_assistant/conversations',
Expand Down Expand Up @@ -66,78 +85,84 @@ export default function ApiTest({ getService, getPageObjects }: FtrProviderConte
}

async function createOldConversation() {
await observabilityAIAssistantAPIClient.editor({
endpoint: 'POST /internal/observability_ai_assistant/conversation',
params: {
body: {
conversation: {
messages: [
{
'@timestamp': '2024-04-18T14:28:50.118Z',
message: {
role: MessageRole.System,
content:
'You are a helpful assistant for Elastic Observability. Your goal is to help the Elastic Observability users to quickly assess what is happening in their observed systems. You can help them visualise and analyze data, investigate their systems, perform root cause analysis or identify optimisation opportunities.\n\nIt\'s very important to not assume what the user is meaning. Ask them for clarification if needed.\n\nIf you are unsure about which function should be used and with what arguments, ask the user for clarification or confirmation.\n\nIn KQL ("kqlFilter")) escaping happens with double quotes, not single quotes. Some characters that need escaping are: \':()\\ /". Always put a field value in double quotes. Best: service.name:"opbeans-go". Wrong: service.name:opbeans-go. This is very important!\n\nYou can use Github-flavored Markdown in your responses. If a function returns an array, consider using a Markdown table to format the response.\n\nNote that ES|QL (the Elasticsearch Query Language which is a new piped language) is the preferred query language.\n\nYou MUST use the "query" function when the user wants to:\n- visualize data\n- run any arbitrary query\n- breakdown or filter ES|QL queries that are displayed on the current page\n- convert queries from another language to ES|QL\n- asks general questions about ES|QL\n\nDO NOT UNDER ANY CIRCUMSTANCES generate ES|QL queries or explain anything about the ES|QL query language yourself.\nDO NOT UNDER ANY CIRCUMSTANCES try to correct an ES|QL query yourself - always use the "query" function for this.\n\nDO NOT UNDER ANY CIRCUMSTANCES USE ES|QL syntax (`service.name == "foo"`) with "kqlFilter" (`service.name:"foo"`).\n\nEven if the "context" function was used before that, follow it up with the "query" function. If a query fails, do not attempt to correct it yourself. Again you should call the "query" function,\neven if it has been called before.\n\nWhen the "visualize_query" function has been called, a visualization has been displayed to the user. DO NOT UNDER ANY CIRCUMSTANCES follow up a "visualize_query" function call with your own visualization attempt.\nIf the "execute_query" function has been called, summarize these results for the user. The user does not see a visualization in this case.\n\nYou MUST use the get_dataset_info function function before calling the "query" or "changes" function.\n\nIf a function requires an index, you MUST use the results from the dataset info functions.\n\n\n\nThe user is able to change the language which they want you to reply in on the settings page of the AI Assistant for Observability, which can be found in the Stack Management app under the option AI Assistants.\nIf the user asks how to change the language, reply in the same language the user asked in.You do not have a working memory. If the user expects you to remember the previous conversations, tell them they can set up the knowledge base.',
},
const { password } = kbnTestConfig.getUrlParts();
const sessionCookie = await login(editor.username, password);
const endpoint = '/internal/observability_ai_assistant/conversation';
const cookie = sessionCookie.cookieString();
const params = {
body: {
conversation: {
messages: [
{
'@timestamp': '2024-04-18T14:28:50.118Z',
message: {
role: MessageRole.System,
content:
'You are a helpful assistant for Elastic Observability. Your goal is to help the Elastic Observability users to quickly assess what is happening in their observed systems. You can help them visualise and analyze data, investigate their systems, perform root cause analysis or identify optimisation opportunities.\n\nIt\'s very important to not assume what the user is meaning. Ask them for clarification if needed.\n\nIf you are unsure about which function should be used and with what arguments, ask the user for clarification or confirmation.\n\nIn KQL ("kqlFilter")) escaping happens with double quotes, not single quotes. Some characters that need escaping are: \':()\\ /". Always put a field value in double quotes. Best: service.name:"opbeans-go". Wrong: service.name:opbeans-go. This is very important!\n\nYou can use Github-flavored Markdown in your responses. If a function returns an array, consider using a Markdown table to format the response.\n\nNote that ES|QL (the Elasticsearch Query Language which is a new piped language) is the preferred query language.\n\nYou MUST use the "query" function when the user wants to:\n- visualize data\n- run any arbitrary query\n- breakdown or filter ES|QL queries that are displayed on the current page\n- convert queries from another language to ES|QL\n- asks general questions about ES|QL\n\nDO NOT UNDER ANY CIRCUMSTANCES generate ES|QL queries or explain anything about the ES|QL query language yourself.\nDO NOT UNDER ANY CIRCUMSTANCES try to correct an ES|QL query yourself - always use the "query" function for this.\n\nDO NOT UNDER ANY CIRCUMSTANCES USE ES|QL syntax (`service.name == "foo"`) with "kqlFilter" (`service.name:"foo"`).\n\nEven if the "context" function was used before that, follow it up with the "query" function. If a query fails, do not attempt to correct it yourself. Again you should call the "query" function,\neven if it has been called before.\n\nWhen the "visualize_query" function has been called, a visualization has been displayed to the user. DO NOT UNDER ANY CIRCUMSTANCES follow up a "visualize_query" function call with your own visualization attempt.\nIf the "execute_query" function has been called, summarize these results for the user. The user does not see a visualization in this case.\n\nYou MUST use the get_dataset_info function function before calling the "query" or "changes" function.\n\nIf a function requires an index, you MUST use the results from the dataset info functions.\n\n\n\nThe user is able to change the language which they want you to reply in on the settings page of the AI Assistant for Observability, which can be found in the Stack Management app under the option AI Assistants.\nIf the user asks how to change the language, reply in the same language the user asked in.You do not have a working memory. If the user expects you to remember the previous conversations, tell them they can set up the knowledge base.',
},
{
'@timestamp': '2024-04-18T14:29:01.615Z',
message: {
content: 'What are SLOs?',
role: MessageRole.User,
},
},
{
'@timestamp': '2024-04-18T14:29:01.876Z',
message: {
role: MessageRole.Assistant,
content: '',
function_call: {
name: 'context',
arguments: '{}',
trigger: MessageRole.Assistant,
},
},
},
{
'@timestamp': '2024-04-18T14:29:01.615Z',
message: {
content: 'What are SLOs?',
role: MessageRole.User,
},
{
'@timestamp': '2024-04-18T14:29:01.876Z',
message: {
content:
'{"screen_description":"The user is looking at http://localhost:5601/ftw/app/observabilityAIAssistant/conversations/new. The current time range is 2024-04-18T14:13:49.815Z - 2024-04-18T14:28:49.815Z.","learnings":[]}',
},
{
'@timestamp': '2024-04-18T14:29:01.876Z',
message: {
role: MessageRole.Assistant,
content: '',
function_call: {
name: 'context',
role: MessageRole.User,
arguments: '{}',
trigger: MessageRole.Assistant,
},
},
{
'@timestamp': '2024-04-18T14:29:22.945Z',
message: {
content:
"SLOs, or Service Level Objectives, are a key part of the Site Reliability Engineering (SRE) methodology. They are a target value or range of values for a service level that is measured by an SLI (Service Level Indicator). \n\nAn SLO is a goal for how often and how much you want your service to meet a particular SLI. For example, you might have an SLO that your service should be up and running 99.9% of the time. \n\nSLOs are important because they set clear expectations for your team and your users about the level of service you aim to provide. They also help you make decisions about where to focus your efforts: if you're meeting your SLOs, you can focus on building new features; if you're not meeting your SLOs, you need to focus on improving reliability. \n\nIn Elastic Observability, you can define and monitor your SLOs to ensure your services are meeting their targets.",
function_call: {
name: '',
arguments: '',
trigger: MessageRole.Assistant,
},
role: MessageRole.Assistant,
},
},
{
'@timestamp': '2024-04-18T14:29:01.876Z',
message: {
content:
'{"screen_description":"The user is looking at http://localhost:5601/ftw/app/observabilityAIAssistant/conversations/new. The current time range is 2024-04-18T14:13:49.815Z - 2024-04-18T14:28:49.815Z.","learnings":[]}',
name: 'context',
role: MessageRole.User,
},
],
conversation: {
title: 'My old conversation',
token_count: {
completion: 1,
prompt: 1,
total: 2,
},
{
'@timestamp': '2024-04-18T14:29:22.945Z',
message: {
content:
"SLOs, or Service Level Objectives, are a key part of the Site Reliability Engineering (SRE) methodology. They are a target value or range of values for a service level that is measured by an SLI (Service Level Indicator). \n\nAn SLO is a goal for how often and how much you want your service to meet a particular SLI. For example, you might have an SLO that your service should be up and running 99.9% of the time. \n\nSLOs are important because they set clear expectations for your team and your users about the level of service you aim to provide. They also help you make decisions about where to focus your efforts: if you're meeting your SLOs, you can focus on building new features; if you're not meeting your SLOs, you need to focus on improving reliability. \n\nIn Elastic Observability, you can define and monitor your SLOs to ensure your services are meeting their targets.",
function_call: {
name: '',
arguments: '',
trigger: MessageRole.Assistant,
},
role: MessageRole.Assistant,
},
},
'@timestamp': '2024-04-18T14:29:22.948',
public: false,
numeric_labels: {},
labels: {},
],
conversation: {
title: 'My old conversation',
token_count: {
completion: 1,
prompt: 1,
total: 2,
},
},
'@timestamp': '2024-04-18T14:29:22.948',
public: false,
numeric_labels: {},
labels: {},
},
},
});
};
await supertestWithoutAuth
.post(endpoint)
.set('kbn-xsrf', 'xxx')
.set('Cookie', cookie)
.send(params.body);
}

describe('Conversations', () => {
Expand Down

0 comments on commit 518dc25

Please sign in to comment.