Skip to content

Commit

Permalink
[8.16] [Security KB] Fix setup KB (#201175) (#202392)
Browse files Browse the repository at this point in the history 
# Backport

This will backport the following commits from `main` to `8.16`:
- [[Security KB] Fix setup KB
(#201175)](#201175)

<!--- Backport version: 8.9.8 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Patryk
Kopyciński","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-11-22T16:58:24Z","message":"[Security
KB] Fix setup KB (#201175)\n\n## Summary\r\n\r\nFix an issue with
auto-recovery of Knowledge Base setup. \r\n\r\nWhen the KB setup was
initialized on an undersized cluster, the model\r\nfailed to deploy
correctly. This resulted in the KB ending up in a\r\nbroken state,
repeatedly displaying the Setup KB
button.\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine
<[email protected]>","sha":"1cb56d7196cf60b03cb539f32f6a466a17141e02","branchLabelMapping":{"^v9.0.0$":"main","^v8.18.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["bug","release_note:fix","v9.0.0","ci:cloud-deploy","Team:Security
Generative
AI","backport:version","v8.17.0","v8.18.0","v8.16.2"],"number":201175,"url":"https://github.com/elastic/kibana/pull/201175","mergeCommit":{"message":"[Security
KB] Fix setup KB (#201175)\n\n## Summary\r\n\r\nFix an issue with
auto-recovery of Knowledge Base setup. \r\n\r\nWhen the KB setup was
initialized on an undersized cluster, the model\r\nfailed to deploy
correctly. This resulted in the KB ending up in a\r\nbroken state,
repeatedly displaying the Setup KB
button.\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine
<[email protected]>","sha":"1cb56d7196cf60b03cb539f32f6a466a17141e02"}},"sourceBranch":"main","suggestedTargetBranches":["8.16"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","labelRegex":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/201175","number":201175,"mergeCommit":{"message":"[Security
KB] Fix setup KB (#201175)\n\n## Summary\r\n\r\nFix an issue with
auto-recovery of Knowledge Base setup. \r\n\r\nWhen the KB setup was
initialized on an undersized cluster, the model\r\nfailed to deploy
correctly. This resulted in the KB ending up in a\r\nbroken state,
repeatedly displaying the Setup KB
button.\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine
<[email protected]>","sha":"1cb56d7196cf60b03cb539f32f6a466a17141e02"}},{"branch":"8.17","label":"v8.17.0","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/201436","number":201436,"state":"MERGED","mergeCommit":{"sha":"252fbfd3de5279b450358e8ffa5c87c58a8767c6","message":"[8.17]
[Security KB] Fix setup KB (#201175) (#201436)\n\n# Backport\n\nThis
will backport the following commits from `main` to `8.17`:\n- [[Security
KB] Fix setup
KB\n(#201175)](https://github.com/elastic/kibana/pull/201175)\n\n<!---
Backport version: 9.4.3 -->\n\n### Questions ?\nPlease refer to the
[Backport
tool\ndocumentation](https://github.com/sqren/backport)\n\n<!--BACKPORT
[{\"author\":{\"name\":\"Patryk\nKopyciński\",\"email\":\"[email protected]\"},\"sourceCommit\":{\"committedDate\":\"2024-11-22T16:58:24Z\",\"message\":\"[Security\nKB]
Fix setup KB (#201175)\\n\\n## Summary\\r\\n\\r\\nFix an issue
with\nauto-recovery of Knowledge Base setup. \\r\\n\\r\\nWhen the KB
setup was\ninitialized on an undersized cluster, the model\\r\\nfailed
to deploy\ncorrectly. This resulted in the KB ending up in a\\r\\nbroken
state,\nrepeatedly displaying the Setup
KB\nbutton.\\r\\n\\r\\n---------\\r\\n\\r\\nCo-authored-by:
kibanamachine\n<[email protected]>\",\"sha\":\"1cb56d7196cf60b03cb539f32f6a466a17141e02\",\"branchLabelMapping\":{\"^v9.0.0$\":\"main\",\"^v8.18.0$\":\"8.x\",\"^v(\\\\d+).(\\\\d+).\\\\d+$\":\"$1.$2\"}},\"sourcePullRequest\":{\"labels\":[\"bug\",\"release_note:fix\",\"v9.0.0\",\"ci:cloud-deploy\",\"Team:Security\nGenerative\nAI\",\"backport:version\",\"v8.17.0\",\"v8.18.0\",\"v8.16.2\"],\"title\":\"[Security\nKB]
Fix
setup\nKB\",\"number\":201175,\"url\":\"https://github.com/elastic/kibana/pull/201175\",\"mergeCommit\":{\"message\":\"[Security\nKB]
Fix setup KB (#201175)\\n\\n## Summary\\r\\n\\r\\nFix an issue
with\nauto-recovery of Knowledge Base setup. \\r\\n\\r\\nWhen the KB
setup was\ninitialized on an undersized cluster, the model\\r\\nfailed
to deploy\ncorrectly. This resulted in the KB ending up in a\\r\\nbroken
state,\nrepeatedly displaying the Setup
KB\nbutton.\\r\\n\\r\\n---------\\r\\n\\r\\nCo-authored-by:
kibanamachine\n<[email protected]>\",\"sha\":\"1cb56d7196cf60b03cb539f32f6a466a17141e02\"}},\"sourceBranch\":\"main\",\"suggestedTargetBranches\":[\"8.17\",\"8.x\",\"8.16\"],\"targetPullRequestStates\":[{\"branch\":\"main\",\"label\":\"v9.0.0\",\"branchLabelMappingKey\":\"^v9.0.0$\",\"isSourceBranch\":true,\"state\":\"MERGED\",\"url\":\"https://github.com/elastic/kibana/pull/201175\",\"number\":201175,\"mergeCommit\":{\"message\":\"[Security\nKB]
Fix setup KB (#201175)\\n\\n## Summary\\r\\n\\r\\nFix an issue
with\nauto-recovery of Knowledge Base setup. \\r\\n\\r\\nWhen the KB
setup was\ninitialized on an undersized cluster, the model\\r\\nfailed
to deploy\ncorrectly. This resulted in the KB ending up in a\\r\\nbroken
state,\nrepeatedly displaying the Setup
KB\nbutton.\\r\\n\\r\\n---------\\r\\n\\r\\nCo-authored-by:
kibanamachine\n<[email protected]>\",\"sha\":\"1cb56d7196cf60b03cb539f32f6a466a17141e02\"}},{\"branch\":\"8.17\",\"label\":\"v8.17.0\",\"branchLabelMappingKey\":\"^v(\\\\d+).(\\\\d+).\\\\d+$\",\"isSourceBranch\":false,\"state\":\"NOT_CREATED\"},{\"branch\":\"8.x\",\"label\":\"v8.18.0\",\"branchLabelMappingKey\":\"^v8.18.0$\",\"isSourceBranch\":false,\"state\":\"NOT_CREATED\"},{\"branch\":\"8.16\",\"label\":\"v8.16.2\",\"branchLabelMappingKey\":\"^v(\\\\d+).(\\\\d+).\\\\d+$\",\"isSourceBranch\":false,\"state\":\"NOT_CREATED\"}]}]\nBACKPORT-->\n\nCo-authored-by:
Patryk Kopyciński
<[email protected]>"}},{"branch":"8.x","label":"v8.18.0","labelRegex":"^v8.18.0$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/201437","number":201437,"state":"MERGED","mergeCommit":{"sha":"7d6c034e25e6d9158edd6c5b1ab905265abb2bdb","message":"[8.x]
[Security KB] Fix setup KB (#201175) (#201437)\n\n# Backport\n\nThis
will backport the following commits from `main` to `8.x`:\n- [[Security
KB] Fix setup
KB\n(#201175)](https://github.com/elastic/kibana/pull/201175)\n\n<!---
Backport version: 9.4.3 -->\n\n### Questions ?\nPlease refer to the
[Backport
tool\ndocumentation](https://github.com/sqren/backport)\n\n<!--BACKPORT
[{\"author\":{\"name\":\"Patryk\nKopyciński\",\"email\":\"[email protected]\"},\"sourceCommit\":{\"committedDate\":\"2024-11-22T16:58:24Z\",\"message\":\"[Security\nKB]
Fix setup KB (#201175)\\n\\n## Summary\\r\\n\\r\\nFix an issue
with\nauto-recovery of Knowledge Base setup. \\r\\n\\r\\nWhen the KB
setup was\ninitialized on an undersized cluster, the model\\r\\nfailed
to deploy\ncorrectly. This resulted in the KB ending up in a\\r\\nbroken
state,\nrepeatedly displaying the Setup
KB\nbutton.\\r\\n\\r\\n---------\\r\\n\\r\\nCo-authored-by:
kibanamachine\n<[email protected]>\",\"sha\":\"1cb56d7196cf60b03cb539f32f6a466a17141e02\",\"branchLabelMapping\":{\"^v9.0.0$\":\"main\",\"^v8.18.0$\":\"8.x\",\"^v(\\\\d+).(\\\\d+).\\\\d+$\":\"$1.$2\"}},\"sourcePullRequest\":{\"labels\":[\"bug\",\"release_note:fix\",\"v9.0.0\",\"ci:cloud-deploy\",\"Team:Security\nGenerative\nAI\",\"backport:version\",\"v8.17.0\",\"v8.18.0\",\"v8.16.2\"],\"title\":\"[Security\nKB]
Fix
setup\nKB\",\"number\":201175,\"url\":\"https://github.com/elastic/kibana/pull/201175\",\"mergeCommit\":{\"message\":\"[Security\nKB]
Fix setup KB (#201175)\\n\\n## Summary\\r\\n\\r\\nFix an issue
with\nauto-recovery of Knowledge Base setup. \\r\\n\\r\\nWhen the KB
setup was\ninitialized on an undersized cluster, the model\\r\\nfailed
to deploy\ncorrectly. This resulted in the KB ending up in a\\r\\nbroken
state,\nrepeatedly displaying the Setup
KB\nbutton.\\r\\n\\r\\n---------\\r\\n\\r\\nCo-authored-by:
kibanamachine\n<[email protected]>\",\"sha\":\"1cb56d7196cf60b03cb539f32f6a466a17141e02\"}},\"sourceBranch\":\"main\",\"suggestedTargetBranches\":[\"8.17\",\"8.x\",\"8.16\"],\"targetPullRequestStates\":[{\"branch\":\"main\",\"label\":\"v9.0.0\",\"branchLabelMappingKey\":\"^v9.0.0$\",\"isSourceBranch\":true,\"state\":\"MERGED\",\"url\":\"https://github.com/elastic/kibana/pull/201175\",\"number\":201175,\"mergeCommit\":{\"message\":\"[Security\nKB]
Fix setup KB (#201175)\\n\\n## Summary\\r\\n\\r\\nFix an issue
with\nauto-recovery of Knowledge Base setup. \\r\\n\\r\\nWhen the KB
setup was\ninitialized on an undersized cluster, the model\\r\\nfailed
to deploy\ncorrectly. This resulted in the KB ending up in a\\r\\nbroken
state,\nrepeatedly displaying the Setup
KB\nbutton.\\r\\n\\r\\n---------\\r\\n\\r\\nCo-authored-by:
kibanamachine\n<[email protected]>\",\"sha\":\"1cb56d7196cf60b03cb539f32f6a466a17141e02\"}},{\"branch\":\"8.17\",\"label\":\"v8.17.0\",\"branchLabelMappingKey\":\"^v(\\\\d+).(\\\\d+).\\\\d+$\",\"isSourceBranch\":false,\"state\":\"NOT_CREATED\"},{\"branch\":\"8.x\",\"label\":\"v8.18.0\",\"branchLabelMappingKey\":\"^v8.18.0$\",\"isSourceBranch\":false,\"state\":\"NOT_CREATED\"},{\"branch\":\"8.16\",\"label\":\"v8.16.2\",\"branchLabelMappingKey\":\"^v(\\\\d+).(\\\\d+).\\\\d+$\",\"isSourceBranch\":false,\"state\":\"NOT_CREATED\"}]}]\nBACKPORT-->\n\nCo-authored-by:
Patryk Kopyciński
<[email protected]>"}},{"branch":"8.16","label":"v8.16.2","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

---------

Co-authored-by: kibanamachine <[email protected]>
  • Loading branch information
@patrykkopycinski @kibanamachine
patrykkopycinski and kibanamachine authored Dec 3, 2024
1 parent 7e0ea54 commit 0c0cb2a
Showing 1 changed file with 37 additions and 17 deletions.
54 changes: 37 additions & 17 deletions x-pack/plugins/elastic_assistant/server/ai_assistant_data_clients/knowledge_base/index.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -225,27 +225,47 @@ export class AIAssistantKnowledgeBaseDataClient extends AIAssistantDataClient {
public createInferenceEndpoint = async () => {
const elserId = await this.options.getElserId();
this.options.logger.debug(`Deploying ELSER model '${elserId}'...`);
try {
const esClient = await this.options.elasticsearchClientPromise;
if (this.isV2KnowledgeBaseEnabled) {
await esClient.inference.put({
task_type: 'sparse_embedding',
const esClient = await this.options.elasticsearchClientPromise;

const inferenceEndpointExists = await this.isInferenceEndpointExists();

if (inferenceEndpointExists) {
try {
await esClient.inference.delete({
inference_id: ASSISTANT_ELSER_INFERENCE_ID,
inference_config: {
service: 'elasticsearch',
service_settings: {
adaptive_allocations: {
enabled: true,
min_number_of_allocations: 0,
max_number_of_allocations: 8,
},
num_threads: 1,
model_id: elserId,
// it's being used in the mapping so we need to force delete
force: true,
});
this.options.logger.debug(
`Deleted existing inference endpoint for ELSER model '${elserId}'`
);
} catch (error) {
this.options.logger.error(
`Error deleting inference endpoint for ELSER model '${elserId}':\n${error}`
);
}
}

try {
await esClient.inference.put({
task_type: 'sparse_embedding',
inference_id: ASSISTANT_ELSER_INFERENCE_ID,
inference_config: {
service: 'elasticsearch',
service_settings: {
adaptive_allocations: {
enabled: true,
min_number_of_allocations: 0,
max_number_of_allocations: 8,
},
task_settings: {},
},
});
}
task_settings: {},
},
});

// await for the model to be deployed
await this.isInferenceEndpointExists();
} catch (error) {
this.options.logger.error(
`Error creating inference endpoint for ELSER model '${elserId}':\n${error}`
Expand Down

0 comments on commit 0c0cb2a

Please sign in to comment.