Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add inference endpoints management page #184614

Conversation

saikatsarkar056
Copy link
Contributor

@saikatsarkar056 saikatsarkar056 commented May 31, 2024

Description

In this PR, we implemented a view for managing inference endpoints. The changes include the following items for both Serverless and Stack.

  • A blank page will be displayed if no inference endpoints are available.
  • A page displaying a list of inference endpoints. The user can view various details about each endpoint, such as the endpoint itself, the provider, and the type. The table supports pagination and sorting.
  • Users can add a new inference endpoint using Elasticsearch models and third-party APIs, including Hugging Face, Cohere, and OpenAI.

To keep the changes in this PR manageable, the following items are out of scope but will be added in subsequent PRs

  • Option to delete an inference endpoint
  • Filtering and Search bar
  • Information about allocations, thread.
  • Icons for Provider
  • Deployment status of underlying trained models

Empty page in Stack Management

stack_empty_page.mov

Page with all inference endpoints in Stack Management

stack_all_inference_endpoints.mov

Inference Endpoints Management in Serverless

serverless_all_inference_endpoints.mov

@saikatsarkar056 saikatsarkar056 requested a review from a team as a code owner May 31, 2024 20:47
@saikatsarkar056 saikatsarkar056 requested review from a team May 31, 2024 20:47
@saikatsarkar056 saikatsarkar056 requested a review from a team as a code owner May 31, 2024 20:47
@saikatsarkar056 saikatsarkar056 marked this pull request as draft May 31, 2024 20:47
@saikatsarkar056 saikatsarkar056 changed the title Inference endpoint manage serverless Add inference endpoints management page May 31, 2024
@saikatsarkar056 saikatsarkar056 force-pushed the inference_endpoint_manage_serverless branch 3 times, most recently from 5543d54 to 33c0e26 Compare June 2, 2024 21:24
@saikatsarkar056 saikatsarkar056 self-assigned this Jun 2, 2024
@saikatsarkar056 saikatsarkar056 added v8.15.0 Team:EnterpriseSearch release_note:skip Skip the PR/issue when compiling release notes labels Jun 2, 2024
@saikatsarkar056 saikatsarkar056 marked this pull request as ready for review June 2, 2024 21:49
@saikatsarkar056 saikatsarkar056 force-pushed the inference_endpoint_manage_serverless branch from 33c0e26 to a06e025 Compare June 3, 2024 18:32
x-pack/plugins/enterprise_search/common/constants.ts Outdated Show resolved Hide resolved
config/serverless.es.yml Outdated Show resolved Hide resolved
'xpack.searchInferenceEndpoints.inferenceEndpoints.addEmptyPrompt.createFirstInferenceEndpointDescription',
{
defaultMessage:
'Connect to your third-party model provider of choice to setup a single entity for semantic search.',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc: @leemthompo copy

x-pack/plugins/serverless_search/public/types.ts Outdated Show resolved Hide resolved
@TattdCodeMonkey
Copy link
Contributor

To keep the changes in this PR manageable, the following items are out of scope but will be added in subsequent PRs

@saikatsarkar056 thats fine, but given that ES3 is deployed weekly this should be disabled until the feature is ready and has adequate test coverage.

@saikatsarkar056 saikatsarkar056 force-pushed the inference_endpoint_manage_serverless branch 3 times, most recently from 7d7be32 to 12e84d6 Compare June 4, 2024 17:47
@saikatsarkar056 saikatsarkar056 requested a review from a team as a code owner June 4, 2024 17:47
@saikatsarkar056 saikatsarkar056 force-pushed the inference_endpoint_manage_serverless branch from 7989bbb to f580c31 Compare June 4, 2024 18:28
@saikatsarkar056 saikatsarkar056 requested a review from a team as a code owner June 4, 2024 18:28
@saikatsarkar056 saikatsarkar056 force-pushed the inference_endpoint_manage_serverless branch from aacd039 to bc01b8c Compare June 4, 2024 23:20
@saikatsarkar056 saikatsarkar056 requested a review from a team as a code owner June 4, 2024 23:20
@saikatsarkar056 saikatsarkar056 force-pushed the inference_endpoint_manage_serverless branch from 132be36 to 35e0891 Compare June 4, 2024 23:39
Copy link
Member

@jbudz jbudz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

limits.yml

@saikatsarkar056 saikatsarkar056 force-pushed the inference_endpoint_manage_serverless branch from 279a498 to db5ef5b Compare June 5, 2024 16:35
@saikatsarkar056 saikatsarkar056 force-pushed the inference_endpoint_manage_serverless branch from 8ec9a48 to e44ffc1 Compare June 13, 2024 15:30
@saikatsarkar056 saikatsarkar056 enabled auto-merge (squash) June 13, 2024 15:31
@pzl pzl requested review from paul-tavares and removed request for pzl June 13, 2024 15:42
@kibana-ci
Copy link
Collaborator

kibana-ci commented Jun 13, 2024

💛 Build succeeded, but was flaky

Failed CI Steps

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id before after diff
enterpriseSearch 2325 2332 +7
searchInferenceEndpoints - 89 +89
total +96

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id before after diff
@kbn/deeplinks-search 16 17 +1
searchInferenceEndpoints - 6 +6
total +7

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
enterpriseSearch 2.7MB 2.7MB +5.2KB
searchInferenceEndpoints - 45.7KB +45.7KB
total +50.9KB

Public APIs missing exports

Total count of every type that is part of your API that should be exported but is not. This will cause broken links in the API documentation system. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats exports for more detailed information.

id before after diff
searchInferenceEndpoints - 1 +1

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id before after diff
enterpriseSearch 49.1KB 50.5KB +1.4KB
searchInferenceEndpoints - 5.2KB +5.2KB
serverlessSearch 19.6KB 19.9KB +327.0B
total +7.0KB
Unknown metric groups

API count

id before after diff
@kbn/deeplinks-search 16 17 +1
searchInferenceEndpoints - 10 +10
total +11

async chunk count

id before after diff
enterpriseSearch 17 18 +1
searchInferenceEndpoints - 4 +4
total +5

ESLint disabled line counts

id before after diff
searchInferenceEndpoints - 3 +3

miscellaneous assets size

id before after diff
searchInferenceEndpoints - 697.1KB ⚠️ +697.1KB

Total ESLint disabled count

id before after diff
searchInferenceEndpoints - 3 +3

History

  • 💛 Build #215633 was flaky 0a4c7910a316d5e4db08b5a2b858882be09c471e
  • 💚 Build #215096 succeeded f3765fdec03fc879ef379606f05f925eda1a0c7a
  • 💔 Build #215058 failed 40bbe501c9e194371f1510280d4967743b86976f
  • 💚 Build #214588 succeeded 11b120d3505a1d72250668aa4d35bbcd81a98353

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @saikatsarkar056

@saikatsarkar056 saikatsarkar056 merged commit db425e3 into elastic:main Jun 13, 2024
44 checks passed
@kibanamachine kibanamachine added the backport:skip This commit does not require backporting label Jun 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:skip This commit does not require backporting ci:project-deploy-observability Create an Observability project release_note:skip Skip the PR/issue when compiling release notes Team:EnterpriseSearch Team:Fleet Team label for Observability Data Collection Fleet team Team:obs-ux-infra_services Observability Infrastructure & Services User Experience Team v8.15.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.