[ML] Merge the pytorch-inference feature branch #73660
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Initial start/stop trained model deployment actions.
Adds the model_type field to TrainedModelConfig for distinguishing between models that can be loaded via the model loading service and those that require a native process.
This adds a temporary API for doing inference against a trained model deployment.
Introduces code for re-assembling the individual chunks a model is stored in and streaming those chunks to the inference process. Re-uses the TrainedModelDefinitionDoc format already defined for boosted tree models
Binary data is stored in lucene base64 encoded, the same data stored in a Java string uses 2 bytes (UTF16) to represent each base64 character consuming twice the amount of memory required. The compressed binary representation of the models can stored in ByteReferences more efficiently. For BWC a new field mapping binary_definition is added .ml-inference-* and the index version incremented.
This adds a location field to TrainedModelConfig for large models that cannot be PUT inline with the config. Large models are reassembled from their location.
Adds tokenisation for BERT models via the WordPiece algorithm using the vocabulary that defined with the model and introduces the concept of NLP tasks. Each task is configured with a BERT model supporting that task, pre-processing and post-processing is defined by the task. Named Entity Recognition and Fill Mask are the 2 task types supported by this PR
davidkyle
changed the title
|
Pinging @elastic/ml-core (Team:ML) |
|
Pinging @elastic/clients-team (Team:Clients) |
sethmlarson
suggested changes
Jun 2, 2021
rest-api-spec/src/main/resources/rest-api-spec/api/ml.start_deployment.json
Outdated
Show resolved
Hide resolved
rest-api-spec/src/main/resources/rest-api-spec/api/ml.start_deployment.json
Outdated
Show resolved
Hide resolved
rest-api-spec/src/main/resources/rest-api-spec/api/ml.stop_deployment.json
Outdated
Show resolved
Hide resolved
|
Thanks for jumping in with an early review @sethmlarson
👍 This makes sense to me I'll raise it with the team I've missed out the spec of the These APIs may be in flux for a short while as we work through all the use cases. Is that a problem for the clients team? Would you prefer us to tell you when we settled on something we like? |
|
@davidkyle It's no problem for us that these APIs may change especially if they're experimental/on |
sethmlarson
approved these changes
Jun 2, 2021
rest-api-spec/src/main/resources/rest-api-spec/api/ml.infer_trained_model_deployment.json
Show resolved
Hide resolved
This was referenced Jul 15, 2021
This was referenced Nov 23, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The feature branch contains changes to configure PyTorch models with a
TrainedModelConfigand defines a format to store the binary models. The_startand_stopdeployment actions control the model lifecycle and the model can be directly evaluated with the_inferendpoint. 2 Types of NLP tasks are supported: Named Entity Recognition and Fill Mask.The feature branch consists of these PRs: