Huggingface optimum prepackaged server

[axsaucedo]

Which issue(s) this PR fixes:

Fixes #4082

Introduces MLServer Runtime: SeldonIO/MLServer#573

HuggingFace Server

Thanks to our collaboration with the HuggingFace team you can now easily deploy your models from the HuggingFace Hub with Seldon Core.

We also support the high performance optimizations provided by the Transformer Optimum framework.

Pipeline parameters

The parameters available:

Name	Description
task	The transformer pipeline task
pretrained_model	The name of the pretrained model in the Hub
pretrained_tokenizer	Transformer name in Hub if different to the one provided with model
optimum_model	Boolean to enable loading model with Optimum framework

Simple Example

You can deploy a HuggingFace model by providing parameters to your pipeline.

apiVersion: machinelearning.seldon.io/v1alpha2
kind: SeldonDeployment
metadata:
  name: gpt2-model
spec:
  protocol: v2
  predictors:
  - graph:
      name: transformer
      implementation: HUGGINGFACE_SERVER
      parameters:
      - name: task
        type: STRING
        value: text-generation
      - name: pretrained_model
        type: STRING
        value: distilgpt2
    name: default
    replicas: 1

Quantized & Optimized Models with Optimum

You can deploy a HuggingFace model loaded using the Optimum library by using the optimum_model parameter

apiVersion: machinelearning.seldon.io/v1alpha2
kind: SeldonDeployment
metadata:
  name: gpt2-model
spec:
  protocol: v2
  predictors:
  - graph:
      name: transformer
      implementation: HUGGINGFACE_SERVER
      parameters:
      - name: task
        type: STRING
        value: text-generation
      - name: pretrained_model
        type: STRING
        value: distilgpt2
      - name: optimum_model
        type: BOOL
        value: true
    name: default
    replicas: 1

[axsaucedo]

/test integration

[axsaucedo]

/test integration

[axsaucedo]

/test integration

[axsaucedo]

It seems failed test is test_label_update which is known to be flaky

/test integration

[axsaucedo]

/test integration

[axsaucedo]

All unit integration tests pass - should be good to merge

[adriangonz]

This looks great! I think it should be ready to go @axsaucedo 👍

[seldondev]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: adriangonz

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [adriangonz]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment