Neu.ro MLFlow2Seldon deployer

An integration service to deploy MLFlow registered model as REST/GRPC API to Kubernetes cluster using Seldon-core.

Usage

This service is running inside of the Kubernetes cluster, where the Seldon-core is deployed. By constantly fetching the MLFlow server registered models (running as a platform job) via MLFlow Python SDK, it synchronizes the MLFlow state to Seldon-core within the Kubernetes cluster.

For instance, if the MLFlow registered model version gets assigned to the Staging/Production stage, the corresponding model binary gets deployed from the MLFlow into the K8s cluster as the SeldonDeployment (exposing REST/GRPC APIs). If the stage assignment gets removed/updated - the corresponding SeldonDeployment is changed respectively.

Given that, all the interaction with the service is done implicitly via the MLFlow server state. There is no need to execute particular commands/workloads against this service directly.

Prerequisites and usage assumptions

• MLFlow
  • is up and running as a platform job
  • disabled platform SSO;
  • artifact store as a platform storage, mounted as local path;
  • mlflow server version is at least 1.11.0;
• Seldon
  • SeldonDeployment container image (model wrapper) should be stored in the platform registry, on the same cluster where MLFlow is runnnig;
  • kubectl tool at the time of this service deployment should be authenticated to communicate with a Kubernetes cluster, where Seldon is deployed;
  • seldon-core-operator version is at least 1.5.0;

Deployment

• make helm_deploy - will ask one several questions (e.g. what is the MLFlow URL, which Neu.ro cluster should be considered, etc.). Alternatively, one might also set the following env vars:
  • M2S_MLFLOW_HOST - MLFlow server host name (example: https://mlflow--user.jobs.cluster.org.neu.ro)/;
  • M2S_MLFLOW_STORAGE_ROOT - artifact root path in the platform storage (storage:myproject/mlruns);
  • M2S_SELDON_NEURO_DEF_IMAGE - docker image, stored in a platform registry, which will be used to deploy the model (image:myproject/seldon:v1). Alternatively, one might configure service to use another platform image for deployment by tagging the respective registerred model (not a model version (!) ) with the tag named after M2S_MLFLOW_DEPLOY_IMG_TAG chart parameter value (for instance, with a tag named "deployment-image" and the value "image:myproject/seldon:v2);
  • M2S_SRC_NEURO_CLUSTER - Neu.ro cluster, where deployment image, MLflow artifacts and MLFlow itself are hosted (demo_cluster);
• Direct use of the helm chart is possible, however less comfortable - all requested by makefile info should be passed as chart values.

Cleanup

• make helm_delete - will delete:
  • all created by this helm chart resources, required for this service and the service itself;

Got questions or suggestions?

Feel free to contact us via 📧 or @ slack.

Maintained by Neu.ro MLOps team with ❤️