Use Elastic Inference for GPU-powered clusters

[arun-gupta]

Why do you want this feature?

Elastic Inference adds GPU acceleration to any Amazon EC2 instance for faster inference at much lower cost (up to 75% savings). EKS customers could have extensive savings by provisioning a cluster using this instead of straight up GPUs.

Not sure, if this even made sense.

What feature/behavior/change do you want?

A GPU-powered cluster is created as:

   eksctl create cluster eks-gpu \
      --node-type=p3.8xlarge \
      --nodes=2 \
      --region=us-west-2 \
      --ssh-access \
      --ssh-public-key=arun-us-west2

A CLI using Elastic Inference could be:

   eksctl create cluster eks-gpu \
      --nodes=2 \
      --region=us-west-2 \
      --vpc-elastic-inference
      --ssh-public-key=arun-us-west2

Do not hesitate, when appropriate, to share the exact commands or API you would like, and/or to share a diagram (e.g.: asciiflow.com): "a picture is worth a thousand words".

[errordeveloper]

Could you please share the docs for CloudFormation along with reference docs?

[aflc]

+1
Here is a sample i found
https://github.com/aws-samples/amazon-elastic-inference-eks

[mumoshu]

Do we have anything other than the ones listed below to be addressed for this?

• Set ElasticInferenceAccelerators per node group (https://github.com/aws-samples/amazon-elastic-inference-eks/blob/475452ba42ebff295a482fb971012f8de21ac162/stack.cfn.yml#L795-L796
• Add a special permission named elastic-inference-connect to the node IAM policy (https://github.com/aws-samples/amazon-elastic-inference-eks/blob/475452ba42ebff295a482fb971012f8de21ac162/stack.cfn.yml#L598

[rbrigden]

+1 this is critical for economic inference workloads

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[github-actions]

This issue was closed because it has been stalled for 5 days with no activity.

[deliahu]

@michaelbeaumont I was just wondering if this feature is still on the roadmap, as supporting Elastic Inference is a common request among cortex users (we use eksctl to help configure users' clusters).

[RobertLucian]

@michaelbeaumont we're very interested in this feature. Any updates on this one?

[deliahu]

@mumoshu @errordeveloper I was just wondering if there were any updates on this. Thank you!