RFVE: Support GPU share

[k82cn]

Is this a BUG REPORT or FEATURE REQUEST?:

/kind feature
/priority important-soon
/area scheduling

Description:

There're several scenarios require GPU share, e.g. inference workload, dev environment; it's better to support that by default.

[k82cn]

/cc @carmark @Jeffwan

[Rui-Tang]

Hi, I found two relevent projects.
https://github.com/AliyunContainerService/gpushare-device-plugin
https://github.com/AliyunContainerService/gpushare-scheduler-extender

[Rui-Tang]

These two projects share GPU by sharing GPU memory

[k82cn]

I known those two projects, that'll be great if we can build gpushare-scheduler-extender as a plugin of Volcano :)

/cc @cheyang

[k82cn]

and that should be fine to import the code from that repo, similar to what we have done to the algorithm of default scheduler :)

[k82cn]

/kind rfve

[Jeffwan]

@k82cn We will probably release a simple vGPU device plugin later. The GPU share idea is pretty straightforward, map physical GPU to virtual GPUs and advertise custom resources to APIServer. Ali's solution need extra extender to handle complex case for multi-GPU nodes. For us, I think we probably ask user only use it on single chip node which simplify the case a lot. I will collect more feedbacks and share here.

I would say for some of the cloud providers who doesn't allow users to change master scheduler configurations, extender support in volcano (secondary scheduler) will help

[k82cn]

I would say for some of the cloud providers who doesn't allow users to change master scheduler configurations, extender support in volcano (secondary scheduler) will help

Are you going to use volcano as default scheduler in your cluster? I'm ok to support extender in volcano :)

[k82cn]

We will probably release a simple vGPU device plugin later

Will you open source it? If so, we'd like to leverage it in volcano :)

[Jeffwan]

I would say for some of the cloud providers who doesn't allow users to change master scheduler configurations, extender support in volcano (secondary scheduler) will help

Are you going to use volcano as default scheduler in your cluster? I'm ok to support extender in volcano :)

I mean on AWS, users can not use scheduler extender since user can not touch default scheduler configs. If volcano provides some scheduling support for custom resources, these users can adopt it as an alternative.

We will probably release a simple vGPU device plugin later

Will you open source it? If so, we'd like to leverage it in volcano :)

It's in internal discussion and I will try to move it forward asap. It's similar like gpushare-device-plugin but simpler. What I understand is volcano aims to provides a generic scheduling solution for these kinds of custom resources? Using gpushare-scheduler-extender as an example, it does some bin pack and filter some cases that virtual GPU resources are across two different physical GPUs.

[k82cn]

I would say for some of the cloud providers who doesn't allow users to change master scheduler configurations, extender support in volcano (secondary scheduler) will help

Are you going to use volcano as default scheduler in your cluster? I'm ok to support extender in volcano :)

I mean on AWS, users can not use scheduler extender since user can not touch default scheduler configs. If volcano provides some scheduling support for custom resources, these users can adopt it as an alternative.

Great! We do need to support gpu share, and that'll be great to be an alternative for your serivce :)

We will probably release a simple vGPU device plugin later

Will you open source it? If so, we'd like to leverage it in volcano :)

It's in internal discussion and I will try to move it forward asap. It's similar like gpushare-device-plugin but simpler. What I understand is volcano aims to provides a generic scheduling solution for these kinds of custom resources? Using gpushare-scheduler-extender as an example, it does some bin pack and filter some cases that virtual GPU resources are across two different physical GPUs.

Vocano aimt to be a batch system, similar to Slurm/YARN; we will include more components in this project, including device-plugin if necessary. As we'd like to provides an e2e solution for batch workload :)

[k82cn]

There's a really long discussion in upstream: kubernetes/kubernetes#52757

[Jeffwan]

That's true. Our solution is based on kubernetes/kubernetes#52757 (comment)

[xiaogaozi]

kubernetes/kubernetes#52757 (comment) provide a new solution from Tencent Cloud team

[stale]

Hello 👋 Looks like there was no activity on this issue for last 90 days.
Do you mind updating us on the status? Is this still reproducible or needed? If yes, just comment on this PR or push a commit. Thanks! 🤗
If there will be no activity for 60 days, this issue will be closed (we can always reopen an issue if we need!).

[stale]

Closing for now as there was no activity for last 60 days after marked as stale, let us know if you need this to be reopened! 🤗