[Bug]: Accelerator enablement in kserve is not working

[lucferbux]

Is there an existing issue for this?

• I have searched the existing issues

Deploy type

OpenDataHub core version (eg. v1.6.0)

Version

2.5.0

Current Behavior

We are currently assigning gpu resources to all the containers in a ServingRuntime spec for KServe, this has been ok for Modelmesh but it's creating an issue where we add more resources than the cluster might have.

Expected Behavior

The outcome will be the following:

For modelmesh

Keep the same flow and get the creation of InferenceServices and ServingRuntimes as it is right now

For kserve

1. Remove the assignation of tolerations and GPU resources in the containers of the serving runtime, for that add a conditional and just add them for modelmesh.
2. Add that logic for inferenceservices:
   • Add tolerations in spec.predictor.tolerations such as this example
   • Add the GPU resrouces in the spec.predictor-model-resources section

Steps To Reproduce

1. Creaate a new project with accelerator
2. Deploy a kserve model
3. Select the maximum number of accelerator nodes

Workaround (if any)

No response

What browsers are you seeing the problem on?

No response

Anything else

No response