Example of batch inferenece

[abduld]

I am trying to find an example of performing batched inference. Currenyly, I am trying shufflenet (in the onnx model zoo), but I get the following

2019-09-27 11:12:07.268257103 [W:onnxruntime:Default, inference_session.cc:595 CheckShapes] Got invalid dimensions for input: gpu_0/data_0 for the following indices
 index: 0 Got: 2 Expected: 1
 Please fix either the inputs or the model.

• ONNX Runtime version (you are using): 0.5

Any help is greattly appreciated

[snnn]

The model doesn't support batch inference. Please try another model. You may pick one from:https://github.com/mlperf/inference/tree/master/v0.5/classification_and_detection

[hariharans29]

Hi @abduld ,

Expanding on the answer, if you open-up the model using Netron (or any other tool) and examine the graph inputs, usually a model that natively supports "batching" has symbolic dimensions in the graph input shape descriptions. For example, like this -

A model that doesn't usually has the input shape fixed like [1, 224, 224, 3].

Closing this issue as this is not an issue for ORT.

Pleas re-open for further clarifications.