How to use batchsize in onnxruntime?

[synsis]

Describe the bug
I read the sample of imagenet and the Q&A(#1632), but I still do not know how to use batchsize.
I think the sample means that the session itself do not care the shape of the input, just set the input_tensor the proper shape with the batchsize, and run the session, like the code below:
auto output_tensors = session.Run(Ort::RunOptions{nullptr}, input_node_names.data(), &input_tensor, 1, output_node_names.data(), 2);

But, it doesn't work for me in the current version. It gives me some errors like:
terminate called after throwing an instance of 'Ort::Exception'
what(): Got invalid dimensions for input: data for the following indices
index: 0 Got: 4 Expected: 1
Please fix either the inputs or the model.

I deleted the check code and re-make the whole project.
But it still doesn't work. In output tensor, it just gives me random result except the first data in a batchsize.

System information

• OS Platform and Distribution (e.g., Linux Ubuntu 18.04):
• ONNX Runtime installed from (source or binary): source
• ONNX Runtime version:1.5.2
• GCC/Compiler version (if compiling from source): g++ 7.5.0
• CUDA/cuDNN version:10.2
• GPU model and memory: 1660Ti 6G

[hariharans29]

"I think the sample means that the session itself do not care the shape of the input, just set the input_tensor the proper shape with the batchsize," - That is true but your model should also support it. In this case, your model does not - it is not flexible in its requirement - it expects batch size to be 1 and you provided 4.

Usually the model input has a "dynamic" shape for the batch dimension (see #2118 #1944). I think some frameworks (Torch for example) support exporting the model with "dynamic" input shape requirements that can be used for batched inferencing.