Multi-thread support for inference

[sunnyxiaohu]

Hi, Sorry to disturbing you. There are two questions always confusing me.
Q1:
I was wondering if this version of Caffe support multi-thread when inference ? In other words, we just need to load the model once, and could use multiple threads to predict the results of input.
Q2:
I also searched a modified version of Caffe with links https://github.com/flx42/caffe.git and https://github.com/NVIDIA/gpu-rest-engine , which claimed that it could be used to properly handle multithreading. However, when testing, we found that it is not real multi-thread, which adopts serialization handling. How about your ideal ?

Sincerely.