Can not load any network for model person-detection-retail-0013

[kafan1986]

System information (version)

• OpenVINO => 2021.1
• Operating System / Platform => Mac OS Catalina
• Compiler => Clang
• Problem classification: Model Loading
• Framework: Intel proprietary
• Model name: person-detection-retail-0013

Detailed description

Steps to reproduce

At the below line all three version of the model (fp32, fp16, fp16-int8) throw different error.

NOTE: Strange thing is almost similar code (in the sample directory object_detection_sample_ssd) works with the model files. Can't seem to figure the issue here.

openvino_ = ie.LoadNetwork(network, device_name);

fp32

libc++abi.dylib: terminating with uncaught exception of type InferenceEngine::details::InferenceEngineException: AssertionFailed: layer->get_output_partial_shape(i).is_static() nGraph Unsqueeze operation with name: mbox1/priorbox cannot be converted to Unsqueeze layer with name: mbox1/priorbox because output with index 0 contains dynamic shapes: ?. Try to use CNNNetwork::reshape() method in order to specialize shapes before the conversion.
/localdisk/jenkins/workspace/private-ci/ie/build-linux-macos/b/repos/openvino/inference-engine/include/details/ie_exception_conversion.hpp:64
Abort trap: 6

fp16

libc++abi.dylib: terminating with uncaught exception of type InferenceEngine::details::InferenceEngineException: Check 'element::Type::merge(element_type, element_type, node->get_input_element_type(i))' failed at ngraph/core/src/op/util/elementwise_args.cpp:36:
While validating node 'v1::Multiply data/norm/bn/mean/Fused_Mul_ (data[0]:f32{1,3,320,544}, data_mul_24443/copy_const[0]:f16{1,3,1,1}) -> (f16{1,3,320,544})' with friendly_name 'data/norm/bn/mean/Fused_Mul_':
Argument element types are inconsistent.

/localdisk/jenkins/workspace/private-ci/ie/build-linux-macos/b/repos/openvino/inference-engine/include/details/ie_exception_conversion.hpp:64
Abort trap: 6

fp16-int8

libc++abi.dylib: terminating with uncaught exception of type InferenceEngine::details::InferenceEngineException: Model file /Users/debasish/OpenCV/workspace/SecurityCamVision/data/models/fp16-int8-person-detection.xml cannot be opened!
/localdisk/jenkins/workspace/private-ci/ie/build-linux-macos/b/repos/openvino/inference-engine/src/inference_engine/ie_network_reader.cpp:172
Abort trap: 6

My Code:

int Openvino::Init(const char *root_path)
    {
        std::cout << "start Init Openvino." << std::endl;

        InferenceEngine::Core ie;

        std::string model_xml = std::string(root_path) + "/fp16-int8-person-detection.xml";
        std::string model_bin = std::string(root_path) + "/fp16-int8-person-detection.bin";

        InferenceEngine::CNNNetwork network = ie.ReadNetwork(model_xml, model_bin);

        InferenceEngine::InputsDataMap inputsInfo(network.getInputsInfo());

        if (inputsInfo.size() != 1) throw std::logic_error("Sample supports topologies only with 1 inputs");

        InferenceEngine::InputInfo::Ptr input_info = nullptr;

        /** Iterating over all input blobs **/
        for (auto & item : inputsInfo) {
            /** Working with first input tensor that stores image **/
            if (item.second->getInputData()->getTensorDesc().getDims().size() == 4) {
                input_name = item.first;
                input_info = item.second;
                
                input_info->getPreProcess().setResizeAlgorithm(InferenceEngine::RESIZE_BILINEAR);
                input_info->setLayout(InferenceEngine::Layout::NHWC);
                input_info->setPrecision(InferenceEngine::Precision::U8);
            }
        }

        if (input_info == nullptr) {
            input_info = inputsInfo.begin()->second;
        }

        InferenceEngine::OutputsDataMap outputsInfo(network.getOutputsInfo());

        InferenceEngine::DataPtr output_info;
        
        if (auto ngraphFunction = network.getFunction()) {
            for (const auto& out : outputsInfo) {
                for (const auto & op : ngraphFunction->get_ops()) {
                    if (op->get_type_info() == ngraph::op::DetectionOutput::type_info &&
                            op->get_friendly_name() == out.second->getName()) {
                        output_name = out.first;
                        output_info = out.second;
                        break;
                    }
                }
            }
        } else {
            output_info = outputsInfo.begin()->second;
            output_name = output_info->getName();
        }


        if (output_info == nullptr) {
            throw std::logic_error("Can't find a DetectionOutput layer in the topology");
        }

        const InferenceEngine::SizeVector outputDims = output_info->getTensorDesc().getDims();

        if (outputDims.size() != 4) {
            throw std::logic_error("Incorrect output dimensions for SSD model");
        }

        maxProposalCount = outputDims[2];
        objectSize = outputDims[3];
        
        if (objectSize != 7) {
            throw std::logic_error("Output item should have 7 as a last dimension");
        }
        
        output_info->setPrecision(InferenceEngine::Precision::FP32);
        // ***** Throws error below *******
        openvino_ = ie.LoadNetwork(network, device_name);
        initialized_ = true;
        std::cout << "end load model Openvino." << std::endl;

        return 0;
    }

[jgespino]

Hi @kafan1986

Could you try loading the model using the inference engine demo application? I was able to load the model to CPU and run inference with the following steps:

Source Environment

source /opt/intel/openvino_2021/bin/setupvars.sh
mkdir ~/test && cd ~/test

Download the model
python3 /opt/intel/openvino_2021/deployment_tools/tools/model_downloader/downloader.py --name person-detection-retail-0013

Build the demo application

cmake /opt/intel/openvino_2021/inference_engine/demos/
make object_detection_demo_ssd_async

Run application
./intel64/Release/object_detection_demo_ssd_async -m intel/person-detection-retail-0013/FP32/person-detection-retail-0013.xml -i people-detection.mp4

Test Environment
MacOS Catalina 10.15.5
cmake 3.16.0
python 3.7.9
Apple clang version 11.0.3

[kafan1986]

Hi @kafan1986

Could you try loading the model using the inference engine demo application? I was able to load the model to CPU and run inference with the following steps:

Source Environment

source /opt/intel/openvino_2021/bin/setupvars.sh
mkdir ~/test && cd ~/test

Download the model
python3 /opt/intel/openvino_2021/deployment_tools/tools/model_downloader/downloader.py --name person-detection-retail-0013

Build the demo application

cmake /opt/intel/openvino_2021/inference_engine/demos/
make object_detection_demo_ssd_async

Run application
./intel64/Release/object_detection_demo_ssd_async -m intel/person-detection-retail-0013/FP32/person-detection-retail-0013.xml -i people-detection.mp4

Test Environment
MacOS Catalina 10.15.5
cmake 3.16.0
python 3.7.9
Apple clang version 11.0.3

I told you in the original post the model files work fine with the sample application. Only when I use it inside my c++ app, it gives following error. Makes no sense but I have tried nearly everything.

[kafan1986]

@jgespino As mentioned earlier the official sample app works but not my own application. I have created a mini sample app that can reproduce the issue at my end. You will need to change the root_path in the main_application.cpp to the model file directory (already present inside the data folder).

Then run

cd build && cmake .. && make -j2

testapp.zip

[kafan1986]

@jgespino Did you get the time to test the above application? I am looking for some guidance towards solution.

[jgespino]

Hi @kafan1986

I haven't had a chance to deep dive into your code and debug. However, the error message recommends to use CNNNetwork::reshape() method in order to specialize shapes before the conversion. Briefly looking through your code, I did not see any reshape methods called to match the input shape.

Please take a look at the following documentation for additional information. https://docs.openvinotoolkit.org/latest/openvino_docs_IE_DG_ShapeInference.html#usage_of_reshape_method

Regards,
Jesus

[kafan1986]

Hi @kafan1986

I haven't had a chance to deep dive into your code and debug. However, the error message recommends to use CNNNetwork::reshape() method in order to specialize shapes before the conversion. Briefly looking through your code, I did not see any reshape methods called to match the input shape.

Please take a look at the following documentation for additional information. https://docs.openvinotoolkit.org/latest/openvino_docs_IE_DG_ShapeInference.html#usage_of_reshape_method

Regards,
Jesus

Hi @jgespino

https://github.com/openvinotoolkit/openvino/blob/master/inference-engine/samples/object_detection_sample_ssd/main.cpp
This is the official sample app and it works fine with the same model file. The code does not need any reshape method. Also, the other model are throwing different errors.

My code is exact same apart from few lines that I removed to not support the Resnet. Can you build the test app at your end and check if you get the error message too. May be it is my environment setup issue.

[kafan1986]

@jgespino After many hours of hair pulling, I finally figured out the issue. I am not sure if this is the intended behaviour of the project. The sample project or any other project that uses it, runs OK, only when build with "Release". You change it to "Debug", the build goes fine but while running it starts throwing different errors, giving an illusion, something is wrong with model/weight files.

May be this is a bug and should be corrected in future release.

To reproduce the issue, go the "build_samples.sh" and change

cmake -DCMAKE_BUILD_TYPE=Release "$SAMPLES_PATH"
to
cmake -DCMAKE_BUILD_TYPE=Debug "$SAMPLES_PATH"

and try to run the generated samples with model files

[jgespino]

Hi @kafan1986

I am glad you figured it out! I was able to reproduce the error with your testapp and by building the Inference Engine samples in debug. I have opened a bug with the development team to take a look.

Regards,
Jesus

Ref. 41509

[jgespino]

Hi @kafan1986

I apologize the delay in my response, the build_samples.sh script was not designed with 'make install' of source code in mind, it was created to make use of the OpenVINO toolkit release packages more convenient for users.

Regards,
Jesus