Managed API: NativeMethods.OrtSetIntraOpNumThreads Crashes Application

[mejas]

Describe the bug
C# Application crashes without exception when setting SessionOptions.IntraOpNumThreads. This does not happen when building from source.

Urgency
High. This used to work back in 1.2.0

System information

• Windows 10 1803
• ONNX Runtime installed from (source or binary): nuget package
• ONNX Runtime version: 1.3.0
• Python version: 3.7.8
• Visual Studio version (if applicable): 16.7 Preview 1.0
• GCC/Compiler version (if compiling from source): Msbuild 16.7.0.22806
• CUDA/cuDNN version: N/A
• GPU model and memory: Intel HD Graphics

To Reproduce

• Initialize InferenceSession object
• Set IntraOpNumThreads to 1
• Application crashes

Expected behavior
Application should not crash and create the InferenceSession object.

Additional context
This happens even when you don't have an ONNX model. See repro steps for procedure.

[snnn]

The official package has openmp. When you build from source by yourself I guess you didn't append "--use_openmp" to the build script?

[mejas]

The official package has openmp. When you build from source by yourself I guess you didn't append "--use_openmp" to the build script?

I did not append the OpenMP flag to the build script. Is this now a requirement for the runtime?

I used the following for building from source:
.\build.bat --config Release --build_shared_lib --parallel --build-csharp

I do find it odd that 1.2.0 did not have this problem. Upgrading to 1.3.0 was when the issue started to happen.

A little further digging around: using the 1.2.0 onnxruntime.dll does not cause the problem as well.

[snnn]

I was trying to explain the difference, why "This does not happen when building from source.".

If you can build it from source with

.\build.bat --config RelWithDebInfo --build_shared_lib --parallel --build-csharp --use_openmp

And repro the bug with newly built binary, it would be easier for us to know why it crashed.

Is this now a requirement for the runtime?

No. It is totally fine to live without it.

[mejas]

Thanks for the recommendation. I built from source with the switches you suggested and was able to replicate the crashing behavior.

Exact switch used:
.\build.bat --config RelWithDebInfo --build_shared_lib --parallel --build_csharp --use_openmp --cmake_generator "Visual Studio 16 2019"

Oddly enough there's no exceptions happening and just an explicit crash, but it does look like OpenMP is the culprit.

[pranavsharma]

I used this sample code https://gist.github.com/pranavsharma/66186e986101022a7bb43201cd1e685e and tried it on my machine using the published pkg on Nuget and couldn't replicate. Here's my csproj file https://gist.github.com/pranavsharma/4d7d0b3964e0386295ed6362636452cf and I'm using

Microsoft Visual Studio Community 2019
Version 16.6.0
VisualStudio.16.Release/16.6.0+30114.105
Microsoft .NET Framework
Version 4.8.03752

[mejas]

Thanks @pranavsharma
I noticed that line 30 sets InterOpNumThreads and not IntraOpNumThreads
Could you try setting IntraOpNumThreads to 1?

[pranavsharma]

Yes, I can repro.

[mejas]

Just to help a bit, I tried debugging deeper by enabling native code debugging in Visual Studio and found the following in the stack before the error was thrown:

The file in the stack is abi_session_options.cc

[pranavsharma]

Btw, when openmp is enabled calling this method has no effect. So if this is blocking you, you can safely comment out the setting of options.IntraOpNumThreads in your C# code. The number of intra op threads are controlled using the openmp settings when openmp is enabled. See release notes https://github.com/microsoft/onnxruntime/releases/tag/v1.3.0.

[mejas]

Thanks. I have commented out the code and proceeded with this as a temporary workaround.