Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ldd-link error when building with dub against MKL-Intel #6

Closed
tastyminerals opened this issue Mar 4, 2020 · 17 comments
Closed

ldd-link error when building with dub against MKL-Intel #6

tastyminerals opened this issue Mar 4, 2020 · 17 comments

Comments

@tastyminerals
Copy link

tastyminerals commented Mar 4, 2020

I would like to use gemm function from mir-blas in the dub project.
Downloaded and installed Intel MKL library for Windows: https://software.intel.com/en-us/mkl/choose-download/windows

dub.json

"dependencies": {
	"mir-algorithm": "~>3.7.18",
	"mir-random": "~>2.2.11",
	"mir-blas": "~>1.1.9"
},

dub.sdl

name "d_benchmarks"

subConfiguration "mir-blas" "mkl-sequential"

lflags "/LIBPATH:C:\Program Files (x86)\IntelSWTools\compilers_and_libraries\windows\mkl\lib\intel64" platform="windows-x86_64-ldc"
lflags "/LIBPATH:C:\Program Files (x86)\IntelSWTools\compilers_and_libraries\windows\mkl\lib\ia32" platform="windows-x86-ldc"

But running $ dub build --compiler=ldc2 throws the following error:

mir_benchmarks>dub build --compiler=ldc2
Performing "debug" build using ldc2 for x86_64.
mir-core 1.1.0: target for configuration "library" is up to date.
mir-algorithm 3.7.18: target for configuration "default" is up to date.
mir-blas 1.1.9: target for configuration "library" is up to date.
mir-linux-kernel 1.0.1: target for configuration "library" is up to date.
mir-random 2.2.11: target for configuration "default" is up to date.
app ~master: building configuration "application"...
blasint = long
lld-link: error: undefined symbol: __security_cookie
>>> referenced by mkl_intel_ilp64.lib(_dgemm.obj):(dgemm)
>>> referenced by mkl_intel_ilp64.lib(_dgemm.obj):(dgemm)
>>> referenced by mkl_intel_ilp64.lib(_dgemm.obj):(dgemm)
>>> referenced by mkl_intel_ilp64.lib(_dgemm.obj):(dgemm)
>>> referenced by mkl_core.lib(mkl_libc.obj):(mkl_serv_printf_s)        
>>> referenced by mkl_core.lib(mkl_libc.obj):(mkl_serv_printf_s)        
>>> referenced by mkl_core.lib(mkl_libc.obj):(mkl_serv_fprintf_s)       
>>> referenced by mkl_core.lib(mkl_libc.obj):(mkl_serv_fprintf_s)       
>>> referenced by mkl_core.lib(mkl_libc.obj):(mkl_serv_fprintf_stdout_s)
>>> referenced by mkl_core.lib(mkl_libc.obj):(mkl_serv_fprintf_stdout_s)
>>> referenced 255 more times

lld-link: error: undefined symbol: __security_check_cookie
>>> referenced by mkl_intel_ilp64.lib(_dgemm.obj):(dgemm)
>>> referenced by mkl_intel_ilp64.lib(_dgemm.obj):(dgemm)
>>> referenced by mkl_core.lib(mkl_libc.obj):(mkl_serv_printf_s)
>>> referenced by mkl_core.lib(mkl_libc.obj):(mkl_serv_fprintf_s)
>>> referenced by mkl_core.lib(mkl_libc.obj):(mkl_serv_fprintf_stdout_s)
>>> referenced by mkl_core.lib(mkl_libc.obj):(mkl_serv_vfprintf_s)
>>> referenced by mkl_core.lib(mkl_libc.obj):(mkl_serv_vfprintf_stdout_s)
>>> referenced by mkl_core.lib(mkl_libc.obj):(mkl_serv_sprintf_s)
>>> referenced by mkl_core.lib(mkl_libc.obj):(mkl_serv_vsprintf_s)
>>> referenced by mkl_core.lib(mkl_libc.obj):(mkl_serv_vsnprintf_s)
>>> referenced 92 more times

lld-link: error: undefined symbol: __GSHandlerCheck
>>> referenced by mkl_intel_ilp64.lib(_dgemm.obj):(.xdata)
>>> referenced by mkl_intel_ilp64.lib(_dgemm.obj):(.xdata)
>>> referenced by mkl_core.lib(mkl_libc.obj):(.xdata)
>>> referenced by mkl_core.lib(mkl_libc.obj):(.xdata)
>>> referenced by mkl_core.lib(mkl_libc.obj):(.xdata)
>>> referenced by mkl_core.lib(mkl_libc.obj):(.xdata)
>>> referenced by mkl_core.lib(mkl_libc.obj):(.xdata)
>>> referenced by mkl_core.lib(mkl_libc.obj):(.xdata)
>>> referenced by mkl_core.lib(mkl_libc.obj):(.xdata)
>>> referenced by mkl_core.lib(mkl_libc.obj):(.xdata)
>>> referenced 94 more times
Error: linking with LLD failed
ldc2 failed with exit code 1.

Am I doing everything correctly?

@9il
Copy link
Member

9il commented Mar 4, 2020

Nope, it looks like a bug. Probably mir-blas dub.sdl needs an update for intel-mkl for linux.

@tastyminerals
Copy link
Author

Nope, it looks like a bug. Probably mir-blas dub.sdl needs an update for intel-mkl for linux.

This is Windows machine. I haven't tried it on Linux.

@9il
Copy link
Member

9il commented Mar 5, 2020

Ah, interesting. Why do you have both dub.sdl and dub.json?

@tastyminerals
Copy link
Author

tastyminerals commented Mar 5, 2020

Because it is not said you cannot have them both and dub.sdl it is given as example on mir-blas github.
I assumed dub.sdl is a config you need to add in order to link with MKL on Windows.

@9il
Copy link
Member

9il commented Mar 6, 2020

dub will use either dub.sdl or dub.json. First, place configs in a single file.
Then check that you have installed intel mkl from the off cite.
Then try to remove lflags and other subConfiguration.

"dependencies": {
	"mir-random": "~>2.2.11",
	"mir-blas": "~>1.1.9"
}

It should work on Windows without any other additional configs if MKL is installed because MKL is the default in Windows configs and we use mir-blas on windows in production every day.

@tastyminerals
Copy link
Author

tastyminerals commented Mar 7, 2020

Sorry, reopening because I mistakenly uncommented import mir.blas : gemm and thought that modified dub.json fixes the linking. It is not.

I removed dub.sdl and moved everything to dub.json. Tried removing the lflags and subConfiguration from dub.json, doesn't help.
The MKL was downloaded and installed from official intel site.

Capture

dub.json

{
	"name": "app",
	"targetType": "executable",
	"dependencies": {
		"mir-algorithm": "~>3.7.18",
		"mir-random": "~>2.2.11",
		"mir-blas": "~>1.1.9"
	},
	"dflags-ldc": ["-mcpu=native"],
	"subConfigurations": {"mir-blas": "mkl-sequential"},
	"buildTypes": {
		"release": {
			"buildOptions": ["releaseMode", "inline", "optimize"],
			"dflags": ["-boundscheck=off"]
		},
		"debug": {
			"buildOptions": ["debugMode", "debugInfo", "optimize"]
		},
		"debug-profile": {
			"buildOptions": ["debugMode", "debugInfo", "profile"]
		},
		"tests": {
			"buildOptions": ["unittests"]
		}

	}
}

Finally, here is the project repo with the latest commits: https://github.com/tastyminerals/mir_benchmarks

@tastyminerals
Copy link
Author

tastyminerals commented Mar 7, 2020

The sparse gemm version from mir.sparse.blas works though

void main(){
    import mir.sparse.blas: gemm
    auto sp = sparse!int(3, 5);
    sp[] = [[-5, 1, 7, 7, -4], [-1, -5, 6, 3, -3], [-5, -2, -3, 6, 0]];

    auto a = sp.compress;

    auto b = slice!double(5, 4);
    b[] = [
        [-5.0, -3, 3, 1], [4.0, 3, 6, 4], [-4.0, -2, -2, 2], [-1.0, 9, 4, 8],
        [9.0, 8, 3, -2]
    ];

    auto c = slice!double(3, 4);
    gemm(1.0, a, b, 0, c); // works fine!
}

@9il
Copy link
Member

9il commented Mar 8, 2020

try remove "subConfigurations": {"mir-blas": "mkl-sequential"}, or replace it with "subConfigurations": {"mir-blas": "mkl-sequential-ilp"},

@tastyminerals
Copy link
Author

Tried removing subConfigurations and replacing with "mkl-sequential-ilp", same result :(

Performing "debug" build using ldc2 for x86_64.
mir-core 1.1.0: target for configuration "library" is up to date.
mir-algorithm 3.7.18: target for configuration "default" is up to date.
mir-linux-kernel 1.0.1: target for configuration "library" is up to date.
mir-random 2.2.11: target for configuration "extended" is up to date.
mir-blas 1.1.9: building configuration "mkl-sequential-ilp"...
blasint = long
app ~master: building configuration "application"...
blasint = long
lld-link: error: undefined symbol: __security_cookie
>>> referenced by mkl_intel_ilp64.lib(_dgemm.obj):(dgemm)
>>> referenced by mkl_intel_ilp64.lib(_dgemm.obj):(dgemm)
>>> referenced by mkl_intel_ilp64.lib(_dgemm.obj):(dgemm)
>>> referenced by mkl_intel_ilp64.lib(_dgemm.obj):(dgemm)
>>> referenced by mkl_core.lib(mkl_libc.obj):(mkl_serv_printf_s)
>>> referenced by mkl_core.lib(mkl_libc.obj):(mkl_serv_printf_s)
>>> referenced by mkl_core.lib(mkl_libc.obj):(mkl_serv_fprintf_s)
>>> referenced by mkl_core.lib(mkl_libc.obj):(mkl_serv_fprintf_s)
>>> referenced by mkl_core.lib(mkl_libc.obj):(mkl_serv_fprintf_stdout_s)
>>> referenced by mkl_core.lib(mkl_libc.obj):(mkl_serv_fprintf_stdout_s)
>>> referenced 255 more times

lld-link: error: undefined symbol: __security_check_cookie
>>> referenced by mkl_intel_ilp64.lib(_dgemm.obj):(dgemm)
>>> referenced by mkl_intel_ilp64.lib(_dgemm.obj):(dgemm)
>>> referenced by mkl_core.lib(mkl_libc.obj):(mkl_serv_printf_s)
>>> referenced by mkl_core.lib(mkl_libc.obj):(mkl_serv_fprintf_s)
>>> referenced by mkl_core.lib(mkl_libc.obj):(mkl_serv_fprintf_stdout_s)
>>> referenced by mkl_core.lib(mkl_libc.obj):(mkl_serv_vfprintf_s)
>>> referenced by mkl_core.lib(mkl_libc.obj):(mkl_serv_vfprintf_stdout_s)
>>> referenced by mkl_core.lib(mkl_libc.obj):(mkl_serv_sprintf_s)
>>> referenced by mkl_core.lib(mkl_libc.obj):(mkl_serv_vsprintf_s)
>>> referenced by mkl_core.lib(mkl_libc.obj):(mkl_serv_vsnprintf_s)
>>> referenced 92 more times

lld-link: error: undefined symbol: __GSHandlerCheck
>>> referenced by mkl_intel_ilp64.lib(_dgemm.obj):(.xdata)
>>> referenced by mkl_intel_ilp64.lib(_dgemm.obj):(.xdata)
>>> referenced by mkl_core.lib(mkl_libc.obj):(.xdata)
>>> referenced by mkl_core.lib(mkl_libc.obj):(.xdata)
>>> referenced by mkl_core.lib(mkl_libc.obj):(.xdata)
>>> referenced by mkl_core.lib(mkl_libc.obj):(.xdata)
>>> referenced by mkl_core.lib(mkl_libc.obj):(.xdata)
>>> referenced by mkl_core.lib(mkl_libc.obj):(.xdata)
>>> referenced by mkl_core.lib(mkl_libc.obj):(.xdata)
>>> referenced by mkl_core.lib(mkl_libc.obj):(.xdata)
>>> referenced 94 more times
Error: linking with LLD failed
ldc2 failed with exit code 1.

@tastyminerals
Copy link
Author

tastyminerals commented Mar 8, 2020

I have found the following section in MKL user docs: https://jp.xlsoft.com/documents/intel/mkl/11.0/mkl_userguide_win.pdf

Linking with System Libraries
If your system is based on the Intel® 64 architecture, be aware that Microsoft SDK builds 1289 or higher provide the bufferoverflowu.lib library to resolve the __security_cookie external references.
Makefiles for examples and tests include this library by using the buf_lib=bufferoverflowu.lib macro. If you are using older SDKs, leave this macro empty on your command line as follows: buf_lib= .

But I think it is not directly related to mir blas though...

@9il
Copy link
Member

9il commented Mar 9, 2020

Interesting.
What VS version and LDC version are used?
Does linking withbufferoverlowu.libor passing /GS or /GS- to the linker help?

@tastyminerals
Copy link
Author

tastyminerals commented Mar 31, 2020

Sorry for the late reply.

VS version: VisualStudio.16.Release/16.4.5+29806.167
LDC version: 1.20.0

I tried adding "lflags" : ["/GS-"] but that doesn't work.

lld-link: error: could not open '/GS': no such file or directory

What, worked is linking to BufferOverflowU.lib

"lflags": ["C:\\Program Files (x86)\\Windows Kits\\10\\Lib\\10.0.18362.0\\um\\x64\\BufferOverflowU.lib"],

However, there is one last remaining exception:

PS C:\Devel\mir_benchmarks> dub build --compiler=ldc2
The sub configuration directive "comment" -> "Also try MKL like 'mkl-tbb-thread'" references a package that is not specified as a dependency and
 will have no effect.
Performing "debug" build using ldc2 for x86_64.
mir-core 1.1.2: target for configuration "library" is up to date.
mir-algorithm 3.7.27: target for configuration "default" is up to date.
mir-blas 1.1.13: target for configuration "library" is up to date.
mir-random 2.2.14: target for configuration "default" is up to date.
app ~master: building configuration "application"...
blasint = long
lld-link: error: undefined symbol: __GSHandlerCheck
>>> referenced by mkl_intel_ilp64.lib(_ddot.obj):(.xdata)
>>> referenced by mkl_intel_ilp64.lib(_ddot.obj):(.xdata)
>>> referenced by mkl_intel_ilp64.lib(_ddot.obj):(.xdata)
>>> referenced by mkl_intel_ilp64.lib(_ddot.obj):(.xdata)
>>> referenced by mkl_intel_ilp64.lib(_ddot.obj):(.xdata)
>>> referenced by mkl_intel_ilp64.lib(_ddot.obj):(.xdata)
>>> referenced by mkl_intel_ilp64.lib(_dgemm.obj):(.xdata)
>>> referenced by mkl_intel_ilp64.lib(_dgemm.obj):(.xdata)
>>> referenced by mkl_core.lib(mkl_libc.obj):(.xdata)
>>> referenced by mkl_core.lib(mkl_libc.obj):(.xdata)
>>> referenced 100 more times
Error: linking with LLD failed
ldc2 failed with exit code 1.

@9il
Copy link
Member

9il commented Apr 1, 2020

Maybe the following can help

From: https://social.msdn.microsoft.com/Forums/en-US/3abba38c-0cb0-4d1c-9b09-e945fda04205/unresolved-external-symbol-gshandlercheck?forum=vssmartdevicesnative

You are doing debug build, so you should use debug versions of the libraries. Have you tried

libcmtd.lib

instead of libcmt.lib?

In my case, the debug build worked fine, so I didn't worry about that. In release build I had to turn the buffer checking off (which seems to use the GSHandlerCheck functions). I can live without that in release version.

/GS-

C/C++ -> Code Generation -> Buffer Security Check = NO

But in debug build it's usefull, so take this as "only for your information".

Best luck.

MAN

@tastyminerals
Copy link
Author

So, linking the libcmt.lib or libcmtd.lib forces you to link two dependent libs as well.
Here are all the linked libs in dub.sdl:

lflags `C:\Program Files (x86)\Windows Kits\10\Lib\10.0.18362.0\um\x64\BufferOverflowU.lib` platform="windows-x86_64"
lflags `C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.25.28610\lib\x64\libcmt.lib` platform="windows-x86_64"
lflags `C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.25.28610\lib\x64\libvcruntime.lib` platform="windows-x86_64"
lflags `C:\Program Files (x86)\Windows Kits\10\Lib\10.0.18362.0\ucrt\x64\libucrt.lib` platform="windows-x86_64"

Now, when I try to compile the project I get:

C:\Devel\mir_benchmarks>dub build --compiler=ldc2 --build=release
Performing "release" build using ldc2 for x86_64.
mir-core 1.1.2: building configuration "library"...
mir-algorithm 3.7.28: building configuration "default"...
mir-blas 1.1.13: building configuration "mkl-tbb-thread-dll"...
mir-random 2.2.14: building configuration "default"...
mir_benchmarks ~master: building configuration "application"...
lld-link: error: undefined symbol: open
>>> referenced by phobos2-ldc.lib(gzlib.c.obj):(gz_open)

lld-link: error: undefined symbol: fileno
>>> referenced by druntime-ldc.lib(dmain2.obj):(_d_print_throwable)
>>> referenced by druntime-ldc.lib(cover.obj):(_D2rt5cover25_sharedStaticDtor_L188_C1FZv)
>>> referenced by druntime-ldc.lib(cover.obj):(_D2rt5cover25_sharedStaticDtor_L188_C1FZv)

lld-link: error: undefined symbol: execv
>>> referenced by phobos2-ldc.lib(process.obj):(_D3std7process6execv_FxAyaxAQfZi)

lld-link: error: undefined symbol: execve
>>> referenced by phobos2-ldc.lib(process.obj):(_D3std7process7execve_FxAyaxAQfxQeZi)

lld-link: error: undefined symbol: execvp
>>> referenced by phobos2-ldc.lib(process.obj):(_D3std7process7execvp_FxAyaxAQfZi)

lld-link: error: undefined symbol: execvpe
>>> referenced by phobos2-ldc.lib(process.obj):(_D3std7process8execvpe_FxAyaxAQfxQeZi)

lld-link: error: undefined symbol: tzset
>>> referenced by phobos2-ldc.lib(timezone.obj):(_D3std8datetime8timezone9LocalTime9singletonFNeZ12__dgliteral4MFNaNbNiNfZOb)
>>> referenced by phobos2-ldc.lib(timezone.obj):(_D3std8datetime8timezone9LocalTime9singletonFNeZ9__lambda3FNbNiNfZb)
Error: linking with LLD failed
ldc2 failed with exit code 1.

@9il
Copy link
Member

9il commented Apr 14, 2020

Asked LDC team ldc-developers/ldc#3402,
BTW, @tastyminerals, I did s small update for your blogpost tastyminerals/tasty-blog#3 ?

@kinke
Copy link

kinke commented Apr 14, 2020

See LDC's README.txt for how to use the MS toolchain, which is the simplest solution when linking other C[++] libs compiled without /GS- /Zl and should fix the linker errors in the original post.

@tastyminerals
Copy link
Author

tastyminerals commented Apr 25, 2020

Thank you. I didn't have to modify ldc2.conf. Everything completes without issues if I build in a dedicated cmd x86_x64 Cross Tools Command Prompt for VS 2019. Also, no need to explicitly link BufferOverflowU.lib.

C:\Devel\mir_benchmarks>dub build --compiler=ldc2
Performing "debug" build using ldc2 for x86_64.
mir-core 1.1.2: target for configuration "library" is up to date.
mir-algorithm 3.7.28: target for configuration "default" is up to date.
mir-blas 1.1.13: target for configuration "mkl-sequential-ilp" is up to date.
mir-random 2.2.14: target for configuration "default" is up to date.
mir_benchmarks ~master: building configuration "application"...
blasint = long
To force a rebuild of up-to-date targets, run again with --force.

C:\Devel\mir_benchmarks>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants