Consider removing dynlib library loading from the language

[alaviss]

{.dynlib: "library_regex".} attempts to recreate dynamic library loading in the language itself.

This have some issues:

• Since the library is dynamically loaded by the program itself, it does not show up in binary inspection tools like readelf or ldd.
• Errors related to not finding a library are not internationalized as well as not having sufficient UI integration (ie. a helpful dialog on Windows).
• Many users specify a range of SONAME supported or use the "generic" version for linking (ie. .so instead of .so.3.0). This risks hidden errors due to mismatched ABI between the user and the library.

Suggestion: Remove {.dynlib: "library".} and rely on link-time linking. This would enable the OS and tools to see exactly what libraries are required at runtime and use their native dynamic linker which would integrate much better with the OS.

We would need an alternative to specify the libraries required at link-time. While {.dynlib.} could be repurposed, using native linking means a static library could be linked as well, making the name misleading.

[haxscramper]

Yes, absolutely. There is no reason for a statically compiled language to try and match what is done by the interpreted languages like python -- some wrappers around dynamic loading can be exposed in the standard library, but that's the extent it makes sense to rely on this feature.

It is already possible to work with the wrapped libraries without relying on this mis-feature and for simpler (aka 99.99%) cases they are transparently swappable.

nimskull/lib/wrappers/openssl.nim

Lines 313 to 349 in 4c0cd14

	# Here we're trying to stay compatible with openssl 1.0.* and 1.1.*. Some
	# symbols are loaded dynamically and we don't use them if not found.
	proc thisModule(): LibHandle {.inline.} =
	var thisMod {.global.}: LibHandle
	if thisMod.isNil: thisMod = loadLib()
	result = thisMod
	proc sslModule(): LibHandle {.inline.} =
	var sslMod {.global.}: LibHandle
	if sslMod.isNil: sslMod = loadLibPattern(DLLSSLName)
	result = sslMod
	proc utilModule(): LibHandle {.inline.} =
	var utilMod {.global.}: LibHandle
	if utilMod.isNil: utilMod = loadLibPattern(DLLUtilName)
	result = utilMod
	proc symNullable(dll: LibHandle, name: string, alternativeName = ""): pointer =
	# Load from DLL.
	if not dll.isNil:
	result = symAddr(dll, name)
	if result.isNil and alternativeName.len > 0:
	result = symAddr(dll, alternativeName)
	# Attempt to load from current exe.
	if result.isNil:
	let thisDynlib = thisModule()
	if thisDynlib.isNil: return nil
	result = symAddr(thisDynlib, name)
	if result.isNil and alternativeName.len > 0:
	result = symAddr(thisDynlib, alternativeName)
	proc sslSymNullable(name: string, alternativeName = ""): pointer =
	sslModule().symNullable(name, alternativeName)

-- I'm not an SSL user, but maintaining things like that is probably overly tedious, as a wrapper generator it is impossible to provide in a generic manner either.

[alaviss]

as a wrapper generator it is impossible to provide in a generic manner either

Yea, OpenSSL is one of those libraries that you need to know what version are you dealing with because of their tendencies to break ABI. Even in Rust they had to resort to generating a wrapper whenever the library is used.

Ideally we have a build tool like meson that can get us the version to configure our wrapper automatically, but that stuff doesn't exist yet.

[disruptek]

IIUC, this design choice is motivated by making the semantics of the posix platforms more similar to those of windows.

[alaviss]

Funny how today's Windows uses import libraries for dlls and is equivalent to how *nix does it.

Sans the library search path, but that's not something we try to abstract anywhere in the language.

[saem]

Ultimately compile time is, or at the very least overlaps with, build time for a project, as described by the project file and friends. Even if it's just overlapping they're going to need to inform each other. So at some point we'd likely need a facility remotely similar to this.

The interface, and likely the implementation, have issues. So I'm fine dropping those for now, as they probably need a do over.

[mrgaturus]

i think removing {.dynlib.} also would make ffi proc importing more simple because nimskull generates runtime initialization and name mangling of each imported proc through {.dynlib.}
for example on x11 module:

N_LIB_PRIVATE N_NIMCALL(void, atmdotdotatsdotdotatsdotdotatsdotdotatsdotnimbleatspkgs2atsx11minus1dot1minus6b41f60091959ada084a92bb7e96e74806f4bffdatsx11atsxlibdotnim_DatInit000)(void) {
if (!((TM__U7j10wf2hJXgu05OSHoOmw_2 = nimLoadLibrary(TM__U7j10wf2hJXgu05OSHoOmw_5))
||(TM__U7j10wf2hJXgu05OSHoOmw_2 = nimLoadLibrary(TM__U7j10wf2hJXgu05OSHoOmw_8))
)) nimLoadLibraryError(TM__U7j10wf2hJXgu05OSHoOmw_10);
	Dl_1862273570_ = (tyProc__B3UvMsa15lHlKTm6yxGhLA) nimGetProcAddr(TM__U7j10wf2hJXgu05OSHoOmw_2, "XUndefineCursor");
	Dl_1862272927_ = (tyProc__tBASvQWQKftWieQirE7ctw) nimGetProcAddr(TM__U7j10wf2hJXgu05OSHoOmw_2, "XFreeCursor");
	Dl_1862272710_ = (tyProc__iWKKtMndGlZOYfb3DEdQHw) nimGetProcAddr(TM__U7j10wf2hJXgu05OSHoOmw_2, "XDefineCursor");
	Dl_1862272190_ = (tyProc__k5R87gmgtW6CNXnc9an6XOg) nimGetProcAddr(TM__U7j10wf2hJXgu05OSHoOmw_2, "XOpenDisplay");
	Dl_1862272306_ = (tyProc__sVIjc8kXTEUfP6NOf0LRaQ) nimGetProcAddr(TM__U7j10wf2hJXgu05OSHoOmw_2, "XCreateWindow");
	Dl_1862273633_ = (tyProc__cTCodr69cPNr9chtH7apXJYg) nimGetProcAddr(TM__U7j10wf2hJXgu05OSHoOmw_2, "XSetLocaleModifiers");
	Dl_1862273823_ = (tyProc__wxZMwYHQCeB2L58s6WjP7A) nimGetProcAddr(TM__U7j10wf2hJXgu05OSHoOmw_2, "XOpenIM");
	Dl_1862273838_ = (tyProc__gQyCi5Qjhu8An0HYgo9bAbQ) nimGetProcAddr(TM__U7j10wf2hJXgu05OSHoOmw_2, "XCreateIC");
	Dl_1862273225_ = (tyProc__bLi89bseeEFucqAHYZzD02g) nimGetProcAddr(TM__U7j10wf2hJXgu05OSHoOmw_2, "XQueryExtension");
	Dl_1862272222_ = (tyProc__rvpy4KyTUsLgOxFuuhdLDA) nimGetProcAddr(TM__U7j10wf2hJXgu05OSHoOmw_2, "XInternAtom");
	Dl_1862273116_ = (tyProc__B3UvMsa15lHlKTm6yxGhLA) nimGetProcAddr(TM__U7j10wf2hJXgu05OSHoOmw_2, "XMapWindow");
	Dl_1862273534_ = (tyProc__9aL17d2gWlKaCdhmRHrQwtQ) nimGetProcAddr(TM__U7j10wf2hJXgu05OSHoOmw_2, "XSync");
	Dl_1862273163_ = (tyProc__p8PCWhVohAHlC8tROLWrEA) nimGetProcAddr(TM__U7j10wf2hJXgu05OSHoOmw_2, "XPending");
	Dl_1862273139_ = (tyProc__4loCjJLVsu9aCLKyCV9b5nLA) nimGetProcAddr(TM__U7j10wf2hJXgu05OSHoOmw_2, "XNextEvent");
	Dl_1862273858_ = (tyProc__P6uJ59bF2Es9bteqw7F1m9ajg) nimGetProcAddr(TM__U7j10wf2hJXgu05OSHoOmw_2, "XFilterEvent");
	Dl_1862273926_ = (tyProc__pNP2RP29chcNsp0ayk5nzCA) nimGetProcAddr(TM__U7j10wf2hJXgu05OSHoOmw_2, "XGetEventData");
	Dl_1862273929_ = (tyProc__EFltRCl3Kwo5aicsf84Fdw) nimGetProcAddr(TM__U7j10wf2hJXgu05OSHoOmw_2, "XFreeEventData");
	Dl_1862273875_ = (tyProc__4UYg151LXj9aEPzgGdF3aEg) nimGetProcAddr(TM__U7j10wf2hJXgu05OSHoOmw_2, "Xutf8LookupString");
	Dl_1862272866_ = (tyProc__9a2ufVRUNOSifMO9cIRxPgHw) nimGetProcAddr(TM__U7j10wf2hJXgu05OSHoOmw_2, "XEventsQueued");
	Dl_1862273155_ = (tyProc__4loCjJLVsu9aCLKyCV9b5nLA) nimGetProcAddr(TM__U7j10wf2hJXgu05OSHoOmw_2, "XPeekEvent");
	Dl_1862272352_ = (tyProc__bQT3aqIQ9aw4GT2aFP714cw) nimGetProcAddr(TM__U7j10wf2hJXgu05OSHoOmw_2, "XLookupKeysym");
	Dl_1862273842_ = (tyProc__kIvABQTYztdzeIhs52BeqA) nimGetProcAddr(TM__U7j10wf2hJXgu05OSHoOmw_2, "XSetICFocus");
	Dl_1862273844_ = (tyProc__kIvABQTYztdzeIhs52BeqA) nimGetProcAddr(TM__U7j10wf2hJXgu05OSHoOmw_2, "XUnsetICFocus");
	Dl_1862273840_ = (tyProc__kIvABQTYztdzeIhs52BeqA) nimGetProcAddr(TM__U7j10wf2hJXgu05OSHoOmw_2, "XDestroyIC");
	Dl_1862273828_ = (tyProc__GvaT4D9cENRmSk7dgsDbJxg) nimGetProcAddr(TM__U7j10wf2hJXgu05OSHoOmw_2, "XCloseIM");
	Dl_1862272718_ = (tyProc__B3UvMsa15lHlKTm6yxGhLA) nimGetProcAddr(TM__U7j10wf2hJXgu05OSHoOmw_2, "XDestroyWindow");
	Dl_1862272659_ = (tyProc__p8PCWhVohAHlC8tROLWrEA) nimGetProcAddr(TM__U7j10wf2hJXgu05OSHoOmw_2, "XCloseDisplay");
}