Discuss symbol naming and visibility

[wrwilliams]

Problem

Symbol names and visibility are part of an ABI. MPI says nothing about the former and only implicitly discusses the latter.

Proposal

We should IMO make the following explicit:

• Procedures defined in the standard may ultimately be called through a C ABI with C names defined by the standard. (Basically, an F08 module should always have a well-defined C-mangled symbol it can use to provide its actual implementation, and a PMPI+F08 tool should be able to intercept the PMPI version of that well-defined symbol. I would however like to phrase this in a way that doesn't automatically bless the C bindings as the One True Bindings.)
• Symbols for procedures, variables and constants specified by the standard shall be publicly visible from an MPI library
• Symbols for implementation details shall not be publicly visible from an MPI library

...the problem on that last point of course is "how do you define implementation details, and how do you distinguish them from stuff like MPI_X?"

Impact on Implementations

Hopefully minimal, if they have already been standard-compliant...

Impact on Users

F08 modules become easier to write as independent things
PMPI tools can avoid a measure of guess-and-check name mangling and/or "just write a thin Fortran wrapper"
In general it becomes possible to iterate the symbols provided by an MPI library and do things with them in the absence of headers

[devreal]

Symbols for implementation details shall not be publicly visible from an MPI library

I'm not sure that is feasible. We have tried hard to reduce the number of implementation-specific symbols visible in OMPI (and to properly namespace them) but hiding them entirely is another beast. What about shared libraries pulled in by libmpi? We cannot hide their symbols and they will become visible in the process. After all, what is the benefit of hiding implementation-specific symbols? I would rather state that implementation-specific symbols shall be properly namespaced to reduce the chance for collisions with application symbols.

[wrwilliams]

I'm not sure that is feasible. We have tried hard to reduce the number of implementation-specific symbols visible in OMPI (and to properly namespace them) but hiding them entirely is another beast.

I think there's an important (but as-yet fuzzy) distinction between "this is implementation-specific" and "this is part of MPI's internals". The former I would want to allow (in particular for extensions); the latter I would want to prohibit or at least discourage.

What about shared libraries pulled in by libmpi? We cannot hide their symbols and they will become visible in the process.

But they won't be symbols in the MPI library; I would say this is fine.

After all, what is the benefit of hiding implementation-specific symbols? I would rather state that implementation-specific symbols shall be properly namespaced to reduce the chance for collisions with application symbols.

The question is rather one of "what symbols may a valid MPI program depend on an MPI library to provide?", and my goal here is "a valid program can compile against any valid ABI-compatible library and run with any other ABI-compatible library". If a program can see an implementation-specific symbol it can (accidentally or otherwise) depend on it...

[besnardjb]

Yes, this double PMPI has always been such a headache. When you see what the wrappers have to do: https://github.com/LLNL/wrap#notes-on-the-fortran-wrappers

Would be a big plus for tools.

Also, it would make the future QMPI approach Fortran compatible for free, I suppose.

[gonzalobg]

Symbols for implementation details shall not be publicly visible from an MPI library

What's the rationale for this being a requirement for an MPI ABI?

I understand that this is "nice to have" but what usages does this requirement enable that cannot be had without it?

---

My preference is to start by providing an a MPI C ABI - only for C - and just say that "this is C". That way the name mangling of function and variables, the call ABI, the layout of types, etc. is already specified by the platform's C ABI specification (e.g. x86_64 psABI for x86_64 x86 on Linux). I think that suffices since a cross-platform ABI seems an unsolvable problem.

A Fortran / Python / Rust/ Julia / Haskell / ... MPI ABI stub could be built on top of the C one by just having a thin fortran wrapper over the MPI C API stub that the MPI C ABI would enable.

[devreal]

Another thought (based on a conversation I had the other day): should default handles be available for dlsym? That would allow some tool/library to use MPI without a header and might be useful for runtime decisions based on whether MPI is available.

This is somewhat cross-cutting with #1 and one does not preclude the other (i.e., we may have compile-time constants and have that same symbol defined publicly within the library).