Multiple rules generated for "dll from shared foreign_archives"

[recoules]

Actual Behavior

The shared library dllutil.so is part of two libraries that are public. The result is that dune generate two rules for installing it and thus fail with:

Error: Multiple rules generated for
_build/install/default/lib/stublibs/dllutil.so:
- dune:6
- dune:12

Expected Behavior

It should be possible to use foreign library more than once per public library.

Reproduction

Here is a minimal example to reproduce the issue:

dune-project

(lang dune 2.7)

(package
 (name test)
 (synopsis "test")
 (description "test")
 (depends
  (ocaml (>= 4.05))
  (conf-gcc :build)
  (conf-g++ :build)))

dune

(foreign_library
 (archive_name util)
 (language c)
 (names :standard))

(library
 (public_name test.a)
 (name a)
 (modules a)
 (foreign_archives util))

(library
 (public_name test.b) ; comment this line and it works fine
 (name b)
 (modules b)
 (foreign_archives util))

util.c

int foo () { return 0; };

a.ml & b.ml

external foo : unit -> int = "foo"

Specifications

• Version of dune (output of dune --version): 2.8.2
• Version of ocaml (output of ocamlc --version): 4.08.1
• Operating system (distribution and version): ubuntu 20.04

[ghost]

Agreed, this seems like a case that should be supported.

Looking at the code, it shouldn't be too difficult: in src/dune_rules/install_rules.ml, you have a function symlink_installed_artifacts_to_build_install that is responsible for creating the symlinks in _build/install/.... Inside this function, there is this code that declares the symlink to Dune:

        Super_context.add_rule sctx ~loc ~dir:ctx.build_dir
          (Action_builder.symlink ~src:(Path.build entry.src) ~dst)

You could adapt the code to first construct a list of symlinks to declare and de-duplicate it.

[recoules]

As a side note of further investigation, ocamlmklib create two objects for foreign libraries (libutil.a and dllutil.so).
The problem is that only the first one respect the linux library name standard used by ld.

In my test for example, when ocamlopt is called with the -shared option, the linker considers that only the static archive exists and so include all the code. This way, the dllutil.so is never used because both cmxa and cmxs have used libutil.a.
Manually renaming dllutil.so in libutil.so and calling again the same ocamlopt command properly add the so in the dynamic section. Still, the question of where the so will be searched at runtime is out of my knowledge.

Can someone confirm this? Does somebody already have reported such behavior? Because if it is exact, the shared part loses all its purpose.

[ghost]

dllxxx.so files are only used for bytecode. When you dynamically load a library inside the OCaml toplevel, the dllxxx.so files will be loaded. When you link a bytecode program, at runtime it will load the various dllxxx.so files it needs, unless the program is linked with -output-complete-exe.

dllxxx.so files are searched at runtime in the directories specified in the file $(ocamlc -where)/ld.conf. The convention is to install all dllxxx.so files in $(ocamlc -where)/stublibs.

[recoules]

Isn't it desirable that cmxs links against shared version of foreign_library?

[ghost]

That would make cmxs files less self-contained

[recoules]

Ok but, in the toy example above, is it possible to link against A et B as they both include their own copy of libutil.a?

[ghost]

You cannot indeed. You can do the following: introduce a third library test.foreign to which you attach the foreign archive, remove the foreign archive from test.a and test.b and make both test.a and test.b depend on test.foreign

[recoules]

Thank you for your answers

I will do that but it would be nice if it was automatically handled by dune (foreign_library treated as a normal library without having to attach it to a dummy library)

[ghost]

You can compress the foreign_library stanza and the library one for the dummy library into one as follow:

(library
 (public_name test.foreign)
 (foreign_stubs
  (language c)
  (names :standard)))

Given the problem you pointed out with not being able to link two libraries that would share a common foreign archive and the simple workaround we came up with, supporting sharing a foreign archive between several OCaml libraries doesn't seem worth the trouble, so I'm closing this ticket.

[bobot]

The fact that we get an internal error is still problematic, but I'm going to open another issue that sums up the different difficult point of foreign_archives