Introduce a strict_package_deps field

[rgrinberg]

In this mode, dune makes sure that all the package dependencies that it
infers are present in the opam file

[rgrinberg]

In the WIP commit, I've attempted to implement this feature when resolving closures for libraries and executables. Then, I've realized that it's much simpler to just reuse the inferred package dependencies from the build system. However, the initial approach leads to better error messages and I have some other uses for plumbing the package field of libraries. So I will rework that commit to keep the plumbing.

[rgrinberg]

Another interesting feature to add:

Dune could also infer package dependencies of external libraries. For example, if a public library foo depends on base which is installed from the package base, then this package dependency should be included in the dune-project file.

[rgrinberg]

I've removed the attempts at detecting the packages of external paths. I'll try this again with a less general approach in lib.ml.

However, I am thinking that perhaps we should think of a better default policy for this feature. What about the following:

If you are using >= 2.3 then this enabled by default but you can turn it off.

This means that upgrading from 2.2 to 2.3 could technically break things, but I feel like it's quite easy to disable and the feature seems desirable and harmless in 99% of cases.

[rgrinberg]

@aalekseyev the implementation is ready for review. The parts about attaching package and strict_package_deps to libraries is unused for now, but will be used in a later PR.

[rgrinberg]

@aalekseyev I've removed the unnecessary code.

@avsm what do you think about my comment? I thought about things a bit more and I tend to agree with you that this option is essentially always desirable and making it off by default will just introduce churn for downstream users.

[aalekseyev]

I think breakage when you bump a minor language version is not a big deal, but I hear that our users expect stronger guarantees. @dim said that we should discuss on discuss before changing the policy to generally allow breakage on minor language version bump. (although perhaps this one is OK on a case-by-case basis, as you say)

(the PR as it currently is is not introducing the breakage because strict package deps default to false)

[rgrinberg]

For now let's be conservative and just make it a separate mode. We could enable it in 2.4 if we change our mind.

[ghost]

I thought a bit more about this question of breakages in minor release of the language. When it comes to errors, in the end it seems fine to me to immediately get more error when you bump the minor language version.

i.e. you bump the minor version and you start getting more errors because Dune got more clever. That seems fine because it's not changing what things mean, we just detect a bit more error cases, which is always beneficial to the user.

However, we have to make sure that there are no false negatives. i.e. that we don't report as errors things that are correct. For this PR, because we don't interpret optional dependencies, it seems that we could have false negative. So making it opt-in seems like the right thing to do.