Removing dependencies on older LF versions

[cocreature]

Currently a DAR built in LF 1.14 (or really any LF version) still pulls in dependencies on LF 1.6 (and versions in between) for types like Archive. You can see the full list of packages at

daml/compiler/damlc/stable-packages/lib/DA/Daml/StablePackages.hs

Line 21 in 7a2a349

allStablePackages =

(note that the exception packages are already in LF 1.14 so they are unaffected).

This has worked fine so far but now we have a concrete reason for a ledger to want to drop support for older LF versions: Canton would like to drop support for everything < LF 1.14 before going GA. This allows them to drop a number of backwards compatibility hacks around byKey, old contract ids and other stuff. This needs to happen before going GA since at that point dropping support is basically impossible (outside of major upgrades at least).

More generally, we should also expect that this is not the last time we want to drop support for an old LF version so the changes here should be repeatable for future deprecations.

Changes for Daml Users

All of this only applies when targetting LF version 1.14. If you target an older LF version absolutely nothing changes.

• Packages will automatically depend on stable packages compiled in LF 1.14. These will all be mapped under DA.Stable.V1_14.*. E.g., DA.Stable.V1_14.Archive would contain the archive type. DA.Stable.V1_14.Types tuples and so on.
• daml-stdlib reexports the new types where we currently reexport the older types.
• The old types are still reexported in daml-stdlib under their old module names. Only if you reference them will you pull in the old stable packages as a dependency.
• For upgrades, you can simply import both of them. Qualified imports allow you to disambiguate.
• data-dependencies will automatically map the type to the correct module.
• Typeclass instances for the old types will have to be pulled in via an import of an old LF version. Current daml-stdlib will only have the instances for the newer types. This is necessary because we need to avoid having a dependency from daml-stdlib on the older LF packages.
• We will slightly relax the import restrictions to allow importing the modules where the old types currently live in directly even if they are under DA.Internal. The name makes no sense anyway since those modules are part of the public API via the ledger API.

Breakage/Impact

1. This is a compiler change that only affects newer LF versions so unless you recompile in LF >= 1.14 there is no impact at all.
2. If you do not have production data, then the main change is that the module name and package id over the ledger API changes. Nobody should be hardcoding package ids any way and with codegens this is a relatively simple change to make guided by the compiler. We have no chance of keeping the package id stable. We could in theory reuse the same module name. However, I think that is more confusing than helpful and makes it harder to disambiguate.
3. If you do have production data you have to deal with two versions of Archive or something similar during the upgrade. These can be disambiguated by qualified imports

Reducing Impact

We could either hide the behavior behind a flag or alternatively enable it by default but provide a way to turn this off. I don’t think this really provides any benefit. If you really want to be stable, don’t target new LF versions. If not, then doing the upgrade once is relatively easy and afterwards things are back to normal.

[sofiafaro-da]

IIRC, we decided to go a different route with dropping support for older LF versions. The approach we're going with is to disallow any packages with LF version < 1.14 (in the engine), unless they are stable packages (from daml-prim / daml-stdlib).

[cocreature]

This has been fixed now, thanks @sofiafaro-da!