solver: Interaction of base-shim qualifier and setup qualifier is not correct #9467
Comments
|
cc @grayjay |
mpickering
added a commit
to mpickering/cabal
that referenced
this issue
Nov 24, 2023
…eraction) This adds two tests for issue haskell#9467
mpickering
added a commit
to mpickering/cabal
that referenced
this issue
Nov 24, 2023
…eraction) This adds two tests for issue haskell#9467
|
I read some of the code related to base shims, and I think I see why setup dependencies and base shims don't work well together. I don't fully understand the log from the second example, though. The issue may have been introduced by #3220, which limited the depth of qualifiers to one to prevent infinite loops. The depth limit means that the solver discards the existing setup qualifier when it adds a base qualifier. I also noticed that when base shims are in use, dependencies of base are always given the top-level qualifier, which could cause inconsistent dependencies for a setup component: cabal/cabal-install-solver/src/Distribution/Solver/Modular/Dependency.hs Lines 217 to 222 in 4f53a2f I think that we could fix these two issues by keeping the limit of one setup or exe qualifier but adding a boolean field to represent the existence of a final base qualifier. |
|
@grayjay I have fixed the issue on a branch I have by clarifying the difference between a "namespace" and a "qualifier".
The structure of the Therefore most qualified goals are distinguished by a new namespace (and it's fine if these replace each other because they introduce a completely new scope). The base-shim goal is the only example of a "qualifier" goal where the goal does not persist into transitive dependencies. This simplifies the implementation of
|
|
I think that the idea of independent goals was that all dependencies of different targets would be independent, including setup and build tool dependencies. So I think that the namespace should continue to always be inherited. We could fix the base shim issue without reducing the set of possible solutions by using types like this: data PackagePath = PackagePath Namespace Qualifier (Maybe BaseQualifier)
data Namespace = DefaultNamespace | Independent PackageName
data Qualifier =
QualToplevel
| QualSetup PackageName
| QualExe PackageName PackageName
newtype BaseQualifier = BaseQualifier PackageName |
alt-romes
pushed a commit
to mpickering/cabal
that referenced
this issue
Mar 11, 2024
…eraction) This adds two tests for issue haskell#9467
alt-romes
pushed a commit
to mpickering/cabal
that referenced
this issue
Mar 14, 2024
…eraction) This adds two tests for issue haskell#9467
alt-romes
pushed a commit
to mpickering/cabal
that referenced
this issue
Apr 3, 2024
…eraction) This adds two tests for issue haskell#9467
alt-romes
pushed a commit
to mpickering/cabal
that referenced
this issue
Apr 3, 2024
…eraction) This adds two tests for issue haskell#9467
alt-romes
pushed a commit
to mpickering/cabal
that referenced
this issue
Apr 4, 2024
…eraction) This adds two tests for issue haskell#9467
alt-romes
pushed a commit
to mpickering/cabal
that referenced
this issue
Apr 5, 2024
…eraction) This adds two tests for issue haskell#9467
alt-romes
pushed a commit
to mpickering/cabal
that referenced
this issue
Apr 5, 2024
This commit introduces so-called "private dependencies".
High-level Overview
~~~~~~~~~~~~~~~~~~~
Since its inception, Cabal has enforced the restriction that a library
must only link against one version of each package it depends on. This
ensures that all of the dependencies in the build plan work together. In
your application you use different libraries together, so it’s of
paramount importance that they all agree on what `Text` means or what a
`ByteString` is.
However, sometimes it’s desirable to allow multiple versions of the same
library into a build plan. In this case, it’s desirable to allow a
library author to specify a private dependency with the promise that its
existence will not leak from the interface of the library which uses it.
The two main use cases of private dependencies are:
- Writing benchmarks and testsuites for your library which test new
versions of your library against old versions.
- Writing libraries which can communicate with processes built against
a range of different library versions (such as cabal-install calling
./Setup).
A user specifies a private dependency in their cabal file using
`private-build-depends`. The specification starts with the name of the
private dependency scope and then contains a list of normal dependency
specifications which dictates what is included in that private scope:
private-build-depends: TEXT1 with (text == 1.2.*), TEXT2 with (text == 2.*)
Each private scope is then solved independently of all other scopes. In
this example the TEXT1 scope can choose a version of text in the 1.2.x
range and the TEXT2 scope can choose a version of text in the 2.* range.
Private scopes do not apply transitively, so the dependencies of text
will be solved in the normal top-level scope. If your program contains a
value of type Bool, that comes from the base package, which text depends
on, because the scopes are not applied transitively the same Bool value
can be passed to functions from the TEXT1 scope and TEXT2 scope.
Dependencies introduced privately can be imported into modules in the
project by prefixing the name of the private scope to an exposed module
name.
import qualified TEXT1.Data.Text as T1
import qualified TEXT2.Data.Text as T2
Closure of Private Scopes
~~~~~~~~~~~~~~~~~~~~~~~~~
Private dependency scopes can contain multiple packages. Packages in the
same scope are solved together. For example, if two packages are tightly
coupled and you need to use compatible versions with each other, then
you can list them in the same private scope. Such packages will then be
solved together, but independently of other packages.
Private scopes must be closed. A scope is closed if, whenever we have a
dependency chain P1 -> Q -> P2, in which P1 and P2 are both in a given
private scope S, then Q also belongs to the private scope S. The solver
checks this property, but doesn’t implicitly add packages into a private
scope.
Implementation
~~~~~~~~~~~~~~
To implement private dependencies we changed
* Cabal-syntax to introduce the new `private-build-depends: ALIAS (packages, in, private, scope)` syntax.
See the new type `Dependencies` and changes in `Distribution.Types.Dependency`.
* cabal-install-solver now considers both public and private
dependencies of a given package (see e.g. `solverPkgLibDeps`), has
a new constructor `PrivateScope` in `ConstraintScope` for goals in a
private scope, and there's a new `Qualifier` for packages introduced
in private scopes (see also [Namespace vs Qualifier refactor] below),
to solve them separately from packages introduced by `build-depends`.
* cabal-install-solver needs to check that the private-scope closure
property holds (the closure of the packages in a private scope is in
the private scope) (see `Distribution.Solver.Modular.PrivateScopeClosure`).
We check that the closure holds by looking at the reverse dependency
map while traversing down the tree, at every node:
For every package in a private scope, traverse up the reverse
dependency map until a package in the same private scope is found.
If one exists, and if along the way up any package was not in the same
private scope as the packages in the two ends, we fail.
* cabal-install understands plans with private dependencies and has a
new `UserQualifier` to support constrainting packages in private
scopes using the `--constraint` flag.
Example: `--constraint=private.pkg-a.TEXT01:text == 1.2.*`
* Cabal the library uses the ghc module-renaming mechanism (also used by
Backpack) to rename modules from the packages in a private scope to
prefix them with the private scope alias. It also ensures `cabal
check` fails if there exist the package has private dependencies, as
it is currently an experimental feature which we don't necessarily
want to allow in hackage yet -- e.g. how will haddock render private
dependencies?
Namespace vs Qualifier refactor
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
We also refactored the `Namespace` vs `Qualifier` types in the solver,
clarifying their interaction such that:
* A package goal with an indepedent namespace is fully solved
indepently from other namespaces, i.e. all the dependency goals
introduced by a goal in a given namespace are also solved in that
namespace.
* In contrast, a package goal with a qualifier is shallow-solved
separately from other goals in the same namespace. The dependency
goals introduced by it will be solved unqualified (`QualTopLevel`) in
that namespace.
For example, goal `pkg-a == 0.1` in `DefaultNamespace+QualTopLevel`, and
goal `pkg-a == 0.2` in the same namespace but with `QualAlias A2 ...`
can be solved together and yield a different version of pkg-a for each
of the goals, however, the dependencies of both will be solved together
-- if they both dependend on `base`, we'd have to find a single
solution. If `pkg-a == 0.2` was in an `Independent` namespace, we could
still solve the two goals with two versions of `pkg-a`, but we could
also pick different versions for all the subdependencies of `pkg-a ==
0.2`.
Besides Namespace vs Qualifier being a welcome refactor that facilitates
implementing private dependencies, it also fixes haskell#9466 and helps with haskell#9467.
---
Co-authored-by: Rodrigo Mesquita (@alt-romes)
alt-romes
pushed a commit
to mpickering/cabal
that referenced
this issue
Apr 5, 2024
This commit introduces so-called "private dependencies".
High-level Overview
~~~~~~~~~~~~~~~~~~~
Since its inception, Cabal has enforced the restriction that a library
must only link against one version of each package it depends on. This
ensures that all of the dependencies in the build plan work together. In
your application you use different libraries together, so it’s of
paramount importance that they all agree on what `Text` means or what a
`ByteString` is.
However, sometimes it’s desirable to allow multiple versions of the same
library into a build plan. In this case, it’s desirable to allow a
library author to specify a private dependency with the promise that its
existence will not leak from the interface of the library which uses it.
The two main use cases of private dependencies are:
- Writing benchmarks and testsuites for your library which test new
versions of your library against old versions.
- Writing libraries which can communicate with processes built against
a range of different library versions (such as cabal-install calling
./Setup).
A user specifies a private dependency in their cabal file using
`private-build-depends`. The specification starts with the name of the
private dependency scope and then contains a list of normal dependency
specifications which dictates what is included in that private scope:
private-build-depends: TEXT1 with (text == 1.2.*), TEXT2 with (text == 2.*)
Each private scope is then solved independently of all other scopes. In
this example the TEXT1 scope can choose a version of text in the 1.2.x
range and the TEXT2 scope can choose a version of text in the 2.* range.
Private scopes do not apply transitively, so the dependencies of text
will be solved in the normal top-level scope. If your program contains a
value of type Bool, that comes from the base package, which text depends
on, because the scopes are not applied transitively the same Bool value
can be passed to functions from the TEXT1 scope and TEXT2 scope.
Dependencies introduced privately can be imported into modules in the
project by prefixing the name of the private scope to an exposed module
name.
import qualified TEXT1.Data.Text as T1
import qualified TEXT2.Data.Text as T2
Closure of Private Scopes
~~~~~~~~~~~~~~~~~~~~~~~~~
Private dependency scopes can contain multiple packages. Packages in the
same scope are solved together. For example, if two packages are tightly
coupled and you need to use compatible versions with each other, then
you can list them in the same private scope. Such packages will then be
solved together, but independently of other packages.
Private scopes must be closed. A scope is closed if, whenever we have a
dependency chain P1 -> Q -> P2, in which P1 and P2 are both in a given
private scope S, then Q also belongs to the private scope S. The solver
checks this property, but doesn’t implicitly add packages into a private
scope.
Implementation
~~~~~~~~~~~~~~
To implement private dependencies we changed
* Cabal-syntax to introduce the new `private-build-depends: ALIAS (packages, in, private, scope)` syntax.
See the new type `Dependencies` and changes in `Distribution.Types.Dependency`.
* cabal-install-solver now considers both public and private
dependencies of a given package (see e.g. `solverPkgLibDeps`), has
a new constructor `PrivateScope` in `ConstraintScope` for goals in a
private scope, and there's a new `Qualifier` for packages introduced
in private scopes (see also [Namespace vs Qualifier refactor] below),
to solve them separately from packages introduced by `build-depends`.
* cabal-install-solver needs to check that the private-scope closure
property holds (the closure of the packages in a private scope is in
the private scope) (see `Distribution.Solver.Modular.PrivateScopeClosure`).
We check that the closure holds by looking at the reverse dependency
map while traversing down the tree, at every node:
For every package in a private scope, traverse up the reverse
dependency map until a package in the same private scope is found.
If one exists, and if along the way up any package was not in the same
private scope as the packages in the two ends, we fail.
* cabal-install understands plans with private dependencies and has a
new `UserQualifier` to support constrainting packages in private
scopes using the `--constraint` flag.
Example: `--constraint=private.pkg-a.TEXT01:text == 1.2.*`
* Cabal the library uses the ghc module-renaming mechanism (also used by
Backpack) to rename modules from the packages in a private scope to
prefix them with the private scope alias. It also ensures `cabal
check` fails if there exist the package has private dependencies, as
it is currently an experimental feature which we don't necessarily
want to allow in hackage yet -- e.g. how will haddock render private
dependencies?
Namespace vs Qualifier refactor
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
We also refactored the `Namespace` vs `Qualifier` types in the solver,
clarifying their interaction such that:
* A package goal with an indepedent namespace is fully solved
indepently from other namespaces, i.e. all the dependency goals
introduced by a goal in a given namespace are also solved in that
namespace.
* In contrast, a package goal with a qualifier is shallow-solved
separately from other goals in the same namespace. The dependency
goals introduced by it will be solved unqualified (`QualTopLevel`) in
that namespace.
For example, goal `pkg-a == 0.1` in `DefaultNamespace+QualTopLevel`, and
goal `pkg-a == 0.2` in the same namespace but with `QualAlias A2 ...`
can be solved together and yield a different version of pkg-a for each
of the goals, however, the dependencies of both will be solved together
-- if they both dependend on `base`, we'd have to find a single
solution. If `pkg-a == 0.2` was in an `Independent` namespace, we could
still solve the two goals with two versions of `pkg-a`, but we could
also pick different versions for all the subdependencies of `pkg-a ==
0.2`.
Besides Namespace vs Qualifier being a welcome refactor that facilitates
implementing private dependencies, it also fixes haskell#9466 and helps with haskell#9467.
---
Co-authored-by: Rodrigo Mesquita (@alt-romes)
alt-romes
added a commit
to mpickering/cabal
that referenced
this issue
Apr 5, 2024
This commit introduces so-called "private dependencies".
High-level Overview
~~~~~~~~~~~~~~~~~~~
Since its inception, Cabal has enforced the restriction that a library
must only link against one version of each package it depends on. This
ensures that all of the dependencies in the build plan work together. In
your application you use different libraries together, so it’s of
paramount importance that they all agree on what `Text` means or what a
`ByteString` is.
However, sometimes it’s desirable to allow multiple versions of the same
library into a build plan. In this case, it’s desirable to allow a
library author to specify a private dependency with the promise that its
existence will not leak from the interface of the library which uses it.
The two main use cases of private dependencies are:
- Writing benchmarks and testsuites for your library which test new
versions of your library against old versions.
- Writing libraries which can communicate with processes built against
a range of different library versions (such as cabal-install calling
./Setup).
A user specifies a private dependency in their cabal file using
`private-build-depends`. The specification starts with the name of the
private dependency scope and then contains a list of normal dependency
specifications which dictates what is included in that private scope:
private-build-depends: TEXT1 with (text == 1.2.*), TEXT2 with (text == 2.*)
Each private scope is then solved independently of all other scopes. In
this example the TEXT1 scope can choose a version of text in the 1.2.x
range and the TEXT2 scope can choose a version of text in the 2.* range.
Private scopes do not apply transitively, so the dependencies of text
will be solved in the normal top-level scope. If your program contains a
value of type Bool, that comes from the base package, which text depends
on, because the scopes are not applied transitively the same Bool value
can be passed to functions from the TEXT1 scope and TEXT2 scope.
Dependencies introduced privately can be imported into modules in the
project by prefixing the name of the private scope to an exposed module
name.
import qualified TEXT1.Data.Text as T1
import qualified TEXT2.Data.Text as T2
Closure of Private Scopes
~~~~~~~~~~~~~~~~~~~~~~~~~
Private dependency scopes can contain multiple packages. Packages in the
same scope are solved together. For example, if two packages are tightly
coupled and you need to use compatible versions with each other, then
you can list them in the same private scope. Such packages will then be
solved together, but independently of other packages.
Private scopes must be closed. A scope is closed if, whenever we have a
dependency chain P1 -> Q -> P2, in which P1 and P2 are both in a given
private scope S, then Q also belongs to the private scope S. The solver
checks this property, but doesn’t implicitly add packages into a private
scope.
Implementation
~~~~~~~~~~~~~~
To implement private dependencies we changed
* Cabal-syntax to introduce the new `private-build-depends: ALIAS (packages, in, private, scope)` syntax.
See the new type `Dependencies` and changes in `Distribution.Types.Dependency`.
* cabal-install-solver now considers both public and private
dependencies of a given package (see e.g. `solverPkgLibDeps`), has
a new constructor `PrivateScope` in `ConstraintScope` for goals in a
private scope, and there's a new `Qualifier` for packages introduced
in private scopes (see also [Namespace vs Qualifier refactor] below),
to solve them separately from packages introduced by `build-depends`.
* cabal-install-solver needs to check that the private-scope closure
property holds (the closure of the packages in a private scope is in
the private scope) (see `Distribution.Solver.Modular.PrivateScopeClosure`).
We check that the closure holds by looking at the reverse dependency
map while traversing down the tree, at every node:
For every package in a private scope, traverse up the reverse
dependency map until a package in the same private scope is found.
If one exists, and if along the way up any package was not in the same
private scope as the packages in the two ends, we fail.
* cabal-install understands plans with private dependencies and has a
new `UserQualifier` to support constrainting packages in private
scopes using the `--constraint` flag.
Example: `--constraint=private.pkg-a.TEXT01:text == 1.2.*`
* Cabal the library uses the ghc module-renaming mechanism (also used by
Backpack) to rename modules from the packages in a private scope to
prefix them with the private scope alias. It also ensures `cabal
check` fails if there exist the package has private dependencies, as
it is currently an experimental feature which we don't necessarily
want to allow in hackage yet -- e.g. how will haddock render private
dependencies?
Namespace vs Qualifier refactor
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
We also refactored the `Namespace` vs `Qualifier` types in the solver,
clarifying their interaction such that:
* A package goal with an indepedent namespace is fully solved
indepently from other namespaces, i.e. all the dependency goals
introduced by a goal in a given namespace are also solved in that
namespace.
* In contrast, a package goal with a qualifier is shallow-solved
separately from other goals in the same namespace. The dependency
goals introduced by it will be solved unqualified (`QualTopLevel`) in
that namespace.
For example, goal `pkg-a == 0.1` in `DefaultNamespace+QualTopLevel`, and
goal `pkg-a == 0.2` in the same namespace but with `QualAlias A2 ...`
can be solved together and yield a different version of pkg-a for each
of the goals, however, the dependencies of both will be solved together
-- if they both dependend on `base`, we'd have to find a single
solution. If `pkg-a == 0.2` was in an `Independent` namespace, we could
still solve the two goals with two versions of `pkg-a`, but we could
also pick different versions for all the subdependencies of `pkg-a ==
0.2`.
Besides Namespace vs Qualifier being a welcome refactor that facilitates
implementing private dependencies, it also fixes haskell#9466 and helps with haskell#9467.
---
Co-authored-by: Rodrigo Mesquita <[email protected]>
Mikolaj
pushed a commit
to mpickering/cabal
that referenced
this issue
Apr 7, 2024
…eraction) This adds two tests for issue haskell#9467
alt-romes
pushed a commit
to mpickering/cabal
that referenced
this issue
Apr 12, 2024
…eraction) This adds two tests for issue haskell#9467
alt-romes
added a commit
to mpickering/cabal
that referenced
this issue
Apr 12, 2024
This commit introduces so-called "private dependencies".
High-level Overview
~~~~~~~~~~~~~~~~~~~
Since its inception, Cabal has enforced the restriction that a library
must only link against one version of each package it depends on. This
ensures that all of the dependencies in the build plan work together. In
your application you use different libraries together, so it’s of
paramount importance that they all agree on what `Text` means or what a
`ByteString` is.
However, sometimes it’s desirable to allow multiple versions of the same
library into a build plan. In this case, it’s desirable to allow a
library author to specify a private dependency with the promise that its
existence will not leak from the interface of the library which uses it.
The two main use cases of private dependencies are:
- Writing benchmarks and testsuites for your library which test new
versions of your library against old versions.
- Writing libraries which can communicate with processes built against
a range of different library versions (such as cabal-install calling
./Setup).
A user specifies a private dependency in their cabal file using
`private-build-depends`. The specification starts with the name of the
private dependency scope and then contains a list of normal dependency
specifications which dictates what is included in that private scope:
private-build-depends: TEXT1 with (text == 1.2.*), TEXT2 with (text == 2.*)
Each private scope is then solved independently of all other scopes. In
this example the TEXT1 scope can choose a version of text in the 1.2.x
range and the TEXT2 scope can choose a version of text in the 2.* range.
Private scopes do not apply transitively, so the dependencies of text
will be solved in the normal top-level scope. If your program contains a
value of type Bool, that comes from the base package, which text depends
on, because the scopes are not applied transitively the same Bool value
can be passed to functions from the TEXT1 scope and TEXT2 scope.
Dependencies introduced privately can be imported into modules in the
project by prefixing the name of the private scope to an exposed module
name.
import qualified TEXT1.Data.Text as T1
import qualified TEXT2.Data.Text as T2
Closure of Private Scopes
~~~~~~~~~~~~~~~~~~~~~~~~~
Private dependency scopes can contain multiple packages. Packages in the
same scope are solved together. For example, if two packages are tightly
coupled and you need to use compatible versions with each other, then
you can list them in the same private scope. Such packages will then be
solved together, but independently of other packages.
Private scopes must be closed. A scope is closed if, whenever we have a
dependency chain P1 -> Q -> P2, in which P1 and P2 are both in a given
private scope S, then Q also belongs to the private scope S. The solver
checks this property, but doesn’t implicitly add packages into a private
scope.
Implementation
~~~~~~~~~~~~~~
To implement private dependencies we changed
* Cabal-syntax to introduce the new `private-build-depends: ALIAS (packages, in, private, scope)` syntax.
See the new type `Dependencies` and changes in `Distribution.Types.Dependency`.
* cabal-install-solver now considers both public and private
dependencies of a given package (see e.g. `solverPkgLibDeps`), has
a new constructor `PrivateScope` in `ConstraintScope` for goals in a
private scope, and there's a new `Qualifier` for packages introduced
in private scopes (see also [Namespace vs Qualifier refactor] below),
to solve them separately from packages introduced by `build-depends`.
* cabal-install-solver needs to check that the private-scope closure
property holds (the closure of the packages in a private scope is in
the private scope) (see `Distribution.Solver.Modular.PrivateScopeClosure`).
We check that the closure holds by looking at the reverse dependency
map while traversing down the tree, at every node:
For every package in a private scope, traverse up the reverse
dependency map until a package in the same private scope is found.
If one exists, and if along the way up any package was not in the same
private scope as the packages in the two ends, we fail.
* cabal-install understands plans with private dependencies and has a
new `UserQualifier` to support constrainting packages in private
scopes using the `--constraint` flag.
Example: `--constraint=private.pkg-a.TEXT01:text == 1.2.*`
* Cabal the library uses the ghc module-renaming mechanism (also used by
Backpack) to rename modules from the packages in a private scope to
prefix them with the private scope alias. It also ensures `cabal
check` fails if there exist the package has private dependencies, as
it is currently an experimental feature which we don't necessarily
want to allow in hackage yet -- e.g. how will haddock render private
dependencies?
Namespace vs Qualifier refactor
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
We also refactored the `Namespace` vs `Qualifier` types in the solver,
clarifying their interaction such that:
* A package goal with an indepedent namespace is fully solved
indepently from other namespaces, i.e. all the dependency goals
introduced by a goal in a given namespace are also solved in that
namespace.
* In contrast, a package goal with a qualifier is shallow-solved
separately from other goals in the same namespace. The dependency
goals introduced by it will be solved unqualified (`QualTopLevel`) in
that namespace.
For example, goal `pkg-a == 0.1` in `DefaultNamespace+QualTopLevel`, and
goal `pkg-a == 0.2` in the same namespace but with `QualAlias A2 ...`
can be solved together and yield a different version of pkg-a for each
of the goals, however, the dependencies of both will be solved together
-- if they both dependend on `base`, we'd have to find a single
solution. If `pkg-a == 0.2` was in an `Independent` namespace, we could
still solve the two goals with two versions of `pkg-a`, but we could
also pick different versions for all the subdependencies of `pkg-a ==
0.2`.
Besides Namespace vs Qualifier being a welcome refactor that facilitates
implementing private dependencies, it also fixes haskell#9466 and helps with haskell#9467.
---
Co-authored-by: Rodrigo Mesquita <[email protected]>
alt-romes
added a commit
to mpickering/cabal
that referenced
this issue
Apr 12, 2024
This commit introduces so-called "private dependencies".
High-level Overview
~~~~~~~~~~~~~~~~~~~
Since its inception, Cabal has enforced the restriction that a library
must only link against one version of each package it depends on. This
ensures that all of the dependencies in the build plan work together. In
your application you use different libraries together, so it’s of
paramount importance that they all agree on what `Text` means or what a
`ByteString` is.
However, sometimes it’s desirable to allow multiple versions of the same
library into a build plan. In this case, it’s desirable to allow a
library author to specify a private dependency with the promise that its
existence will not leak from the interface of the library which uses it.
The two main use cases of private dependencies are:
- Writing benchmarks and testsuites for your library which test new
versions of your library against old versions.
- Writing libraries which can communicate with processes built against
a range of different library versions (such as cabal-install calling
./Setup).
A user specifies a private dependency in their cabal file using
`private-build-depends`. The specification starts with the name of the
private dependency scope and then contains a list of normal dependency
specifications which dictates what is included in that private scope:
private-build-depends: TEXT1 with (text == 1.2.*), TEXT2 with (text == 2.*)
Each private scope is then solved independently of all other scopes. In
this example the TEXT1 scope can choose a version of text in the 1.2.x
range and the TEXT2 scope can choose a version of text in the 2.* range.
Private scopes do not apply transitively, so the dependencies of text
will be solved in the normal top-level scope. If your program contains a
value of type Bool, that comes from the base package, which text depends
on, because the scopes are not applied transitively the same Bool value
can be passed to functions from the TEXT1 scope and TEXT2 scope.
Dependencies introduced privately can be imported into modules in the
project by prefixing the name of the private scope to an exposed module
name.
import qualified TEXT1.Data.Text as T1
import qualified TEXT2.Data.Text as T2
Closure of Private Scopes
~~~~~~~~~~~~~~~~~~~~~~~~~
Private dependency scopes can contain multiple packages. Packages in the
same scope are solved together. For example, if two packages are tightly
coupled and you need to use compatible versions with each other, then
you can list them in the same private scope. Such packages will then be
solved together, but independently of other packages.
Private scopes must be closed. A scope is closed if, whenever we have a
dependency chain P1 -> Q -> P2, in which P1 and P2 are both in a given
private scope S, then Q also belongs to the private scope S. The solver
checks this property, but doesn’t implicitly add packages into a private
scope.
Implementation
~~~~~~~~~~~~~~
To implement private dependencies we changed
* Cabal-syntax to introduce the new `private-build-depends: ALIAS (packages, in, private, scope)` syntax.
See the new type `Dependencies` and changes in `Distribution.Types.Dependency`.
* cabal-install-solver now considers both public and private
dependencies of a given package (see e.g. `solverPkgLibDeps`), has
a new constructor `PrivateScope` in `ConstraintScope` for goals in a
private scope, and there's a new `Qualifier` for packages introduced
in private scopes (see also [Namespace vs Qualifier refactor] below),
to solve them separately from packages introduced by `build-depends`.
* cabal-install-solver needs to check that the private-scope closure
property holds (the closure of the packages in a private scope is in
the private scope) (see `Distribution.Solver.Modular.PrivateScopeClosure`).
We check that the closure holds by looking at the reverse dependency
map while traversing down the tree, at every node:
For every package in a private scope, traverse up the reverse
dependency map until a package in the same private scope is found.
If one exists, and if along the way up any package was not in the same
private scope as the packages in the two ends, we fail.
* cabal-install understands plans with private dependencies and has a
new `UserQualifier` to support constrainting packages in private
scopes using the `--constraint` flag.
Example: `--constraint=private.pkg-a.TEXT01:text == 1.2.*`
* Cabal the library uses the ghc module-renaming mechanism (also used by
Backpack) to rename modules from the packages in a private scope to
prefix them with the private scope alias. It also ensures `cabal
check` fails if there exist the package has private dependencies, as
it is currently an experimental feature which we don't necessarily
want to allow in hackage yet -- e.g. how will haddock render private
dependencies?
Namespace vs Qualifier refactor
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
We also refactored the `Namespace` vs `Qualifier` types in the solver,
clarifying their interaction such that:
* A package goal with an indepedent namespace is fully solved
indepently from other namespaces, i.e. all the dependency goals
introduced by a goal in a given namespace are also solved in that
namespace.
* In contrast, a package goal with a qualifier is shallow-solved
separately from other goals in the same namespace. The dependency
goals introduced by it will be solved unqualified (`QualTopLevel`) in
that namespace.
For example, goal `pkg-a == 0.1` in `DefaultNamespace+QualTopLevel`, and
goal `pkg-a == 0.2` in the same namespace but with `QualAlias A2 ...`
can be solved together and yield a different version of pkg-a for each
of the goals, however, the dependencies of both will be solved together
-- if they both dependend on `base`, we'd have to find a single
solution. If `pkg-a == 0.2` was in an `Independent` namespace, we could
still solve the two goals with two versions of `pkg-a`, but we could
also pick different versions for all the subdependencies of `pkg-a ==
0.2`.
Besides Namespace vs Qualifier being a welcome refactor that facilitates
implementing private dependencies, it also fixes haskell#9466 and helps with haskell#9467.
---
Co-authored-by: Rodrigo Mesquita <[email protected]>
alt-romes
added a commit
to mpickering/cabal
that referenced
this issue
Apr 12, 2024
This commit introduces so-called "private dependencies".
High-level Overview
~~~~~~~~~~~~~~~~~~~
Since its inception, Cabal has enforced the restriction that a library
must only link against one version of each package it depends on. This
ensures that all of the dependencies in the build plan work together. In
your application you use different libraries together, so it’s of
paramount importance that they all agree on what `Text` means or what a
`ByteString` is.
However, sometimes it’s desirable to allow multiple versions of the same
library into a build plan. In this case, it’s desirable to allow a
library author to specify a private dependency with the promise that its
existence will not leak from the interface of the library which uses it.
The two main use cases of private dependencies are:
- Writing benchmarks and testsuites for your library which test new
versions of your library against old versions.
- Writing libraries which can communicate with processes built against
a range of different library versions (such as cabal-install calling
./Setup).
A user specifies a private dependency in their cabal file using
`private-build-depends`. The specification starts with the name of the
private dependency scope and then contains a list of normal dependency
specifications which dictates what is included in that private scope:
private-build-depends: TEXT1 with (text == 1.2.*), TEXT2 with (text == 2.*)
Each private scope is then solved independently of all other scopes. In
this example the TEXT1 scope can choose a version of text in the 1.2.x
range and the TEXT2 scope can choose a version of text in the 2.* range.
Private scopes do not apply transitively, so the dependencies of text
will be solved in the normal top-level scope. If your program contains a
value of type Bool, that comes from the base package, which text depends
on, because the scopes are not applied transitively the same Bool value
can be passed to functions from the TEXT1 scope and TEXT2 scope.
Dependencies introduced privately can be imported into modules in the
project by prefixing the name of the private scope to an exposed module
name.
import qualified TEXT1.Data.Text as T1
import qualified TEXT2.Data.Text as T2
Closure of Private Scopes
~~~~~~~~~~~~~~~~~~~~~~~~~
Private dependency scopes can contain multiple packages. Packages in the
same scope are solved together. For example, if two packages are tightly
coupled and you need to use compatible versions with each other, then
you can list them in the same private scope. Such packages will then be
solved together, but independently of other packages.
Private scopes must be closed. A scope is closed if, whenever we have a
dependency chain P1 -> Q -> P2, in which P1 and P2 are both in a given
private scope S, then Q also belongs to the private scope S. The solver
checks this property, but doesn’t implicitly add packages into a private
scope.
Implementation
~~~~~~~~~~~~~~
To implement private dependencies we changed
* Cabal-syntax to introduce the new `private-build-depends: ALIAS (packages, in, private, scope)` syntax.
See the new type `Dependencies` and changes in `Distribution.Types.Dependency`.
* cabal-install-solver now considers both public and private
dependencies of a given package (see e.g. `solverPkgLibDeps`), has
a new constructor `PrivateScope` in `ConstraintScope` for goals in a
private scope, and there's a new `Qualifier` for packages introduced
in private scopes (see also [Namespace vs Qualifier refactor] below),
to solve them separately from packages introduced by `build-depends`.
* cabal-install-solver needs to check that the private-scope closure
property holds (the closure of the packages in a private scope is in
the private scope) (see `Distribution.Solver.Modular.PrivateScopeClosure`).
We check that the closure holds by looking at the reverse dependency
map while traversing down the tree, at every node:
For every package in a private scope, traverse up the reverse
dependency map until a package in the same private scope is found.
If one exists, and if along the way up any package was not in the same
private scope as the packages in the two ends, we fail.
* cabal-install understands plans with private dependencies and has a
new `UserQualifier` to support constrainting packages in private
scopes using the `--constraint` flag.
Example: `--constraint=private.pkg-a.TEXT01:text == 1.2.*`
* Cabal the library uses the ghc module-renaming mechanism (also used by
Backpack) to rename modules from the packages in a private scope to
prefix them with the private scope alias. It also ensures `cabal
check` fails if there exist the package has private dependencies, as
it is currently an experimental feature which we don't necessarily
want to allow in hackage yet -- e.g. how will haddock render private
dependencies?
Namespace vs Qualifier refactor
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
We also refactored the `Namespace` vs `Qualifier` types in the solver,
clarifying their interaction such that:
* A package goal with an indepedent namespace is fully solved
indepently from other namespaces, i.e. all the dependency goals
introduced by a goal in a given namespace are also solved in that
namespace.
* In contrast, a package goal with a qualifier is shallow-solved
separately from other goals in the same namespace. The dependency
goals introduced by it will be solved unqualified (`QualTopLevel`) in
that namespace.
For example, goal `pkg-a == 0.1` in `DefaultNamespace+QualTopLevel`, and
goal `pkg-a == 0.2` in the same namespace but with `QualAlias A2 ...`
can be solved together and yield a different version of pkg-a for each
of the goals, however, the dependencies of both will be solved together
-- if they both dependend on `base`, we'd have to find a single
solution. If `pkg-a == 0.2` was in an `Independent` namespace, we could
still solve the two goals with two versions of `pkg-a`, but we could
also pick different versions for all the subdependencies of `pkg-a ==
0.2`.
Besides Namespace vs Qualifier being a welcome refactor that facilitates
implementing private dependencies, it also fixes haskell#9466 and helps with haskell#9467.
---
Co-authored-by: Rodrigo Mesquita <[email protected]>
alt-romes
added a commit
to mpickering/cabal
that referenced
this issue
Apr 12, 2024
This commit introduces so-called "private dependencies".
High-level Overview
~~~~~~~~~~~~~~~~~~~
Since its inception, Cabal has enforced the restriction that a library
must only link against one version of each package it depends on. This
ensures that all of the dependencies in the build plan work together. In
your application you use different libraries together, so it’s of
paramount importance that they all agree on what `Text` means or what a
`ByteString` is.
However, sometimes it’s desirable to allow multiple versions of the same
library into a build plan. In this case, it’s desirable to allow a
library author to specify a private dependency with the promise that its
existence will not leak from the interface of the library which uses it.
The two main use cases of private dependencies are:
- Writing benchmarks and testsuites for your library which test new
versions of your library against old versions.
- Writing libraries which can communicate with processes built against
a range of different library versions (such as cabal-install calling
./Setup).
A user specifies a private dependency in their cabal file using
`private-build-depends`. The specification starts with the name of the
private dependency scope and then contains a list of normal dependency
specifications which dictates what is included in that private scope:
private-build-depends: TEXT1 with (text == 1.2.*), TEXT2 with (text == 2.*)
Each private scope is then solved independently of all other scopes. In
this example the TEXT1 scope can choose a version of text in the 1.2.x
range and the TEXT2 scope can choose a version of text in the 2.* range.
Private scopes do not apply transitively, so the dependencies of text
will be solved in the normal top-level scope. If your program contains a
value of type Bool, that comes from the base package, which text depends
on, because the scopes are not applied transitively the same Bool value
can be passed to functions from the TEXT1 scope and TEXT2 scope.
Dependencies introduced privately can be imported into modules in the
project by prefixing the name of the private scope to an exposed module
name.
import qualified TEXT1.Data.Text as T1
import qualified TEXT2.Data.Text as T2
Closure of Private Scopes
~~~~~~~~~~~~~~~~~~~~~~~~~
Private dependency scopes can contain multiple packages. Packages in the
same scope are solved together. For example, if two packages are tightly
coupled and you need to use compatible versions with each other, then
you can list them in the same private scope. Such packages will then be
solved together, but independently of other packages.
Private scopes must be closed. A scope is closed if, whenever we have a
dependency chain P1 -> Q -> P2, in which P1 and P2 are both in a given
private scope S, then Q also belongs to the private scope S. The solver
checks this property, but doesn’t implicitly add packages into a private
scope.
Implementation
~~~~~~~~~~~~~~
To implement private dependencies we changed
* Cabal-syntax to introduce the new `private-build-depends: ALIAS (packages, in, private, scope)` syntax.
See the new type `Dependencies` and changes in `Distribution.Types.Dependency`.
* cabal-install-solver now considers both public and private
dependencies of a given package (see e.g. `solverPkgLibDeps`), has
a new constructor `PrivateScope` in `ConstraintScope` for goals in a
private scope, and there's a new `Qualifier` for packages introduced
in private scopes (see also [Namespace vs Qualifier refactor] below),
to solve them separately from packages introduced by `build-depends`.
* cabal-install-solver needs to check that the private-scope closure
property holds (the closure of the packages in a private scope is in
the private scope) (see `Distribution.Solver.Modular.PrivateScopeClosure`).
We check that the closure holds by looking at the reverse dependency
map while traversing down the tree, at every node:
For every package in a private scope, traverse up the reverse
dependency map until a package in the same private scope is found.
If one exists, and if along the way up any package was not in the same
private scope as the packages in the two ends, we fail.
* cabal-install understands plans with private dependencies and has a
new `UserQualifier` to support constrainting packages in private
scopes using the `--constraint` flag.
Example: `--constraint=private.pkg-a.TEXT01:text == 1.2.*`
* Cabal the library uses the ghc module-renaming mechanism (also used by
Backpack) to rename modules from the packages in a private scope to
prefix them with the private scope alias. It also ensures `cabal
check` fails if there exist the package has private dependencies, as
it is currently an experimental feature which we don't necessarily
want to allow in hackage yet -- e.g. how will haddock render private
dependencies?
Namespace vs Qualifier refactor
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
We also refactored the `Namespace` vs `Qualifier` types in the solver,
clarifying their interaction such that:
* A package goal with an indepedent namespace is fully solved
indepently from other namespaces, i.e. all the dependency goals
introduced by a goal in a given namespace are also solved in that
namespace.
* In contrast, a package goal with a qualifier is shallow-solved
separately from other goals in the same namespace. The dependency
goals introduced by it will be solved unqualified (`QualTopLevel`) in
that namespace.
For example, goal `pkg-a == 0.1` in `DefaultNamespace+QualTopLevel`, and
goal `pkg-a == 0.2` in the same namespace but with `QualAlias A2 ...`
can be solved together and yield a different version of pkg-a for each
of the goals, however, the dependencies of both will be solved together
-- if they both dependend on `base`, we'd have to find a single
solution. If `pkg-a == 0.2` was in an `Independent` namespace, we could
still solve the two goals with two versions of `pkg-a`, but we could
also pick different versions for all the subdependencies of `pkg-a ==
0.2`.
Besides Namespace vs Qualifier being a welcome refactor that facilitates
implementing private dependencies, it also fixes haskell#9466 and helps with haskell#9467.
---
Co-authored-by: Rodrigo Mesquita <[email protected]>
alt-romes
added a commit
to mpickering/cabal
that referenced
this issue
Apr 15, 2024
This commit introduces so-called "private dependencies".
High-level Overview
~~~~~~~~~~~~~~~~~~~
Since its inception, Cabal has enforced the restriction that a library
must only link against one version of each package it depends on. This
ensures that all of the dependencies in the build plan work together. In
your application you use different libraries together, so it’s of
paramount importance that they all agree on what `Text` means or what a
`ByteString` is.
However, sometimes it’s desirable to allow multiple versions of the same
library into a build plan. In this case, it’s desirable to allow a
library author to specify a private dependency with the promise that its
existence will not leak from the interface of the library which uses it.
The two main use cases of private dependencies are:
- Writing benchmarks and testsuites for your library which test new
versions of your library against old versions.
- Writing libraries which can communicate with processes built against
a range of different library versions (such as cabal-install calling
./Setup).
A user specifies a private dependency in their cabal file using
`private-build-depends`. The specification starts with the name of the
private dependency scope and then contains a list of normal dependency
specifications which dictates what is included in that private scope:
private-build-depends: TEXT1 with (text == 1.2.*), TEXT2 with (text == 2.*)
Each private scope is then solved independently of all other scopes. In
this example the TEXT1 scope can choose a version of text in the 1.2.x
range and the TEXT2 scope can choose a version of text in the 2.* range.
Private scopes do not apply transitively, so the dependencies of text
will be solved in the normal top-level scope. If your program contains a
value of type Bool, that comes from the base package, which text depends
on, because the scopes are not applied transitively the same Bool value
can be passed to functions from the TEXT1 scope and TEXT2 scope.
Dependencies introduced privately can be imported into modules in the
project by prefixing the name of the private scope to an exposed module
name.
import qualified TEXT1.Data.Text as T1
import qualified TEXT2.Data.Text as T2
Closure of Private Scopes
~~~~~~~~~~~~~~~~~~~~~~~~~
Private dependency scopes can contain multiple packages. Packages in the
same scope are solved together. For example, if two packages are tightly
coupled and you need to use compatible versions with each other, then
you can list them in the same private scope. Such packages will then be
solved together, but independently of other packages.
Private scopes must be closed. A scope is closed if, whenever we have a
dependency chain P1 -> Q -> P2, in which P1 and P2 are both in a given
private scope S, then Q also belongs to the private scope S. The solver
checks this property, but doesn’t implicitly add packages into a private
scope.
Implementation
~~~~~~~~~~~~~~
To implement private dependencies we changed
* Cabal-syntax to introduce the new `private-build-depends: ALIAS (packages, in, private, scope)` syntax.
See the new type `Dependencies` and changes in `Distribution.Types.Dependency`.
* cabal-install-solver now considers both public and private
dependencies of a given package (see e.g. `solverPkgLibDeps`), has
a new constructor `PrivateScope` in `ConstraintScope` for goals in a
private scope, and there's a new `Qualifier` for packages introduced
in private scopes (see also [Namespace vs Qualifier refactor] below),
to solve them separately from packages introduced by `build-depends`.
* cabal-install-solver needs to check that the private-scope closure
property holds (the closure of the packages in a private scope is in
the private scope) (see `Distribution.Solver.Modular.PrivateScopeClosure`).
We check that the closure holds by looking at the reverse dependency
map while traversing down the tree, at every node:
For every package in a private scope, traverse up the reverse
dependency map until a package in the same private scope is found.
If one exists, and if along the way up any package was not in the same
private scope as the packages in the two ends, we fail.
* cabal-install understands plans with private dependencies and has a
new `UserQualifier` to support constrainting packages in private
scopes using the `--constraint` flag.
Example: `--constraint=private.pkg-a.TEXT01:text == 1.2.*`
* Cabal the library uses the ghc module-renaming mechanism (also used by
Backpack) to rename modules from the packages in a private scope to
prefix them with the private scope alias. It also ensures `cabal
check` fails if there exist the package has private dependencies, as
it is currently an experimental feature which we don't necessarily
want to allow in hackage yet -- e.g. how will haddock render private
dependencies?
Namespace vs Qualifier refactor
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
We also refactored the `Namespace` vs `Qualifier` types in the solver,
clarifying their interaction such that:
* A package goal with an indepedent namespace is fully solved
indepently from other namespaces, i.e. all the dependency goals
introduced by a goal in a given namespace are also solved in that
namespace.
* In contrast, a package goal with a qualifier is shallow-solved
separately from other goals in the same namespace. The dependency
goals introduced by it will be solved unqualified (`QualTopLevel`) in
that namespace.
For example, goal `pkg-a == 0.1` in `DefaultNamespace+QualTopLevel`, and
goal `pkg-a == 0.2` in the same namespace but with `QualAlias A2 ...`
can be solved together and yield a different version of pkg-a for each
of the goals, however, the dependencies of both will be solved together
-- if they both dependend on `base`, we'd have to find a single
solution. If `pkg-a == 0.2` was in an `Independent` namespace, we could
still solve the two goals with two versions of `pkg-a`, but we could
also pick different versions for all the subdependencies of `pkg-a ==
0.2`.
Besides Namespace vs Qualifier being a welcome refactor that facilitates
implementing private dependencies, it also fixes haskell#9466 and helps with haskell#9467.
---
Co-authored-by: Rodrigo Mesquita <[email protected]>
alt-romes
added a commit
to mpickering/cabal
that referenced
this issue
Apr 15, 2024
This commit introduces so-called "private dependencies".
High-level Overview
~~~~~~~~~~~~~~~~~~~
Since its inception, Cabal has enforced the restriction that a library
must only link against one version of each package it depends on. This
ensures that all of the dependencies in the build plan work together. In
your application you use different libraries together, so it’s of
paramount importance that they all agree on what `Text` means or what a
`ByteString` is.
However, sometimes it’s desirable to allow multiple versions of the same
library into a build plan. In this case, it’s desirable to allow a
library author to specify a private dependency with the promise that its
existence will not leak from the interface of the library which uses it.
The two main use cases of private dependencies are:
- Writing benchmarks and testsuites for your library which test new
versions of your library against old versions.
- Writing libraries which can communicate with processes built against
a range of different library versions (such as cabal-install calling
./Setup).
A user specifies a private dependency in their cabal file using
`private-build-depends`. The specification starts with the name of the
private dependency scope and then contains a list of normal dependency
specifications which dictates what is included in that private scope:
private-build-depends: TEXT1 with (text == 1.2.*), TEXT2 with (text == 2.*)
Each private scope is then solved independently of all other scopes. In
this example the TEXT1 scope can choose a version of text in the 1.2.x
range and the TEXT2 scope can choose a version of text in the 2.* range.
Private scopes do not apply transitively, so the dependencies of text
will be solved in the normal top-level scope. If your program contains a
value of type Bool, that comes from the base package, which text depends
on, because the scopes are not applied transitively the same Bool value
can be passed to functions from the TEXT1 scope and TEXT2 scope.
Dependencies introduced privately can be imported into modules in the
project by prefixing the name of the private scope to an exposed module
name.
import qualified TEXT1.Data.Text as T1
import qualified TEXT2.Data.Text as T2
Closure of Private Scopes
~~~~~~~~~~~~~~~~~~~~~~~~~
Private dependency scopes can contain multiple packages. Packages in the
same scope are solved together. For example, if two packages are tightly
coupled and you need to use compatible versions with each other, then
you can list them in the same private scope. Such packages will then be
solved together, but independently of other packages.
Private scopes must be closed. A scope is closed if, whenever we have a
dependency chain P1 -> Q -> P2, in which P1 and P2 are both in a given
private scope S, then Q also belongs to the private scope S. The solver
checks this property, but doesn’t implicitly add packages into a private
scope.
Implementation
~~~~~~~~~~~~~~
To implement private dependencies we changed
* Cabal-syntax to introduce the new `private-build-depends: ALIAS (packages, in, private, scope)` syntax.
See the new type `Dependencies` and changes in `Distribution.Types.Dependency`.
* cabal-install-solver now considers both public and private
dependencies of a given package (see e.g. `solverPkgLibDeps`), has
a new constructor `PrivateScope` in `ConstraintScope` for goals in a
private scope, and there's a new `Qualifier` for packages introduced
in private scopes (see also [Namespace vs Qualifier refactor] below),
to solve them separately from packages introduced by `build-depends`.
* cabal-install-solver needs to check that the private-scope closure
property holds (the closure of the packages in a private scope is in
the private scope) (see `Distribution.Solver.Modular.PrivateScopeClosure`).
We check that the closure holds by looking at the reverse dependency
map while traversing down the tree, at every node:
For every package in a private scope, traverse up the reverse
dependency map until a package in the same private scope is found.
If one exists, and if along the way up any package was not in the same
private scope as the packages in the two ends, we fail.
* cabal-install understands plans with private dependencies and has a
new `UserQualifier` to support constrainting packages in private
scopes using the `--constraint` flag.
Example: `--constraint=private.pkg-a.TEXT01:text == 1.2.*`
* Cabal the library uses the ghc module-renaming mechanism (also used by
Backpack) to rename modules from the packages in a private scope to
prefix them with the private scope alias. It also ensures `cabal
check` fails if there exist the package has private dependencies, as
it is currently an experimental feature which we don't necessarily
want to allow in hackage yet -- e.g. how will haddock render private
dependencies?
Namespace vs Qualifier refactor
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
We also refactored the `Namespace` vs `Qualifier` types in the solver,
clarifying their interaction such that:
* A package goal with an indepedent namespace is fully solved
indepently from other namespaces, i.e. all the dependency goals
introduced by a goal in a given namespace are also solved in that
namespace.
* In contrast, a package goal with a qualifier is shallow-solved
separately from other goals in the same namespace. The dependency
goals introduced by it will be solved unqualified (`QualTopLevel`) in
that namespace.
For example, goal `pkg-a == 0.1` in `DefaultNamespace+QualTopLevel`, and
goal `pkg-a == 0.2` in the same namespace but with `QualAlias A2 ...`
can be solved together and yield a different version of pkg-a for each
of the goals, however, the dependencies of both will be solved together
-- if they both dependend on `base`, we'd have to find a single
solution. If `pkg-a == 0.2` was in an `Independent` namespace, we could
still solve the two goals with two versions of `pkg-a`, but we could
also pick different versions for all the subdependencies of `pkg-a ==
0.2`.
Besides Namespace vs Qualifier being a welcome refactor that facilitates
implementing private dependencies, it also fixes haskell#9466 and helps with haskell#9467.
---
Co-authored-by: Rodrigo Mesquita <[email protected]>
alt-romes
added a commit
to mpickering/cabal
that referenced
this issue
Apr 15, 2024
This commit introduces so-called "private dependencies".
High-level Overview
~~~~~~~~~~~~~~~~~~~
Since its inception, Cabal has enforced the restriction that a library
must only link against one version of each package it depends on. This
ensures that all of the dependencies in the build plan work together. In
your application you use different libraries together, so it’s of
paramount importance that they all agree on what `Text` means or what a
`ByteString` is.
However, sometimes it’s desirable to allow multiple versions of the same
library into a build plan. In this case, it’s desirable to allow a
library author to specify a private dependency with the promise that its
existence will not leak from the interface of the library which uses it.
The two main use cases of private dependencies are:
- Writing benchmarks and testsuites for your library which test new
versions of your library against old versions.
- Writing libraries which can communicate with processes built against
a range of different library versions (such as cabal-install calling
./Setup).
A user specifies a private dependency in their cabal file using
`private-build-depends`. The specification starts with the name of the
private dependency scope and then contains a list of normal dependency
specifications which dictates what is included in that private scope:
private-build-depends: TEXT1 with (text == 1.2.*), TEXT2 with (text == 2.*)
Each private scope is then solved independently of all other scopes. In
this example the TEXT1 scope can choose a version of text in the 1.2.x
range and the TEXT2 scope can choose a version of text in the 2.* range.
Private scopes do not apply transitively, so the dependencies of text
will be solved in the normal top-level scope. If your program contains a
value of type Bool, that comes from the base package, which text depends
on, because the scopes are not applied transitively the same Bool value
can be passed to functions from the TEXT1 scope and TEXT2 scope.
Dependencies introduced privately can be imported into modules in the
project by prefixing the name of the private scope to an exposed module
name.
import qualified TEXT1.Data.Text as T1
import qualified TEXT2.Data.Text as T2
Closure of Private Scopes
~~~~~~~~~~~~~~~~~~~~~~~~~
Private dependency scopes can contain multiple packages. Packages in the
same scope are solved together. For example, if two packages are tightly
coupled and you need to use compatible versions with each other, then
you can list them in the same private scope. Such packages will then be
solved together, but independently of other packages.
Private scopes must be closed. A scope is closed if, whenever we have a
dependency chain P1 -> Q -> P2, in which P1 and P2 are both in a given
private scope S, then Q also belongs to the private scope S. The solver
checks this property, but doesn’t implicitly add packages into a private
scope.
Implementation
~~~~~~~~~~~~~~
To implement private dependencies we changed
* Cabal-syntax to introduce the new `private-build-depends: ALIAS (packages, in, private, scope)` syntax.
See the new type `Dependencies` and changes in `Distribution.Types.Dependency`.
* cabal-install-solver now considers both public and private
dependencies of a given package (see e.g. `solverPkgLibDeps`), has
a new constructor `PrivateScope` in `ConstraintScope` for goals in a
private scope, and there's a new `Qualifier` for packages introduced
in private scopes (see also [Namespace vs Qualifier refactor] below),
to solve them separately from packages introduced by `build-depends`.
* cabal-install-solver needs to check that the private-scope closure
property holds (the closure of the packages in a private scope is in
the private scope) (see `Distribution.Solver.Modular.PrivateScopeClosure`).
We check that the closure holds by looking at the reverse dependency
map while traversing down the tree, at every node:
For every package in a private scope, traverse up the reverse
dependency map until a package in the same private scope is found.
If one exists, and if along the way up any package was not in the same
private scope as the packages in the two ends, we fail.
* cabal-install understands plans with private dependencies and has a
new `UserQualifier` to support constrainting packages in private
scopes using the `--constraint` flag.
Example: `--constraint=private.pkg-a.TEXT01:text == 1.2.*`
* Cabal the library uses the ghc module-renaming mechanism (also used by
Backpack) to rename modules from the packages in a private scope to
prefix them with the private scope alias. It also ensures `cabal
check` fails if there exist the package has private dependencies, as
it is currently an experimental feature which we don't necessarily
want to allow in hackage yet -- e.g. how will haddock render private
dependencies?
Namespace vs Qualifier refactor
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
We also refactored the `Namespace` vs `Qualifier` types in the solver,
clarifying their interaction such that:
* A package goal with an indepedent namespace is fully solved
indepently from other namespaces, i.e. all the dependency goals
introduced by a goal in a given namespace are also solved in that
namespace.
* In contrast, a package goal with a qualifier is shallow-solved
separately from other goals in the same namespace. The dependency
goals introduced by it will be solved unqualified (`QualTopLevel`) in
that namespace.
For example, goal `pkg-a == 0.1` in `DefaultNamespace+QualTopLevel`, and
goal `pkg-a == 0.2` in the same namespace but with `QualAlias A2 ...`
can be solved together and yield a different version of pkg-a for each
of the goals, however, the dependencies of both will be solved together
-- if they both dependend on `base`, we'd have to find a single
solution. If `pkg-a == 0.2` was in an `Independent` namespace, we could
still solve the two goals with two versions of `pkg-a`, but we could
also pick different versions for all the subdependencies of `pkg-a ==
0.2`.
Besides Namespace vs Qualifier being a welcome refactor that facilitates
implementing private dependencies, it also fixes haskell#9466 and helps with haskell#9467.
---
Co-authored-by: Rodrigo Mesquita <[email protected]>
alt-romes
added a commit
to mpickering/cabal
that referenced
this issue
Apr 16, 2024
This commit introduces so-called "private dependencies".
High-level Overview
~~~~~~~~~~~~~~~~~~~
Since its inception, Cabal has enforced the restriction that a library
must only link against one version of each package it depends on. This
ensures that all of the dependencies in the build plan work together. In
your application you use different libraries together, so it’s of
paramount importance that they all agree on what `Text` means or what a
`ByteString` is.
However, sometimes it’s desirable to allow multiple versions of the same
library into a build plan. In this case, it’s desirable to allow a
library author to specify a private dependency with the promise that its
existence will not leak from the interface of the library which uses it.
The two main use cases of private dependencies are:
- Writing benchmarks and testsuites for your library which test new
versions of your library against old versions.
- Writing libraries which can communicate with processes built against
a range of different library versions (such as cabal-install calling
./Setup).
A user specifies a private dependency in their cabal file using
`private-build-depends`. The specification starts with the name of the
private dependency scope and then contains a list of normal dependency
specifications which dictates what is included in that private scope:
private-build-depends: TEXT1 with (text == 1.2.*), TEXT2 with (text == 2.*)
Each private scope is then solved independently of all other scopes. In
this example the TEXT1 scope can choose a version of text in the 1.2.x
range and the TEXT2 scope can choose a version of text in the 2.* range.
Private scopes do not apply transitively, so the dependencies of text
will be solved in the normal top-level scope. If your program contains a
value of type Bool, that comes from the base package, which text depends
on, because the scopes are not applied transitively the same Bool value
can be passed to functions from the TEXT1 scope and TEXT2 scope.
Dependencies introduced privately can be imported into modules in the
project by prefixing the name of the private scope to an exposed module
name.
import qualified TEXT1.Data.Text as T1
import qualified TEXT2.Data.Text as T2
Closure of Private Scopes
~~~~~~~~~~~~~~~~~~~~~~~~~
Private dependency scopes can contain multiple packages. Packages in the
same scope are solved together. For example, if two packages are tightly
coupled and you need to use compatible versions with each other, then
you can list them in the same private scope. Such packages will then be
solved together, but independently of other packages.
Private scopes must be closed. A scope is closed if, whenever we have a
dependency chain P1 -> Q -> P2, in which P1 and P2 are both in a given
private scope S, then Q also belongs to the private scope S. The solver
checks this property, but doesn’t implicitly add packages into a private
scope.
Implementation
~~~~~~~~~~~~~~
To implement private dependencies we changed
* Cabal-syntax to introduce the new `private-build-depends: ALIAS (packages, in, private, scope)` syntax.
See the new type `Dependencies` and changes in `Distribution.Types.Dependency`.
* cabal-install-solver now considers both public and private
dependencies of a given package (see e.g. `solverPkgLibDeps`), has
a new constructor `PrivateScope` in `ConstraintScope` for goals in a
private scope, and there's a new `Qualifier` for packages introduced
in private scopes (see also [Namespace vs Qualifier refactor] below),
to solve them separately from packages introduced by `build-depends`.
* cabal-install-solver needs to check that the private-scope closure
property holds (the closure of the packages in a private scope is in
the private scope) (see `Distribution.Solver.Modular.PrivateScopeClosure`).
We check that the closure holds by looking at the reverse dependency
map while traversing down the tree, at every node:
For every package in a private scope, traverse up the reverse
dependency map until a package in the same private scope is found.
If one exists, and if along the way up any package was not in the same
private scope as the packages in the two ends, we fail.
* cabal-install understands plans with private dependencies and has a
new `UserQualifier` to support constrainting packages in private
scopes using the `--constraint` flag.
Example: `--constraint=private.pkg-a.TEXT01:text == 1.2.*`
* Cabal the library uses the ghc module-renaming mechanism (also used by
Backpack) to rename modules from the packages in a private scope to
prefix them with the private scope alias. It also ensures `cabal
check` fails if there exist the package has private dependencies, as
it is currently an experimental feature which we don't necessarily
want to allow in hackage yet -- e.g. how will haddock render private
dependencies?
Namespace vs Qualifier refactor
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
We also refactored the `Namespace` vs `Qualifier` types in the solver,
clarifying their interaction such that:
* A package goal with an indepedent namespace is fully solved
indepently from other namespaces, i.e. all the dependency goals
introduced by a goal in a given namespace are also solved in that
namespace.
* In contrast, a package goal with a qualifier is shallow-solved
separately from other goals in the same namespace. The dependency
goals introduced by it will be solved unqualified (`QualTopLevel`) in
that namespace.
For example, goal `pkg-a == 0.1` in `DefaultNamespace+QualTopLevel`, and
goal `pkg-a == 0.2` in the same namespace but with `QualAlias A2 ...`
can be solved together and yield a different version of pkg-a for each
of the goals, however, the dependencies of both will be solved together
-- if they both dependend on `base`, we'd have to find a single
solution. If `pkg-a == 0.2` was in an `Independent` namespace, we could
still solve the two goals with two versions of `pkg-a`, but we could
also pick different versions for all the subdependencies of `pkg-a ==
0.2`.
Besides Namespace vs Qualifier being a welcome refactor that facilitates
implementing private dependencies, it also fixes haskell#9466 and helps with haskell#9467.
---
Co-authored-by: Rodrigo Mesquita <[email protected]>
erikd
pushed a commit
to erikd/cabal
that referenced
this issue
Apr 22, 2024
…eraction) This adds two tests for issue haskell#9467
alt-romes
added a commit
to mpickering/cabal
that referenced
this issue
Apr 26, 2024
This commit introduces so-called "private dependencies".
High-level Overview
~~~~~~~~~~~~~~~~~~~
Since its inception, Cabal has enforced the restriction that a library
must only link against one version of each package it depends on. This
ensures that all of the dependencies in the build plan work together. In
your application you use different libraries together, so it’s of
paramount importance that they all agree on what `Text` means or what a
`ByteString` is.
However, sometimes it’s desirable to allow multiple versions of the same
library into a build plan. In this case, it’s desirable to allow a
library author to specify a private dependency with the promise that its
existence will not leak from the interface of the library which uses it.
The two main use cases of private dependencies are:
- Writing benchmarks and testsuites for your library which test new
versions of your library against old versions.
- Writing libraries which can communicate with processes built against
a range of different library versions (such as cabal-install calling
./Setup).
A user specifies a private dependency in their cabal file using
`private-build-depends`. The specification starts with the name of the
private dependency scope and then contains a list of normal dependency
specifications which dictates what is included in that private scope:
private-build-depends: TEXT1 with (text == 1.2.*), TEXT2 with (text == 2.*)
Each private scope is then solved independently of all other scopes. In
this example the TEXT1 scope can choose a version of text in the 1.2.x
range and the TEXT2 scope can choose a version of text in the 2.* range.
Private scopes do not apply transitively, so the dependencies of text
will be solved in the normal top-level scope. If your program contains a
value of type Bool, that comes from the base package, which text depends
on, because the scopes are not applied transitively the same Bool value
can be passed to functions from the TEXT1 scope and TEXT2 scope.
Dependencies introduced privately can be imported into modules in the
project by prefixing the name of the private scope to an exposed module
name.
import qualified TEXT1.Data.Text as T1
import qualified TEXT2.Data.Text as T2
Closure of Private Scopes
~~~~~~~~~~~~~~~~~~~~~~~~~
Private dependency scopes can contain multiple packages. Packages in the
same scope are solved together. For example, if two packages are tightly
coupled and you need to use compatible versions with each other, then
you can list them in the same private scope. Such packages will then be
solved together, but independently of other packages.
Private scopes must be closed. A scope is closed if, whenever we have a
dependency chain P1 -> Q -> P2, in which P1 and P2 are both in a given
private scope S, then Q also belongs to the private scope S. The solver
checks this property, but doesn’t implicitly add packages into a private
scope.
Implementation
~~~~~~~~~~~~~~
To implement private dependencies we changed
* Cabal-syntax to introduce the new `private-build-depends: ALIAS (packages, in, private, scope)` syntax.
See the new type `Dependencies` and changes in `Distribution.Types.Dependency`.
* cabal-install-solver now considers both public and private
dependencies of a given package (see e.g. `solverPkgLibDeps`), has
a new constructor `PrivateScope` in `ConstraintScope` for goals in a
private scope, and there's a new `Qualifier` for packages introduced
in private scopes (see also [Namespace vs Qualifier refactor] below),
to solve them separately from packages introduced by `build-depends`.
* cabal-install-solver needs to check that the private-scope closure
property holds (the closure of the packages in a private scope is in
the private scope) (see `Distribution.Solver.Modular.PrivateScopeClosure`).
We check that the closure holds by looking at the reverse dependency
map while traversing down the tree, at every node:
For every package in a private scope, traverse up the reverse
dependency map until a package in the same private scope is found.
If one exists, and if along the way up any package was not in the same
private scope as the packages in the two ends, we fail.
* cabal-install understands plans with private dependencies and has a
new `UserQualifier` to support constrainting packages in private
scopes using the `--constraint` flag.
Example: `--constraint=private.pkg-a.TEXT01:text == 1.2.*`
* Cabal the library uses the ghc module-renaming mechanism (also used by
Backpack) to rename modules from the packages in a private scope to
prefix them with the private scope alias. It also ensures `cabal
check` fails if there exist the package has private dependencies, as
it is currently an experimental feature which we don't necessarily
want to allow in hackage yet -- e.g. how will haddock render private
dependencies?
Namespace vs Qualifier refactor
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
We also refactored the `Namespace` vs `Qualifier` types in the solver,
clarifying their interaction such that:
* A package goal with an indepedent namespace is fully solved
indepently from other namespaces, i.e. all the dependency goals
introduced by a goal in a given namespace are also solved in that
namespace.
* In contrast, a package goal with a qualifier is shallow-solved
separately from other goals in the same namespace. The dependency
goals introduced by it will be solved unqualified (`QualTopLevel`) in
that namespace.
For example, goal `pkg-a == 0.1` in `DefaultNamespace+QualTopLevel`, and
goal `pkg-a == 0.2` in the same namespace but with `QualAlias A2 ...`
can be solved together and yield a different version of pkg-a for each
of the goals, however, the dependencies of both will be solved together
-- if they both dependend on `base`, we'd have to find a single
solution. If `pkg-a == 0.2` was in an `Independent` namespace, we could
still solve the two goals with two versions of `pkg-a`, but we could
also pick different versions for all the subdependencies of `pkg-a ==
0.2`.
Besides Namespace vs Qualifier being a welcome refactor that facilitates
implementing private dependencies, it also fixes haskell#9466 and helps with haskell#9467.
---
Co-authored-by: Rodrigo Mesquita <[email protected]>
alt-romes
added a commit
to mpickering/cabal
that referenced
this issue
Apr 30, 2024
This commit introduces so-called "private dependencies".
High-level Overview
~~~~~~~~~~~~~~~~~~~
Since its inception, Cabal has enforced the restriction that a library
must only link against one version of each package it depends on. This
ensures that all of the dependencies in the build plan work together. In
your application you use different libraries together, so it’s of
paramount importance that they all agree on what `Text` means or what a
`ByteString` is.
However, sometimes it’s desirable to allow multiple versions of the same
library into a build plan. In this case, it’s desirable to allow a
library author to specify a private dependency with the promise that its
existence will not leak from the interface of the library which uses it.
The two main use cases of private dependencies are:
- Writing benchmarks and testsuites for your library which test new
versions of your library against old versions.
- Writing libraries which can communicate with processes built against
a range of different library versions (such as cabal-install calling
./Setup).
A user specifies a private dependency in their cabal file using
`private-build-depends`. The specification starts with the name of the
private dependency scope and then contains a list of normal dependency
specifications which dictates what is included in that private scope:
private-build-depends: TEXT1 with (text == 1.2.*), TEXT2 with (text == 2.*)
Each private scope is then solved independently of all other scopes. In
this example the TEXT1 scope can choose a version of text in the 1.2.x
range and the TEXT2 scope can choose a version of text in the 2.* range.
Private scopes do not apply transitively, so the dependencies of text
will be solved in the normal top-level scope. If your program contains a
value of type Bool, that comes from the base package, which text depends
on, because the scopes are not applied transitively the same Bool value
can be passed to functions from the TEXT1 scope and TEXT2 scope.
Dependencies introduced privately can be imported into modules in the
project by prefixing the name of the private scope to an exposed module
name.
import qualified TEXT1.Data.Text as T1
import qualified TEXT2.Data.Text as T2
Closure of Private Scopes
~~~~~~~~~~~~~~~~~~~~~~~~~
Private dependency scopes can contain multiple packages. Packages in the
same scope are solved together. For example, if two packages are tightly
coupled and you need to use compatible versions with each other, then
you can list them in the same private scope. Such packages will then be
solved together, but independently of other packages.
Private scopes must be closed. A scope is closed if, whenever we have a
dependency chain P1 -> Q -> P2, in which P1 and P2 are both in a given
private scope S, then Q also belongs to the private scope S. The solver
checks this property, but doesn’t implicitly add packages into a private
scope.
Implementation
~~~~~~~~~~~~~~
To implement private dependencies we changed
* Cabal-syntax to introduce the new `private-build-depends: ALIAS (packages, in, private, scope)` syntax.
See the new type `Dependencies` and changes in `Distribution.Types.Dependency`.
* cabal-install-solver now considers both public and private
dependencies of a given package (see e.g. `solverPkgLibDeps`), has
a new constructor `PrivateScope` in `ConstraintScope` for goals in a
private scope, and there's a new `Qualifier` for packages introduced
in private scopes (see also [Namespace vs Qualifier refactor] below),
to solve them separately from packages introduced by `build-depends`.
* cabal-install-solver needs to check that the private-scope closure
property holds (the closure of the packages in a private scope is in
the private scope) (see `Distribution.Solver.Modular.PrivateScopeClosure`).
We check that the closure holds by looking at the reverse dependency
map while traversing down the tree, at every node:
For every package in a private scope, traverse up the reverse
dependency map until a package in the same private scope is found.
If one exists, and if along the way up any package was not in the same
private scope as the packages in the two ends, we fail.
* cabal-install understands plans with private dependencies and has a
new `UserQualifier` to support constrainting packages in private
scopes using the `--constraint` flag.
Example: `--constraint=private.pkg-a.TEXT01:text == 1.2.*`
* Cabal the library uses the ghc module-renaming mechanism (also used by
Backpack) to rename modules from the packages in a private scope to
prefix them with the private scope alias. It also ensures `cabal
check` fails if there exist the package has private dependencies, as
it is currently an experimental feature which we don't necessarily
want to allow in hackage yet -- e.g. how will haddock render private
dependencies?
Namespace vs Qualifier refactor
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
We also refactored the `Namespace` vs `Qualifier` types in the solver,
clarifying their interaction such that:
* A package goal with an indepedent namespace is fully solved
indepently from other namespaces, i.e. all the dependency goals
introduced by a goal in a given namespace are also solved in that
namespace.
* In contrast, a package goal with a qualifier is shallow-solved
separately from other goals in the same namespace. The dependency
goals introduced by it will be solved unqualified (`QualTopLevel`) in
that namespace.
For example, goal `pkg-a == 0.1` in `DefaultNamespace+QualTopLevel`, and
goal `pkg-a == 0.2` in the same namespace but with `QualAlias A2 ...`
can be solved together and yield a different version of pkg-a for each
of the goals, however, the dependencies of both will be solved together
-- if they both dependend on `base`, we'd have to find a single
solution. If `pkg-a == 0.2` was in an `Independent` namespace, we could
still solve the two goals with two versions of `pkg-a`, but we could
also pick different versions for all the subdependencies of `pkg-a ==
0.2`.
Besides Namespace vs Qualifier being a welcome refactor that facilitates
implementing private dependencies, it also fixes haskell#9466 and helps with haskell#9467.
---
Co-authored-by: Rodrigo Mesquita <[email protected]>
alt-romes
added a commit
to mpickering/cabal
that referenced
this issue
Apr 30, 2024
This commit introduces so-called "private dependencies".
High-level Overview
~~~~~~~~~~~~~~~~~~~
Since its inception, Cabal has enforced the restriction that a library
must only link against one version of each package it depends on. This
ensures that all of the dependencies in the build plan work together. In
your application you use different libraries together, so it’s of
paramount importance that they all agree on what `Text` means or what a
`ByteString` is.
However, sometimes it’s desirable to allow multiple versions of the same
library into a build plan. In this case, it’s desirable to allow a
library author to specify a private dependency with the promise that its
existence will not leak from the interface of the library which uses it.
The two main use cases of private dependencies are:
- Writing benchmarks and testsuites for your library which test new
versions of your library against old versions.
- Writing libraries which can communicate with processes built against
a range of different library versions (such as cabal-install calling
./Setup).
A user specifies a private dependency in their cabal file using
`private-build-depends`. The specification starts with the name of the
private dependency scope and then contains a list of normal dependency
specifications which dictates what is included in that private scope:
private-build-depends: TEXT1 with (text == 1.2.*), TEXT2 with (text == 2.*)
Each private scope is then solved independently of all other scopes. In
this example the TEXT1 scope can choose a version of text in the 1.2.x
range and the TEXT2 scope can choose a version of text in the 2.* range.
Private scopes do not apply transitively, so the dependencies of text
will be solved in the normal top-level scope. If your program contains a
value of type Bool, that comes from the base package, which text depends
on, because the scopes are not applied transitively the same Bool value
can be passed to functions from the TEXT1 scope and TEXT2 scope.
Dependencies introduced privately can be imported into modules in the
project by prefixing the name of the private scope to an exposed module
name.
import qualified TEXT1.Data.Text as T1
import qualified TEXT2.Data.Text as T2
Closure of Private Scopes
~~~~~~~~~~~~~~~~~~~~~~~~~
Private dependency scopes can contain multiple packages. Packages in the
same scope are solved together. For example, if two packages are tightly
coupled and you need to use compatible versions with each other, then
you can list them in the same private scope. Such packages will then be
solved together, but independently of other packages.
Private scopes must be closed. A scope is closed if, whenever we have a
dependency chain P1 -> Q -> P2, in which P1 and P2 are both in a given
private scope S, then Q also belongs to the private scope S. The solver
checks this property, but doesn’t implicitly add packages into a private
scope.
Implementation
~~~~~~~~~~~~~~
To implement private dependencies we changed
* Cabal-syntax to introduce the new `private-build-depends: ALIAS (packages, in, private, scope)` syntax.
See the new type `Dependencies` and changes in `Distribution.Types.Dependency`.
* cabal-install-solver now considers both public and private
dependencies of a given package (see e.g. `solverPkgLibDeps`), has
a new constructor `PrivateScope` in `ConstraintScope` for goals in a
private scope, and there's a new `Qualifier` for packages introduced
in private scopes (see also [Namespace vs Qualifier refactor] below),
to solve them separately from packages introduced by `build-depends`.
* cabal-install-solver needs to check that the private-scope closure
property holds (the closure of the packages in a private scope is in
the private scope) (see `Distribution.Solver.Modular.PrivateScopeClosure`).
We check that the closure holds by looking at the reverse dependency
map while traversing down the tree, at every node:
For every package in a private scope, traverse up the reverse
dependency map until a package in the same private scope is found.
If one exists, and if along the way up any package was not in the same
private scope as the packages in the two ends, we fail.
* cabal-install understands plans with private dependencies and has a
new `UserQualifier` to support constrainting packages in private
scopes using the `--constraint` flag.
Example: `--constraint=private.pkg-a.TEXT01:text == 1.2.*`
* Cabal the library uses the ghc module-renaming mechanism (also used by
Backpack) to rename modules from the packages in a private scope to
prefix them with the private scope alias. It also ensures `cabal
check` fails if there exist the package has private dependencies, as
it is currently an experimental feature which we don't necessarily
want to allow in hackage yet -- e.g. how will haddock render private
dependencies?
Namespace vs Qualifier refactor
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
We also refactored the `Namespace` vs `Qualifier` types in the solver,
clarifying their interaction such that:
* A package goal with an indepedent namespace is fully solved
indepently from other namespaces, i.e. all the dependency goals
introduced by a goal in a given namespace are also solved in that
namespace.
* In contrast, a package goal with a qualifier is shallow-solved
separately from other goals in the same namespace. The dependency
goals introduced by it will be solved unqualified (`QualTopLevel`) in
that namespace.
For example, goal `pkg-a == 0.1` in `DefaultNamespace+QualTopLevel`, and
goal `pkg-a == 0.2` in the same namespace but with `QualAlias A2 ...`
can be solved together and yield a different version of pkg-a for each
of the goals, however, the dependencies of both will be solved together
-- if they both dependend on `base`, we'd have to find a single
solution. If `pkg-a == 0.2` was in an `Independent` namespace, we could
still solve the two goals with two versions of `pkg-a`, but we could
also pick different versions for all the subdependencies of `pkg-a ==
0.2`.
Besides Namespace vs Qualifier being a welcome refactor that facilitates
implementing private dependencies, it also fixes haskell#9466 and helps with haskell#9467.
---
Co-authored-by: Rodrigo Mesquita <[email protected]>
alt-romes
added a commit
to mpickering/cabal
that referenced
this issue
Apr 30, 2024
This commit introduces so-called "private dependencies".
High-level Overview
~~~~~~~~~~~~~~~~~~~
Since its inception, Cabal has enforced the restriction that a library
must only link against one version of each package it depends on. This
ensures that all of the dependencies in the build plan work together. In
your application you use different libraries together, so it’s of
paramount importance that they all agree on what `Text` means or what a
`ByteString` is.
However, sometimes it’s desirable to allow multiple versions of the same
library into a build plan. In this case, it’s desirable to allow a
library author to specify a private dependency with the promise that its
existence will not leak from the interface of the library which uses it.
The two main use cases of private dependencies are:
- Writing benchmarks and testsuites for your library which test new
versions of your library against old versions.
- Writing libraries which can communicate with processes built against
a range of different library versions (such as cabal-install calling
./Setup).
A user specifies a private dependency in their cabal file using
`private-build-depends`. The specification starts with the name of the
private dependency scope and then contains a list of normal dependency
specifications which dictates what is included in that private scope:
private-build-depends: TEXT1 with (text == 1.2.*), TEXT2 with (text == 2.*)
Each private scope is then solved independently of all other scopes. In
this example the TEXT1 scope can choose a version of text in the 1.2.x
range and the TEXT2 scope can choose a version of text in the 2.* range.
Private scopes do not apply transitively, so the dependencies of text
will be solved in the normal top-level scope. If your program contains a
value of type Bool, that comes from the base package, which text depends
on, because the scopes are not applied transitively the same Bool value
can be passed to functions from the TEXT1 scope and TEXT2 scope.
Dependencies introduced privately can be imported into modules in the
project by prefixing the name of the private scope to an exposed module
name.
import qualified TEXT1.Data.Text as T1
import qualified TEXT2.Data.Text as T2
Closure of Private Scopes
~~~~~~~~~~~~~~~~~~~~~~~~~
Private dependency scopes can contain multiple packages. Packages in the
same scope are solved together. For example, if two packages are tightly
coupled and you need to use compatible versions with each other, then
you can list them in the same private scope. Such packages will then be
solved together, but independently of other packages.
Private scopes must be closed. A scope is closed if, whenever we have a
dependency chain P1 -> Q -> P2, in which P1 and P2 are both in a given
private scope S, then Q also belongs to the private scope S. The solver
checks this property, but doesn’t implicitly add packages into a private
scope.
Implementation
~~~~~~~~~~~~~~
To implement private dependencies we changed
* Cabal-syntax to introduce the new `private-build-depends: ALIAS (packages, in, private, scope)` syntax.
See the new type `Dependencies` and changes in `Distribution.Types.Dependency`.
* cabal-install-solver now considers both public and private
dependencies of a given package (see e.g. `solverPkgLibDeps`), has
a new constructor `PrivateScope` in `ConstraintScope` for goals in a
private scope, and there's a new `Qualifier` for packages introduced
in private scopes (see also [Namespace vs Qualifier refactor] below),
to solve them separately from packages introduced by `build-depends`.
* cabal-install-solver needs to check that the private-scope closure
property holds (the closure of the packages in a private scope is in
the private scope) (see `Distribution.Solver.Modular.PrivateScopeClosure`).
We check that the closure holds by looking at the reverse dependency
map while traversing down the tree, at every node:
For every package in a private scope, traverse up the reverse
dependency map until a package in the same private scope is found.
If one exists, and if along the way up any package was not in the same
private scope as the packages in the two ends, we fail.
* cabal-install understands plans with private dependencies and has a
new `UserQualifier` to support constrainting packages in private
scopes using the `--constraint` flag.
Example: `--constraint=private.pkg-a.TEXT01:text == 1.2.*`
* Cabal the library uses the ghc module-renaming mechanism (also used by
Backpack) to rename modules from the packages in a private scope to
prefix them with the private scope alias. It also ensures `cabal
check` fails if there exist the package has private dependencies, as
it is currently an experimental feature which we don't necessarily
want to allow in hackage yet -- e.g. how will haddock render private
dependencies?
Namespace vs Qualifier refactor
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
We also refactored the `Namespace` vs `Qualifier` types in the solver,
clarifying their interaction such that:
* A package goal with an indepedent namespace is fully solved
indepently from other namespaces, i.e. all the dependency goals
introduced by a goal in a given namespace are also solved in that
namespace.
* In contrast, a package goal with a qualifier is shallow-solved
separately from other goals in the same namespace. The dependency
goals introduced by it will be solved unqualified (`QualTopLevel`) in
that namespace.
For example, goal `pkg-a == 0.1` in `DefaultNamespace+QualTopLevel`, and
goal `pkg-a == 0.2` in the same namespace but with `QualAlias A2 ...`
can be solved together and yield a different version of pkg-a for each
of the goals, however, the dependencies of both will be solved together
-- if they both dependend on `base`, we'd have to find a single
solution. If `pkg-a == 0.2` was in an `Independent` namespace, we could
still solve the two goals with two versions of `pkg-a`, but we could
also pick different versions for all the subdependencies of `pkg-a ==
0.2`.
Besides Namespace vs Qualifier being a welcome refactor that facilitates
implementing private dependencies, it also fixes haskell#9466 and helps with haskell#9467.
---
Co-authored-by: Rodrigo Mesquita <[email protected]>
mergify bot
added a commit
that referenced
this issue
May 16, 2024
* testsuite: Add tests for #9467 (base shim, setup qualifier interaction) This adds two tests for issue #9467 (cherry picked from commit 3851043) * testsuite: Add two tests for independent goals (#9466) These tests check how constraints interact with the --independent-goals flag. (cherry picked from commit b169cd4) * testsuite: Add some tests for setup component scope interacts with stanza flags These tests check how the setup qualified scope interacts with the stanza flags (specified on the top-level and with the any qualifier) (cherry picked from commit 573c15d) --------- Co-authored-by: Matthew Pickering <[email protected]> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
alt-romes
added a commit
to alt-romes/cabal
that referenced
this issue
Jul 1, 2024
This commit introduces so-called "private dependencies".
High-level Overview
~~~~~~~~~~~~~~~~~~~
Since its inception, Cabal has enforced the restriction that a library
must only link against one version of each package it depends on. This
ensures that all of the dependencies in the build plan work together. In
your application you use different libraries together, so it’s of
paramount importance that they all agree on what `Text` means or what a
`ByteString` is.
However, sometimes it’s desirable to allow multiple versions of the same
library into a build plan. In this case, it’s desirable to allow a
library author to specify a private dependency with the promise that its
existence will not leak from the interface of the library which uses it.
The two main use cases of private dependencies are:
- Writing benchmarks and testsuites for your library which test new
versions of your library against old versions.
- Writing libraries which can communicate with processes built against
a range of different library versions (such as cabal-install calling
./Setup).
A user specifies a private dependency in their cabal file using
`private-build-depends`. The specification starts with the name of the
private dependency scope and then contains a list of normal dependency
specifications which dictates what is included in that private scope:
private-build-depends: TEXT1 with (text == 1.2.*), TEXT2 with (text == 2.*)
Each private scope is then solved independently of all other scopes. In
this example the TEXT1 scope can choose a version of text in the 1.2.x
range and the TEXT2 scope can choose a version of text in the 2.* range.
Private scopes do not apply transitively, so the dependencies of text
will be solved in the normal top-level scope. If your program contains a
value of type Bool, that comes from the base package, which text depends
on, because the scopes are not applied transitively the same Bool value
can be passed to functions from the TEXT1 scope and TEXT2 scope.
Dependencies introduced privately can be imported into modules in the
project by prefixing the name of the private scope to an exposed module
name.
import qualified TEXT1.Data.Text as T1
import qualified TEXT2.Data.Text as T2
Closure of Private Scopes
~~~~~~~~~~~~~~~~~~~~~~~~~
Private dependency scopes can contain multiple packages. Packages in the
same scope are solved together. For example, if two packages are tightly
coupled and you need to use compatible versions with each other, then
you can list them in the same private scope. Such packages will then be
solved together, but independently of other packages.
Private scopes must be closed. A scope is closed if, whenever we have a
dependency chain P1 -> Q -> P2, in which P1 and P2 are both in a given
private scope S, then Q also belongs to the private scope S. The solver
checks this property, but doesn’t implicitly add packages into a private
scope.
Implementation
~~~~~~~~~~~~~~
To implement private dependencies we changed
* Cabal-syntax to introduce the new `private-build-depends: ALIAS (packages, in, private, scope)` syntax.
See the new type `Dependencies` and changes in `Distribution.Types.Dependency`.
* cabal-install-solver now considers both public and private
dependencies of a given package (see e.g. `solverPkgLibDeps`), has
a new constructor `PrivateScope` in `ConstraintScope` for goals in a
private scope, and there's a new `Qualifier` for packages introduced
in private scopes (see also [Namespace vs Qualifier refactor] below),
to solve them separately from packages introduced by `build-depends`.
* cabal-install-solver needs to check that the private-scope closure
property holds (the closure of the packages in a private scope is in
the private scope) (see `Distribution.Solver.Modular.PrivateScopeClosure`).
We check that the closure holds by looking at the reverse dependency
map while traversing down the tree, at every node:
For every package in a private scope, traverse up the reverse
dependency map until a package in the same private scope is found.
If one exists, and if along the way up any package was not in the same
private scope as the packages in the two ends, we fail.
* cabal-install understands plans with private dependencies and has a
new `UserQualifier` to support constrainting packages in private
scopes using the `--constraint` flag.
Example: `--constraint=private.pkg-a.TEXT01:text == 1.2.*`
* Cabal the library uses the ghc module-renaming mechanism (also used by
Backpack) to rename modules from the packages in a private scope to
prefix them with the private scope alias. It also ensures `cabal
check` fails if there exist the package has private dependencies, as
it is currently an experimental feature which we don't necessarily
want to allow in hackage yet -- e.g. how will haddock render private
dependencies?
Namespace vs Qualifier refactor
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
We also refactored the `Namespace` vs `Qualifier` types in the solver,
clarifying their interaction such that:
* A package goal with an indepedent namespace is fully solved
indepently from other namespaces, i.e. all the dependency goals
introduced by a goal in a given namespace are also solved in that
namespace.
* In contrast, a package goal with a qualifier is shallow-solved
separately from other goals in the same namespace. The dependency
goals introduced by it will be solved unqualified (`QualTopLevel`) in
that namespace.
For example, goal `pkg-a == 0.1` in `DefaultNamespace+QualTopLevel`, and
goal `pkg-a == 0.2` in the same namespace but with `QualAlias A2 ...`
can be solved together and yield a different version of pkg-a for each
of the goals, however, the dependencies of both will be solved together
-- if they both dependend on `base`, we'd have to find a single
solution. If `pkg-a == 0.2` was in an `Independent` namespace, we could
still solve the two goals with two versions of `pkg-a`, but we could
also pick different versions for all the subdependencies of `pkg-a ==
0.2`.
Besides Namespace vs Qualifier being a welcome refactor that facilitates
implementing private dependencies, it also fixes haskell#9466 and helps with haskell#9467.
---
Co-authored-by: Rodrigo Mesquita <[email protected]>
alt-romes
added a commit
to alt-romes/cabal
that referenced
this issue
Aug 2, 2024
This commit introduces so-called "private dependencies".
High-level Overview
~~~~~~~~~~~~~~~~~~~
Since its inception, Cabal has enforced the restriction that a library
must only link against one version of each package it depends on. This
ensures that all of the dependencies in the build plan work together. In
your application you use different libraries together, so it’s of
paramount importance that they all agree on what `Text` means or what a
`ByteString` is.
However, sometimes it’s desirable to allow multiple versions of the same
library into a build plan. In this case, it’s desirable to allow a
library author to specify a private dependency with the promise that its
existence will not leak from the interface of the library which uses it.
The two main use cases of private dependencies are:
- Writing benchmarks and testsuites for your library which test new
versions of your library against old versions.
- Writing libraries which can communicate with processes built against
a range of different library versions (such as cabal-install calling
./Setup).
A user specifies a private dependency in their cabal file using
`private-build-depends`. The specification starts with the name of the
private dependency scope and then contains a list of normal dependency
specifications which dictates what is included in that private scope:
private-build-depends: TEXT1 with (text == 1.2.*), TEXT2 with (text == 2.*)
Each private scope is then solved independently of all other scopes. In
this example the TEXT1 scope can choose a version of text in the 1.2.x
range and the TEXT2 scope can choose a version of text in the 2.* range.
Private scopes do not apply transitively, so the dependencies of text
will be solved in the normal top-level scope. If your program contains a
value of type Bool, that comes from the base package, which text depends
on, because the scopes are not applied transitively the same Bool value
can be passed to functions from the TEXT1 scope and TEXT2 scope.
Dependencies introduced privately can be imported into modules in the
project by prefixing the name of the private scope to an exposed module
name.
import qualified TEXT1.Data.Text as T1
import qualified TEXT2.Data.Text as T2
Closure of Private Scopes
~~~~~~~~~~~~~~~~~~~~~~~~~
Private dependency scopes can contain multiple packages. Packages in the
same scope are solved together. For example, if two packages are tightly
coupled and you need to use compatible versions with each other, then
you can list them in the same private scope. Such packages will then be
solved together, but independently of other packages.
Private scopes must be closed. A scope is closed if, whenever we have a
dependency chain P1 -> Q -> P2, in which P1 and P2 are both in a given
private scope S, then Q also belongs to the private scope S. The solver
checks this property, but doesn’t implicitly add packages into a private
scope.
Implementation
~~~~~~~~~~~~~~
To implement private dependencies we changed
* Cabal-syntax to introduce the new `private-build-depends: ALIAS (packages, in, private, scope)` syntax.
See the new type `Dependencies` and changes in `Distribution.Types.Dependency`.
* cabal-install-solver now considers both public and private
dependencies of a given package (see e.g. `solverPkgLibDeps`), has
a new constructor `PrivateScope` in `ConstraintScope` for goals in a
private scope, and there's a new `Qualifier` for packages introduced
in private scopes (see also [Namespace vs Qualifier refactor] below),
to solve them separately from packages introduced by `build-depends`.
* cabal-install-solver needs to check that the private-scope closure
property holds (the closure of the packages in a private scope is in
the private scope) (see `Distribution.Solver.Modular.PrivateScopeClosure`).
We check that the closure holds by looking at the reverse dependency
map while traversing down the tree, at every node:
For every package in a private scope, traverse up the reverse
dependency map until a package in the same private scope is found.
If one exists, and if along the way up any package was not in the same
private scope as the packages in the two ends, we fail.
* cabal-install understands plans with private dependencies and has a
new `UserQualifier` to support constrainting packages in private
scopes using the `--constraint` flag.
Example: `--constraint=private.pkg-a.TEXT01:text == 1.2.*`
* Cabal the library uses the ghc module-renaming mechanism (also used by
Backpack) to rename modules from the packages in a private scope to
prefix them with the private scope alias. It also ensures `cabal
check` fails if there exist the package has private dependencies, as
it is currently an experimental feature which we don't necessarily
want to allow in hackage yet -- e.g. how will haddock render private
dependencies?
Namespace vs Qualifier refactor
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
We also refactored the `Namespace` vs `Qualifier` types in the solver,
clarifying their interaction such that:
* A package goal with an indepedent namespace is fully solved
indepently from other namespaces, i.e. all the dependency goals
introduced by a goal in a given namespace are also solved in that
namespace.
* In contrast, a package goal with a qualifier is shallow-solved
separately from other goals in the same namespace. The dependency
goals introduced by it will be solved unqualified (`QualTopLevel`) in
that namespace.
For example, goal `pkg-a == 0.1` in `DefaultNamespace+QualTopLevel`, and
goal `pkg-a == 0.2` in the same namespace but with `QualAlias A2 ...`
can be solved together and yield a different version of pkg-a for each
of the goals, however, the dependencies of both will be solved together
-- if they both dependend on `base`, we'd have to find a single
solution. If `pkg-a == 0.2` was in an `Independent` namespace, we could
still solve the two goals with two versions of `pkg-a`, but we could
also pick different versions for all the subdependencies of `pkg-a ==
0.2`.
Besides Namespace vs Qualifier being a welcome refactor that facilitates
implementing private dependencies, it also fixes haskell#9466 and helps with haskell#9467.
---
Co-authored-by: Rodrigo Mesquita <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
The setup component should be solved independently of the other components, so the decisions about constraints taken in the setup component should not be influenced by decisions taken in other components.
This is witnessed by the following test where the setup and library depend on different versions of
base.However, if the situation is slightly more complicated, and we have the situation where
base-3is a shim forbase-4then things quickly go wrong:The issue is that with the current structure of qualified goals, the qualified goal introduced by the base shim obliterates the qualified goal introduced by the setup dependency and therefore the
basedependency in the main package and setup stanza is solved under the same qualifier (and hence fails).I uncovered this issue when working on extending the qualified goals mechanism, so on it's own it can be filled into the niche but it highlights a more general problem with how the code around qualified goals is structured.
The text was updated successfully, but these errors were encountered: