New plotting tools

Added new plot functionalities, following adapted work from PR #21

Modification in Sinkhorn of how implicit parameters are passed onto solver.

changes in how parameters are passed on to implicit solver. Instead of passing these few parameters (the solver, ridge regularizations and the specification of whether the system was symmetric or not) as a dictionary linear_solve_kwargs, the parameters are now passed directly to the sinkhorn function. This removes a bug that was prevening the jit=True option when running sinkhorn to work when doing implicit differentiation.

Addition of ICNN, correction of bugs related to passing empty arguments in Geometry and GW

What's Changed

• fix point_clouds.ipynb for now. by @alantian in #16
• Solving Issue #17 when no sinkhorn_kwargs is passed to gromov_wasserstein by @marcocuturi in #19
• Solving Issue #18 when no epsilon is passed to a geometry defined with a kernel matrix by @marcocuturi in #20
• Input Convex Neural Network Architecture and Tests by @bunnech in #15

New Contributors

• @marcocuturi made their first contribution in #19
• @bunnech made their first contribution in #15

Full Changelog: 0.1.17...0.1.18

New cost function for geometry and tools to plot

In this release:

• a new cost function (Cosine distance)
• tools to plot OT are migrated from a NB to codebase.

preconditioning the implicit function theorem linear system using a function

In this release, we provide a new approach to stabilize the resolution of the linear system encountered when differentiating the optimal transport solution w.r.t. solutions.

To that end, we introduce a precondition_fun argument in the sinkhorn function to improve the resolution of the implicit function theorem differentiation of the outputs of the sinkhorn algorithm. Sinkhorn's first order condition (FOC) amounts to ∇ Energy = (marginal of transport) - (marginal fitting penalty) = 0.

This commit introduces the possibility of differentiating instead
precondition_fun(marginal of transport) - precondition_fun(marginal of transport) = 0.

For Sinkhorn, we draw inspiration from this paper to use by default the function 𝜀 log. This provides a numerically more stable approach.

Anderson acceleration, a few changes in epsilon scheduling, segment sinkhorn divergence

In this new release:

• Anderson acceleration implemented for Sinkhorn.
• Simplifications of the way epsilon scheduling is now defined. the init value should be understood as a multiple of the scale parameter, if one has been passed.
• Addition of segment_sinkhorn_divergence, a convenient way to compute several sinkhorn divergences of various subsets of points in two point clouds X, Y.

new tests to check jacobian, small fixes in: transport interface, handling of kernel_ridge in implicit differentiation of sinkhorn

In this release:

• more tests to check accuracy of differentiation of the optimal potentials w.r.t. input parameters.
• fixes issues in transport.Transport handling of **kwargs, that must be routed to the epsilon_scheduler object.
• cancel ridge_kernel in the unbalanced case when doing implicit differentiation.

More flexibility in soft_sorting operations

In this release:

• it is now possible to parameterize the squashing function used by the sort.py module to squash/squeeze input values in [0,1]. The default behavior remains sigmoid of whitened values, but this can be now changed to any other approach, including using the values directly without squashing them (e.g. squashing_fun= lambda x : x).
• handling of keyword arguments kwargs is facilitated in the sort module. Before user was supposed to pass a dictionary, now kwargs are passed directly.

New rules to set momentum

The momentum parameter in sinkhorn can be useful to speed up its iterations. This new API allows to define that parameter with a more fine grained tuning.

It is now not only possible to set the momentum (instead of momentum_strategy), it's also possible to set a number of iterations after which that value is refreshed using a simple rule on the decrease of the marginal error of the algorithm. If you are using the momentum_strategy='Lehmann' option then this would be equivalent to momentum=1.0, chg_momentum_from=100

Sqrt matrix jacobian + autotune options for epsilon

In this release:

• implicit differentiation for sqrtm. Since sqrtm operates on n x n matrices, and an implicit solve is typically cubic, complexity is O(n^6), therefore only tractable for very small n.
• autotune epsilon options. When running Transport or instantiating a Geometry, the epsilon value is by default 1/20th of the mean of the cost matrix. When passing an epsilon value, Geometry assumes the value is the correct one. If setting relative_epsilon = True, that value will be interpreted as a fraction of the mean of the cost matrix.