diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index a9c53187d8..ec4a34664c 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -39,7 +39,7 @@ Even providing a single new method is a good contribution. A main contribution you can provide is another algorithm that is not yet included in the package. -An algorithm is always based on a concrete type of a [`Problem`](https://manoptjl.org/stable/plans/index.html#Problems-1) storing the main information of the task and a concrete type of an [`Option`](https://manoptjl.org/stable/plans/index.html#Options-1from) storing all information that needs to be known to the solver in general. The actual algorithm is split into an initialization phase, see [`initialize_solver!`](https://manoptjl.org/stable/solvers/index.html#Manopt.initialize_solver!), and the implementation of the `i`th step of the solver itself, see before the iterative procedure, see [`step_solver!`](https://manoptjl.org/stable/solvers/index.html#Manopt.step_solver!). +An algorithm is always based on a concrete type of a [`AbstractManoptProblem`](https://manoptjl.org/stable/plans/index.html#AbstractManoptProblems-1) storing the main information of the task and a concrete type of an [`Option`](https://manoptjl.org/stable/plans/index.html#Options-1from) storing all information that needs to be known to the solver in general. The actual algorithm is split into an initialization phase, see [`initialize_solver!`](https://manoptjl.org/stable/solvers/index.html#Manopt.initialize_solver!), and the implementation of the `i`th step of the solver itself, see before the iterative procedure, see [`step_solver!`](https://manoptjl.org/stable/solvers/index.html#Manopt.step_solver!). For these two functions, it would be great if a new algorithm uses functions from the [`ManifoldsBase.jl`](https://juliamanifolds.github.io/Manifolds.jl/latest/interface.html) interface as generically as possible. For example, if possible use [`retract!(M,q,p,X)`](https://juliamanifolds.github.io/Manifolds.jl/latest/interface.html#ManifoldsBase.retract!-Tuple{AbstractManifold,Any,Any,Any}) in favor of [`exp!(M,q,p,X)`](https://juliamanifolds.github.io/Manifolds.jl/latest/interface.html#ManifoldsBase.exp!-Tuple{AbstractManifold,Any,Any,Any}) to perform a step starting in `p` in direction `X` (in place of `q`), since the exponential map might be too expensive to evaluate or might not be available on a certain manifold. See [Retractions and inverse retractions](https://juliamanifolds.github.io/Manifolds.jl/latest/interface.html#Retractions-and-inverse-Retractions) for more details. Further, if possible, prefer [`retract!(M,q,p,X)`](https://juliamanifolds.github.io/Manifolds.jl/latest/interface.html#ManifoldsBase.retract!-Tuple{AbstractManifold,Any,Any,Any}) in favor of [`retract(M,p,X)`](https://juliamanifolds.github.io/Manifolds.jl/latest/interface.html#ManifoldsBase.retract-Tuple{AbstractManifold,Any,Any}), since a computation in place of a suitable variable `q` reduces memory allocations. @@ -58,9 +58,9 @@ We run [`JuliaFormatter.jl`](https://github.com/domluna/JuliaFormatter.jl) on th We also follow a few internal conventions: -- It is preferred that the `Problem`'s struct contains information about the general structure of the problem. +- It is preferred that the `AbstractManoptProblem`'s struct contains information about the general structure of the problem. - Any implemented function should be accompanied by its mathematical formulae if a closed form exists. -- Problem and option structures are stored within the `plan/` folder and sorted by properties of the problem and/or solver at hand. +- AbstractManoptProblem and option structures are stored within the `plan/` folder and sorted by properties of the problem and/or solver at hand. - Within the source code of one algorithm, the high level interface should be first, then the initialization, then the step. - Otherwise an alphabetical order is preferable. - The above implies that the mutating variant of a function follows the non-mutating variant. diff --git a/Project.toml b/Project.toml index e185bccba2..1a3b845e51 100644 --- a/Project.toml +++ b/Project.toml @@ -1,7 +1,7 @@ name = "Manopt" uuid = "0fc0a36d-df90-57f3-8f93-d78a9fc72bb5" authors = ["Ronny Bergmann "] -version = "0.3.47" +version = "0.4.0" [deps] ColorSchemes = "35d6a980-a343-548e-a6ea-1d62b119f2f4" diff --git a/benchmarks/benchmark_subgradient.jl b/benchmarks/benchmark_subgradient.jl index 4bc19aa944..540505087f 100644 --- a/benchmarks/benchmark_subgradient.jl +++ b/benchmarks/benchmark_subgradient.jl @@ -23,8 +23,8 @@ function ∂f!(M, X, y) return X end x2 = copy(x0) -subgradient_method!(M, f, ∂f!, x2; evaluation=MutatingEvaluation()) -@btime subgradient_method!($M, $f, $∂f!, x3; evaluation=$(MutatingEvaluation())) setup = ( +subgradient_method!(M, f, ∂f!, x2; evaluation=InplaceEvaluation()) +@btime subgradient_method!($M, $f, $∂f!, x3; evaluation=$(InplaceEvaluation())) setup = ( x3 = deepcopy($x0) ) diff --git a/benchmarks/benchmark_trust_regions.jl b/benchmarks/benchmark_trust_regions.jl index d0bbf50018..9f102dd7b0 100644 --- a/benchmarks/benchmark_trust_regions.jl +++ b/benchmarks/benchmark_trust_regions.jl @@ -15,20 +15,20 @@ x_opt = trust_regions(M, cost, rgrad, rhess, x; max_trust_region_radius=8.0) h = RHess(M, A, p) g = RGrad(M, A) x_opt2 = trust_regions( - M, cost, g, h, x; max_trust_region_radius=8.0, evaluation=MutatingEvaluation() + M, cost, g, h, x; max_trust_region_radius=8.0, evaluation=InplaceEvaluation() ) @btime trust_regions( - $M, $cost, $g, $h, x2; max_trust_region_radius=8.0, evaluation=$(MutatingEvaluation()) + $M, $cost, $g, $h, x2; max_trust_region_radius=8.0, evaluation=$(InplaceEvaluation()) ) setup = (x2 = deepcopy($x)) x3 = deepcopy(x) trust_regions!( - M, cost, g, h, x3; max_trust_region_radius=8.0, evaluation=MutatingEvaluation() + M, cost, g, h, x3; max_trust_region_radius=8.0, evaluation=InplaceEvaluation() ) @btime trust_regions!( - $M, $cost, $g, $h, x3; max_trust_region_radius=8.0, evaluation=$(MutatingEvaluation()) + $M, $cost, $g, $h, x3; max_trust_region_radius=8.0, evaluation=$(InplaceEvaluation()) ) setup = (x3 = deepcopy($x)) @test distance(M, x_opt, x_opt2) ≈ 0 diff --git a/docs/src/index.md b/docs/src/index.md index 8bf46855e4..53af927069 100644 --- a/docs/src/index.md +++ b/docs/src/index.md @@ -69,7 +69,7 @@ Several functions are available, implemented on an arbitrary manifold, [cost fun ### Optimization Algorithms (Solvers) -For every optimization algorithm, a [solver](@ref SolversSection) is implemented based on a [`Problem`](@ref) that describes the problem to solve and its [`Options`](@ref) that set up the solver, store interims values. Together they +For every optimization algorithm, a [solver](@ref SolversSection) is implemented based on a [`AbstractManoptProblem`](@ref) that describes the problem to solve and its [`Options`](@ref) that set up the solver, store interims values. Together they form a [plan](@ref planSection). ### Visualization diff --git a/docs/src/plans/index.md b/docs/src/plans/index.md index 6d5308d269..5f5fb0486a 100644 --- a/docs/src/plans/index.md +++ b/docs/src/plans/index.md @@ -4,7 +4,7 @@ CurrentModule = Manopt ``` -In order to start a solver, both a [Problem](@ref ProblemSection) and [Options](@ref OptionsSection) are required. +In order to start a solver, both a [AbstractManoptProblem](@ref AbstractManoptProblemSection) and [Options](@ref OptionsSection) are required. Together they form a __plan__. Everything related to problems, options, and their tools in general, is explained in this section and its subpages. The specific Options related to a certain (concrete) solver can be diff --git a/docs/src/plans/problem.md b/docs/src/plans/problem.md index d5850ec070..983e6be4fe 100644 --- a/docs/src/plans/problem.md +++ b/docs/src/plans/problem.md @@ -1,4 +1,4 @@ -# [Problems](@id ProblemSection) +# [AbstractManoptProblems](@id ProblemSection) ```@meta CurrentModule = Manopt @@ -19,7 +19,7 @@ allocation function `X = gradF(x)` allocates memory for its result `X`, while `g ```@docs AbstractEvaluationType AllocatingEvaluation -MutatingEvaluation +InplaceEvaluation ``` ## Cost based problem diff --git a/docs/src/plans/stopping_criteria.md b/docs/src/plans/stopping_criteria.md index f896caa0ef..3617b8d187 100644 --- a/docs/src/plans/stopping_criteria.md +++ b/docs/src/plans/stopping_criteria.md @@ -13,8 +13,8 @@ StoppingCriterionSet ``` Then the stopping criteria `s` might have certain internal values to check against, -and this is done when calling them as a function `s(p::Problem, o::Options)`, -where the [`Problem`](@ref) and the [`Options`](@ref) together represent +and this is done when calling them as a function `s(p::AbstractManoptProblem, o::Options)`, +where the [`AbstractManoptProblem`](@ref) and the [`Options`](@ref) together represent the current state of the solver. The functor returns either `false` when the stopping criterion is not fulfilled or `true` otherwise. One field all criteria should have is the `s.reason`, a string giving the reason to stop, see [`get_reason`](@ref). diff --git a/docs/src/solvers/index.md b/docs/src/solvers/index.md index a781cf1d3e..cc3233608c 100644 --- a/docs/src/solvers/index.md +++ b/docs/src/solvers/index.md @@ -5,7 +5,7 @@ CurrentModule = Manopt ``` -Solvers can be applied to [`Problem`](@ref)s with solver +Solvers can be applied to [`AbstractManoptProblem`](@ref)s with solver specific [`Options`](@ref). # List of Algorithms @@ -40,7 +40,7 @@ Note that the solvers (or their [`Options`](@ref), to be precise) can also be de The main function a solver calls is ```@docs -solve(p::Problem, o::Options) +solve!(p::AbstractManoptProblem, o::Options) ``` which is a framework that you in general should not change or redefine. @@ -52,5 +52,5 @@ initialize_solver! step_solver! get_solver_result get_solver_return -stop_solver!(p::Problem, o::Options, i::Int) +stop_solver!(p::AbstractManoptProblem, o::Options, i::Int) ``` diff --git a/examples/FrankWolfeSPDMean.jl b/examples/FrankWolfeSPDMean.jl index bcc673a00b..b52dab2886 100644 --- a/examples/FrankWolfeSPDMean.jl +++ b/examples/FrankWolfeSPDMean.jl @@ -180,7 +180,7 @@ PlutoUI.with_terminal() do 50, ], record=[:Iteration, :Iterate, :Cost], - evaluation=MutatingEvaluation(), + evaluation=InplaceEvaluation(), return_options=true, ) end @@ -201,7 +201,7 @@ statsF20 = @timed Frank_Wolfe_method!( grad_weighted_mean!, q1; subtask=special_oracle!, - evaluation=MutatingEvaluation(), + evaluation=InplaceEvaluation(), stopping_criterion=StopAfterIteration(20), ); @@ -225,7 +225,7 @@ PlutoUI.with_terminal() do :Stop, 1, ], - evaluation=MutatingEvaluation(), + evaluation=InplaceEvaluation(), stopping_criterion=StopAfterIteration(200) | StopWhenGradientNormLess(1e-12), return_options=true, ) @@ -240,7 +240,7 @@ statsG = @timed gradient_descent!( weighted_mean_cost, grad_weighted_mean!, q2; - evaluation=MutatingEvaluation(), + evaluation=InplaceEvaluation(), stopping_criterion=StopAfterIteration(200) | StopWhenGradientNormLess(1e-12), ); diff --git a/src/Manopt.jl b/src/Manopt.jl index 225c92795e..e1c951cfa1 100644 --- a/src/Manopt.jl +++ b/src/Manopt.jl @@ -185,7 +185,7 @@ export Problem, StochasticGradientProblem, AbstractEvaluationType, AllocatingEvaluation, - MutatingEvaluation + InplaceEvaluation # # Options export Options, diff --git a/src/functions/proximal_maps.jl b/src/functions/proximal_maps.jl index 1408dc183a..1b9712aff1 100644 --- a/src/functions/proximal_maps.jl +++ b/src/functions/proximal_maps.jl @@ -349,7 +349,7 @@ function prox_TV2!( ∂F!, y; stopping_criterion=stopping_criterion, - evaluation=MutatingEvaluation(), + evaluation=InplaceEvaluation(), kwargs..., ) return y diff --git a/src/plans/alm_plan.jl b/src/plans/alm_plan.jl index 7a5eb8b2f3..cff85fb981 100644 --- a/src/plans/alm_plan.jl +++ b/src/plans/alm_plan.jl @@ -39,7 +39,7 @@ in the keyword arguments. [`augmented_Lagrangian_method`](@ref) """ mutable struct AugmentedLagrangianMethodOptions{ - P,Pr<:Problem,Op<:Options,TStopping<:StoppingCriterion + P,Pr<:AbstractManoptProblem,Op<:Options,TStopping<:StoppingCriterion } <: Options x::P sub_problem::Pr @@ -78,7 +78,7 @@ mutable struct AugmentedLagrangianMethodOptions{ stopping_criterion::StoppingCriterion=StopAfterIteration(300) | ( StopWhenSmallerOrEqual(:ϵ, ϵ_min) & StopWhenChangeLess(1e-10) ), - ) where {P,Pr<:Problem,Op<:Options} + ) where {P,Pr<:AbstractManoptProblem,Op<:Options} o = new{P,Pr,Op,typeof(stopping_criterion)}() o.x = x0 o.sub_problem = sub_problem @@ -209,7 +209,7 @@ end # mutating vector -> we can omit a few of the ineq gradients and allocations. function ( LG::AugmentedLagrangianGrad{ - <:ConstrainedProblem{<:MutatingEvaluation,<:VectorConstraint} + <:ConstrainedProblem{<:InplaceEvaluation,<:VectorConstraint} } )( M::AbstractManifold, X, p diff --git a/src/plans/alternating_gradient_plan.jl b/src/plans/alternating_gradient_plan.jl index ee08daaa4e..ec3f1d02ef 100644 --- a/src/plans/alternating_gradient_plan.jl +++ b/src/plans/alternating_gradient_plan.jl @@ -1,5 +1,5 @@ @doc raw""" - AlternatingGradientProblem <: Problem + AlternatingGradientProblem <:AbstractManoptProblem An alternating gradient problem consists of * a `ProductManifold M` ``=\mathcal M = \mathcal M_1 × ⋯ × M_n`` @@ -91,15 +91,13 @@ function get_gradient!( return X end function get_gradient( - p::AlternatingGradientProblem{MutatingEvaluation,<:AbstractManifold,TC,<:Function}, x + p::AlternatingGradientProblem{InplaceEvaluation,<:AbstractManifold,TC,<:Function}, x ) where {TC} Y = zero_vector(p.M, x) return p.gradient!!(p.M, Y, x) end function get_gradient( - p::AlternatingGradientProblem{ - MutatingEvaluation,<:AbstractManifold,TC,<:AbstractVector - }, + p::AlternatingGradientProblem{InplaceEvaluation,<:AbstractManifold,TC,<:AbstractVector}, x, ) where {TC} Y = zero_vector(p.M, x) @@ -107,14 +105,12 @@ function get_gradient( return Y end function get_gradient!( - p::AlternatingGradientProblem{MutatingEvaluation,<:AbstractManifold,TC,<:Function}, X, x + p::AlternatingGradientProblem{InplaceEvaluation,<:AbstractManifold,TC,<:Function}, X, x ) where {TC} return p.gradient!!(p.M, X, x) end function get_gradient!( - p::AlternatingGradientProblem{ - MutatingEvaluation,<:AbstractManifold,TC,<:AbstractVector - }, + p::AlternatingGradientProblem{InplaceEvaluation,<:AbstractManifold,TC,<:AbstractVector}, X, x, ) where {TC} @@ -167,14 +163,14 @@ function get_gradient!( return X end function get_gradient( - p::AlternatingGradientProblem{MutatingEvaluation,<:AbstractManifold,TC}, k, x + p::AlternatingGradientProblem{InplaceEvaluation,<:AbstractManifold,TC}, k, x ) where {TC} X = zero_vector(p.M[k], x[p.M, k]) get_gradient!(p, X, k, x) return X end function get_gradient!( - p::AlternatingGradientProblem{MutatingEvaluation,<:AbstractManifold,TC,<:Function}, + p::AlternatingGradientProblem{InplaceEvaluation,<:AbstractManifold,TC,<:Function}, X, k, x, @@ -186,9 +182,7 @@ function get_gradient!( return X end function get_gradient!( - p::AlternatingGradientProblem{ - MutatingEvaluation,<:AbstractManifold,TC,<:AbstractVector - }, + p::AlternatingGradientProblem{InplaceEvaluation,<:AbstractManifold,TC,<:AbstractVector}, X, k, x, diff --git a/src/plans/constrained_plan.jl b/src/plans/constrained_plan.jl index c3e480455b..0ff78052a0 100644 --- a/src/plans/constrained_plan.jl +++ b/src/plans/constrained_plan.jl @@ -356,7 +356,7 @@ evaluate the gradient of the `j` th equality constraint ``(\operatorname{grad} h !!! note For the [`FunctionConstraint`](@ref) variant of the problem, this function still evaluates the full gradient. - For the [`MutatingEvaluation`](@ref) and [`FunctionConstraint`](@ref) of the problem, this function currently also calls [`get_equality_constraints`](@ref), + For the [`InplaceEvaluation`](@ref) and [`FunctionConstraint`](@ref) of the problem, this function currently also calls [`get_equality_constraints`](@ref), since this is the only way to determine the number of cconstraints. It also allocates a full tangent vector. """ get_grad_equality_constraint(P::ConstrainedProblem, p, j) @@ -371,14 +371,14 @@ function get_grad_equality_constraint( return P.gradH!![j](P.M, p) end function get_grad_equality_constraint( - P::ConstrainedProblem{MutatingEvaluation,FunctionConstraint}, p, j + P::ConstrainedProblem{InplaceEvaluation,FunctionConstraint}, p, j ) X = [zero_vector(P.M, p) for _ in 1:length(P.H(P.M, p))] P.gradH!!(P.M, X, p) return X[j] end function get_grad_equality_constraint( - P::ConstrainedProblem{MutatingEvaluation,VectorConstraint}, p, j + P::ConstrainedProblem{InplaceEvaluation,VectorConstraint}, p, j ) X = zero_vector(P.M, p) P.gradH!![j](P.M, X, p) @@ -392,7 +392,7 @@ Evaluate the gradient of the `j`th equality constraint ``(\operatorname{grad} h( !!! note For the [`FunctionConstraint`](@ref) variant of the problem, this function still evaluates the full gradient. - For the [`MutatingEvaluation`](@ref) of the [`FunctionConstraint`](@ref) of the problem, this function currently also calls [`get_inequality_constraints`](@ref), + For the [`InplaceEvaluation`](@ref) of the [`FunctionConstraint`](@ref) of the problem, this function currently also calls [`get_inequality_constraints`](@ref), since this is the only way to determine the number of cconstraints and allocates a full vector of tangent vectors """ get_grad_equality_constraint!(P::ConstrainedProblem, p, j) @@ -409,7 +409,7 @@ function get_grad_equality_constraint!( return X end function get_grad_equality_constraint!( - P::ConstrainedProblem{MutatingEvaluation,FunctionConstraint}, X, p, j + P::ConstrainedProblem{InplaceEvaluation,FunctionConstraint}, X, p, j ) Y = [zero_vector(P.M, p) for _ in 1:length(P.H(P.M, p))] P.gradH!!(P.M, Y, p) @@ -417,7 +417,7 @@ function get_grad_equality_constraint!( return X end function get_grad_equality_constraint!( - P::ConstrainedProblem{MutatingEvaluation,VectorConstraint}, X, p, j + P::ConstrainedProblem{InplaceEvaluation,VectorConstraint}, X, p, j ) P.gradH!![j](P.M, X, p) return X @@ -431,7 +431,7 @@ eevaluate all gradients of the equality constraints ``\operatorname{grad} h(x)`` of the [`ConstrainedProblem`](@ref) `P` at `p`. !!! note - for the [`MutatingEvaluation`](@ref) and [`FunctionConstraint`](@ref) variant of the problem, + for the [`InplaceEvaluation`](@ref) and [`FunctionConstraint`](@ref) variant of the problem, this function currently also calls [`get_equality_constraints`](@ref), since this is the only way to determine the number of cconstraints. """ @@ -447,14 +447,14 @@ function get_grad_equality_constraints( return [grad_hi(P.M, p) for grad_hi in P.gradH!!] end function get_grad_equality_constraints( - P::ConstrainedProblem{MutatingEvaluation,FunctionConstraint}, p + P::ConstrainedProblem{InplaceEvaluation,FunctionConstraint}, p ) X = [zero_vector(P.M, p) for _ in 1:length(P.H(P.M, p))] P.gradH!!(P.M, X, p) return X end function get_grad_equality_constraints( - P::ConstrainedProblem{MutatingEvaluation,VectorConstraint}, p + P::ConstrainedProblem{InplaceEvaluation,VectorConstraint}, p ) X = [zero_vector(P.M, p) for _ in 1:length(P.H)] [grad_hi(P.M, Xj, p) for (Xj, grad_hi) in zip(X, P.gradH!!)] @@ -482,13 +482,13 @@ function get_grad_equality_constraints!( return X end function get_grad_equality_constraints!( - P::ConstrainedProblem{MutatingEvaluation,FunctionConstraint}, X, p + P::ConstrainedProblem{InplaceEvaluation,FunctionConstraint}, X, p ) P.gradH!!(P.M, X, p) return X end function get_grad_equality_constraints!( - P::ConstrainedProblem{MutatingEvaluation,VectorConstraint}, X, p + P::ConstrainedProblem{InplaceEvaluation,VectorConstraint}, X, p ) for (Xj, grad_hj) in zip(X, P.gradH!!) grad_hj(P.M, Xj, p) @@ -503,7 +503,7 @@ Evaluate the gradient of the `i` th inequality constraints ``(\operatorname{grad !!! note For the [`FunctionConstraint`](@ref) variant of the problem, this function still evaluates the full gradient. - For the [`MutatingEvaluation`](@ref) and [`FunctionConstraint`](@ref) of the problem, this function currently also calls [`get_inequality_constraints`](@ref), + For the [`InplaceEvaluation`](@ref) and [`FunctionConstraint`](@ref) of the problem, this function currently also calls [`get_inequality_constraints`](@ref), since this is the only way to determine the number of cconstraints. """ get_grad_inequality_constraint(P::ConstrainedProblem, p, i) @@ -518,14 +518,14 @@ function get_grad_inequality_constraint( return P.gradG!![i](P.M, p) end function get_grad_inequality_constraint( - P::ConstrainedProblem{MutatingEvaluation,FunctionConstraint}, p, i + P::ConstrainedProblem{InplaceEvaluation,FunctionConstraint}, p, i ) X = [zero_vector(P.M, p) for _ in 1:length(P.G(P.M, p))] P.gradG!!(P.M, X, p) return X[i] end function get_grad_inequality_constraint( - P::ConstrainedProblem{MutatingEvaluation,VectorConstraint}, p, i + P::ConstrainedProblem{InplaceEvaluation,VectorConstraint}, p, i ) X = zero_vector(P.M, p) P.gradG!![i](P.M, X, p) @@ -540,7 +540,7 @@ of the [`ConstrainedProblem`](@ref) `P` in place of ``X`` !!! note For the [`FunctionConstraint`](@ref) variant of the problem, this function still evaluates the full gradient. - For the [`MutatingEvaluation`](@ref) and [`FunctionConstraint`](@ref) of the problem, this function currently also calls [`get_inequality_constraints`](@ref), + For the [`InplaceEvaluation`](@ref) and [`FunctionConstraint`](@ref) of the problem, this function currently also calls [`get_inequality_constraints`](@ref), since this is the only way to determine the number of cconstraints. evaluate all gradients of the inequality constraints ``\operatorname{grad} h(x)`` or ``\bigl(g_1(x), g_2(x),\ldots,g_m(x)\bigr)`` of the [`ConstrainedProblem`](@ref) ``p`` at ``x`` in place of `X``, which is a vector of ``m`` tangent vectors . @@ -558,7 +558,7 @@ function get_grad_inequality_constraint!( return X end function get_grad_inequality_constraint!( - P::ConstrainedProblem{MutatingEvaluation,FunctionConstraint}, X, p, i + P::ConstrainedProblem{InplaceEvaluation,FunctionConstraint}, X, p, i ) Y = [zero_vector(P.M, p) for _ in 1:length(P.G(P.M, p))] P.gradG!!(P.M, Y, p) @@ -566,7 +566,7 @@ function get_grad_inequality_constraint!( return X end function get_grad_inequality_constraint!( - P::ConstrainedProblem{MutatingEvaluation,VectorConstraint}, X, p, i + P::ConstrainedProblem{InplaceEvaluation,VectorConstraint}, X, p, i ) P.gradG!![i](P.M, X, p) return X @@ -579,7 +579,7 @@ evaluate all gradients of the inequality constraints ``\operatorname{grad} g(p)` of the [`ConstrainedProblem`](@ref) ``P`` at ``p``. !!! note - for the [`MutatingEvaluation`](@ref) and [`FunctionConstraint`](@ref) variant of the problem, + for the [`InplaceEvaluation`](@ref) and [`FunctionConstraint`](@ref) variant of the problem, this function currently also calls [`get_equality_constraints`](@ref), since this is the only way to determine the number of cconstraints. """ @@ -595,14 +595,14 @@ function get_grad_inequality_constraints( return [grad_gi(P.M, p) for grad_gi in P.gradG!!] end function get_grad_inequality_constraints( - P::ConstrainedProblem{MutatingEvaluation,FunctionConstraint}, p + P::ConstrainedProblem{InplaceEvaluation,FunctionConstraint}, p ) X = [zero_vector(P.M, p) for _ in 1:length(P.G(P.M, p))] P.gradG!!(P.M, X, p) return X end function get_grad_inequality_constraints( - P::ConstrainedProblem{MutatingEvaluation,VectorConstraint}, p + P::ConstrainedProblem{InplaceEvaluation,VectorConstraint}, p ) X = [zero_vector(P.M, p) for _ in 1:length(P.G)] [grad_gi(P.M, Xi, p) for (Xi, grad_gi) in zip(X, P.gradG!!)] @@ -630,13 +630,13 @@ function get_grad_inequality_constraints!( return X end function get_grad_inequality_constraints!( - P::ConstrainedProblem{MutatingEvaluation,FunctionConstraint}, X, p + P::ConstrainedProblem{InplaceEvaluation,FunctionConstraint}, X, p ) P.gradG!!(P.M, X, p) return X end function get_grad_inequality_constraints!( - P::ConstrainedProblem{MutatingEvaluation,VectorConstraint}, X, p + P::ConstrainedProblem{InplaceEvaluation,VectorConstraint}, X, p ) for (Xi, grad_gi!) in zip(X, P.gradG!!) grad_gi!(P.M, Xi, p) diff --git a/src/plans/cost_plan.jl b/src/plans/cost_plan.jl index 3fc2f48d69..db904d4287 100644 --- a/src/plans/cost_plan.jl +++ b/src/plans/cost_plan.jl @@ -1,16 +1,15 @@ @doc raw""" - CostProblem{T, Manifold, TCost} <: Problem{T} + ManifoldCostObjective{T,Tcost} <: AbstractManifoldObjective -speficy a problem for solvers just based on cost functions, i.e. -gradient free ones. -# Fields +speficy an [`AbstractManifoldObjective`](@ref) that does only have information about +the cost function ``f\colon \mathbb M → ℝ`` implemented as a function `(M, p) -> c` +to compute the cost value `c` at `p` on the manifold `M`. -* `M` – a manifold ``\mathcal M`` -* `cost` – a function ``F: \mathcal M → ℝ`` to minimize +* `cost` – a function ``f: \mathcal M → ℝ`` to minimize # Constructors - CostProblem(M, cost; evaluation=AllocatingEvaluation()) + ManifoldCostObjective(f) Generate a problem. While this Problem does not have any allocating functions, the type `T` can be set for consistency reasons with other problems. @@ -18,14 +17,11 @@ the type `T` can be set for consistency reasons with other problems. # See also [`NelderMead`](@ref) """ -struct CostProblem{T,mT<:AbstractManifold,Tcost} <: Problem{T} - M::mT +struct ManifoldCostObjective{T,Tcost} <: AbstractManifoldObjective{T} cost::Tcost end -function CostProblem( - M::mT, cost::T; evaluation::AbstractEvaluationType=AllocatingEvaluation() -) where {mT<:AbstractManifold,T} - return CostProblem{typeof(evaluation),mT,T}(M, cost) +function CostProblem(cost::Tcost) where {Tcost} + return CostProblem{AllocatingEvaluation,Tcost}(cost) end @doc raw""" NelderMeadOptions <: Options diff --git a/src/plans/debug_options.jl b/src/plans/debug_options.jl index 8522f095cf..8d0b8e23bb 100644 --- a/src/plans/debug_options.jl +++ b/src/plans/debug_options.jl @@ -83,7 +83,7 @@ mutable struct DebugGroup <: DebugAction group::Array{DebugAction,1} DebugGroup(g::Array{<:DebugAction,1}) = new(g) end -function (d::DebugGroup)(p::Problem, o::Options, i) +function (d::DebugGroup)(p::AbstractManoptProblem, o::Options, i) for di in d.group di(p, o, i) end @@ -105,7 +105,7 @@ mutable struct DebugEvery <: DebugAction return new(d, every, alwaysUpdate) end end -function (d::DebugEvery)(p::Problem, o::Options, i) +function (d::DebugEvery)(p::AbstractManoptProblem, o::Options, i) if (rem(i, d.every) == 0) d.debug(p, o, i) elseif d.alwaysUpdate @@ -153,7 +153,7 @@ end @deprecate DebugChange(a::StoreOptionsAction, pre::String="Last Change: ", io::IO=stdout) DebugChange(; storage=a, prefix=pre, io=io ) -function (d::DebugChange)(p::Problem, o::Options, i) +function (d::DebugChange)(p::AbstractManoptProblem, o::Options, i) (i > 0) && Printf.format( d.io, Printf.Format(d.format), @@ -187,7 +187,7 @@ mutable struct DebugGradientChange <: DebugAction return new(io, format, storage) end end -function (d::DebugGradientChange)(p::Problem, o::Options, i) +function (d::DebugGradientChange)(p::AbstractManoptProblem, o::Options, i) (i > 0) && Printf.format( d.io, Printf.Format(d.format), @@ -222,7 +222,7 @@ mutable struct DebugIterate <: DebugAction end end @deprecate DebugIterate(io::IO, long::Bool=false) DebugIterate(; io=io, long=long) -function (d::DebugIterate)(::Problem, o::Options, i::Int) +function (d::DebugIterate)(::AbstractManoptProblem, o::Options, i::Int) (i > 0) && Printf.format(d.io, Printf.Format(d.format), get_iterate(o)) return nothing end @@ -247,7 +247,7 @@ mutable struct DebugIteration <: DebugAction DebugIteration(; io::IO=stdout, format="# %-6d") = new(io, format) end @deprecate DebugIteration(io::IO) DebugIteration(; io=io) -function (d::DebugIteration)(::Problem, ::Options, i::Int) +function (d::DebugIteration)(::AbstractManoptProblem, ::Options, i::Int) (i == 0) && print(d.io, "Initial ") (i > 0) && Printf.format(d.io, Printf.Format(d.format), i) return nothing @@ -279,7 +279,7 @@ end @deprecate DebugCost(pre::String) DebugCost(; format="$pre %f") @deprecate DebugCost(pre::String, io::IO) DebugCost(; format="$pre %f", io=op) @deprecate DebugCost(long::Bool, io::IO) DebugCost(; long=long, io=io) -function (d::DebugCost)(p::Problem, o::Options, i::Int) +function (d::DebugCost)(p::AbstractManoptProblem, o::Options, i::Int) (i >= 0) && Printf.format(d.io, Printf.Format(d.format), get_cost(p, get_iterate(o))) return nothing end @@ -298,7 +298,7 @@ mutable struct DebugDivider <: DebugAction divider::String DebugDivider(divider=" | ", io::IO=stdout) = new(io, divider) end -function (d::DebugDivider)(::Problem, ::Options, i::Int) +function (d::DebugDivider)(::AbstractManoptProblem, ::Options, i::Int) print(d.io, (i >= 0) ? d.divider : "") return nothing end @@ -327,7 +327,7 @@ mutable struct DebugEntry <: DebugAction end @deprecate DebugEntry(f, prefix="$f:", io=stdout) DebugEntry(f; prefix=prefix, io=io) -function (d::DebugEntry)(::Problem, o::Options, i) +function (d::DebugEntry)(::AbstractManoptProblem, o::Options, i) (i >= 0) && Printf.format(d.io, Printf.Format(d.format), getfield(o, d.field)) return nothing end @@ -380,7 +380,7 @@ mutable struct DebugEntryChange <: DebugAction end end -function (d::DebugEntryChange)(p::Problem, o::Options, i::Int) +function (d::DebugEntryChange)(p::AbstractManoptProblem, o::Options, i::Int) if i == 0 # on init if field not present -> generate !has_storage(d.storage, d.field) && d.storage(p, o, i) @@ -403,7 +403,7 @@ mutable struct DebugStoppingCriterion <: DebugAction io::IO DebugStoppingCriterion(io::IO=stdout) = new(io) end -function (d::DebugStoppingCriterion)(::Problem, o::Options, i::Int) +function (d::DebugStoppingCriterion)(::AbstractManoptProblem, o::Options, i::Int) print(d.io, (i >= 0 || i == typemin(Int)) ? get_reason(o) : "") return nothing end @@ -442,7 +442,7 @@ mutable struct DebugTime <: DebugAction return new(io, format, Nanosecond(start ? time_ns() : 0), time_accuracy, mode) end end -function (d::DebugTime)(::Problem, ::Options, i) +function (d::DebugTime)(::AbstractManoptProblem, ::Options, i) if i == 0 || d.last_time == Nanosecond(0) # init d.last_time = Nanosecond(time_ns()) else @@ -504,7 +504,7 @@ mutable struct DebugWarnIfCostIncreases <: DebugAction tol::Float64 DebugWarnIfCostIncreases(warn::Symbol=:Once; tol=1e-13) = new(warn, Float64(Inf), tol) end -function (d::DebugWarnIfCostIncreases)(p::Problem, o::Options, i::Int) +function (d::DebugWarnIfCostIncreases)(p::AbstractManoptProblem, o::Options, i::Int) if d.status !== :No cost = get_cost(p, get_iterate(o)) if cost > d.old_cost + d.tol @@ -547,7 +547,7 @@ mutable struct DebugWarnIfCostNotFinite <: DebugAction status::Symbol DebugWarnIfCostNotFinite(warn::Symbol=:Once) = new(warn) end -function (d::DebugWarnIfCostNotFinite)(p::Problem, o::Options, i::Int) +function (d::DebugWarnIfCostNotFinite)(p::AbstractManoptProblem, o::Options, i::Int) if d.status !== :No cost = get_cost(p, get_iterate(o)) if !isfinite(cost) @@ -586,7 +586,7 @@ mutable struct DebugWarnIfFieldNotFinite <: DebugAction field::Symbol DebugWarnIfFieldNotFinite(field::Symbol, warn::Symbol=:Once) = new(warn, field) end -function (d::DebugWarnIfFieldNotFinite)(::Problem, o::Options, i::Int) +function (d::DebugWarnIfFieldNotFinite)(::AbstractManoptProblem, o::Options, i::Int) if d.status !== :No v = getproperty(o, d.field) if !all(isfinite.(v)) diff --git a/src/plans/epm_plan.jl b/src/plans/epm_plan.jl index f3e83b57f0..102c489774 100644 --- a/src/plans/epm_plan.jl +++ b/src/plans/epm_plan.jl @@ -30,7 +30,7 @@ arguments. [`exact_penalty_method`](@ref) """ mutable struct ExactPenaltyMethodOptions{ - P,Pr<:Problem,Op<:Options,TStopping<:StoppingCriterion + P,Pr<:AbstractManoptProblem,Op<:Options,TStopping<:StoppingCriterion } <: Options x::P sub_problem::Pr @@ -63,7 +63,7 @@ mutable struct ExactPenaltyMethodOptions{ StopAfterIteration(300), StopWhenAll(StopWhenSmallerOrEqual(:ϵ, ϵ_min), StopWhenChangeLess(1e-10)), ), - ) where {P,Pr<:Problem,Op<:Options} + ) where {P,Pr<:AbstractManoptProblem,Op<:Options} o = new{P,Pr,Op,typeof(stopping_criterion)}() o.x = x0 o.sub_problem = sub_problem @@ -276,7 +276,7 @@ end # Variant 3: Vectors of mutating gradients - we can spare a few gradient evaluations and allocations function ( EG::ExactPenaltyGrad{ - <:LinearQuadraticHuber,<:ConstrainedProblem{<:MutatingEvaluation,<:VectorConstraint} + <:LinearQuadraticHuber,<:ConstrainedProblem{<:InplaceEvaluation,<:VectorConstraint} } )( M::AbstractManifold, X, p::P diff --git a/src/plans/frank_wolfe_plan.jl b/src/plans/frank_wolfe_plan.jl index 837452d08d..13d15c185b 100644 --- a/src/plans/frank_wolfe_plan.jl +++ b/src/plans/frank_wolfe_plan.jl @@ -26,7 +26,7 @@ where currently two variants are supported 1. `subtask(M, q, X, p)` is a mutating function, i.e. we have a closed form solution of the optimization problem given `M`, `X` and `p` which is computed in place of `q`, which even works correctly, if we pass the same memory to `p` and `q`. -2. `subtask::Tuple{<:Problem,<:Options}` specifies a plan to solve the subtask with a subsolver, +2. `subtask::Tuple{<:AbstractManoptProblem,<:Options}` specifies a plan to solve the subtask with a subsolver, i.e. the cost within `subtask[1]` is a [`FrankWolfeCost`](@ref) using references to `p`and `X`, that is to the current iterate and gradient internally. Similarly for gradient based functions using the [`FrankWolfeGradient`](@ref). @@ -96,7 +96,7 @@ mutable struct FrankWolfeOptions{ inverse_retraction_method::ITM=default_inverse_retraction_method(M), ) where { P, - S<:Tuple{<:Problem,<:Options}, + S<:Tuple{<:AbstractManoptProblem,<:Options}, T, TStop<:StoppingCriterion, TStep<:Stepsize, diff --git a/src/plans/gradient_plan.jl b/src/plans/gradient_plan.jl index b853f32c4a..44f838d4a2 100644 --- a/src/plans/gradient_plan.jl +++ b/src/plans/gradient_plan.jl @@ -1,10 +1,10 @@ """ - AbstractGradientProblem{T} <: Problem{T} + AbstractGradientProblem{T} <: AbstractManoptProblem An abstract type for all functions that provide a (full) gradient, where `T` is a [`AbstractEvaluationType`](@ref) for the gradient function. """ -abstract type AbstractGradientProblem{T} <: Problem{T} end +abstract type AbstractGradientProblem{T} <: AbstractManoptProblem end @doc raw""" GradientProblem{T} <: AbstractGradientProblem{T} @@ -19,7 +19,7 @@ specify a problem for gradient based algorithms. Depending on the [`AbstractEvaluationType`](@ref) `T` the gradient has to be provided * as a function `x -> X` that allocates memory for `X` itself for an [`AllocatingEvaluation`](@ref) -* as a function `(X,x) -> X` that work in place of `X` for an [`MutatingEvaluation`](@ref) +* as a function `(X,x) -> X` that work in place of `X` for an [`InplaceEvaluation`](@ref) # Constructors GradientProblem(M, cost, gradient; evaluation=AllocatingEvaluation()) @@ -46,7 +46,7 @@ evaluate the gradient of a [`AbstractGradientProblem{T}`](@ref)`p` at the point The evaluation is done in place of `X` for the `!`-variant. The `T=`[`AllocatingEvaluation`](@ref) problem might still allocate memory within. -When the non-mutating variant is called with a `T=`[`MutatingEvaluation`](@ref) +When the non-mutating variant is called with a `T=`[`InplaceEvaluation`](@ref) memory for the result is allocated. """ get_gradient(p::AbstractGradientProblem, x) @@ -54,7 +54,7 @@ get_gradient(p::AbstractGradientProblem, x) function get_gradient(p::AbstractGradientProblem{AllocatingEvaluation}, x) return p.gradient!!(p.M, x) end -function get_gradient(p::AbstractGradientProblem{MutatingEvaluation}, x) +function get_gradient(p::AbstractGradientProblem{InplaceEvaluation}, x) X = zero_vector(p.M, x) return p.gradient!!(p.M, X, x) end @@ -63,7 +63,7 @@ function get_gradient!(p::AbstractGradientProblem{AllocatingEvaluation}, X, x) return copyto!(p.M, X, x, p.gradient!!(p.M, x)) end -function get_gradient!(p::AbstractGradientProblem{MutatingEvaluation}, X, x) +function get_gradient!(p::AbstractGradientProblem{InplaceEvaluation}, X, x) return p.gradient!!(p.M, X, x) end @@ -271,7 +271,7 @@ function MomentumGradient( gradient, deepcopy(x0), momentum, s, vector_transport_method ) end -function (m::MomentumGradient)(p::Problem, o::AbstractGradientOptions, i) +function (m::MomentumGradient)(p::AbstractManoptProblem, o::AbstractGradientOptions, i) s, d = m.direction(p, o, i) #get inner direction and step size old_d = m.momentum * @@ -352,7 +352,7 @@ function AverageGradient( gradients, deepcopy(x0), s, vector_transport_method ) end -function (a::AverageGradient)(p::Problem, o::AbstractGradientOptions, i) +function (a::AverageGradient)(p::AbstractManoptProblem, o::AbstractGradientOptions, i) pop!(a.gradients) s, d = a.direction(p, o, i) #get inner gradient and step a.gradients = vcat([deepcopy(d)], a.gradients) @@ -494,7 +494,7 @@ mutable struct DebugGradientNorm <: DebugAction return new(io, format) end end -function (d::DebugGradientNorm)(p::Problem, o::Options, i::Int) +function (d::DebugGradientNorm)(p::AbstractManoptProblem, o::Options, i::Int) (i < 1) && return nothing Printf.format(d.io, Printf.Format(d.format), norm(p.M, get_iterate(o), get_gradient(o))) return nothing @@ -561,7 +561,7 @@ mutable struct RecordGradientNorm <: RecordAction recorded_values::Array{Float64,1} RecordGradientNorm() = new(Array{Float64,1}()) end -function (r::RecordGradientNorm)(p::Problem, o::Options, i::Int) +function (r::RecordGradientNorm)(p::AbstractManoptProblem, o::Options, i::Int) return record_or_reset!(r, norm(p.M, get_iterate(o), get_gradient(o)), i) end diff --git a/src/plans/hessian_plan.jl b/src/plans/hessian_plan.jl index 3ae9acf258..f0525927d6 100644 --- a/src/plans/hessian_plan.jl +++ b/src/plans/hessian_plan.jl @@ -1,6 +1,6 @@ @doc raw""" - HessianProblem <: Problem + HessianProblem <:AbstractManoptProblem specify a problem for hessian based algorithms. @@ -332,20 +332,20 @@ applied to a tangent vector `X`, i.e. ``\operatorname{Hess}f(q)[X]``. The evaluation is done in place of `Y` for the `!`-variant. The `T=`[`AllocatingEvaluation`](@ref) problem might still allocate memory within. -When the non-mutating variant is called with a `T=`[`MutatingEvaluation`](@ref) +When the non-mutating variant is called with a `T=`[`InplaceEvaluation`](@ref) memory for the result is allocated. """ function get_hessian(p::HessianProblem{AllocatingEvaluation}, q, X) return p.hessian!!(p.M, q, X) end -function get_hessian(p::HessianProblem{MutatingEvaluation}, q, X) +function get_hessian(p::HessianProblem{InplaceEvaluation}, q, X) Y = zero_vector(p.M, q) return p.hessian!!(p.M, Y, q, X) end function get_hessian!(p::HessianProblem{AllocatingEvaluation}, Y, q, X) return copyto!(p.M, Y, p.hessian!!(p.M, q, X)) end -function get_hessian!(p::HessianProblem{MutatingEvaluation}, Y, q, X) +function get_hessian!(p::HessianProblem{InplaceEvaluation}, Y, q, X) return p.hessian!!(p.M, Y, q, X) end @@ -422,7 +422,7 @@ function (f::ApproxHessianFiniteDifference{AllocatingEvaluation})(M, x, X) ) return (1 / c) * (f.grad_tmp_dir - f.grad_tmp) end -function (f::ApproxHessianFiniteDifference{MutatingEvaluation})(M, Y, x, X) +function (f::ApproxHessianFiniteDifference{InplaceEvaluation})(M, Y, x, X) norm_X = norm(M, x, X) (norm_X ≈ zero(norm_X)) && return zero_vector!(M, X, x) c = f.steplength / norm_X diff --git a/src/plans/higher_order_primal_dual_plan.jl b/src/plans/higher_order_primal_dual_plan.jl index 679dcc0eff..18b40fb017 100644 --- a/src/plans/higher_order_primal_dual_plan.jl +++ b/src/plans/higher_order_primal_dual_plan.jl @@ -187,7 +187,7 @@ function get_differential_primal_prox( return p.diff_prox_F!!(p.M, σ, x, X) end function get_differential_primal_prox( - p::PrimalDualSemismoothNewtonProblem{MutatingEvaluation}, σ, x, X + p::PrimalDualSemismoothNewtonProblem{InplaceEvaluation}, σ, x, X ) y = allocate_result(p.M, get_differential_primal_prox, x, X) return p.diff_prox_F!!(p.M, y, σ, x, X) @@ -198,7 +198,7 @@ function get_differential_primal_prox!( return copyto!(p.M, y, p.diff_prox_F!!(p.M, σ, x, X)) end function get_differential_primal_prox!( - p::PrimalDualSemismoothNewtonProblem{MutatingEvaluation}, y, σ, x, X + p::PrimalDualSemismoothNewtonProblem{InplaceEvaluation}, y, σ, x, X ) return p.diff_prox_F!!(p.M, y, σ, x, X) end @@ -223,7 +223,7 @@ function get_differential_dual_prox( return p.diff_prox_G_dual!!(p.N, n, τ, ξ, Ξ) end function get_differential_dual_prox( - p::PrimalDualSemismoothNewtonProblem{<:MutatingEvaluation}, n, τ, ξ, Ξ + p::PrimalDualSemismoothNewtonProblem{<:InplaceEvaluation}, n, τ, ξ, Ξ ) η = allocate_result(p.N, get_differential_dual_prox, ξ, Ξ) return p.diff_prox_G_dual!!(p.N, η, n, τ, ξ, Ξ) @@ -234,7 +234,7 @@ function get_differential_dual_prox!( return copyto!(p.N, η, p.diff_prox_G_dual!!(p.N, n, τ, ξ, Ξ)) end function get_differential_dual_prox!( - p::PrimalDualSemismoothNewtonProblem{<:MutatingEvaluation}, η, n, τ, ξ, Ξ + p::PrimalDualSemismoothNewtonProblem{<:InplaceEvaluation}, η, n, τ, ξ, Ξ ) return p.diff_prox_G_dual!!(p.N, η, n, τ, ξ, Ξ) end diff --git a/src/plans/options.jl b/src/plans/options.jl index 7fa1bdd5dd..85c7d17332 100644 --- a/src/plans/options.jl +++ b/src/plans/options.jl @@ -218,7 +218,9 @@ mutable struct StoreOptionsAction <: AbstractOptionsAction return new(Dict{Symbol,Any}(), keys, once, -1) end end -function (a::StoreOptionsAction)(::P, o::O, i::Int) where {P<:Problem,O<:Options} +function (a::StoreOptionsAction)( + ::P, o::O, i::Int +) where {P<:AbstractManoptProblem,O<:Options} #update values (maybe only once) if !a.once || a.last_stored != i for key in a.keys diff --git a/src/plans/primal_dual_plan.jl b/src/plans/primal_dual_plan.jl index b4ed80273b..8a14728b16 100644 --- a/src/plans/primal_dual_plan.jl +++ b/src/plans/primal_dual_plan.jl @@ -1,9 +1,9 @@ @doc raw""" - AbstractPrimalDualProblem{T} <: Problem{T} + AbstractPrimalDualProblem{T} <: AbstractManoptProblem An abstract type for primal-dual-based problems. """ -abstract type AbstractPrimalDualProblem{T} <: Problem{T} end +abstract type AbstractPrimalDualProblem{T} <: AbstractManoptProblem end @doc raw""" PrimalDualProblem {T, mT <: AbstractManifold, nT <: AbstractManifold} <: AbstractPrimalDualProblem @@ -101,14 +101,14 @@ get_primal_prox(::AbstractPrimalDualProblem, ::Any...) function get_primal_prox(p::AbstractPrimalDualProblem{<:AllocatingEvaluation}, σ, x) return p.prox_F!!(p.M, σ, x) end -function get_primal_prox(p::AbstractPrimalDualProblem{<:MutatingEvaluation}, σ, x) +function get_primal_prox(p::AbstractPrimalDualProblem{<:InplaceEvaluation}, σ, x) y = allocate_result(p.M, get_primal_prox, x) return p.prox_F!!(p.M, y, σ, x) end function get_primal_prox!(p::AbstractPrimalDualProblem{<:AllocatingEvaluation}, y, σ, x) return copyto!(p.M, y, p.prox_F!!(p.M, σ, x)) end -function get_primal_prox!(p::AbstractPrimalDualProblem{<:MutatingEvaluation}, y, σ, x) +function get_primal_prox!(p::AbstractPrimalDualProblem{<:InplaceEvaluation}, y, σ, x) return p.prox_F!!(p.M, y, σ, x) end @@ -129,14 +129,14 @@ get_dual_prox(::AbstractPrimalDualProblem, ::Any...) function get_dual_prox(p::AbstractPrimalDualProblem{<:AllocatingEvaluation}, n, τ, ξ) return p.prox_G_dual!!(p.N, n, τ, ξ) end -function get_dual_prox(p::AbstractPrimalDualProblem{<:MutatingEvaluation}, n, τ, ξ) +function get_dual_prox(p::AbstractPrimalDualProblem{<:InplaceEvaluation}, n, τ, ξ) η = allocate_result(p.N, get_dual_prox, ξ) return p.prox_G_dual!!(p.N, η, n, τ, ξ) end function get_dual_prox!(p::AbstractPrimalDualProblem{<:AllocatingEvaluation}, η, n, τ, ξ) return copyto!(p.N, η, p.prox_G_dual!!(p.N, n, τ, ξ)) end -function get_dual_prox!(p::AbstractPrimalDualProblem{<:MutatingEvaluation}, η, n, τ, ξ) +function get_dual_prox!(p::AbstractPrimalDualProblem{<:InplaceEvaluation}, η, n, τ, ξ) return p.prox_G_dual!!(p.N, η, n, τ, ξ) end @doc raw""" @@ -154,7 +154,7 @@ function linearized_forward_operator( return p.linearized_forward_operator!!(p.M, m, X) end function linearized_forward_operator( - p::AbstractPrimalDualProblem{<:MutatingEvaluation}, m, X, ::Any + p::AbstractPrimalDualProblem{<:InplaceEvaluation}, m, X, ::Any ) y = random_point(p.N) forward_operator!(p, y, m) @@ -167,7 +167,7 @@ function linearized_forward_operator!( return copyto!(p.N, Y, n, p.linearized_forward_operator!!(p.M, m, X)) end function linearized_forward_operator!( - p::AbstractPrimalDualProblem{<:MutatingEvaluation}, Y, m, X, ::Any + p::AbstractPrimalDualProblem{<:InplaceEvaluation}, Y, m, X, ::Any ) return p.linearized_forward_operator!!(p.M, Y, m, X) end @@ -184,14 +184,14 @@ forward_operator(::AbstractPrimalDualProblem{<:AllocatingEvaluation}, ::Any...) function forward_operator(p::AbstractPrimalDualProblem{<:AllocatingEvaluation}, x) return p.Λ!!(p.M, x) end -function forward_operator(p::AbstractPrimalDualProblem{<:MutatingEvaluation}, x) +function forward_operator(p::AbstractPrimalDualProblem{<:InplaceEvaluation}, x) y = random_point(p.N) return p.Λ!!(p.M, y, x) end function forward_operator!(p::AbstractPrimalDualProblem{<:AllocatingEvaluation}, y, x) return copyto!(p.N, y, p.Λ!!(p.M, x)) end -function forward_operator!(p::AbstractPrimalDualProblem{<:MutatingEvaluation}, y, x) +function forward_operator!(p::AbstractPrimalDualProblem{<:InplaceEvaluation}, y, x) return p.Λ!!(p.M, y, x) end @@ -212,7 +212,7 @@ function adjoint_linearized_operator( return p.adjoint_linearized_operator!!(p.N, m, n, Y) end function adjoint_linearized_operator( - p::AbstractPrimalDualProblem{<:MutatingEvaluation}, m, n, Y + p::AbstractPrimalDualProblem{<:InplaceEvaluation}, m, n, Y ) X = zero_vector(p.M, m) return p.adjoint_linearized_operator!!(p.N, X, m, n, Y) @@ -223,7 +223,7 @@ function adjoint_linearized_operator!( return copyto!(p.M, X, p.adjoint_linearized_operator!!(p.N, m, n, Y)) end function adjoint_linearized_operator!( - p::AbstractPrimalDualProblem{<:MutatingEvaluation}, X, m, n, Y + p::AbstractPrimalDualProblem{<:InplaceEvaluation}, X, m, n, Y ) return p.adjoint_linearized_operator!!(p.N, X, m, n, Y) end diff --git a/src/plans/problem.jl b/src/plans/problem.jl index 626f5d8037..1c8489cf05 100644 --- a/src/plans/problem.jl +++ b/src/plans/problem.jl @@ -3,29 +3,45 @@ # # --- -""" - Problem{T} +@doc raw""" + AbstractManoptProblem + +Describe a Riemannian optimization problem with all static (not-changing) properties. -Describe the problem that should be optimized by stating all properties, that do not change -during an optimization or that are dependent of a certain solver. +The most prominent features that should always be stated here are -The parameter `T` can be used to distinguish problems with different representations -or implementations. -The default parameter [`AllocatingEvaluation`](@ref), which might be slower but easier to use. -The usually faster parameter value is [`MutatingEvaluation`](@ref) +* the `AbstractManifold` ``\mathcal M`` (cf. [ManifoldsBase.jl#AbstractManifold](https://juliamanifolds.github.io/ManifoldsBase.jl/stable/types.html#The-AbstractManifold)) +* the cost function ``f\colon \mathcal M → ℝ`` -See [`Options`](@ref) for the changing and solver dependent properties. +Usually the cost should be within an [`AbstractManifoldObjective`](@ref). """ -abstract type Problem{T} end +abstract type AbstractManoptProblem end + +@doc raw""" + AbstractManifoldObjective{T} + +Describe the collection of the optimization function ``f\colon \mathcal M → \bbR` (or even a vectorial range) +and its corresponding elements, which might for example be a gradient or (one or more) prxomial maps. + +All these elements should usually be implemented as functions +`(M, p) -> ...`, or `(M, X, p) -> ...` that is + +* the first argument of these functions should be the manifold `M` they are defined on +* the argument `X` is present, if the computation is performed inplace of `X` (see [`InplaceEvaluation`](@ref)) +* the argument `p` is the place the function (``f`` or one of its elements) is evaluated __at__. +the type `T` indicates the global [`AbstractEvaluationType`](@ref). """ +abstract type AbstractManifoldObjective{T} end + +@doc raw""" AbstractEvaluationType An abstract type to specify the kind of evaluation a [`Problem`](@ref) supports. """ abstract type AbstractEvaluationType end -""" +@doc raw""" AllocatingEvaluation <: AbstractEvaluationType A parameter for a [`Problem`](@ref) indicating that the problem uses functions that @@ -33,19 +49,56 @@ allocate memory for their result, i.e. they work out of place. """ struct AllocatingEvaluation <: AbstractEvaluationType end -""" - MutatingEvaluation +@doc raw""" + InplaceEvaluation A parameter for a [`Problem`](@ref) indicating that the problem uses functions that do not allocate memory but work on their input, i.e. in place. """ -struct MutatingEvaluation <: AbstractEvaluationType end +struct InplaceEvaluation <: AbstractEvaluationType end +@doc raw""" + DefaultManoptProblem{TM <: AbstractManifold, Objective <: AbstractManifoldObjective} + +Model a default manifold problem, that (just) consists of the domain of optimisatio, +that is an `AbstractManifold` and a [`AbstractManifoldObjective`](@ref) """ - get_cost(P::Problem, p) +struct DefaultManoptProblem{TM<:AbstractManifold,Objective<:AbstractManifoldObjective} + manifold::TM + objetive::Objective +end + +@doc raw""" + get_manifold(mp::AbstractManoptProblem) -evaluate the cost function `F` stored within a [`Problem`](@ref) `P` at the point `p`. +return the manifold stored within an [`AbstractManoptProblem`](@ref) """ -function get_cost(P::Problem, p) - return P.cost(P.M, p) +get_manifold(::AbstractManoptProblem) + +get_manifold(mp::DefaultManoptProblem) = mp.manifold + +@doc raw""" + get_objective(mp::AbstractManoptProblem) + +return the objective [`AbstractManifoldObjective`](@ref) stored within an [`AbstractManoptProblem`](@ref). +""" +get_objective(::AbstractManoptProblem) + +get_objective(mp::DefaultManoptProblem) = mp.objective + +@doc raw""" + get_cost(mp::AbstractManoptProblem, p) + +evaluate the cost function `f` stored within the [`AbstractManifoldObjective`](@ref) of an +[`AbstractManoptProblem`](@ref) `mp` at the point `p`. +""" +function get_cost(mp::AbstractManoptProblem, p) + return get_cost(get_manifold(mp), get_objective(mp), p) end + +@doc raw""" + get_cost(M::AbstractManifold, obj::AbstractManifoldObjective, p) + +evaluate the cost function `f` defined on `M` stored within the [`AbstractManifoldObjective`](@ref) at the point `p`. +""" +get_cost(::AbstractManifold, ::AbstractManifoldObjective, p) diff --git a/src/plans/proximal_plan.jl b/src/plans/proximal_plan.jl index e5ca726f18..e048baad3e 100644 --- a/src/plans/proximal_plan.jl +++ b/src/plans/proximal_plan.jl @@ -4,7 +4,7 @@ # # @doc raw""" - ProximalProblem <: Problem + ProximalProblem <:AbstractManoptProblem specify a problem for solvers based on the evaluation of proximal map(s). # Fields @@ -22,7 +22,7 @@ specify a problem for solvers based on the evaluation of proximal map(s). """ mutable struct ProximalProblem{ T,mT<:AbstractManifold,TCost,TProxes<:Union{Tuple,AbstractVector} -} <: Problem{T} +} <: AbstractManoptProblem M::mT cost::TCost proximal_maps!!::TProxes @@ -72,12 +72,12 @@ function get_proximal_map!(p::ProximalProblem{AllocatingEvaluation}, y, λ, x, i check_prox_number(length(p.proximal_maps!!), i) return copyto!(p.M, y, p.proximal_maps!![i](p.M, λ, x)) end -function get_proximal_map(p::ProximalProblem{MutatingEvaluation}, λ, x, i) +function get_proximal_map(p::ProximalProblem{InplaceEvaluation}, λ, x, i) check_prox_number(length(p.proximal_maps!!), i) y = allocate_result(p.M, get_proximal_map, x) return p.proximal_maps!![i](p.M, y, λ, x) end -function get_proximal_map!(p::ProximalProblem{MutatingEvaluation}, y, λ, x, i) +function get_proximal_map!(p::ProximalProblem{InplaceEvaluation}, y, λ, x, i) check_prox_number(length(p.proximal_maps!!), i) return p.proximal_maps!![i](p.M, y, λ, x) end diff --git a/src/plans/record_options.jl b/src/plans/record_options.jl index f89b40633e..24ff043ad3 100644 --- a/src/plans/record_options.jl +++ b/src/plans/record_options.jl @@ -217,7 +217,7 @@ mutable struct RecordGroup <: RecordAction RecordGroup() = new(Array{RecordAction,1}(), Dict{Symbol,Int}()) end -function (d::RecordGroup)(p::P, o::O, i::Int) where {P<:Problem,O<:Options} +function (d::RecordGroup)(p::P, o::O, i::Int) where {P<:AbstractManoptProblem,O<:Options} for ri in d.group ri(p, o, i) end @@ -280,7 +280,7 @@ mutable struct RecordEvery <: RecordAction return new(r, every, alwaysUpdate) end end -function (d::RecordEvery)(p::P, o::O, i::Int) where {P<:Problem,O<:Options} +function (d::RecordEvery)(p::P, o::O, i::Int) where {P<:AbstractManoptProblem,O<:Options} if i <= 0 d.record(p, o, i) elseif (rem(i, d.every) == 0) @@ -337,7 +337,7 @@ mutable struct RecordChange{TInvRetr<:AbstractInverseRetractionMethod} <: Record return new{typeof(invretr)}(Vector{Float64}(), a, invretr) end end -function (r::RecordChange)(p::P, o::O, i::Int) where {P<:Problem,O<:Options} +function (r::RecordChange)(p::P, o::O, i::Int) where {P<:AbstractManoptProblem,O<:Options} record_or_reset!( r, if has_storage(r.storage, :Iterate) @@ -368,7 +368,7 @@ mutable struct RecordEntry{T} <: RecordAction end RecordEntry(::T, f::Symbol) where {T} = RecordEntry{T}(f) RecordEntry(d::DataType, f::Symbol) = RecordEntry{d}(f) -function (r::RecordEntry{T})(::Problem, o::Options, i) where {T} +function (r::RecordEntry{T})(::AbstractManoptProblem, o::Options, i) where {T} return record_or_reset!(r, getfield(o, r.field), i) end @@ -398,7 +398,9 @@ mutable struct RecordEntryChange <: RecordAction return new(Float64[], f, d, a) end end -function (r::RecordEntryChange)(p::P, o::O, i::Int) where {P<:Problem,O<:Options} +function (r::RecordEntryChange)( + p::P, o::O, i::Int +) where {P<:AbstractManoptProblem,O<:Options} value = 0.0 if has_storage(r.storage, r.field) value = r.distance(p, o, getfield(o, r.field), get_storage(r.storage, r.field)) @@ -434,7 +436,7 @@ function RecordIterate() ) end -function (r::RecordIterate{T})(::Problem, o::Options, i) where {T} +function (r::RecordIterate{T})(::AbstractManoptProblem, o::Options, i) where {T} return record_or_reset!(r, get_iterate(o), i) end @@ -447,7 +449,7 @@ mutable struct RecordIteration <: RecordAction recorded_values::Array{Int,1} RecordIteration() = new(Array{Int,1}()) end -function (r::RecordIteration)(::Problem, ::Options, i::Int) +function (r::RecordIteration)(::AbstractManoptProblem, ::Options, i::Int) return record_or_reset!(r, i, i) end @@ -460,7 +462,7 @@ mutable struct RecordCost <: RecordAction recorded_values::Array{Float64,1} RecordCost() = new(Array{Float64,1}()) end -function (r::RecordCost)(p::P, o::O, i::Int) where {P<:Problem,O<:Options} +function (r::RecordCost)(p::P, o::O, i::Int) where {P<:AbstractManoptProblem,O<:Options} return record_or_reset!(r, get_cost(p, get_iterate(o)), i) end @@ -488,7 +490,7 @@ mutable struct RecordTime <: RecordAction return new(Array{Nanosecond,1}(), Nanosecond(time_ns()), mode) end end -function (r::RecordTime)(p::P, o::O, i::Int) where {P<:Problem,O<:Options} +function (r::RecordTime)(p::P, o::O, i::Int) where {P<:AbstractManoptProblem,O<:Options} # At iteartion zero also reset start (i == 0) && (r.start = Nanosecond(time_ns())) t = Nanosecond(time_ns()) - r.start diff --git a/src/plans/stepsize.jl b/src/plans/stepsize.jl index 697acdf440..c0eff8af7c 100644 --- a/src/plans/stepsize.jl +++ b/src/plans/stepsize.jl @@ -29,7 +29,7 @@ function ConstantStepsize( end function (s::ConstantStepsize)( p::P, o::O, i::Int, args...; kwargs... -) where {P<:Problem,O<:Options} +) where {P<:AbstractManoptProblem,O<:Options} return s.length end get_initial_stepsize(s::ConstantStepsize) = s.length @@ -90,7 +90,7 @@ function DecreasingStepsize( end function (s::DecreasingStepsize)( ::P, ::O, i::Int, args...; kwargs... -) where {P<:Problem,O<:Options} +) where {P<:AbstractManoptProblem,O<:Options} return (s.length - i * s.subtrahend) * (s.factor^i) / ((i + s.shift)^(s.exponent)) end get_initial_stepsize(s::DecreasingStepsize) = s.length @@ -797,50 +797,66 @@ function (a::WolfePowellBinaryLinesearch)( end @doc raw""" - get_stepsize(p::Problem, o::Options, vars...) + get_stepsize(p::AbstractManoptProblem, o::Options, vars...) return the stepsize stored within [`Options`](@ref) `o` when solving [`Problem`](@ref) `p`. This method also works for decorated options and the [`Stepsize`](@ref) function within the options, by default stored in `o.stepsize`. """ -function get_stepsize(p::Problem, o::Options, vars...; kwargs...) +function get_stepsize(p::AbstractManoptProblem, o::Options, vars...; kwargs...) return get_stepsize(p, o, dispatch_options_decorator(o), vars...; kwargs...) end -function get_stepsize(p::Problem, o::Options, ::Val{true}, vars...; kwargs...) +function get_stepsize(p::AbstractManoptProblem, o::Options, ::Val{true}, vars...; kwargs...) return get_stepsize(p, o.options, vars...; kwargs...) end -function get_stepsize(p::Problem, o::Options, ::Val{false}, vars...; kwargs...) +function get_stepsize( + p::AbstractManoptProblem, o::Options, ::Val{false}, vars...; kwargs... +) return o.stepsize(p, o, vars...; kwargs...) end -function get_initial_stepsize(p::Problem, o::Options, vars...; kwargs...) +function get_initial_stepsize(p::AbstractManoptProblem, o::Options, vars...; kwargs...) return get_initial_stepsize( - p::Problem, o::Options, dispatch_options_decorator(o), vars...; kwargs... + p::AbstractManoptProblem, + o::Options, + dispatch_options_decorator(o), + vars...; + kwargs..., ) end -function get_initial_stepsize(p::Problem, o::Options, ::Val{true}, vars...) +function get_initial_stepsize(p::AbstractManoptProblem, o::Options, ::Val{true}, vars...) return get_initial_stepsize(p, o.options) end -function get_initial_stepsize(p::Problem, o::Options, ::Val{false}, vars...) +function get_initial_stepsize(p::AbstractManoptProblem, o::Options, ::Val{false}, vars...) return get_initial_stepsize(o.stepsize) end -function get_last_stepsize(p::Problem, o::Options, vars...) +function get_last_stepsize(p::AbstractManoptProblem, o::Options, vars...) return get_last_stepsize(p, o, dispatch_options_decorator(o), vars...) end -function get_last_stepsize(p::Problem, o::Options, ::Val{true}, vars...) +function get_last_stepsize(p::AbstractManoptProblem, o::Options, ::Val{true}, vars...) return get_last_stepsize(p, o.options, vars...) end -function get_last_stepsize(p::Problem, o::Options, ::Val{false}, vars...) +function get_last_stepsize(p::AbstractManoptProblem, o::Options, ::Val{false}, vars...) return get_last_stepsize(p, o, o.stepsize, vars...) end # # dispatch on stepsize -get_last_stepsize(p::Problem, o::Options, s::Stepsize, vars...) = s(p, o, vars...) -get_last_stepsize(::Problem, ::Options, s::ArmijoLinesearch, ::Any...) = s.last_stepsize -function get_last_stepsize(::Problem, ::Options, s::WolfePowellLinesearch, ::Any...) +function get_last_stepsize(p::AbstractManoptProblem, o::Options, s::Stepsize, vars...) + return s(p, o, vars...) +end +function get_last_stepsize( + ::AbstractManoptProblem, ::Options, s::ArmijoLinesearch, ::Any... +) return s.last_stepsize end -function get_last_stepsize(::Problem, ::Options, s::WolfePowellBinaryLinesearch, ::Any...) +function get_last_stepsize( + ::AbstractManoptProblem, ::Options, s::WolfePowellLinesearch, ::Any... +) + return s.last_stepsize +end +function get_last_stepsize( + ::AbstractManoptProblem, ::Options, s::WolfePowellBinaryLinesearch, ::Any... +) return s.last_stepsize end diff --git a/src/plans/stochastic_gradient_plan.jl b/src/plans/stochastic_gradient_plan.jl index 4da1118c88..009762562b 100644 --- a/src/plans/stochastic_gradient_plan.jl +++ b/src/plans/stochastic_gradient_plan.jl @@ -1,5 +1,5 @@ @doc raw""" - StochasticGradientProblem <: Problem + StochasticGradientProblem <:AbstractManoptProblem A stochastic gradient problem consists of * a `Manifold M` @@ -50,7 +50,7 @@ end Evaluate all summands gradients ``\{\operatorname{grad}f_i\}_{i=1}^n`` at `x` (in place of `Y`). -Note that for the [`MutatingEvaluation`](@ref) based problem and a single function for the +Note that for the [`InplaceEvaluation`](@ref) based problem and a single function for the stochastic gradient, the allocating variant is not available. """ function get_gradients( @@ -85,26 +85,26 @@ function get_gradients!( return X end function get_gradients( - ::StochasticGradientProblem{MutatingEvaluation,<:AbstractManifold,TC,<:Function}, ::Any + ::StochasticGradientProblem{InplaceEvaluation,<:AbstractManifold,TC,<:Function}, ::Any ) where {TC} return error( "For a mutating function type stochastic gradient, the allocating variant is not possible.", ) end function get_gradients( - p::StochasticGradientProblem{MutatingEvaluation,<:AbstractManifold,TC,<:AbstractVector}, + p::StochasticGradientProblem{InplaceEvaluation,<:AbstractManifold,TC,<:AbstractVector}, x, ) where {TC} X = [zero_vector(p.M, x) for _ in 1:length(p.gradient!!)] return get_gradients!(p, X, x) end function get_gradients!( - p::StochasticGradientProblem{MutatingEvaluation,<:AbstractManifold,TC,<:Function}, X, x + p::StochasticGradientProblem{InplaceEvaluation,<:AbstractManifold,TC,<:Function}, X, x ) where {TC} return p.gradient!!(p.M, X, x) end function get_gradients!( - p::StochasticGradientProblem{MutatingEvaluation,<:AbstractManifold,TC,<:AbstractVector}, + p::StochasticGradientProblem{InplaceEvaluation,<:AbstractManifold,TC,<:AbstractVector}, X, x, ) where {TC} @@ -120,7 +120,7 @@ end Evaluate one of the summands gradients ``\operatorname{grad}f_k``, ``k∈\{1,…,n\}``, at `x` (in place of `Y`). -Note that for the [`MutatingEvaluation`](@ref) based problem and a single function for the +Note that for the [`InplaceEvaluation`](@ref) based problem and a single function for the stochastic gradient mutating variant is not available, since it would require too many allocations. """ function get_gradient( @@ -160,13 +160,13 @@ function get_gradient!( return X end function get_gradient( - p::StochasticGradientProblem{MutatingEvaluation,<:AbstractManifold,TC}, k, x + p::StochasticGradientProblem{InplaceEvaluation,<:AbstractManifold,TC}, k, x ) where {TC} X = zero_vector(p.M, x) return get_gradient!(p, X, k, x) end function get_gradient!( - ::StochasticGradientProblem{MutatingEvaluation,<:AbstractManifold,TC,<:Function}, + ::StochasticGradientProblem{InplaceEvaluation,<:AbstractManifold,TC,<:Function}, ::Any, ::Any, ::Any, @@ -176,7 +176,7 @@ function get_gradient!( ) end function get_gradient!( - p::StochasticGradientProblem{MutatingEvaluation,<:AbstractManifold,TC,<:AbstractVector}, + p::StochasticGradientProblem{InplaceEvaluation,<:AbstractManifold,TC,<:AbstractVector}, X, k, x, diff --git a/src/plans/stopping_criterion.jl b/src/plans/stopping_criterion.jl index aa40c8c6e3..4ed3bb122a 100644 --- a/src/plans/stopping_criterion.jl +++ b/src/plans/stopping_criterion.jl @@ -25,7 +25,9 @@ mutable struct StopAfterIteration <: StoppingCriterion reason::String StopAfterIteration(mIter::Int) = new(mIter, "") end -function (c::StopAfterIteration)(::P, ::O, i::Int) where {P<:Problem,O<:Options} +function (c::StopAfterIteration)( + ::P, ::O, i::Int +) where {P<:AbstractManoptProblem,O<:Options} (i == 0) && (c.reason = "") # reset on init if i >= c.maxIter c.reason = "The algorithm reached its maximal number of iterations ($(c.maxIter)).\n" @@ -55,7 +57,7 @@ mutable struct StopWhenGradientNormLess <: StoppingCriterion reason::String StopWhenGradientNormLess(ε::Float64) = new(ε, "") end -function (c::StopWhenGradientNormLess)(p::Problem, o::Options, i::Int) +function (c::StopWhenGradientNormLess)(p::AbstractManoptProblem, o::Options, i::Int) (i == 0) && (c.reason = "") # reset on init if norm(p.M, get_iterate(o), get_gradient(o)) < c.threshold c.reason = "The algorithm reached approximately critical point after $i iterations; the gradient norm ($(norm(p.M,get_iterate(o),get_gradient(o)))) is less than $(c.threshold).\n" @@ -101,7 +103,7 @@ mutable struct StopWhenChangeLess <: StoppingCriterion return new(ε, "", a) end end -function (c::StopWhenChangeLess)(P::Problem, O::Options, i) +function (c::StopWhenChangeLess)(P::AbstractManoptProblem, O::Options, i) (i == 0) && (c.reason = "") # reset on init if has_storage(c.storage, :Iterate) x_old = get_storage(c.storage, :Iterate) @@ -145,7 +147,9 @@ mutable struct StopWhenStepsizeLess <: StoppingCriterion return new(ε, "") end end -function (c::StopWhenStepsizeLess)(p::P, o::O, i::Int) where {P<:Problem,O<:Options} +function (c::StopWhenStepsizeLess)( + p::P, o::O, i::Int +) where {P<:AbstractManoptProblem,O<:Options} (i == 0) && (c.reason = "") # reset on init s = get_last_stepsize(p, o, i) if s < c.threshold && i > 0 @@ -182,7 +186,9 @@ mutable struct StopWhenCostLess <: StoppingCriterion reason::String StopWhenCostLess(ε::Float64) = new(ε, "") end -function (c::StopWhenCostLess)(p::P, o::O, i::Int) where {P<:Problem,O<:Options} +function (c::StopWhenCostLess)( + p::P, o::O, i::Int +) where {P<:AbstractManoptProblem,O<:Options} (i == 0) && (c.reason = "") # reset on init if i > 0 && get_cost(p, get_iterate(o)) < c.threshold c.reason = "The algorithm reached a cost function value ($(get_cost(p,get_iterate(o)))) less than the threshold ($(c.threshold)).\n" @@ -224,7 +230,9 @@ mutable struct StopWhenSmallerOrEqual <: StoppingCriterion reason::String StopWhenSmallerOrEqual(value::Symbol, mValue::Real) = new(value, mValue, "") end -function (c::StopWhenSmallerOrEqual)(::P, o::O, i::Int) where {P<:Problem,O<:Options} +function (c::StopWhenSmallerOrEqual)( + ::P, o::O, i::Int +) where {P<:AbstractManoptProblem,O<:Options} (i == 0) && (c.reason = "") # reset on init if getfield(o, c.value) <= c.minValue c.reason = "The value of the variable ($(string(c.value))) is smaller than or equal to its threshold ($(c.minValue)).\n" @@ -258,7 +266,7 @@ mutable struct StopAfter <: StoppingCriterion end end end -function (c::StopAfter)(p::P, o::O, i::Int) where {P<:Problem,O<:Options} +function (c::StopAfter)(p::P, o::O, i::Int) where {P<:AbstractManoptProblem,O<:Options} if value(c.start) == 0 || i <= 0 # (re)start timer c.reason = "" c.start = Nanosecond(time_ns()) @@ -304,7 +312,7 @@ mutable struct StopWhenAll{TCriteria<:Tuple} <: StoppingCriterionSet StopWhenAll(c::Vector{StoppingCriterion}) = new{typeof(tuple(c...))}(tuple(c...), "") StopWhenAll(c...) = new{typeof(c)}(c, "") end -function (c::StopWhenAll)(p::P, o::O, i::Int) where {P<:Problem,O<:Options} +function (c::StopWhenAll)(p::P, o::O, i::Int) where {P<:AbstractManoptProblem,O<:Options} (i == 0) && (c.reason = "") # reset on init if all(subC -> subC(p, o, i), c.criteria) c.reason = string([get_reason(subC) for subC in c.criteria]...) @@ -357,7 +365,7 @@ mutable struct StopWhenAny{TCriteria<:Tuple} <: StoppingCriterionSet StopWhenAny(c::Vector{StoppingCriterion}) = new{typeof(tuple(c...))}(tuple(c...), "") StopWhenAny(c::StoppingCriterion...) = new{typeof(c)}(c) end -function (c::StopWhenAny)(p::P, o::O, i::Int) where {P<:Problem,O<:Options} +function (c::StopWhenAny)(p::P, o::O, i::Int) where {P<:AbstractManoptProblem,O<:Options} (i == 0) && (c.reason = "") # reset on init if any(subC -> subC(p, o, i), c.criteria) c.reason = string([get_reason(subC) for subC in c.criteria]...) diff --git a/src/plans/subgradient_plan.jl b/src/plans/subgradient_plan.jl index b330006b5b..3e5d11df51 100644 --- a/src/plans/subgradient_plan.jl +++ b/src/plans/subgradient_plan.jl @@ -1,5 +1,5 @@ @doc raw""" - SubGradientProblem <: Problem + SubGradientProblem <:AbstractManoptProblem A structure to store information about a subgradient based optimization problem @@ -16,7 +16,7 @@ Generate the [`Problem`] for a subgradient problem, i.e. a function `f` on the manifold `M` and a function `∂f` that returns an element from the subdifferential at a point. """ -struct SubGradientProblem{T,mT<:AbstractManifold,C,S} <: Problem{T} +struct SubGradientProblem{T,mT<:AbstractManifold,C,S} <: AbstractManoptProblem M::mT cost::C subgradient!!::S @@ -37,20 +37,20 @@ Evaluate the (sub)gradient of a [`SubGradientProblem`](@ref)` p` at the point `q The evaluation is done in place of `X` for the `!`-variant. The `T=`[`AllocatingEvaluation`](@ref) problem might still allocate memory within. -When the non-mutating variant is called with a `T=`[`MutatingEvaluation`](@ref) +When the non-mutating variant is called with a `T=`[`InplaceEvaluation`](@ref) memory for the result is allocated. """ function get_subgradient(p::SubGradientProblem{AllocatingEvaluation}, q) return p.subgradient!!(p.M, q) end -function get_subgradient(p::SubGradientProblem{MutatingEvaluation}, q) +function get_subgradient(p::SubGradientProblem{InplaceEvaluation}, q) X = zero_vector(p.M, q) return p.subgradient!!(p.M, X, q) end function get_subgradient!(p::SubGradientProblem{AllocatingEvaluation}, X, q) return copyto!(p.M, X, p.subgradient!!(p.M, q)) end -function get_subgradient!(p::SubGradientProblem{MutatingEvaluation}, X, q) +function get_subgradient!(p::SubGradientProblem{InplaceEvaluation}, X, q) return p.subgradient!!(p.M, X, q) end diff --git a/src/solvers/ChambollePock.jl b/src/solvers/ChambollePock.jl index 3ef88bd773..b0e10cf0fb 100644 --- a/src/solvers/ChambollePock.jl +++ b/src/solvers/ChambollePock.jl @@ -168,7 +168,7 @@ function ChambollePock!( vector_transport_method=vector_transport_method, ) o = decorate_options(o; kwargs...) - return get_solver_return(solve(p, o)) + return get_solver_return(solve!(p, o)) end function initialize_solver!(::PrimalDualProblem, ::ChambollePockOptions) end diff --git a/src/solvers/DouglasRachford.jl b/src/solvers/DouglasRachford.jl index ad4308d1b1..e7b5ef5cad 100644 --- a/src/solvers/DouglasRachford.jl +++ b/src/solvers/DouglasRachford.jl @@ -21,7 +21,7 @@ i.e. component wise in a vector. # Optional values the default parameter is given in brackets * `evaluation` – ([`AllocatingEvaluation`](@ref)) specify whether the proximal maps work by allocation (default) form `prox(M, λ, x)` - or [`MutatingEvaluation`](@ref) in place, i.e. is of the form `prox!(M, y, λ, x)`. + or [`InplaceEvaluation`](@ref) in place, i.e. is of the form `prox!(M, y, λ, x)`. * `λ` – (`(iter) -> 1.0`) function to provide the value for the proximal parameter during the calls * `α` – (`(iter) -> 0.9`) relaxation of the step from old to new iterate, i.e. @@ -118,7 +118,7 @@ function DouglasRachford!( parallel=parallel > 0, ) o = decorate_options(o; kwargs...) - return get_solver_return(solve(p, o)) + return get_solver_return(solve!(p, o)) end function initialize_solver!(::ProximalProblem, ::DouglasRachfordOptions) end function step_solver!(p::ProximalProblem, o::DouglasRachfordOptions, iter) diff --git a/src/solvers/FrankWolfe.jl b/src/solvers/FrankWolfe.jl index 3ba8f821b3..5d447c610e 100644 --- a/src/solvers/FrankWolfe.jl +++ b/src/solvers/FrankWolfe.jl @@ -76,19 +76,19 @@ function Frank_Wolfe_method!( evaluation=evaluation, ) O = decorate_options(O; kwargs...) - return get_solver_return(solve(P, O)) + return get_solver_return(solve!(P, O)) end function initialize_solver!(P::GradientProblem, O::FrankWolfeOptions) get_gradient!(P, O.X, O.p) return O end function step_solver!( - P::GradientProblem, O::FrankWolfeOptions{<:Tuple{<:Problem,<:Options}}, i + P::GradientProblem, O::FrankWolfeOptions{<:Tuple{<:AbstractManoptProblem,<:Options}}, i ) # update gradient get_gradient!(P, O.X, O.p) # evaluate grad F(p), store the result in O.X # solve subtask - solve(O.subtask[1], O.subtask[2]) # call the subsolver + solve!(O.subtask[1], O.subtask[2]) # call the subsolver q = get_solver_result(O.subtask[2]) s = O.stepsize(P, O, i) # step along the geodesic @@ -105,7 +105,7 @@ end # Variant II: subtask is a mutating function providing a closed form soltuion # function step_solver!( - P::GradientProblem, O::FrankWolfeOptions{<:Tuple{S,<:MutatingEvaluation}}, i + P::GradientProblem, O::FrankWolfeOptions{<:Tuple{S,<:InplaceEvaluation}}, i ) where {S} get_gradient!(P, O.X, O.p) # evaluate grad F in place for O.X q = copy(P.M, O.p) diff --git a/src/solvers/NelderMead.jl b/src/solvers/NelderMead.jl index 445c2a1f1f..d9ba74156c 100644 --- a/src/solvers/NelderMead.jl +++ b/src/solvers/NelderMead.jl @@ -1,6 +1,6 @@ @doc raw""" - NelderMead(M, F [, p]) -perform a nelder mead minimization problem for the cost function `F` on the + NelderMead(M, f [, population]) +perform a Nelder-Mead minimization problem for the cost function ``f\colon \mathcal M`` on the manifold `M`. If the initial population `p` is not given, a random set of points is chosen. @@ -12,8 +12,8 @@ and # Input * `M` – a manifold ``\mathcal M`` -* `F` – a cost function ``F:\mathcal M→ℝ`` to minimize -* `population` – (n+1 `random_point(M)`) an initial population of ``n+1`` points, where ``n`` +* `f` – a cost function to minimize +* `population` – (n+1 `random_point(M)`s) an initial population of ``n+1`` points, where ``n`` is the dimension of the manifold `M`. # Optional @@ -38,18 +38,18 @@ the obtained (approximate) minimizer ``x^*``, see [`get_solver_return`](@ref) fo """ function NelderMead( M::AbstractManifold, - F::TF, + f::TF, population=[random_point(M) for i in 1:(manifold_dimension(M) + 1)]; kwargs..., ) where {TF} res_population = copy.(Ref(M), population) - return NelderMead!(M, F, res_population; kwargs...) + return NelderMead!(M, f, res_population; kwargs...) end @doc raw""" - NelderMead(M, F [, p]) -perform a Nelder Mead minimization problem for the cost function `F` on the -manifold `M`. If the initial population `p` is not given, a random set of -points is chosen. If it is given, the computation is done in place of `p`. + NelderMead(M, F [, population]) +perform a Nelder Mead minimization problem for the cost function `f` on the +manifold `M`. If the initial population `population` is not given, a random set of +points is chosen. If it is given, the computation is done in place of `population`. For more options see [`NelderMead`](@ref). """ @@ -68,7 +68,7 @@ function NelderMead!( ), kwargs..., #collect rest ) where {TF} - p = CostProblem(M, F) + mp = DefaultManoptProblem(M, ManifoldCostObjective(f)) o = NelderMeadOptions( M, population; @@ -81,22 +81,27 @@ function NelderMead!( inverse_retraction_method=inverse_retraction_method, ) o = decorate_options(o; kwargs...) - return get_solver_return(solve(p, o)) + return get_solver_return(solve!(mp, o)) end # # Solver functions # -function initialize_solver!(p::CostProblem, o::NelderMeadOptions) +function initialize_solver!( + mp::DefaultManoptProblem{TM,<:ManifoldCostObjective}, o::NelderMeadOptions +) where {TM} # init cost and x - o.costs = get_cost.(Ref(p), o.population) + o.costs = get_cost.(Ref(mp), o.population) return o.x = o.population[argmin(o.costs)] # select min end -function step_solver!(p::CostProblem, o::NelderMeadOptions, ::Any) - m = mean(p.M, o.population) +function step_solver!( + mp::DefaultManoptProblem{TM,<:ManifoldCostObjective}, o::NelderMeadOptions, ::Any +) where {TM} + M = get_manifold(mp) + m = mean(M, o.population) ind = sortperm(o.costs) # reordering for cost and p, i.e. minimizer is at ind[1] - ξ = inverse_retract(p.M, m, o.population[last(ind)], o.inverse_retraction_method) + ξ = inverse_retract(M, m, o.population[last(ind)], o.inverse_retraction_method) # reflect last - xr = retract(p.M, m, -o.α * ξ, o.retraction_method) + xr = retract(M, m, -o.α * ξ, o.retraction_method) Costr = get_cost(p, xr) # is it better than the worst but not better than the best? if Costr >= o.costs[first(ind)] && Costr < o.costs[last(ind)] @@ -106,7 +111,7 @@ function step_solver!(p::CostProblem, o::NelderMeadOptions, ::Any) end # --- Expansion --- if Costr < o.costs[first(ind)] # reflected is better than fist -> expand - xe = retract(p.M, m, -o.γ * o.α * ξ, o.retraction_method) + xe = retract(M, m, -o.γ * o.α * ξ, o.retraction_method) Coste = get_cost(p, xe) # successful? use the expanded, otherwise still use xr o.population[last(ind)] .= Coste < Costr ? xe : xr @@ -115,7 +120,7 @@ function step_solver!(p::CostProblem, o::NelderMeadOptions, ::Any) # --- Contraction --- if Costr > o.costs[ind[end - 1]] # even worse than second worst s = (Costr < o.costs[last(ind)] ? -o.ρ : o.ρ) - xc = retract(p.M, m, s * ξ, o.retraction_method) + xc = retract(M, m, s * ξ, o.retraction_method) Costc = get_cost(p, xc) if Costc < o.costs[last(ind)] # better than last ? -> store o.population[last(ind)] = xc @@ -128,7 +133,7 @@ function step_solver!(p::CostProblem, o::NelderMeadOptions, ::Any) p.M, o.population[ind[1]], inverse_retract( - p.M, o.population[ind[1]], o.population[ind[i]], o.inverse_retraction_method + M, o.population[ind[1]], o.population[ind[i]], o.inverse_retraction_method ), o.σ, o.retraction_method, diff --git a/src/solvers/alternating_gradient_descent.jl b/src/solvers/alternating_gradient_descent.jl index 01a0d079dd..36c55f48eb 100644 --- a/src/solvers/alternating_gradient_descent.jl +++ b/src/solvers/alternating_gradient_descent.jl @@ -14,7 +14,7 @@ perform an alternating gradient descent # Optional * `evaluation` – ([`AllocatingEvaluation`](@ref)) specify whether the gradient(s) works by - allocation (default) form `gradF(M, x)` or [`MutatingEvaluation`](@ref) in place, i.e. + allocation (default) form `gradF(M, x)` or [`InplaceEvaluation`](@ref) in place, i.e. is of the form `gradF!(M, X, x)` (elementwise). * `evaluation_order` – (`:Linear`) – whether to use a randomly permuted sequence (`:FixedRandom`), a per @@ -88,7 +88,7 @@ function alternating_gradient_descent!( retraction_method=retraction_method, ) o = decorate_options(o; kwargs...) - return get_solver_return(solve(p, o)) + return get_solver_return(solve!(p, o)) end function initialize_solver!( ::AlternatingGradientProblem, o::AlternatingGradientDescentOptions diff --git a/src/solvers/augmented_Lagrangian_method.jl b/src/solvers/augmented_Lagrangian_method.jl index 99a26723a2..7a4c596922 100644 --- a/src/solvers/augmented_Lagrangian_method.jl +++ b/src/solvers/augmented_Lagrangian_method.jl @@ -142,7 +142,9 @@ function augmented_Lagrangian_method!( ); sub_kwargs..., ), - sub_problem::Problem=GradientProblem(M, sub_cost, sub_grad; evaluation=evaluation), + sub_problem::AbstractManoptProblem=GradientProblem( + M, sub_cost, sub_grad; evaluation=evaluation + ), stopping_criterion::StoppingCriterion=StopAfterIteration(300) | ( StopWhenSmallerOrEqual(:ϵ, ϵ_min) & StopWhenChangeLess(1e-10) ), @@ -168,7 +170,7 @@ function augmented_Lagrangian_method!( stopping_criterion=stopping_criterion, ) o = decorate_options(o; kwargs...) - return get_solver_return(solve(_problem, o)) + return get_solver_return(solve!(_problem, o)) end # @@ -190,7 +192,7 @@ function step_solver!(p::ConstrainedProblem, o::AugmentedLagrangianMethodOptions update_stopping_criterion!(o, :MinIterateChange, o.ϵ) - o.x = get_solver_result(solve(o.sub_problem, o.sub_options)) + o.x = get_solver_result(solve!(o.sub_problem, o.sub_options)) # update multipliers cost_ineq = get_inequality_constraints(p, o.x) diff --git a/src/solvers/conjugate_gradient_descent.jl b/src/solvers/conjugate_gradient_descent.jl index 23f3ddfac0..c787498f73 100644 --- a/src/solvers/conjugate_gradient_descent.jl +++ b/src/solvers/conjugate_gradient_descent.jl @@ -34,7 +34,7 @@ They all compute ``β_k`` such that this algorithm updates the search direction `p` is the current [`GradientProblem`](@ref), `o` are the [`ConjugateGradientDescentOptions`](@ref) `o` and `i` is the current iterate. * `evaluation` – ([`AllocatingEvaluation`](@ref)) specify whether the gradient works by allocation (default) form `gradF(M, x)` - or [`MutatingEvaluation`](@ref) in place, i.e. is of the form `gradF!(M, X, x)`. + or [`InplaceEvaluation`](@ref) in place, i.e. is of the form `gradF!(M, X, x)`. * `retraction_method` - (`default_retraction_method(M`) a retraction method to use. * `stepsize` - (`Constant(1.)`) A [`Stepsize`](@ref) function applied to the search direction. The default is a constant step size 1. @@ -99,7 +99,7 @@ function conjugate_gradient_descent!( X, ) o = decorate_options(o; kwargs...) - return get_solver_return(solve(p, o)) + return get_solver_return(solve!(p, o)) end function initialize_solver!(p::GradientProblem, o::ConjugateGradientDescentOptions) o.gradient = get_gradient(p, o.x) diff --git a/src/solvers/cyclic_proximal_point.jl b/src/solvers/cyclic_proximal_point.jl index d66964418e..b8ba6102ec 100644 --- a/src/solvers/cyclic_proximal_point.jl +++ b/src/solvers/cyclic_proximal_point.jl @@ -13,7 +13,7 @@ perform a cyclic proximal point algorithm. # Optional the default values are given in brackets * `evaluation` – ([`AllocatingEvaluation`](@ref)) specify whether the proximal maps work by allocation (default) form `prox(M, λ, x)` - or [`MutatingEvaluation`](@ref) in place, i.e. is of the form `prox!(M, y, λ, x)`. + or [`InplaceEvaluation`](@ref) in place, i.e. is of the form `prox!(M, y, λ, x)`. * `evaluation_order` – (`:Linear`) – whether to use a randomly permuted sequence (`:FixedRandom`), a per cycle permuted sequence (`:Random`) or the default linear one. @@ -66,7 +66,7 @@ function cyclic_proximal_point!( M, x0; stopping_criterion=stopping_criterion, λ=λ, evaluation_order=evaluation_order ) o = decorate_options(o; kwargs...) - return get_solver_return(solve(p, o)) + return get_solver_return(solve!(p, o)) end function initialize_solver!(p::ProximalProblem, o::CyclicProximalPointOptions) c = length(p.proximal_maps!!) diff --git a/src/solvers/debug_solver.jl b/src/solvers/debug_solver.jl index 4f2ca49919..7175027d67 100644 --- a/src/solvers/debug_solver.jl +++ b/src/solvers/debug_solver.jl @@ -1,5 +1,5 @@ """ - initialize_solver!(p::Problem, o::DebugOptions) + initialize_solver!(p::AbstractManoptProblem, o::DebugOptions) Initialize the solver to the optimization [`Problem`](@ref) by initializing all values in the [`DebugOptions`](@ref)` o`. @@ -7,19 +7,19 @@ values in the [`DebugOptions`](@ref)` o`. Since debug acts as a decorator this also calls the `initialize_solver!` of the correpsonding internally stored options """ -function initialize_solver!(p::Problem, o::DebugOptions) +function initialize_solver!(p::AbstractManoptProblem, o::DebugOptions) initialize_solver!(p, o.options) get(o.debugDictionary, :Start, DebugDivider(""))(p, get_options(o), 0) get(o.debugDictionary, :All, DebugDivider(""))(p, get_options(o), 0) return o end """ - step_solver!(p::Problem, o::DebugOptions, i) + step_solver!(p::AbstractManoptProblem, o::DebugOptions, i) Do one iteration step (the `i`th) for [`Problem`](@ref)` p` by modifying the values in the [`Options`](@ref)` o.options` and print the debug specified """ -function step_solver!(p::Problem, o::DebugOptions, i) +function step_solver!(p::AbstractManoptProblem, o::DebugOptions, i) step_solver!(p, o.options, i) get(o.debugDictionary, :Step, DebugDivider(""))(p, get_options(o), i) get(o.debugDictionary, :All, DebugDivider(""))(p, get_options(o), i) @@ -33,7 +33,7 @@ determine whether the solver for [`Problem`](@ref) `p` and the [`DebugOptions`]( should stop at iteration `i` by calling the function corresponding to the internally stored [`Options`](@ref). If so, print debug from `:All` and `:Stop`. """ -function stop_solver!(p::Problem, o::DebugOptions, i::Int) +function stop_solver!(p::AbstractManoptProblem, o::DebugOptions, i::Int) s = stop_solver!(p, o.options, i) if s get(o.debugDictionary, :Stop, DebugDivider(""))(p, get_options(o), typemin(Int)) diff --git a/src/solvers/exact_penalty_method.jl b/src/solvers/exact_penalty_method.jl index 1e30bf2c09..3902e9731e 100644 --- a/src/solvers/exact_penalty_method.jl +++ b/src/solvers/exact_penalty_method.jl @@ -116,7 +116,9 @@ function exact_penalty_method!( problem=ConstrainedProblem(M, F, gradF, G, gradG, H, gradH; evaluation=evaluation), sub_cost=ExactPenaltyCost(problem, ρ, u; smoothing=smoothing), sub_grad=ExactPenaltyGrad(problem, ρ, u; smoothing=smoothing), - sub_problem::Problem=GradientProblem(M, sub_cost, sub_grad; evaluation=evaluation), + sub_problem::AbstractManoptProblem=GradientProblem( + M, sub_cost, sub_grad; evaluation=evaluation + ), sub_kwargs=[], sub_stopping_criterion=StopAfterIteration(300) | StopWhenGradientNormLess(ϵ) | @@ -155,7 +157,7 @@ function exact_penalty_method!( stopping_criterion=stopping_criterion, ) o = decorate_options(o; kwargs...) - return get_solver_return(solve(problem, o)) + return get_solver_return(solve!(problem, o)) end # # Solver functions @@ -172,7 +174,7 @@ function step_solver!(p::ConstrainedProblem, o::ExactPenaltyMethodOptions, iter) set_iterate!(o.sub_options, copy(p.M, o.x)) update_stopping_criterion!(o, :MinIterateChange, o.ϵ) - o.x = get_solver_result(solve(o.sub_problem, o.sub_options)) + o.x = get_solver_result(solve!(o.sub_problem, o.sub_options)) # get new evaluation of penalty cost_ineq = get_inequality_constraints(p, o.x) diff --git a/src/solvers/gradient_descent.jl b/src/solvers/gradient_descent.jl index 602ba1b9da..d5ec360aa7 100644 --- a/src/solvers/gradient_descent.jl +++ b/src/solvers/gradient_descent.jl @@ -18,7 +18,7 @@ with different choices of ``s_k`` available (see `stepsize` option below). # Optional * `direction` – [`IdentityUpdateRule`](@ref) perform a processing of the direction, e.g. * `evaluation` – ([`AllocatingEvaluation`](@ref)) specify whether the gradient works by allocation (default) form `gradF(M, x)` - or [`MutatingEvaluation`](@ref) in place, i.e. is of the form `gradF!(M, X, x)`. + or [`InplaceEvaluation`](@ref) in place, i.e. is of the form `gradF!(M, X, x)`. * `retraction_method` – (`default_retraction_method(M)`) a `retraction(M,x,ξ)` to use. * `stepsize` – ([`ConstantStepsize`](@ref)`(1.)`) specify a [`Stepsize`](@ref) functor. @@ -80,7 +80,7 @@ function gradient_descent!( retraction_method=retraction_method, ) o = decorate_options(o; debug=debug, kwargs...) - return get_solver_return(solve(p, o)) + return get_solver_return(solve!(p, o)) end # # Solver functions diff --git a/src/solvers/particle_swarm.jl b/src/solvers/particle_swarm.jl index 613dc343fe..6600a5ec47 100644 --- a/src/solvers/particle_swarm.jl +++ b/src/solvers/particle_swarm.jl @@ -133,32 +133,37 @@ function particle_swarm!( vector_transport_method=vector_transport_method, ) o = decorate_options(o; kwargs...) - return get_solver_return(solve(p, o)) + return get_solver_return(solve!(p, o)) end # # Solver functions # -function initialize_solver!(p::CostProblem, o::ParticleSwarmOptions) - j = argmin([get_cost(p, y) for y in o.x]) +function initialize_solver!( + mp::DefaultManoptProblem{TM,<:ManifoldCostObjective}, o::ParticleSwarmOptions +) where {TM} + j = argmin([get_cost(mp, p) for p in o.x]) return o.g = deepcopy(o.x[j]) end -function step_solver!(p::CostProblem, o::ParticleSwarmOptions, iter) +function step_solver!( + mp::DefaultManoptProblem{TM,<:ManifoldCostObjective}, o::ParticleSwarmOptions, ::Any +) where {TM} + M = get_manifold(mp) for i in 1:length(o.x) o.velocity[i] = o.inertia .* o.velocity[i] + o.cognitive_weight * rand(1) .* - inverse_retract(p.M, o.x[i], o.p[i], o.inverse_retraction_method) + + inverse_retract(M, o.x[i], o.p[i], o.inverse_retraction_method) + o.social_weight * rand(1) .* - inverse_retract(p.M, o.x[i], o.g, o.inverse_retraction_method) + inverse_retract(M, o.x[i], o.g, o.inverse_retraction_method) xOld = o.x[i] - o.x[i] = retract(p.M, o.x[i], o.velocity[i], o.retraction_method) + o.x[i] = retract(M, o.x[i], o.velocity[i], o.retraction_method) o.velocity[i] = vector_transport_to( - p.M, xOld, o.velocity[i], o.x[i], o.vector_transport_method + M, xOld, o.velocity[i], o.x[i], o.vector_transport_method ) - if get_cost(p, o.x[i]) < get_cost(p, o.p[i]) + if get_cost(mp, o.x[i]) < get_cost(mp, o.p[i]) o.p[i] = o.x[i] - if get_cost(p, o.p[i]) < get_cost(p, o.g) + if get_cost(mp, o.p[i]) < get_cost(mp, o.g) o.g = o.p[i] end end @@ -170,7 +175,7 @@ get_solver_result(o::ParticleSwarmOptions) = o.g # Change not only refers to different iterates (x=the whole population) # but also lives in the power manifold on M, so we have to adapt StopWhenChangeless # -function (c::StopWhenChangeLess)(P::CostProblem, O::ParticleSwarmOptions, i) +function (c::StopWhenChangeLess)(P::AbstractManoptProblem, O::ParticleSwarmOptions, i) if has_storage(c.storage, :Iterate) x_old = get_storage(c.storage, :Iterate) n = length(O.x) diff --git a/src/solvers/primal_dual_semismooth_Newton.jl b/src/solvers/primal_dual_semismooth_Newton.jl index e7bbd3af58..c01374a056 100644 --- a/src/solvers/primal_dual_semismooth_Newton.jl +++ b/src/solvers/primal_dual_semismooth_Newton.jl @@ -155,7 +155,7 @@ function primal_dual_semismooth_Newton!( vector_transport_method=vector_transport_method, ) o = decorate_options(o; kwargs...) - return get_solver_return(solve(p, o)) + return get_solver_return(solve!(p, o)) end function initialize_solver!( diff --git a/src/solvers/quasi_Newton.jl b/src/solvers/quasi_Newton.jl index 389ce3a867..508e489634 100644 --- a/src/solvers/quasi_Newton.jl +++ b/src/solvers/quasi_Newton.jl @@ -26,7 +26,7 @@ The ``k``th iteration consists of at 0 and strictly increasing at 0 for the cautious update. * `direction_update` – ([`InverseBFGS`](@ref)`()`) the update rule to use. * `evaluation` – ([`AllocatingEvaluation`](@ref)) specify whether the gradient works by - allocation (default) form `gradF(M, x)` or [`MutatingEvaluation`](@ref) in place, i.e. + allocation (default) form `gradF(M, x)` or [`InplaceEvaluation`](@ref) in place, i.e. is of the form `gradF!(M, X, x)`. * `initial_operator` – (`Matrix{Float64}(I,n,n)`) initial matrix to use die the approximation, where `n=manifold_dimension(M)`, see also `scale_initial_operator`. @@ -134,7 +134,7 @@ function quasi_Newton!( vector_transport_method=vector_transport_method, ) o = decorate_options(o; kwargs...) - return get_solver_return(solve(p, o)) + return get_solver_return(solve!(p, o)) end function initialize_solver!(p::GradientProblem, o::QuasiNewtonOptions) diff --git a/src/solvers/record_solver.jl b/src/solvers/record_solver.jl index 7053e0a098..d53de7e3a2 100644 --- a/src/solvers/record_solver.jl +++ b/src/solvers/record_solver.jl @@ -4,7 +4,7 @@ Initialize the solver to the optimization [`Problem`](@ref) by initializing the encapsulated `options` from within the [`RecordOptions`](@ref)` o`. """ -function initialize_solver!(p::Problem, o::RecordOptions) +function initialize_solver!(p::AbstractManoptProblem, o::RecordOptions) initialize_solver!(p, o.options) get(o.recordDictionary, :Start, RecordGroup())(p, get_options(o), 0) return o @@ -15,7 +15,7 @@ end Do one iteration step (the `iter`th) for [`Problem`](@ref)` p` by modifying the values in the [`Options`](@ref)` o.options` and record the result(s). """ -function step_solver!(p::Problem, o::RecordOptions, i) +function step_solver!(p::AbstractManoptProblem, o::RecordOptions, i) step_solver!(p, o.options, i) get(o.recordDictionary, :Iteration, RecordGroup())(p, get_options(o), i) return o @@ -28,7 +28,7 @@ determine whether the solver for [`Problem`](@ref) `p` and the [`RecordOptions`](@ref) `o` should stop at iteration `i`. If so, do a (final) record to `:All` and `:Stop`. """ -function stop_solver!(p::Problem, o::RecordOptions, i::Int) +function stop_solver!(p::AbstractManoptProblem, o::RecordOptions, i::Int) s = stop_solver!(p, o.options, i) s && get(o.recordDictionary, :Stop, RecordGroup())(p, get_options(o), i) return s diff --git a/src/solvers/solver.jl b/src/solvers/solver.jl index 1422e7da2b..6fa49ad12d 100644 --- a/src/solvers/solver.jl +++ b/src/solvers/solver.jl @@ -75,16 +75,16 @@ depending on the current [`Problem`](@ref) `p`, the current state of the solver stored in [`Options`](@ref) `o` and the current iterate `i` this function determines whether to stop the solver by calling the [`StoppingCriterion`](@ref). """ -stop_solver!(p::Problem, o::Options, i::Int) = o.stop(p, o, i) +stop_solver!(p::AbstractManoptProblem, o::Options, i::Int) = o.stop(p, o, i) """ - solve(p,o) + solve!(p,o) run the solver implemented for the [`Problem`](@ref)` p` and the [`Options`](@ref)` o` employing [`initialize_solver!`](@ref), [`step_solver!`](@ref), as well as the [`stop_solver!`](@ref) of the solver. """ -function solve(p::Problem, o::Options) +function solve!(p::AbstractManoptProblem, o::Options) iter::Integer = 0 initialize_solver!(p, o) while !stop_solver!(p, o, iter) @@ -94,6 +94,10 @@ function solve(p::Problem, o::Options) return o end -initialize_solver!(p::Problem, o::ReturnOptions) = initialize_solver!(p, o.options) -step_solver!(p::Problem, o::ReturnOptions, i) = step_solver!(p, o.options, i) -stop_solver!(p::Problem, o::ReturnOptions, i::Int) = stop_solver!(p, o.options, i) +function initialize_solver!(p::AbstractManoptProblem, o::ReturnOptions) + return initialize_solver!(p, o.options) +end +step_solver!(p::AbstractManoptProblem, o::ReturnOptions, i) = step_solver!(p, o.options, i) +function stop_solver!(p::AbstractManoptProblem, o::ReturnOptions, i::Int) + return stop_solver!(p, o.options, i) +end diff --git a/src/solvers/stochastic_gradient_descent.jl b/src/solvers/stochastic_gradient_descent.jl index e05d5f5cc2..59c21ce28f 100644 --- a/src/solvers/stochastic_gradient_descent.jl +++ b/src/solvers/stochastic_gradient_descent.jl @@ -13,7 +13,7 @@ perform a stochastic gradient descent # Optional * `cost` – (`missing`) you can provide a cost function for example to track the function value * `evaluation` – ([`AllocatingEvaluation`](@ref)) specify whether the gradient(s) works by - allocation (default) form `gradF(M, x)` or [`MutatingEvaluation`](@ref) in place, i.e. + allocation (default) form `gradF(M, x)` or [`InplaceEvaluation`](@ref) in place, i.e. is of the form `gradF!(M, X, x)` (elementwise). * `evaluation_order` – (`:Random`) – whether to use a randomly permuted sequence (`:FixedRandom`), a per @@ -79,7 +79,7 @@ function stochastic_gradient_descent!( retraction_method=retraction_method, ) o = decorate_options(o; kwargs...) - return get_solver_return(solve(p, o)) + return get_solver_return(solve!(p, o)) end function initialize_solver!( ::StochasticGradientProblem, o::StochasticGradientDescentOptions diff --git a/src/solvers/subgradient.jl b/src/solvers/subgradient.jl index f768bfe4e6..1c409475ad 100644 --- a/src/solvers/subgradient.jl +++ b/src/solvers/subgradient.jl @@ -19,7 +19,7 @@ not necessarily deterministic. # Optional * `evaluation` – ([`AllocatingEvaluation`](@ref)) specify whether the subgradient works by - allocation (default) form `∂F(M, y)` or [`MutatingEvaluation`](@ref) in place, i.e. is + allocation (default) form `∂F(M, y)` or [`InplaceEvaluation`](@ref) in place, i.e. is of the form `∂F!(M, X, x)`. * `stepsize` – ([`ConstantStepsize`](@ref)`(1.)`) specify a [`Stepsize`](@ref) * `retraction` – (`default_retraction_method(M)`) a `retraction(M,x,ξ)` to use. @@ -74,7 +74,7 @@ function subgradient_method!( retraction_method=retraction_method, ) o = decorate_options(o; kwargs...) - return get_solver_return(solve(p, o)) + return get_solver_return(solve!(p, o)) end function initialize_solver!(p::SubGradientProblem, o::SubGradientMethodOptions) o.x_optimal = o.x diff --git a/src/solvers/truncated_conjugate_gradient_descent.jl b/src/solvers/truncated_conjugate_gradient_descent.jl index 3eac37b2b5..76495b9362 100644 --- a/src/solvers/truncated_conjugate_gradient_descent.jl +++ b/src/solvers/truncated_conjugate_gradient_descent.jl @@ -35,7 +35,7 @@ see the reference: # Optional * `evaluation` – ([`AllocatingEvaluation`](@ref)) specify whether the gradient and hessian work by - allocation (default) or [`MutatingEvaluation`](@ref) in place + allocation (default) or [`InplaceEvaluation`](@ref) in place * `preconditioner` – a preconditioner for the hessian H * `θ` – (`1.0`) 1+θ is the superlinear convergence target rate. The algorithm will terminate early if the residual was reduced by a power of 1+theta. @@ -128,7 +128,7 @@ function truncated_conjugate_gradient_descent!( (project!)=project!, ) o = decorate_options(o; kwargs...) - return get_solver_return(solve(p, o)) + return get_solver_return(solve!(p, o)) end @deprecate truncated_conjugate_gradient_descent!(M, F, gradF, x, η, H, r; kwargs...) truncated_conjugate_gradient_descent!( M, F, gradF, x, η, H; trust_region_radius=r, kwargs... diff --git a/src/solvers/trust_regions.jl b/src/solvers/trust_regions.jl index 980b64e7e7..b777cf1b5d 100644 --- a/src/solvers/trust_regions.jl +++ b/src/solvers/trust_regions.jl @@ -24,7 +24,7 @@ For a description of the algorithm and more details see # Optional * `evaluation` – ([`AllocatingEvaluation`](@ref)) specify whether the gradient and hessian work by - allocation (default) or [`MutatingEvaluation`](@ref) in place + allocation (default) or [`InplaceEvaluation`](@ref) in place * `max_trust_region_radius` – the maximum trust-region radius * `preconditioner` – a preconditioner (a symmetric, positive definite operator that should approximate the inverse of the Hessian) @@ -131,7 +131,7 @@ function trust_regions!( (project!)=project!, ) o = decorate_options(o; kwargs...) - return get_solver_return(solve(p, o)) + return get_solver_return(solve!(p, o)) end function initialize_solver!(p::HessianProblem, o::TrustRegionsOptions) @@ -172,7 +172,7 @@ function step_solver!(p::HessianProblem, o::TrustRegionsOptions, iter) o.tcg_options.x = o.x o.tcg_options.η = o.η o.tcg_options.trust_region_radius = o.trust_region_radius - solve(p, o.tcg_options) + solve!(p, o.tcg_options) # o.η = o.tcg_options.η o.Hη = o.tcg_options.Hη diff --git a/test/plans/test_constrained_plan.jl b/test/plans/test_constrained_plan.jl index fcdd7b7c4a..36759ead8f 100644 --- a/test/plans/test_constrained_plan.jl +++ b/test/plans/test_constrained_plan.jl @@ -34,7 +34,7 @@ using Manopt, ManifoldsBase, Test gradH1!(M, X, p) = (X .= [0.0, 0.0, 2.0]) Pfa = ConstrainedProblem(M, F, gradF, G, gradG, H, gradH) Pfm = ConstrainedProblem( - M, F, gradF!, G, gradG!, H, gradH!; evaluation=MutatingEvaluation() + M, F, gradF!, G, gradG!, H, gradH!; evaluation=InplaceEvaluation() ) Pva = ConstrainedProblem(M, F, gradF, [G1, G2], [gradG1, gradG2], [H1], [gradH1]) Pvm = ConstrainedProblem( @@ -45,12 +45,12 @@ using Manopt, ManifoldsBase, Test [gradG1!, gradG2!], [H1], [gradH1!]; - evaluation=MutatingEvaluation(), + evaluation=InplaceEvaluation(), ) @test repr(Pfa) === "ConstrainedProblem{AllocatingEvaluation,FunctionConstraint}." - @test repr(Pfm) === "ConstrainedProblem{MutatingEvaluation,FunctionConstraint}." + @test repr(Pfm) === "ConstrainedProblem{InplaceEvaluation,FunctionConstraint}." @test repr(Pva) === "ConstrainedProblem{AllocatingEvaluation,VectorConstraint}." - @test repr(Pvm) === "ConstrainedProblem{MutatingEvaluation,VectorConstraint}." + @test repr(Pvm) === "ConstrainedProblem{InplaceEvaluation,VectorConstraint}." p = [1.0, 2.0, 3.0] c = [[0.0, -3.0], [5.0]] @@ -62,7 +62,7 @@ using Manopt, ManifoldsBase, Test # At least one constraint necessary @test_throws ErrorException ConstrainedProblem(M, F, gradF) @test_throws ErrorException ConstrainedProblem( - M, F, gradF!; evaluation=MutatingEvaluation() + M, F, gradF!; evaluation=InplaceEvaluation() ) p1f = ConstrainedProblem(M, F, gradF!; G=G, gradG=gradG) @test get_constraints(p1f, p) == [c[1], []] diff --git a/test/plans/test_hessian_plan.jl b/test/plans/test_hessian_plan.jl index 3be39083bb..15f6fafdf8 100644 --- a/test/plans/test_hessian_plan.jl +++ b/test/plans/test_hessian_plan.jl @@ -9,7 +9,7 @@ using Manopt, Manifolds, Test, Random HessF!(M, Y, p, X) = copyto!(M, Y, p, X) precon = (M, p, X) -> X P1 = HessianProblem(M, F, gradF, HessF, precon) - P2 = HessianProblem(M, F, gradF!, HessF!, precon; evaluation=MutatingEvaluation()) + P2 = HessianProblem(M, F, gradF!, HessF!, precon; evaluation=InplaceEvaluation()) p = zeros(2) X = ones(2) diff --git a/test/plans/test_higher_order_primal_dual_plan.jl b/test/plans/test_higher_order_primal_dual_plan.jl index d7bb83e6f2..0eff44444b 100644 --- a/test/plans/test_higher_order_primal_dual_plan.jl +++ b/test/plans/test_higher_order_primal_dual_plan.jl @@ -164,7 +164,7 @@ using Manopt, Manifolds, ManifoldsBase, Test Dprox_G_dual!, DΛ!, adjoint_DΛ!; - evaluation=MutatingEvaluation(), + evaluation=InplaceEvaluation(), ) x1 = get_differential_primal_prox(p1, 1.0, x0, X) x2 = get_differential_primal_prox(p2, 1.0, x0, X) diff --git a/test/plans/test_options.jl b/test/plans/test_options.jl index 611915ed19..ed23f6b2a0 100644 --- a/test/plans/test_options.jl +++ b/test/plans/test_options.jl @@ -2,7 +2,7 @@ using Manifolds, Manopt, Test, ManifoldsBase using Dates -struct TestProblem <: Problem{AllocatingEvaluation} end +struct TestProblem <: AbstractManoptProblem{AllocatingEvaluation} end struct TestOptions <: Options end @testset "generic Options test" begin diff --git a/test/plans/test_primal_dual_plan.jl b/test/plans/test_primal_dual_plan.jl index cd864cf4e9..3f60a6ced6 100644 --- a/test/plans/test_primal_dual_plan.jl +++ b/test/plans/test_primal_dual_plan.jl @@ -89,7 +89,7 @@ using Manopt, Manifolds, ManifoldsBase, Test adjoint_DΛ!; linearized_forward_operator=DΛ!, Λ=Λ!, - evaluation=MutatingEvaluation(), + evaluation=InplaceEvaluation(), ) x1 = get_primal_prox(p1, 1.0, x0) x2 = get_primal_prox(p2, 1.0, x0) diff --git a/test/plans/test_stopping_criteria.jl b/test/plans/test_stopping_criteria.jl index 4466bdbbb0..2f106d3050 100644 --- a/test/plans/test_stopping_criteria.jl +++ b/test/plans/test_stopping_criteria.jl @@ -1,6 +1,6 @@ using Manifolds, Manopt, Test, ManifoldsBase, Dates -struct TestProblem <: Problem{AllocatingEvaluation} end +struct TestProblem <: AbstractManoptProblem{AllocatingEvaluation} end struct TestOptions <: Options end @testset "StoppingCriteria" begin diff --git a/test/solvers/test_Frank_Wolfe.jl b/test/solvers/test_Frank_Wolfe.jl index 01f28b21cd..085d6f0a25 100644 --- a/test/solvers/test_Frank_Wolfe.jl +++ b/test/solvers/test_Frank_Wolfe.jl @@ -39,7 +39,7 @@ using ManifoldsBase, Manopt, Test, LinearAlgebra @testset "Two small Test runs" begin @testset "Testing with an Oracle" begin p2a = Frank_Wolfe_method( - M, f, grad_f!, p; subtask=oracle!, evaluation=MutatingEvaluation() + M, f, grad_f!, p; subtask=oracle!, evaluation=InplaceEvaluation() ) @test f(M, p2a) < f(M, p) p2b = Frank_Wolfe_method(M, f, grad_f, p; subtask=oracle) @@ -55,7 +55,7 @@ using ManifoldsBase, Manopt, Test, LinearAlgebra f, grad_f!, p; - evaluation=MutatingEvaluation(), + evaluation=InplaceEvaluation(), stopping_criterion=StopAfterIteration(1), ) @test is_point(M, p3) diff --git a/test/solvers/test_alternating_gradient.jl b/test/solvers/test_alternating_gradient.jl index e410dc28dc..963b54d8e2 100644 --- a/test/solvers/test_alternating_gradient.jl +++ b/test/solvers/test_alternating_gradient.jl @@ -23,9 +23,9 @@ using Manopt, Manifolds, Test @testset "Test gradient access" begin Pf = AlternatingGradientProblem(N, F, gradF) Pv = AlternatingGradientProblem(N, F, [gradF1, gradF2]) - Pf! = AlternatingGradientProblem(N, F, gradF!; evaluation=MutatingEvaluation()) + Pf! = AlternatingGradientProblem(N, F, gradF!; evaluation=InplaceEvaluation()) Pv! = AlternatingGradientProblem( - N, F, [gradF1!, gradF2!]; evaluation=MutatingEvaluation() + N, F, [gradF1!, gradF2!]; evaluation=InplaceEvaluation() ) for P in [Pf, Pv, Pf!, Pv!] Y = zero_vector(N, x) @@ -49,15 +49,10 @@ using Manopt, Manifolds, Test y3 = allocate(x) copyto!(N, y3, x) y = alternating_gradient_descent( - N, F, [gradF1!, gradF2!], x; order_type=:Linear, evaluation=MutatingEvaluation() + N, F, [gradF1!, gradF2!], x; order_type=:Linear, evaluation=InplaceEvaluation() ) alternating_gradient_descent!( - N, - F, - [gradF1!, gradF2!], - y2; - order_type=:Linear, - evaluation=MutatingEvaluation(), + N, F, [gradF1!, gradF2!], y2; order_type=:Linear, evaluation=InplaceEvaluation() ) @test isapprox(N, y, y2) o = alternating_gradient_descent!( @@ -65,7 +60,7 @@ using Manopt, Manifolds, Test F, [gradF1!, gradF2!], y3; - evaluation=MutatingEvaluation(), + evaluation=InplaceEvaluation(), order_type=:Linear, return_options=true, ) diff --git a/test/solvers/test_cyclic_proximal_point.jl b/test/solvers/test_cyclic_proximal_point.jl index 91ef9f6a86..51cc6aa9b5 100644 --- a/test/solvers/test_cyclic_proximal_point.jl +++ b/test/solvers/test_cyclic_proximal_point.jl @@ -59,7 +59,7 @@ using Manifolds, Manopt, Test, Dates f; λ=i -> π / (2 * i), stopping_criterion=StopAfterIteration(100), - evaluation=MutatingEvaluation(), + evaluation=InplaceEvaluation(), ) @test isapprox(N, s1, s2) end @@ -81,7 +81,7 @@ using Manifolds, Manopt, Test, Dates g = deepcopy(f) get_proximal_map!(p1, g, 1.0, g, i) @test isapprox(N, g, get_proximal_map(p1, 1.0, f, i)) - p2 = ProximalProblem(N, F, proxes!; evaluation=MutatingEvaluation()) + p2 = ProximalProblem(N, F, proxes!; evaluation=InplaceEvaluation()) g = deepcopy(f) get_proximal_map!(p2, g, 1.0, g, i) @test isapprox(N, g, get_proximal_map(p2, 1.0, f, i)) diff --git a/test/solvers/test_stochastic_gradient_descent.jl b/test/solvers/test_stochastic_gradient_descent.jl index 59a9de5547..988c9d425c 100644 --- a/test/solvers/test_stochastic_gradient_descent.jl +++ b/test/solvers/test_stochastic_gradient_descent.jl @@ -27,9 +27,9 @@ using Manopt, Manifolds, Test @testset "Constructors" begin p1 = StochasticGradientProblem(M, sgradF1) - p1e = StochasticGradientProblem(M, sgradF1!; evaluation=MutatingEvaluation()) + p1e = StochasticGradientProblem(M, sgradF1!; evaluation=InplaceEvaluation()) p2 = StochasticGradientProblem(M, sgradF2) - p2m = StochasticGradientProblem(M, sgradF2!; evaluation=MutatingEvaluation()) + p2m = StochasticGradientProblem(M, sgradF2!; evaluation=InplaceEvaluation()) @test get_gradient(p1, 1, p) == zeros(3) @test get_gradient(p2, 1, p) == zeros(3) for pr in [p1, p2, p2m] diff --git a/test/solvers/test_subgradient_method.jl b/test/solvers/test_subgradient_method.jl index 519931b890..fdfb691dcc 100644 --- a/test/solvers/test_subgradient_method.jl +++ b/test/solvers/test_subgradient_method.jl @@ -19,7 +19,7 @@ using Manopt, ManifoldsBase, Manifolds, Test Y = get_subgradient(p, x) get_subgradient!(p, X, x) @test isapprox(M, x, X, Y) - oR = solve(p, o) + oR = solve!(p, o) xHat = get_solver_result(oR) @test get_initial_stepsize(p, o) == 1.0 @test get_stepsize(p, o, 1) == 1.0 @@ -43,12 +43,12 @@ using Manopt, ManifoldsBase, Manifolds, Test X .*= -2 / max(10 * eps(Float64), d) return X end - p = SubGradientProblem(M, f, ∂f!; evaluation=MutatingEvaluation()) + p = SubGradientProblem(M, f, ∂f!; evaluation=InplaceEvaluation()) X = zero_vector(M, x) Y = get_subgradient(p, x) get_subgradient!(p, X, x) @test isapprox(M, x, X, Y) - oR = solve(p, o) + oR = solve!(p, o) xHat = get_solver_result(oR) # Check Fallbacks of Problen @test get_cost(p, x) == 0.0 @@ -56,7 +56,7 @@ using Manopt, ManifoldsBase, Manifolds, Test @test_throws MethodError get_gradient(p, o.x) @test_throws MethodError get_proximal_map(p, 1.0, o.x, 1) o2 = subgradient_method( - M, f, ∂f!, copy(x0); evaluation=MutatingEvaluation(), return_options=true + M, f, ∂f!, copy(x0); evaluation=InplaceEvaluation(), return_options=true ) xhat2 = get_solver_result(o2) @test f(M, xhat2) <= f(M, x0) diff --git a/test/solvers/test_trust_regions.jl b/test/solvers/test_trust_regions.jl index a471c5858b..6616b8cffc 100644 --- a/test/solvers/test_trust_regions.jl +++ b/test/solvers/test_trust_regions.jl @@ -96,7 +96,7 @@ include("trust_region_model.jl") h, x3; max_trust_region_radius=8.0, - evaluation=MutatingEvaluation(), + evaluation=InplaceEvaluation(), debug=[:Stop], ) x4 = deepcopy(x) @@ -107,7 +107,7 @@ include("trust_region_model.jl") h, x4; max_trust_region_radius=8.0, - evaluation=MutatingEvaluation(), + evaluation=InplaceEvaluation(), return_options=true, ) @test isapprox(M, x3, x4) @@ -122,12 +122,12 @@ include("trust_region_model.jl") g; steplength=2^(-9), vector_transport_method=ProjectionTransport(), - evaluation=MutatingEvaluation(), + evaluation=InplaceEvaluation(), ), XaH; stopping_criterion=StopAfterIteration(2000) | StopWhenGradientNormLess(1e-6), max_trust_region_radius=8.0, - evaluation=MutatingEvaluation(), + evaluation=InplaceEvaluation(), ) @test cost(M, XaH) ≈ cost(M, x3) end diff --git a/tutorials/Benchmark.jl b/tutorials/Benchmark.jl index f40e5daf35..f780d39ca8 100644 --- a/tutorials/Benchmark.jl +++ b/tutorials/Benchmark.jl @@ -101,10 +101,10 @@ begin gradF2! = grad!(data, similar(data[1])) m2 = deepcopy(x0) gradient_descent!( - M, F, gradF2!, m2; evaluation=MutatingEvaluation(), stopping_criterion=sc + M, F, gradF2!, m2; evaluation=InplaceEvaluation(), stopping_criterion=sc ) @benchmark gradient_descent!( - $M, $F, $gradF2!, m2; evaluation=$(MutatingEvaluation()), stopping_criterion=$sc + $M, $F, $gradF2!, m2; evaluation=$(InplaceEvaluation()), stopping_criterion=$sc ) setup = (m2 = deepcopy($x0)) end diff --git a/tutorials/ConstrainedOptimization.jl b/tutorials/ConstrainedOptimization.jl index 79ff82d54a..69a63efa08 100644 --- a/tutorials/ConstrainedOptimization.jl +++ b/tutorials/ConstrainedOptimization.jl @@ -195,14 +195,14 @@ grad_g2! = [ # ╔═╡ ce8f1156-a350-4fde-bd39-b08a16b2821d with_terminal() do @time global v2 = augmented_Lagrangian_method( - M, f, grad_f!, x0; G=g2, gradG=grad_g2!, evaluation=MutatingEvaluation(), + M, f, grad_f!, x0; G=g2, gradG=grad_g2!, evaluation=InplaceEvaluation(), debug=[:Iteration, :Cost, :Stop, " | ", :Change, 50, "\n"], ); end # ╔═╡ f6617b0f-3688-4429-974b-990e0279cb38 md""" -As a technical remark: Note that (by default) the change to [`MutatingEvaluation`](https://manoptjl.org/stable/plans/problem/#Manopt.MutatingEvaluation)s affects both the constrained solver as well as the inner solver of the subproblem in each iteration. +As a technical remark: Note that (by default) the change to [`InplaceEvaluation`](https://manoptjl.org/stable/plans/problem/#Manopt.InplaceEvaluation)s affects both the constrained solver as well as the inner solver of the subproblem in each iteration. """ # ╔═╡ f4b3f8c4-8cf8-493e-a6ac-ea9673609a9c @@ -227,7 +227,7 @@ and [`LinearQuadraticHuber`](https://manoptjl.org/stable/solvers/exact_penalty_m # ╔═╡ e9847e43-d4ef-4a90-a51d-ce527787d467 with_terminal() do @time global v3 = exact_penalty_method( - M, f, grad_f!, x0; G=g2, gradG=grad_g2!, evaluation=MutatingEvaluation(), + M, f, grad_f!, x0; G=g2, gradG=grad_g2!, evaluation=InplaceEvaluation(), debug=[:Iteration, :Cost, :Stop, " | ", :Change, 50, "\n"], ); end @@ -249,7 +249,7 @@ The second smoothing technique is often beneficial, when we have a lot of constr # ╔═╡ fac5894c-250e-447d-aab8-1bfab7aae78c with_terminal() do @time global v4 = exact_penalty_method( - M, f, grad_f!, x0; G=g2, gradG=grad_g2!, evaluation=MutatingEvaluation(), + M, f, grad_f!, x0; G=g2, gradG=grad_g2!, evaluation=InplaceEvaluation(), smoothing=LinearQuadraticHuber(), debug=[:Iteration, :Cost, :Stop, " | ", :Change, 50, "\n"], ); @@ -278,7 +278,7 @@ Note that this is much faster, since every iteration of the algorithms above doe # ╔═╡ 70fd7a56-ebce-43b9-b75e-f47c7a277a07 with_terminal() do @time global w1 = quasi_Newton( - M, f, grad_f!, x0; evaluation=MutatingEvaluation() + M, f, grad_f!, x0; evaluation=InplaceEvaluation() ); end diff --git a/tutorials/GeodesicRegression.jl b/tutorials/GeodesicRegression.jl index 67fdb7ee16..73e8887198 100644 --- a/tutorials/GeodesicRegression.jl +++ b/tutorials/GeodesicRegression.jl @@ -223,7 +223,7 @@ with_terminal() do RegressionCost(data, t), RegressionGradient!(data, t), x0; - evaluation=MutatingEvaluation(), + evaluation=InplaceEvaluation(), stepsize=ArmijoLinesearch( M; initial_stepsize=1.0, @@ -319,7 +319,7 @@ with_terminal() do RegressionCost(data2, t2), RegressionGradient!(data2, t2), x1; - evaluation=MutatingEvaluation(), + evaluation=InplaceEvaluation(), stepsize=ArmijoLinesearch( M; initial_stepsize=1.0, @@ -505,7 +505,7 @@ with_terminal() do F3, gradF3_vector, x2; - evaluation=MutatingEvaluation(), + evaluation=InplaceEvaluation(), debug=[:Iteration, " | ", :Cost, "\n", :Stop, 50], stepsize=ArmijoLinesearch(1.0, ExponentialRetraction(), 0.999, 0.066, 1e-11), inner_iterations=1, diff --git a/tutorials/HowToRecord.jl b/tutorials/HowToRecord.jl index c8f36b576f..23b0307923 100644 --- a/tutorials/HowToRecord.jl +++ b/tutorials/HowToRecord.jl @@ -57,7 +57,7 @@ R = gradient_descent(M, F, gradF, data[1]; record=:Cost, return_options=true) # ╔═╡ 7552c7ba-6d45-4ed9-856c-b00be28a84a0 md""" -From the returned options, we see that the `Options` are encapsulated (decorated) with +From the returned options, we see that the `Options` are encapsulated (decorated) with `RecordOptions`. You can attach different recorders to some operations (`:Start`. `:Stop`, `:Iteration` at time of @@ -163,7 +163,7 @@ r = RecordOptions(o, Dict(:Iteration => rI, :Stop => sI)) md"""We now call the solver""" # ╔═╡ 45ee91a6-0377-46e8-bdac-faea18b120e0 -res = solve(p, r) +res = solve!(p, r) # ╔═╡ 1c2c65f0-ccbd-4566-a8b3-d1449b683707 md""" @@ -202,7 +202,7 @@ end # ╔═╡ 05284884-e05c-4714-bf53-2da071c664f7 md""" -and we define the following RecordAction, which is a functor, i.e. a struct that is also a function. The function we have to implement is similar to a single solver step in signature, since it might get called every iteration: +and we define the following RecordAction, which is a functor, i.e. a struct that is also a function. The function we have to implement is similar to a single solver step in signature, since it might get called every iteration: """ # ╔═╡ e0fe662d-edc5-4d1f-9fd6-987f37098cc4 @@ -211,7 +211,7 @@ begin recorded_values::Vector{Int} RecordCount() = new(Vector{Int}()) end - function (r::RecordCount)(p::Problem, ::Options, i) + function (r::RecordCount)(p::AbstractManoptProblem, ::Options, i) if i > 0 push!(r.recorded_values, p.cost.count) elseif i < 0 # reset if negative diff --git a/tutorials/Optimize!.jl b/tutorials/Optimize!.jl index faa3d957b7..b0aa13e023 100644 --- a/tutorials/Optimize!.jl +++ b/tutorials/Optimize!.jl @@ -34,7 +34,7 @@ __plan__. A __plan__ uniquely determines the algorithm to use and provides all necessary information to run it. You can either follow the code here and look at the preprinted output, or, if you want to experiment -things for yourself, you can directly access the Pluto notebooks related to the tutorials by going +things for yourself, you can directly access the Pluto notebooks related to the tutorials by going to "/your`_Julia_`installation_folder/packages/Manopt/tutorials/". """ @@ -73,7 +73,7 @@ both the stopping criterion ([`StopAfterIteration`](@ref)`(100)`) as well as the stepsize ([`ConstantStepsize`](@ref)`(1.)`) are quite conservative, but are chosen to be as simple as possible. -With these two at hand, running the algorithm just requires to call `x_opt = solve(p,o)`. +With these two at hand, running the algorithm just requires to call `x_opt = solve!(p,o)`. In the following two examples we will see, how to use a higher level interface that allows to more easily activate, for example, a debug output or record values during the iterations.