Add a KrylovSolver for each method

[amontoison]

#264
@geoffroyleconte After #269 I will rebase this PR and we could easily add in-place method for all krylov solvers 🚀

For all solver, the syntax is the same and we will be able to do

KnameSolver(A, b)
kname!(solver, A, b)

instead of

kname(A, b)

and reuse the storage, which is quite user-friendly.

[codecov]

Codecov Report

Merging #271 (38e56c5) into master (ac55536) will increase coverage by 0.49%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #271      +/-   ##
==========================================
+ Coverage   95.73%   96.22%   +0.49%     
==========================================
  Files          33       33              
  Lines        3516     3974     +458     
==========================================
+ Hits         3366     3824     +458     
  Misses        150      150              

Impacted Files	Coverage Δ
src/variants.jl	75.00% <ø> (-5.77%)	⬇️
src/bicgstab.jl	100.00% <100.00%> (ø)
src/bilq.jl	100.00% <100.00%> (ø)
src/bilqr.jl	100.00% <100.00%> (ø)
src/cg.jl	100.00% <100.00%> (ø)
src/cg_lanczos.jl	98.06% <100.00%> (+0.23%)	⬆️
src/cgls.jl	100.00% <100.00%> (ø)
src/cgne.jl	100.00% <100.00%> (ø)
src/cgs.jl	100.00% <100.00%> (ø)
src/cr.jl	64.19% <100.00%> (+1.53%)	⬆️
... and 23 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ac55536...38e56c5. Read the comment docs.

[amontoison]

Some benchmarks are needed to see if struct is not more relevant than mutable struct.

[dpo]

Before launching ahead, we should coordinate with @abelsiqueira and JuliaSmoothOptimizers/SolverCore.jl#6, where he’s proposing an AbstractSolver struct that we should be able to import here as a base solver.

[github-actions]

Package name	latest	stable
CaNNOLeS.jl
DCI.jl
JSOSolvers.jl
Percival.jl

[amontoison]

@abelsiqueira For the solver revamp we only need to update the KrylovSolvers : https://github.com/JuliaSmoothOptimizers/Krylov.jl/blob/a4c91b8c7f3f37e5de9eac9e4b7edf725e9c687c/src/krylov_solvers.jl

[github-actions]

Package name	latest	stable
CaNNOLeS.jl
DCI.jl
JSOSolvers.jl
Percival.jl

[github-actions]

Package name	latest	stable
CaNNOLeS.jl
DCI.jl
JSOSolvers.jl
Percival.jl

[github-actions]

Package name	latest	stable
CaNNOLeS.jl
DCI.jl
JSOSolvers.jl
Percival.jl

[amontoison]

@tmigot, @dpo, @geoffroyleconte
The PR is ready to be review, don't be afraid of the number of modifications.

Note : I defined a new function called ktypeof that returned the type of a vector v if it's dense or a dense storage type if it's sparse (SparseVector / CuSparseVector) or a SubArray (#290). With this new function we don't need to convert the rhs b (or c) anymore and it removes some variants in src/variants.jl.

[geoffroyleconte]

Thank you for all these changes! Maybe you could add the stats to the Solver type in order to remove more allocations, like it was done with MINRES?

[amontoison]

Thank you for all these changes! Maybe you could add the stats to the Solver type in order to remove more allocations, like it was done with MINRES?

I would like to use a generic KrylovStats for all Krylov methods before and avoid to modify all files 2 times.

[github-actions]

Package name	latest	stable
CaNNOLeS.jl
DCI.jl
JSOSolvers.jl
Percival.jl

[github-actions]

Package name	latest	stable
CaNNOLeS.jl
DCI.jl
JSOSolvers.jl
Percival.jl

[github-actions]

Package name	latest	stable
CaNNOLeS.jl
DCI.jl
JSOSolvers.jl
Percival.jl