Add backtracking line search #2
Conversation
src/Backtracking.jl
Outdated
| end | ||
|
|
||
| # Determine step size by backtracking line search | ||
| function Backtracking(backtrack::Backtracking) |
There was a problem hiding this comment.
| function Backtracking(backtrack::Backtracking) | |
| function Backtracking(;x, g, α, p , f) |
There was a problem hiding this comment.
Since this should be a constructor for the method
There was a problem hiding this comment.
I feel confused about this comment. In the function Backtracking, there are five arguments including f. Here f is a function. A function cannot be an argument of a function. That is where I feel confused.
src/Backtracking.jl
Outdated
| iteration = 0 | ||
|
|
||
| # Backtracking line search | ||
| while ϕ_α > ϕ_0 + c1 * α * g * p |
There was a problem hiding this comment.
Move this oput to a separate function that takes as argument the function and a Backtracking object
src/Backtracking.jl
Outdated
|
|
||
| # Ensure termination | ||
| if iteration > iterations | ||
| throw(LineSearchException("Linesearch failed to converge, reached maximum iterations $(iterations).", α)) |
There was a problem hiding this comment.
Don't throw an error, return NaN
There was a problem hiding this comment.
Take a look at #3 for some more context for the review comments
|
I rewrote the backtracking line search function as you suggest. There is one minor point that is unsatisfactory, and I have not changed that. That is hyperparameters are not put into |
I add a basic backtracking line search method and add a single test (quadratic function) to validate the correctness of the implementation.