Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Mutating arrays is not supported when trying to calculate the derivative of a function #1425

Closed
jarroyoe opened this issue May 24, 2023 · 4 comments
Closed

Mutating arrays is not supported when trying to calculate the derivative of a function #1425

jarroyoe opened this issue May 24, 2023 · 4 comments

Comments

@jarroyoe
Copy link

jarroyoe commented May 24, 2023 •
edited
Loading

I have a problem where I am trying to optimize the derivative of a function. The derivative of the function is calculated using the jacobian function in Zygote. When I try to run this MWE:

using Zygote, Optimization, OptimizationOptimisers

f(p,x) = [x*p[1];x*p[2]+p[3]]

data = rand(4,10)

function loss(p)
    dfdt(t) = jacobian(x->f(p,x),t)[1]
    sum(abs2,sum(dfdt.(data[1,:])))
end

pinit = rand(3)
adtype = Optimization.AutoZygote()
optf = Optimization.OptimizationFunction((x, p) -> loss(x), adtype)

optprob = Optimization.OptimizationProblem(optf,pinit)

optimizedParameters = Optimization.solve(optprob,
                                ADAM(0.01),
                                maxiters = 10)

Yields the following error

ERROR: Mutating arrays is not supported -- called copyto!(SubArray{Float64, 1, Matrix{Float64}, Tuple{Int64, Base.Slice{Base.OneTo{Int64}}}, true}, ...)
This error occurs when you ask Zygote to differentiate operations that change
the elements of arrays in place (e.g. setting values with x .= ...)

Possible fixes:
- avoid mutating operations (preferred)
- or read the documentation and solutions for this error
  https://fluxml.ai/Zygote.jl/latest/limitations

Stacktrace:
  [1] _throw_mutation_error(f::Function, args::SubArray{Float64, 1, Matrix{Float64}, Tuple{Int64, Base.Slice{Base.OneTo{Int64}}}, true})
    @ Zygote ~/.julia/packages/Zygote/SuKWp/src/lib/array.jl:88
  [2] (::Zygote.var"#554#555"{SubArray{Float64, 1, Matrix{Float64}, Tuple{Int64, Base.Slice{Base.OneTo{Int64}}}, true}})(#unused#::Nothing)
    @ Zygote ~/.julia/packages/Zygote/SuKWp/src/lib/array.jl:103
  [3] (::Zygote.var"#2651#back#556"{Zygote.var"#554#555"{SubArray{Float64, 1, Matrix{Float64}, Tuple{Int64, Base.Slice{Base.OneTo{Int64}}}, true}}})(Δ::Nothing)
    @ Zygote ~/.julia/packages/ZygoteRules/OgCVT/src/adjoint.jl:71
  [4] Pullback
    @ ~/.julia/packages/Zygote/SuKWp/src/lib/grad.jl:172 [inlined]
  [5] (::Zygote.Pullback{Tuple{typeof(Zygote._gradcopy!), SubArray{Float64, 1, Matrix{Float64}, Tuple{Int64, Base.Slice{Base.OneTo{Int64}}}, true}, Float64}, Tuple{Zygote.var"#2651#back#556"{Zygote.var"#554#555"{SubArray{Float64, 1, Matrix{Float64}, Tuple{Int64, Base.Slice{Base.OneTo{Int64}}}, true}}}}})(Δ::Nothing)
    @ Zygote ~/.julia/packages/Zygote/SuKWp/src/compiler/interface2.jl:0
  [6] Pullback
    @ ~/.julia/packages/Zygote/SuKWp/src/lib/grad.jl:153 [inlined]
  [7] (::Zygote.Pullback{Tuple{typeof(withjacobian), var"#15#17"{Vector{Float64}}, Float64}, Any})(Δ::NamedTuple{(:val, :grad), Tuple{Nothing, Tuple{Vector{Float64}}}})
    @ Zygote ~/.julia/packages/Zygote/SuKWp/src/compiler/interface2.jl:0
  [8] #287
    @ ~/.julia/packages/Zygote/SuKWp/src/lib/lib.jl:206 [inlined]
  [9] #2156#back
    @ ~/.julia/packages/ZygoteRules/OgCVT/src/adjoint.jl:71 [inlined]
 [10] Pullback
    @ ~/.julia/packages/Zygote/SuKWp/src/lib/grad.jl:128 [inlined]
 [11] (::Zygote.Pullback{Tuple{typeof(jacobian), var"#15#17"{Vector{Float64}}, Float64}, Tuple{Zygote.var"#2156#back#289"{Zygote.var"#287#288"{Tuple{Tuple{Nothing}, Tuple{Nothing}}, Zygote.Pullback{Tuple{typeof(withjacobian), var"#15#17"{Vector{Float64}}, Float64}, Any}}}, Zygote.var"#2000#back#200"{typeof(identity)}, Zygote.var"#2167#back#299"{Zygote.var"#back#298"{:grad, Zygote.Context{false}, NamedTuple{(:val, :grad), Tuple{Vector{Float64}, Tuple{Vector{Float64}}}}, Tuple{Vector{Float64}}}}}})(Δ::Tuple{Vector{Float64}})
    @ Zygote ~/.julia/packages/Zygote/SuKWp/src/compiler/interface2.jl:0
 [12] Pullback
    @ ~/Documentos/2023/Phytoplankton_PINN/test.jl:8 [inlined]
 [13] (::Zygote.Pullback{Tuple{var"#dfdt#16"{Vector{Float64}}, Float64}, Tuple{Zygote.var"#2167#back#299"{Zygote.var"#back#298"{:p, Zygote.Context{false}, var"#dfdt#16"{Vector{Float64}}, Vector{Float64}}}, Zygote.var"#2197#back#309"{Zygote.Jnew{var"#15#17"{Vector{Float64}}, Nothing, false}}, Zygote.var"#2167#back#299"{Zygote.var"#back#298"{:p, Zygote.Context{false}, var"#dfdt#16"{Vector{Float64}}, Vector{Float64}}}, Zygote.var"#2016#back#209"{Zygote.var"#back#207"{1, 1, Zygote.Context{false}, Vector{Float64}}}, Zygote.Pullback{Tuple{typeof(jacobian), var"#15#17"{Vector{Float64}}, Float64}, Tuple{Zygote.var"#2156#back#289"{Zygote.var"#287#288"{Tuple{Tuple{Nothing}, Tuple{Nothing}}, Zygote.Pullback{Tuple{typeof(withjacobian), var"#15#17"{Vector{Float64}}, Float64}, Any}}}, Zygote.var"#2000#back#200"{typeof(identity)}, Zygote.var"#2167#back#299"{Zygote.var"#back#298"{:grad, Zygote.Context{false}, NamedTuple{(:val, :grad), Tuple{Vector{Float64}, Tuple{Vector{Float64}}}}, Tuple{Vector{Float64}}}}}}}})(Δ::Vector{Float64})
    @ Zygote ~/.julia/packages/Zygote/SuKWp/src/compiler/interface2.jl:0
 [14] #1369
    @ ~/.julia/packages/Zygote/SuKWp/src/lib/broadcast.jl:218 [inlined]
 [15] #4
    @ ./generator.jl:36 [inlined]
 [16] iterate
    @ ./generator.jl:47 [inlined]
 [17] collect(itr::Base.Generator{Base.Iterators.Zip{Tuple{Vector{Tuple{Vector{Float64}, Zygote.Pullback{Tuple{var"#dfdt#16"{Vector{Float64}}, Float64}, Tuple{Zygote.var"#2167#back#299"{Zygote.var"#back#298"{:p, Zygote.Context{false}, var"#dfdt#16"{Vector{Float64}}, Vector{Float64}}}, Zygote.var"#2197#back#309"{Zygote.Jnew{var"#15#17"{Vector{Float64}}, Nothing, false}}, Zygote.var"#2167#back#299"{Zygote.var"#back#298"{:p, Zygote.Context{false}, var"#dfdt#16"{Vector{Float64}}, Vector{Float64}}}, Zygote.var"#2016#back#209"{Zygote.var"#back#207"{1, 1, Zygote.Context{false}, Vector{Float64}}}, Zygote.Pullback{Tuple{typeof(jacobian), var"#15#17"{Vector{Float64}}, Float64}, Tuple{Zygote.var"#2156#back#289"{Zygote.var"#287#288"{Tuple{Tuple{Nothing}, Tuple{Nothing}}, Zygote.Pullback{Tuple{typeof(withjacobian), var"#15#17"{Vector{Float64}}, Float64}, Any}}}, Zygote.var"#2000#back#200"{typeof(identity)}, Zygote.var"#2167#back#299"{Zygote.var"#back#298"{:grad, Zygote.Context{false}, NamedTuple{(:val, :grad), Tuple{Vector{Float64}, Tuple{Vector{Float64}}}}, Tuple{Vector{Float64}}}}}}}}}}, FillArrays.Fill{Vector{Float64}, 1, Tuple{Base.OneTo{Int64}}}}}, Base.var"#4#5"{Zygote.var"#1369#1375"}})
    @ Base ./array.jl:782
 [18] map
    @ ./abstractarray.jl:3383 [inlined]
 [19] (::Zygote.var"#∇broadcasted#1374"{Tuple{Vector{Float64}}, Vector{Tuple{Vector{Float64}, Zygote.Pullback{Tuple{var"#dfdt#16"{Vector{Float64}}, Float64}, Tuple{Zygote.var"#2167#back#299"{Zygote.var"#back#298"{:p, Zygote.Context{false}, var"#dfdt#16"{Vector{Float64}}, Vector{Float64}}}, Zygote.var"#2197#back#309"{Zygote.Jnew{var"#15#17"{Vector{Float64}}, Nothing, false}}, Zygote.var"#2167#back#299"{Zygote.var"#back#298"{:p, Zygote.Context{false}, var"#dfdt#16"{Vector{Float64}}, Vector{Float64}}}, Zygote.var"#2016#back#209"{Zygote.var"#back#207"{1, 1, Zygote.Context{false}, Vector{Float64}}}, Zygote.Pullback{Tuple{typeof(jacobian), var"#15#17"{Vector{Float64}}, Float64}, Tuple{Zygote.var"#2156#back#289"{Zygote.var"#287#288"{Tuple{Tuple{Nothing}, Tuple{Nothing}}, Zygote.Pullback{Tuple{typeof(withjacobian), var"#15#17"{Vector{Float64}}, Float64}, Any}}}, Zygote.var"#2000#back#200"{typeof(identity)}, Zygote.var"#2167#back#299"{Zygote.var"#back#298"{:grad, Zygote.Context{false}, NamedTuple{(:val, :grad), Tuple{Vector{Float64}, Tuple{Vector{Float64}}}}, Tuple{Vector{Float64}}}}}}}}}}, Val{2}})(ȳ::FillArrays.Fill{Vector{Float64}, 1, Tuple{Base.OneTo{Int64}}})
    @ Zygote ~/.julia/packages/Zygote/SuKWp/src/lib/broadcast.jl:218
 [20] #4121#back
    @ ~/.julia/packages/ZygoteRules/OgCVT/src/adjoint.jl:71 [inlined]
 [21] #287
    @ ~/.julia/packages/Zygote/SuKWp/src/lib/lib.jl:206 [inlined]
 [22] #2156#back
    @ ~/.julia/packages/ZygoteRules/OgCVT/src/adjoint.jl:71 [inlined]
 [23] Pullback
    @ ./broadcast.jl:1311 [inlined]
 [24] Pullback
    @ ~/Documentos/2023/Phytoplankton_PINN/test.jl:9 [inlined]
 [25] (::Zygote.Pullback{Tuple{typeof(loss), Vector{Float64}}, Tuple{Zygote.var"#3009#back#777"{Zygote.var"#771#775"{Vector{Vector{Float64}}}}, Zygote.Pullback{Tuple{typeof(Base.Broadcast.materialize), Vector{Vector{Float64}}}, Tuple{}}, Zygote.var"#2197#back#309"{Zygote.Jnew{var"#dfdt#16"{Vector{Float64}}, Nothing, false}}, Zygote.ZBack{ChainRules.var"#sum_abs2_pullback#1681"{Vector{Float64}}}, Zygote.var"#2592#back#528"{Zygote.var"#538#540"{2, Float64, Matrix{Float64}, Tuple{Int64, Colon}}}, Zygote.Pullback{Tuple{typeof(Base.Broadcast.broadcasted), var"#dfdt#16"{Vector{Float64}}, Vector{Float64}}, Tuple{Zygote.var"#2863#back#683"{Zygote.var"#map_back#677"{typeof(Base.Broadcast.broadcastable), 1, Tuple{Tuple{}}, Tuple{Val{0}}, Tuple{}}}, Zygote.var"#2156#back#289"{Zygote.var"#287#288"{Tuple{Tuple{Nothing, Nothing, Nothing}, Tuple{}}, Zygote.var"#4121#back#1363"{Zygote.var"#∇broadcasted#1374"{Tuple{Vector{Float64}}, Vector{Tuple{Vector{Float64}, Zygote.Pullback{Tuple{var"#dfdt#16"{Vector{Float64}}, Float64}, Tuple{Zygote.var"#2167#back#299"{Zygote.var"#back#298"{:p, Zygote.Context{false}, var"#dfdt#16"{Vector{Float64}}, Vector{Float64}}}, Zygote.var"#2197#back#309"{Zygote.Jnew{var"#15#17"{Vector{Float64}}, Nothing, false}}, Zygote.var"#2167#back#299"{Zygote.var"#back#298"{:p, Zygote.Context{false}, var"#dfdt#16"{Vector{Float64}}, Vector{Float64}}}, Zygote.var"#2016#back#209"{Zygote.var"#back#207"{1, 1, Zygote.Context{false}, Vector{Float64}}}, Zygote.Pullback{Tuple{typeof(jacobian), var"#15#17"{Vector{Float64}}, Float64}, Tuple{Zygote.var"#2156#back#289"{Zygote.var"#287#288"{Tuple{Tuple{Nothing}, Tuple{Nothing}}, Zygote.Pullback{Tuple{typeof(withjacobian), var"#15#17"{Vector{Float64}}, Float64}, Any}}}, Zygote.var"#2000#back#200"{typeof(identity)}, Zygote.var"#2167#back#299"{Zygote.var"#back#298"{:grad, Zygote.Context{false}, NamedTuple{(:val, :grad), Tuple{Vector{Float64}, Tuple{Vector{Float64}}}}, Tuple{Vector{Float64}}}}}}}}}}, Val{2}}}}}, Zygote.var"#2000#back#200"{typeof(identity)}, Zygote.Pullback{Tuple{typeof(Base.Broadcast.broadcastable), Vector{Float64}}, Tuple{}}, Zygote.var"#2000#back#200"{typeof(identity)}, Zygote.var"#2156#back#289"{Zygote.var"#287#288"{Tuple{Tuple{Nothing}, Tuple{}}, Zygote.var"#combine_styles_pullback#1169"{Tuple{Nothing, Nothing}}}}}}, Zygote.var"#1973#back#190"{Zygote.var"#186#189"{Zygote.Context{false}, GlobalRef, Matrix{Float64}}}}})(Δ::Float64)
    @ Zygote ~/.julia/packages/Zygote/SuKWp/src/compiler/interface2.jl:0
 [26] Pullback
    @ ~/Documentos/2023/Phytoplankton_PINN/test.jl:14 [inlined]
 [27] #287
    @ ~/.julia/packages/Zygote/SuKWp/src/lib/lib.jl:206 [inlined]
 [28] #2156#back
    @ ~/.julia/packages/ZygoteRules/OgCVT/src/adjoint.jl:71 [inlined]
 [29] Pullback
    @ ~/.julia/packages/SciMLBase/VdcHg/src/scimlfunctions.jl:3626 [inlined]
 [30] #287
    @ ~/.julia/packages/Zygote/SuKWp/src/lib/lib.jl:206 [inlined]
 [31] (::Zygote.var"#2156#back#289"{Zygote.var"#287#288"{Tuple{Tuple{Nothing, Nothing}, Tuple{}}, Zygote.Pullback{Tuple{OptimizationFunction{true, Optimization.AutoZygote, var"#18#19", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Vector{Float64}, SciMLBase.NullParameters}, Tuple{Zygote.var"#2167#back#299"{Zygote.var"#back#298"{:f, Zygote.Context{false}, OptimizationFunction{true, Optimization.AutoZygote, var"#18#19", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, var"#18#19"}}, Zygote.var"#2156#back#289"{Zygote.var"#287#288"{Tuple{Tuple{Nothing, Nothing}}, Zygote.Pullback{Tuple{var"#18#19", Vector{Float64}, SciMLBase.NullParameters}, Tuple{Zygote.Pullback{Tuple{typeof(loss), Vector{Float64}}, Tuple{Zygote.var"#3009#back#777"{Zygote.var"#771#775"{Vector{Vector{Float64}}}}, Zygote.Pullback{Tuple{typeof(Base.Broadcast.materialize), Vector{Vector{Float64}}}, Tuple{}}, Zygote.var"#2197#back#309"{Zygote.Jnew{var"#dfdt#16"{Vector{Float64}}, Nothing, false}}, Zygote.ZBack{ChainRules.var"#sum_abs2_pullback#1681"{Vector{Float64}}}, Zygote.var"#2592#back#528"{Zygote.var"#538#540"{2, Float64, Matrix{Float64}, Tuple{Int64, Colon}}}, Zygote.Pullback{Tuple{typeof(Base.Broadcast.broadcasted), var"#dfdt#16"{Vector{Float64}}, Vector{Float64}}, Tuple{Zygote.var"#2863#back#683"{Zygote.var"#map_back#677"{typeof(Base.Broadcast.broadcastable), 1, Tuple{Tuple{}}, Tuple{Val{0}}, Tuple{}}}, Zygote.var"#2156#back#289"{Zygote.var"#287#288"{Tuple{Tuple{Nothing, Nothing, Nothing}, Tuple{}}, Zygote.var"#4121#back#1363"{Zygote.var"#∇broadcasted#1374"{Tuple{Vector{Float64}}, Vector{Tuple{Vector{Float64}, Zygote.Pullback{Tuple{var"#dfdt#16"{Vector{Float64}}, Float64}, Tuple{Zygote.var"#2167#back#299"{Zygote.var"#back#298"{:p, Zygote.Context{false}, var"#dfdt#16"{Vector{Float64}}, Vector{Float64}}}, Zygote.var"#2197#back#309"{Zygote.Jnew{var"#15#17"{Vector{Float64}}, Nothing, false}}, Zygote.var"#2167#back#299"{Zygote.var"#back#298"{:p, Zygote.Context{false}, var"#dfdt#16"{Vector{Float64}}, Vector{Float64}}}, Zygote.var"#2016#back#209"{Zygote.var"#back#207"{1, 1, Zygote.Context{false}, Vector{Float64}}}, Zygote.Pullback{Tuple{typeof(jacobian), var"#15#17"{Vector{Float64}}, Float64}, Tuple{Zygote.var"#2156#back#289"{Zygote.var"#287#288"{Tuple{Tuple{Nothing}, Tuple{Nothing}}, Zygote.Pullback{Tuple{typeof(withjacobian), var"#15#17"{Vector{Float64}}, Float64}, Any}}}, Zygote.var"#2000#back#200"{typeof(identity)}, Zygote.var"#2167#back#299"{Zygote.var"#back#298"{:grad, Zygote.Context{false}, NamedTuple{(:val, :grad), Tuple{Vector{Float64}, Tuple{Vector{Float64}}}}, Tuple{Vector{Float64}}}}}}}}}}, Val{2}}}}}, Zygote.var"#2000#back#200"{typeof(identity)}, Zygote.Pullback{Tuple{typeof(Base.Broadcast.broadcastable), Vector{Float64}}, Tuple{}}, Zygote.var"#2000#back#200"{typeof(identity)}, Zygote.var"#2156#back#289"{Zygote.var"#287#288"{Tuple{Tuple{Nothing}, Tuple{}}, Zygote.var"#combine_styles_pullback#1169"{Tuple{Nothing, Nothing}}}}}}, Zygote.var"#1973#back#190"{Zygote.var"#186#189"{Zygote.Context{false}, GlobalRef, Matrix{Float64}}}}}}}}}}}}})(Δ::Float64)
    @ Zygote ~/.julia/packages/ZygoteRules/OgCVT/src/adjoint.jl:71
 [32] Pullback
    @ ~/.julia/packages/Optimization/vFala/src/function/zygote.jl:31 [inlined]
 [33] (::Zygote.Pullback{Tuple{Optimization.var"#144#153"{OptimizationFunction{true, Optimization.AutoZygote, var"#18#19", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, SciMLBase.NullParameters}, Vector{Float64}}, Tuple{Zygote.var"#2167#back#299"{Zygote.var"#back#298"{:p, Zygote.Context{false}, Optimization.var"#144#153"{OptimizationFunction{true, Optimization.AutoZygote, var"#18#19", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, SciMLBase.NullParameters}, SciMLBase.NullParameters}}, Zygote.var"#2167#back#299"{Zygote.var"#back#298"{:f, Zygote.Context{false}, Optimization.var"#144#153"{OptimizationFunction{true, Optimization.AutoZygote, var"#18#19", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, SciMLBase.NullParameters}, OptimizationFunction{true, Optimization.AutoZygote, var"#18#19", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}}}, Zygote.var"#2156#back#289"{Zygote.var"#287#288"{Tuple{Tuple{Nothing, Nothing}, Tuple{}}, Zygote.Pullback{Tuple{OptimizationFunction{true, Optimization.AutoZygote, var"#18#19", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Vector{Float64}, SciMLBase.NullParameters}, Tuple{Zygote.var"#2167#back#299"{Zygote.var"#back#298"{:f, Zygote.Context{false}, OptimizationFunction{true, Optimization.AutoZygote, var"#18#19", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, var"#18#19"}}, Zygote.var"#2156#back#289"{Zygote.var"#287#288"{Tuple{Tuple{Nothing, Nothing}}, Zygote.Pullback{Tuple{var"#18#19", Vector{Float64}, SciMLBase.NullParameters}, Tuple{Zygote.Pullback{Tuple{typeof(loss), Vector{Float64}}, Tuple{Zygote.var"#3009#back#777"{Zygote.var"#771#775"{Vector{Vector{Float64}}}}, Zygote.Pullback{Tuple{typeof(Base.Broadcast.materialize), Vector{Vector{Float64}}}, Tuple{}}, Zygote.var"#2197#back#309"{Zygote.Jnew{var"#dfdt#16"{Vector{Float64}}, Nothing, false}}, Zygote.ZBack{ChainRules.var"#sum_abs2_pullback#1681"{Vector{Float64}}}, Zygote.var"#2592#back#528"{Zygote.var"#538#540"{2, Float64, Matrix{Float64}, Tuple{Int64, Colon}}}, Zygote.Pullback{Tuple{typeof(Base.Broadcast.broadcasted), var"#dfdt#16"{Vector{Float64}}, Vector{Float64}}, Tuple{Zygote.var"#2863#back#683"{Zygote.var"#map_back#677"{typeof(Base.Broadcast.broadcastable), 1, Tuple{Tuple{}}, Tuple{Val{0}}, Tuple{}}}, Zygote.var"#2156#back#289"{Zygote.var"#287#288"{Tuple{Tuple{Nothing, Nothing, Nothing}, Tuple{}}, Zygote.var"#4121#back#1363"{Zygote.var"#∇broadcasted#1374"{Tuple{Vector{Float64}}, Vector{Tuple{Vector{Float64}, Zygote.Pullback{Tuple{var"#dfdt#16"{Vector{Float64}}, Float64}, Tuple{Zygote.var"#2167#back#299"{Zygote.var"#back#298"{:p, Zygote.Context{false}, var"#dfdt#16"{Vector{Float64}}, Vector{Float64}}}, Zygote.var"#2197#back#309"{Zygote.Jnew{var"#15#17"{Vector{Float64}}, Nothing, false}}, Zygote.var"#2167#back#299"{Zygote.var"#back#298"{:p, Zygote.Context{false}, var"#dfdt#16"{Vector{Float64}}, Vector{Float64}}}, Zygote.var"#2016#back#209"{Zygote.var"#back#207"{1, 1, Zygote.Context{false}, Vector{Float64}}}, Zygote.Pullback{Tuple{typeof(jacobian), var"#15#17"{Vector{Float64}}, Float64}, Tuple{Zygote.var"#2156#back#289"{Zygote.var"#287#288"{Tuple{Tuple{Nothing}, Tuple{Nothing}}, Zygote.Pullback{Tuple{typeof(withjacobian), var"#15#17"{Vector{Float64}}, Float64}, Any}}}, Zygote.var"#2000#back#200"{typeof(identity)}, Zygote.var"#2167#back#299"{Zygote.var"#back#298"{:grad, Zygote.Context{false}, NamedTuple{(:val, :grad), Tuple{Vector{Float64}, Tuple{Vector{Float64}}}}, Tuple{Vector{Float64}}}}}}}}}}, Val{2}}}}}, Zygote.var"#2000#back#200"{typeof(identity)}, Zygote.Pullback{Tuple{typeof(Base.Broadcast.broadcastable), Vector{Float64}}, Tuple{}}, Zygote.var"#2000#back#200"{typeof(identity)}, Zygote.var"#2156#back#289"{Zygote.var"#287#288"{Tuple{Tuple{Nothing}, Tuple{}}, Zygote.var"#combine_styles_pullback#1169"{Tuple{Nothing, Nothing}}}}}}, Zygote.var"#1973#back#190"{Zygote.var"#186#189"{Zygote.Context{false}, GlobalRef, Matrix{Float64}}}}}}}}}}}}}, Zygote.Pullback{Tuple{typeof(Zygote.literal_getindex), Float64, Val{1}}, Tuple{Zygote.Pullback{Tuple{typeof(getindex), Float64, Int64}, Any}}}, Zygote.var"#2000#back#200"{typeof(identity)}}})(Δ::Float64)
    @ Zygote ~/.julia/packages/Zygote/SuKWp/src/compiler/interface2.jl:0
 [34] #287
    @ ~/.julia/packages/Zygote/SuKWp/src/lib/lib.jl:206 [inlined]
 [35] #2156#back
    @ ~/.julia/packages/ZygoteRules/OgCVT/src/adjoint.jl:71 [inlined]
 [36] Pullback
    @ ~/.julia/packages/Optimization/vFala/src/function/zygote.jl:35 [inlined]
 [37] (::Zygote.Pullback{Tuple{Optimization.var"#146#155"{Tuple{}, Optimization.var"#144#153"{OptimizationFunction{true, Optimization.AutoZygote, var"#18#19", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, SciMLBase.NullParameters}}, Vector{Float64}}, Tuple{Zygote.var"#2167#back#299"{Zygote.var"#back#298"{:_f, Zygote.Context{false}, Optimization.var"#146#155"{Tuple{}, Optimization.var"#144#153"{OptimizationFunction{true, Optimization.AutoZygote, var"#18#19", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, SciMLBase.NullParameters}}, Optimization.var"#144#153"{OptimizationFunction{true, Optimization.AutoZygote, var"#18#19", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, SciMLBase.NullParameters}}}, Zygote.var"#2167#back#299"{Zygote.var"#back#298"{:args, Zygote.Context{false}, Optimization.var"#146#155"{Tuple{}, Optimization.var"#144#153"{OptimizationFunction{true, Optimization.AutoZygote, var"#18#19", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, SciMLBase.NullParameters}}, Tuple{}}}, Zygote.var"#2000#back#200"{typeof(identity)}, Zygote.var"#2156#back#289"{Zygote.var"#287#288"{Tuple{Tuple{Nothing}, Tuple{}}, Zygote.Pullback{Tuple{Optimization.var"#144#153"{OptimizationFunction{true, Optimization.AutoZygote, var"#18#19", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, SciMLBase.NullParameters}, Vector{Float64}}, Tuple{Zygote.var"#2167#back#299"{Zygote.var"#back#298"{:p, Zygote.Context{false}, Optimization.var"#144#153"{OptimizationFunction{true, Optimization.AutoZygote, var"#18#19", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, SciMLBase.NullParameters}, SciMLBase.NullParameters}}, Zygote.var"#2167#back#299"{Zygote.var"#back#298"{:f, Zygote.Context{false}, Optimization.var"#144#153"{OptimizationFunction{true, Optimization.AutoZygote, var"#18#19", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, SciMLBase.NullParameters}, OptimizationFunction{true, Optimization.AutoZygote, var"#18#19", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}}}, Zygote.var"#2156#back#289"{Zygote.var"#287#288"{Tuple{Tuple{Nothing, Nothing}, Tuple{}}, Zygote.Pullback{Tuple{OptimizationFunction{true, Optimization.AutoZygote, var"#18#19", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Vector{Float64}, SciMLBase.NullParameters}, Tuple{Zygote.var"#2167#back#299"{Zygote.var"#back#298"{:f, Zygote.Context{false}, OptimizationFunction{true, Optimization.AutoZygote, var"#18#19", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, var"#18#19"}}, Zygote.var"#2156#back#289"{Zygote.var"#287#288"{Tuple{Tuple{Nothing, Nothing}}, Zygote.Pullback{Tuple{var"#18#19", Vector{Float64}, SciMLBase.NullParameters}, Tuple{Zygote.Pullback{Tuple{typeof(loss), Vector{Float64}}, Tuple{Zygote.var"#3009#back#777"{Zygote.var"#771#775"{Vector{Vector{Float64}}}}, Zygote.Pullback{Tuple{typeof(Base.Broadcast.materialize), Vector{Vector{Float64}}}, Tuple{}}, Zygote.var"#2197#back#309"{Zygote.Jnew{var"#dfdt#16"{Vector{Float64}}, Nothing, false}}, Zygote.ZBack{ChainRules.var"#sum_abs2_pullback#1681"{Vector{Float64}}}, Zygote.var"#2592#back#528"{Zygote.var"#538#540"{2, Float64, Matrix{Float64}, Tuple{Int64, Colon}}}, Zygote.Pullback{Tuple{typeof(Base.Broadcast.broadcasted), var"#dfdt#16"{Vector{Float64}}, Vector{Float64}}, Tuple{Zygote.var"#2863#back#683"{Zygote.var"#map_back#677"{typeof(Base.Broadcast.broadcastable), 1, Tuple{Tuple{}}, Tuple{Val{0}}, Tuple{}}}, Zygote.var"#2156#back#289"{Zygote.var"#287#288"{Tuple{Tuple{Nothing, Nothing, Nothing}, Tuple{}}, Zygote.var"#4121#back#1363"{Zygote.var"#∇broadcasted#1374"{Tuple{Vector{Float64}}, Vector{Tuple{Vector{Float64}, Zygote.Pullback{Tuple{var"#dfdt#16"{Vector{Float64}}, Float64}, Tuple{Zygote.var"#2167#back#299"{Zygote.var"#back#298"{:p, Zygote.Context{false}, var"#dfdt#16"{Vector{Float64}}, Vector{Float64}}}, Zygote.var"#2197#back#309"{Zygote.Jnew{var"#15#17"{Vector{Float64}}, Nothing, false}}, Zygote.var"#2167#back#299"{Zygote.var"#back#298"{:p, Zygote.Context{false}, var"#dfdt#16"{Vector{Float64}}, Vector{Float64}}}, Zygote.var"#2016#back#209"{Zygote.var"#back#207"{1, 1, Zygote.Context{false}, Vector{Float64}}}, Zygote.Pullback{Tuple{typeof(jacobian), var"#15#17"{Vector{Float64}}, Float64}, Tuple{Zygote.var"#2156#back#289"{Zygote.var"#287#288"{Tuple{Tuple{Nothing}, Tuple{Nothing}}, Zygote.Pullback{Tuple{typeof(withjacobian), var"#15#17"{Vector{Float64}}, Float64}, Any}}}, Zygote.var"#2000#back#200"{typeof(identity)}, Zygote.var"#2167#back#299"{Zygote.var"#back#298"{:grad, Zygote.Context{false}, NamedTuple{(:val, :grad), Tuple{Vector{Float64}, Tuple{Vector{Float64}}}}, Tuple{Vector{Float64}}}}}}}}}}, Val{2}}}}}, Zygote.var"#2000#back#200"{typeof(identity)}, Zygote.Pullback{Tuple{typeof(Base.Broadcast.broadcastable), Vector{Float64}}, Tuple{}}, Zygote.var"#2000#back#200"{typeof(identity)}, Zygote.var"#2156#back#289"{Zygote.var"#287#288"{Tuple{Tuple{Nothing}, Tuple{}}, Zygote.var"#combine_styles_pullback#1169"{Tuple{Nothing, Nothing}}}}}}, Zygote.var"#1973#back#190"{Zygote.var"#186#189"{Zygote.Context{false}, GlobalRef, Matrix{Float64}}}}}}}}}}}}}, Zygote.Pullback{Tuple{typeof(Zygote.literal_getindex), Float64, Val{1}}, Tuple{Zygote.Pullback{Tuple{typeof(getindex), Float64, Int64}, Any}}}, Zygote.var"#2000#back#200"{typeof(identity)}}}}}}})(Δ::Float64)
    @ Zygote ~/.julia/packages/Zygote/SuKWp/src/compiler/interface2.jl:0
 [38] (::Zygote.var"#75#76"{Zygote.Pullback{Tuple{Optimization.var"#146#155"{Tuple{}, Optimization.var"#144#153"{OptimizationFunction{true, Optimization.AutoZygote, var"#18#19", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, SciMLBase.NullParameters}}, Vector{Float64}}, Tuple{Zygote.var"#2167#back#299"{Zygote.var"#back#298"{:_f, Zygote.Context{false}, Optimization.var"#146#155"{Tuple{}, Optimization.var"#144#153"{OptimizationFunction{true, Optimization.AutoZygote, var"#18#19", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, SciMLBase.NullParameters}}, Optimization.var"#144#153"{OptimizationFunction{true, Optimization.AutoZygote, var"#18#19", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, SciMLBase.NullParameters}}}, Zygote.var"#2167#back#299"{Zygote.var"#back#298"{:args, Zygote.Context{false}, Optimization.var"#146#155"{Tuple{}, Optimization.var"#144#153"{OptimizationFunction{true, Optimization.AutoZygote, var"#18#19", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, SciMLBase.NullParameters}}, Tuple{}}}, Zygote.var"#2000#back#200"{typeof(identity)}, Zygote.var"#2156#back#289"{Zygote.var"#287#288"{Tuple{Tuple{Nothing}, Tuple{}}, Zygote.Pullback{Tuple{Optimization.var"#144#153"{OptimizationFunction{true, Optimization.AutoZygote, var"#18#19", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, SciMLBase.NullParameters}, Vector{Float64}}, Tuple{Zygote.var"#2167#back#299"{Zygote.var"#back#298"{:p, Zygote.Context{false}, Optimization.var"#144#153"{OptimizationFunction{true, Optimization.AutoZygote, var"#18#19", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, SciMLBase.NullParameters}, SciMLBase.NullParameters}}, Zygote.var"#2167#back#299"{Zygote.var"#back#298"{:f, Zygote.Context{false}, Optimization.var"#144#153"{OptimizationFunction{true, Optimization.AutoZygote, var"#18#19", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, SciMLBase.NullParameters}, OptimizationFunction{true, Optimization.AutoZygote, var"#18#19", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}}}, Zygote.var"#2156#back#289"{Zygote.var"#287#288"{Tuple{Tuple{Nothing, Nothing}, Tuple{}}, Zygote.Pullback{Tuple{OptimizationFunction{true, Optimization.AutoZygote, var"#18#19", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Vector{Float64}, SciMLBase.NullParameters}, Tuple{Zygote.var"#2167#back#299"{Zygote.var"#back#298"{:f, Zygote.Context{false}, OptimizationFunction{true, Optimization.AutoZygote, var"#18#19", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, var"#18#19"}}, Zygote.var"#2156#back#289"{Zygote.var"#287#288"{Tuple{Tuple{Nothing, Nothing}}, Zygote.Pullback{Tuple{var"#18#19", Vector{Float64}, SciMLBase.NullParameters}, Tuple{Zygote.Pullback{Tuple{typeof(loss), Vector{Float64}}, Tuple{Zygote.var"#3009#back#777"{Zygote.var"#771#775"{Vector{Vector{Float64}}}}, Zygote.Pullback{Tuple{typeof(Base.Broadcast.materialize), Vector{Vector{Float64}}}, Tuple{}}, Zygote.var"#2197#back#309"{Zygote.Jnew{var"#dfdt#16"{Vector{Float64}}, Nothing, false}}, Zygote.ZBack{ChainRules.var"#sum_abs2_pullback#1681"{Vector{Float64}}}, Zygote.var"#2592#back#528"{Zygote.var"#538#540"{2, Float64, Matrix{Float64}, Tuple{Int64, Colon}}}, Zygote.Pullback{Tuple{typeof(Base.Broadcast.broadcasted), var"#dfdt#16"{Vector{Float64}}, Vector{Float64}}, Tuple{Zygote.var"#2863#back#683"{Zygote.var"#map_back#677"{typeof(Base.Broadcast.broadcastable), 1, Tuple{Tuple{}}, Tuple{Val{0}}, Tuple{}}}, Zygote.var"#2156#back#289"{Zygote.var"#287#288"{Tuple{Tuple{Nothing, Nothing, Nothing}, Tuple{}}, Zygote.var"#4121#back#1363"{Zygote.var"#∇broadcasted#1374"{Tuple{Vector{Float64}}, Vector{Tuple{Vector{Float64}, Zygote.Pullback{Tuple{var"#dfdt#16"{Vector{Float64}}, Float64}, Tuple{Zygote.var"#2167#back#299"{Zygote.var"#back#298"{:p, Zygote.Context{false}, var"#dfdt#16"{Vector{Float64}}, Vector{Float64}}}, Zygote.var"#2197#back#309"{Zygote.Jnew{var"#15#17"{Vector{Float64}}, Nothing, false}}, Zygote.var"#2167#back#299"{Zygote.var"#back#298"{:p, Zygote.Context{false}, var"#dfdt#16"{Vector{Float64}}, Vector{Float64}}}, Zygote.var"#2016#back#209"{Zygote.var"#back#207"{1, 1, Zygote.Context{false}, Vector{Float64}}}, Zygote.Pullback{Tuple{typeof(jacobian), var"#15#17"{Vector{Float64}}, Float64}, Tuple{Zygote.var"#2156#back#289"{Zygote.var"#287#288"{Tuple{Tuple{Nothing}, Tuple{Nothing}}, Zygote.Pullback{Tuple{typeof(withjacobian), var"#15#17"{Vector{Float64}}, Float64}, Any}}}, Zygote.var"#2000#back#200"{typeof(identity)}, Zygote.var"#2167#back#299"{Zygote.var"#back#298"{:grad, Zygote.Context{false}, NamedTuple{(:val, :grad), Tuple{Vector{Float64}, Tuple{Vector{Float64}}}}, Tuple{Vector{Float64}}}}}}}}}}, Val{2}}}}}, Zygote.var"#2000#back#200"{typeof(identity)}, Zygote.Pullback{Tuple{typeof(Base.Broadcast.broadcastable), Vector{Float64}}, Tuple{}}, Zygote.var"#2000#back#200"{typeof(identity)}, Zygote.var"#2156#back#289"{Zygote.var"#287#288"{Tuple{Tuple{Nothing}, Tuple{}}, Zygote.var"#combine_styles_pullback#1169"{Tuple{Nothing, Nothing}}}}}}, Zygote.var"#1973#back#190"{Zygote.var"#186#189"{Zygote.Context{false}, GlobalRef, Matrix{Float64}}}}}}}}}}}}}, Zygote.Pullback{Tuple{typeof(Zygote.literal_getindex), Float64, Val{1}}, Tuple{Zygote.Pullback{Tuple{typeof(getindex), Float64, Int64}, Any}}}, Zygote.var"#2000#back#200"{typeof(identity)}}}}}}}})(Δ::Float64)
    @ Zygote ~/.julia/packages/Zygote/SuKWp/src/compiler/interface.jl:45
 [39] gradient(f::Function, args::Vector{Float64})
    @ Zygote ~/.julia/packages/Zygote/SuKWp/src/compiler/interface.jl:97
 [40] (::Optimization.var"#145#154"{Optimization.var"#144#153"{OptimizationFunction{true, Optimization.AutoZygote, var"#18#19", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, SciMLBase.NullParameters}})(::Vector{Float64}, ::Vector{Float64})
    @ Optimization ~/.julia/packages/Optimization/vFala/src/function/zygote.jl:33
 [41] macro expansion
    @ ~/.julia/packages/OptimizationOptimisers/FWIuf/src/OptimizationOptimisers.jl:31 [inlined]
 [42] macro expansion
    @ ~/.julia/packages/Optimization/vFala/src/utils.jl:37 [inlined]
 [43] __solve(prob::OptimizationProblem{true, OptimizationFunction{true, Optimization.AutoZygote, var"#18#19", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Vector{Float64}, SciMLBase.NullParameters, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}}, opt::Adam{Float64}, data::Base.Iterators.Cycle{Tuple{Optimization.NullData}}; maxiters::Int64, callback::Function, progress::Bool, save_best::Bool, kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ OptimizationOptimisers ~/.julia/packages/OptimizationOptimisers/FWIuf/src/OptimizationOptimisers.jl:30
 [44] __solve (repeats 2 times)
    @ ~/.julia/packages/OptimizationOptimisers/FWIuf/src/OptimizationOptimisers.jl:7 [inlined]
 [45] #solve#553
    @ ~/.julia/packages/SciMLBase/VdcHg/src/solve.jl:86 [inlined]
 [46] top-level scope
    @ ~/Documentos/2023/Phytoplankton_PINN/test.jl:18

If I just try to run dfdt(t) = jacobian(x->f(p,x),t)[1], it works as expected. The problem arises when it is passed through the Optimization problem.

Is there any solution/workaround to have a loss function that involves the jacobian of the neural network?
@jarroyoe jarroyoe changed the title Mutating arrays is not supported when trying to calculate the derivative of a neural network Mutating arrays is not supported when trying to calculate the derivative of a function May 25, 2023
@ToucheSir
Copy link
Member

Generally I would say that no, you can not differentiate through Zygote.jacobian using Zygote. However, I feel like this question has come up before and people have found workarounds which don't involve Zygote over Zygote. Since you're using mostly SciML libraries, I would suggest asking your question in one of their help channels, as they'd be best suited to give specific suggestions about your code.

@jarroyoe
Copy link
Author

Thanks for the heads-up. Following this issue #953 the solution for my specific case ended up being changing jacobian to Zygote.forwarddiff.

This issue may keep raising as more people test different PINN architectures. Maybe it's worth adding some troubleshooting related to this on the limitations section of the documentation?

@ToucheSir
Copy link
Member

ToucheSir commented May 25, 2023 •
edited
Loading

Maybe it's worth adding some troubleshooting related to this on the limitations section of the documentation?

I'm not sure what we would add! #953 seems to be somewhat different and the last few posts there don't even use jacobian. We already have https://fluxml.ai/Zygote.jl/latest/limitations/#Second-derivatives-1 too.

This issue may keep raising as more people test different PINN architectures.

That's part of the problem, no two PINN-related issues seem to be alike so it's difficult to nail down what would be required to address all of them. For example, there was a Discourse thread last week about nested diff and PINNs, but the formulation there is totally different from the one you presented here.

That's why I recommended seeking help on SciML channels. They have the most experience out of anyone in the Julia community when it comes to working with AD for PINNs, and should be able to provide concrete advice on how to make your code work. You might be able to get a little more help here, but it'll be on the level of #953 instead of PINN-specific.

@jarroyoe
Copy link
Author

Thanks for the input. That discourse thread is a great illustration of how difficult this is from the engineering standpoint that we users aren't always able to grasp

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@ToucheSir @jarroyoe