Nested Differentiation Bug

[SebastianAment]

We cannot use Taylor series to take nested derivatives of functions. This is very similar to this issue in ForwardDiff.jl.

Taking their example:

using TaylorSeries
TD(f, x) = derivative(1, f(x+Taylor1(1)))
TD(x -> x * TD(y -> x + y, 1), 1)

does not produce the correct result. I believe that the solution for TaylorSeries could be similar to the one which is implemented in ForwardDiff. That is, every Taylor1 object could be equipped with a tag parameter.

Of course, if all we needed was the first derivative, we could just use ForwardDiff. However, I am looking for an efficient way to get nested higher-order derivatives.

[ChrisRackauckas]

Why would that make sense here? This repo already computes the higher order terms in the Taylor series.

[dpsanders]

What exactly do you want to take higher-order derivatives of?

As Chris says, doing this is one of the main points of the package: the nth coefficient in a Taylor series expansion is the nth derivative (except for a factorial factor).

[dpsanders]

Note that there are also multi-variate Taylor series available in the package. Please give an example of the kind of calculation you would like to do.

[SebastianAment]

Thank you for your comments.

I wanted to use TaylorSeries to implement certain differential operators on functions and be able to apply them in a nested way. As a simple example, I implemented del and divergence with ForwardDiff, and can get the laplacian simply by chaining the two operators, saving me an implementation.

I wanted to chain more complex operators involving much higher orders and thought I could do that easily with TaylorSeries. However, that fails for the reason above. Of course TaylorN can be used to implement laplacian and other operators, but at the expense of additional coding time. It also does not allow us to apply another operator to the laplacian or any other result.

On a higher level, it could be a useful functionality to be able to compute a Taylor series of a function which itself uses TaylorSeries for its calculation. For the time being, I believe there should be a disclaimer that TaylorSeries should not be used in a nested way (maybe this issue serves this purpose).

Regardless, I am impressed by the functionality this package offers. Thank you for maintaining it!

[dpsanders]

Ah I see; thanks for reporting this. Yes it would be nice to be able to do that, but there are certainly no plans to implement tagging or anything like that. Maybe that's something that can be extracted from ForwardDiff and made into a general purpose tool, which could then be directly applied to TaylorSeries?

[dpsanders]

Note that it is also possible to nest one Taylor1 inside another, although unfortunately the output will currently use the same variable:

julia> x = Taylor1(Taylor1, 10)
 ( 1 + 𝒪(t¹)) t + 𝒪(t¹¹)

julia> x[1] = Taylor1([3, 4, 5])
 3 + 4 t + 5 t² + 𝒪(t³)

julia> x[2] = Taylor1([6, 7, 8])
 6 + 7 t + 8 t² + 𝒪(t³)

julia> x
 ( 3 + 4 t + 5 t² + 𝒪(t³)) t + ( 6 + 7 t + 8 t² + 𝒪(t³)) t² + 𝒪(t¹¹)

[dpsanders]

See also #126

[lbenet]

This is a follow up on @dpsanders's idea, which is a workaround that effectively corresponds to the tagging solution for the perturbation confussion.

The idea is to have two distinct Taylor1 variables, t1::Taylor1{Float64} and t2::Taylor1{Taylor1{Float64}}:

julia> t1 = Taylor1(1)
 1.0 t + 𝒪(t²)

julia> t2 = Taylor1(Taylor1{Float64}, 2) # equivalent to Taylor1([zero(t1), one(t1)], 2)
 ( 1.0 + 𝒪(t¹)) t + 𝒪(t³)

Admittedly, the current output is pretty confusing and awful, since both variables are displayed using t; to help distinguishing them, t1 is of order 1 and t2 of order 2.

Now, TD is rewritten as:

julia> TD(f, t, x) = differentiate(1, f(x+t))
TD (generic function with 1 method)

which corresponds to differentiate with respect to the t::Taylor1 variable.

Finally:

julia> TD(x -> x * TD(y -> x + y, t2, 1), t1, 1)
1.0

julia> TD(x -> x * TD(y -> x + y, t1, 1), t2, 1)
 1.0 + 𝒪(t²)

which are both correct, though the results correspond to different types.

I am not sure if this is a practical solution if the nested differentiation involves many differentials...

[dpsanders]

Can we specify the output variable name?

[lbenet]

Not yet, but I have it in my to-do list...

[lbenet]

We have set_taylor1_varname, but it does not (yet) play nicely with nested Taylor1s...