Fix up type inference and other issues for the new DFT API

[yuyichao]

The type instability/inference issue is mentioned in #12087 (comment) and #12087 (comment).

The code can probably be cleaned up a little to make the type inference more effective without the type assertion but the current version should be good enough.

This also fixes a deprecation warning for int64.

Close #9772

CC. @stevengj

[stevengj]

Looks good, thanks for working on this; sorry it's taken me so long to get back to you on this.

[yuyichao]

I'm also trying to implement some other tweaks that I think can probably make this more compiler/type inference friendly. I guess I'll just do a separate PR when I'm ready.

[stevengj]

Sorry, it looked ready to merge to me.

[stevengj]

With the latest version it seems there is still a little allocation:

A = rand(Complex128, 300,300,50);
X = similar(A)
p = plan_fft(A)
@time A_mul_B!(X, p, A);
@time A_mul_B!(X, p, A);

gives 137.505 milliseconds (4 allocations: 144 bytes).

[yuyichao]

the 4 allocations: 144 bytes comes from @time

[yuyichao]

In global scope

[yuyichao]

julia> function f()
       A = rand(Complex128, 300,300,50);
       X = similar(A)
       p = plan_fft(A)
       @time A_mul_B!(X, p, A);
       @time A_mul_B!(X, p, A);
       end
f (generic function with 1 method)

julia> f();
 119.674 milliseconds
 105.739 milliseconds

[stevengj]

Ah, right. (cc: @HomerReid)