Algebraic differentiation

[BigFav]

Still has some work left, but a large portion is complete (finally got back from vacation). I have a few questions:

• I return the same expression I received if the arguments length is bad, but it doesn't throw an error. How can I get the function of derivative to be the function of the expression returned?
• I wasn't sure about what style of testing you wanted, comparing against math.parse or the written out nodes. Is there a preference?

Any comments are welcome (this PR is mostly meant for feedback).

[josdejong]

Thanks, very cool.

I'm not sure what you mean with your first point.

As for the tests: from unit test perspective it is best to write out the actual Nodes like you did: have as less indirections as possible. But it's quite cumbersome to write the nodes out, and I think we should be pragmatic too and I'm ok with testing against math.parse too: it's less puristic, but makes it way easier to write (and read) the unit tests.

[BigFav]

To see what I mean take a look at the commented out test cases. I think the problem is that I am only returning the nodes, and not actually running the functions (e.g. calling log); I didn't realize this was something I had to do. Is this the problem? If this is the problem, how do I fix it? I tried changing derivative.transform to be:

'Array, Object, Object': function (args, math, scope) {
      var fnScope = Object.create(scope);
      var fnCode = derivative.apply(null, args).compile();
      return function (x) {
        fnScope[variable] = x;
        return fnCode.eval(fnScope);
      };
    }
});

in an attempt to mimic integrate, but that still returns a function, not a value (i.e. f(2) would still return a function).

[BigFav]

After that, how can I/is it possible to get the nodes from this function?

[josdejong]

For these functions for which you've implemented the derivative, like log, you will indeed have to check yourself whether the Node contains the right number of arguments, and throw these errors yourself in derivative.

I'm not sure what you mean with "get the nodes from this function". Do you mean the integrate function you are working on? I suppose you can do this in a similar way as derivative?

[BigFav]

I'm not explaining my point well. The problem is that derivative(expr, var) returns a Node, not a runnable function. This runs into problems during use:

f(x) = derivative(2x + 3x, x)  // should return a function (I think),
                               // based on the nodes of the evaluated (2 + 3)
f(1)  // should print 5

but instead of printing 5, it just prints the OperatorNode.

Do you mean the integrate function you are working on?

No I mean this. I was trying to use this so that derivative would return numeric values.

I'm not sure what you mean with "get the nodes from this function"

I still want to be able to test based on the Nodes, even though this will need to provide numeric answers at times (which I think requires returning a function). So I want to be able to still get the backend Nodes.

[josdejong]

Ah, I get your point. We should think this through.

• If we want this to be really about symbolic computation, the functions derivative and integrate should return a Node tree.
• If we don't want that, there is a way easier solution: just do this numerically... (your integrate looks like that so far?)

I think the real power lies in the first option: being able to get the derivative of a symbolic expression (in the form of a node tree) as an expression, like derivative(2x^3) -> 6x^2. So these symbolic computations act on node trees and return node trees. To do some actual evaluation with it, we may need some extra tooling.

It may be a good idea to scetch some usage scenarios. Here some first ideas:

// JS API

// symbolic computation
var f = math.parse('2x^3');
var df = math.derivative(f, 'x'); 
  // math.derivative could also allow to pass a string, 
  // which will then first call math.parse
console.log('f', f.toString()); // 2x^3
console.log('df', df.toString()); // 6x^2

// evaluation
var x = 2;
var y = f.compile().eval({x: x}); // 16
var dy = df.compile().eval(x: x}); // 24

// how about passing a FunctionNode:
var f = math.parse('f(x) = 2x^3');
var df = math.derivative(f); 
  // derivative can know the variable x from the FunctionNode, 
  // and returns a FunctionNode of the derivative? 
var fCompiled = f.compile().eval();   // compile into an executable function
var dfCompiled = df.compile().eval(); // compile into an executable function

// evalutation
var y = fCompiled(x); // 16
var dy = dfCompiled(x); // 24

The expression parser currently doesn't have any equipment to work with symbolic variables.

# EXPRESSION PARSER API
f = parse("2x^3")       # returns a Node tree 2x^3
df = derivative(f, x);  # returns a Node tree 6x^2

# to evaluate the symbolic expression into an executable function we need extra 
# functionality. The following is not yet supported
dy = eval(df, {x: 2})  # 24 
# (note that JSON objects aren't yet supported in the expression parser)

# we need some way to compile an expression like `2x^3` as a FunctionNode, 
# because it's quite inconvenient to have to enter `f(x) = 2x^3`.
# Maybe extend `eval` and `compile` for this? like:
dfCompiled = eval(df, x)     # second and following arguments 
                             # can be used to specify function arguments 
                             # -> eval will hen auto generate a FunctionNode for this
                             #     and return an executable function

dy = dfCompiled(2)  # 24

# for example Matlab has as special notation to create symbolic variables:
x = sym("x")   # x will be a SymbolNode (same as doing `parse('x')` right now)
f = 2x^3       # because x is symbolic, f will become symbolic too (a node tree)
df = derivative(f, x)

dfCompiled = eval(df, x)  # returns an executable function
dfCompiled(2)  # 24

[BigFav]

Most of what you said seemed fine, but I'm just unsure why this should be any different than everything else. Why should derivative(x^2 + x, x) fundamentally differ from 2x + 1? Doesn't parse already return nodes, and eval return functions?

Also I somewhat disagree with:

The expression parser currently doesn't have any equipment to work with symbolic variables.

as math.eval(['f(x) = 2x + 1', 'f(1)']) works quite well; there is enough equipment for returning Nodes to break consistency.

In short, I think math.eval(['f(x) = derivative(x^2 + x, x)', 'f(1)']) should output a 3, just as math.eval(['f(x) = 2x + 1', 'f(1)']) does.

[josdejong]

What usage scenarios do you have in mind?

If all you want to support is getting the result of math.eval(['f(x) = derivative(x^2 + x, x)', 'f(1)']), I can write a simple numeric solution for you in 10 lines of code. No need to implement a full-fledged algebraic solution then.

[BigFav]

Oh wow, I thought I replied a long time ago.

Something like a convergence test would suffice, but I more thinking about consistency with Math Notepad. If this isn't that important, then I'm fine with leaving it returning Nodes since it seems like it is not possible to decompile the functions back to Nodes, and a (non-Math Notepad) user can always compile it to a function if they want that; I was under a potential misapprehension that you could decompile because plot works with functions.

If there are any style things let me know, I think I'm going to try to lower the character amount since there is a bit of redundant code here.

[BigFav]

I adjusted it now. It's ready to be merged passing inspection.

[josdejong]

Ow, I just cleaned up the v2 branch not realizing your PR was towards v2 (v2 has been moved to develop). Sorry.

Good to hear from you :) I will review your code tomorrow.
If it's easy for you to redo the PR that would be nice, else I will merge your code by hand.

Returning Nodes has my preference, and an algebraic solution is just soo cool. No worries about Math Notepad: it has to adapt to math.js, not the other way around.

[BigFav]

The PR was trivial to redo lol

#411

[josdejong]

The PR was trivial to redo

thanks you're a pro ;)

[BigFav]

@josdejong I've added some comments and test cases, no code though. Just giving you a heads up.

[josdejong]

Thanks for the update @BigFav.