0.5.0

This release brings some big changes to deeplearn.js. Largest of which is moving from a graph centric API to an ‘eager mode' imperative one that mirrors Tensorflow Eager.

We believe the eager style programming model is more suited to faster development and rapid prototyping. We have also found eager style code easier to debug in the ways we as web developers typically debug things. Another big change is moving more functionality to the top level of the library. All ops are directly exported at the top level now.

These and more changes are detailed below. We'd love to hear your feedback so feel free to make issues on the Github repo if you run into any problems or have other feedback.

What’s New

• Eager API: This release introduces our new eager API and all optimizers support eager mode. This allows you to write imperative code to do inference and training without a deferred graph execution model. Eager mode also makes debugging easier.

   import * as dl from ‘deeplearn’;
  
    // y = a * x^2 + b * x + c.
   const f = x => a.mul(x.square()).add(b.mul(x)).add(c);
   const loss = (pred, label) => pred.sub(label).square().mean();
  
   const learningRate = 0.01;
   const optimizer = dl.train.sgd(learningRate);
  
   for (let i = 0; i < 10; i++) {
     optimizer.minimize(() => loss(f(xs), ys));
   }

• Chain API: Tensors now provide a chaining API for ops. Which means you can write

  import * as dl from ‘deeplearn’;
  
  dl.tensor([[2, 3], [5, 1]])
      .mul(dl.scalar(3))
      .square();

• New Ops: dl.gather, dl.less, dl.logicalAnd, dl.where, dl.range, dl.linspace, dl.logicalXor, dl.logicalNot, dl.stack, dl.squeeze and dl.expandDims.

• New utility functions: dl.memory, dl.time and dl.nextFrame.

• Backend selection: Use dl.setBackend(‘webgl’|’cpu’). Also we automatically choose the best backend for you so you will not usually need to call this.

• New API Docs!: We are super excited about these, we are adding code snippets to our api docs and you can run them inline! Power to the browser!

What’s Changing

• NDArray has been renamed to Tensor: Tensors are now immutable meaning operations on tensors return new tensors rather than modifying the original tensors. The NDArray identifier will still work in 0.5 but will be removed in future releases.

• Tensor Creation: We now provide helper functions to create tensors. Use dl.tensor and not NDArray.new. We also have dl.scalar, dl.tensor1d, dl.tensor2d, dl.tensor3d and dl.tensor4d for your convenience.

• Top Level Ops: All ops are now on the top level object of the library. That means you can now do the following

  import * as dl from ‘deeplearn’;
  
  dl.randomNormal([2,2]).print();
  

• dl.clone() now does a shallow copy

• For typescript users: Tensors no longer have a dtype generic. dtype is now only a property of a tensor.

What’s deprecated

This version represents a big shift to a new style API, thus a number of things we are deprecating will be removed in 0.6. Please start upgrading your code now, or lock down your dependencies if you aren't yet ready to upgrade.

• Graph Mode: With the introduction of eager mode we are deprecating graph mode. The entire graph API will be removed in 0.6.

• NDArrayMath: As mentioned above, all ops are now at the top level. So no longer use code like this dl.ENV.math.square(). ** If you want to use a different backend use dl.setBackend(). We will be removing the NDArrayMath in 0.6.**

Acknowledgements

Thanks to all our contributors that helped make this happen! Thanks @manrajgrover for your work on optimizers, gradients, improving tests and more, @reiinakano for work on dl.gather, @jaassoon for work on improving our with unit tests, @skwbc for improving error messages in our checkpoint dump script, @gena for work on gradients for dl.sigmoid, @davidsoergel for work on an upcoming dl.contrib.data package @caisq for work on gradients for dl.clipByValue and dl.batchNormalization, @weizhihuang for fixing a bug in one of demos.

We also want to thank the contributors for 0.4.2 for which we didn't do official release notes. Thanks @chaosmail for your work on automatic Caffe model porting, improvements to conv2d, improvements to our testing infrastructure, and for dl.reverse, @Lewuathe for concat backprop, documentation, and better error messaging, @nkreeger for your work on random number generation, array ops, windows tooling support, and the game of life demo, @minsukkahng for fixing graph-mode broadcasting backprop for arithmetic, @shancarter for the awesome fonts demo and for giving us the idea to implement operation chaining, @jameswex for the fonts demo, @experiencor for mobilenet and yolo models, @easadler for conv1d, @LukasDrgon for adding jsdeliver CDN links, @pyu10055 for work on dl.pow and dl.squeeze, @wang2bo2 for work on dl.prelu, @vuoristo for work on softmax gradients, @haasdo95 for fixing a bug in our docs, @jimbojw for helping with docs, @iaroslav-ai for helping with docs

Finally, welcome to the team @tafsiri, @nkreeger and @pyu10055!