Knet-the-Julia-dope

Abstract

This repo is the Julia translation of the mxnet-the-straight-dope repo, a collection of notebooks designed to teach deep learning, MXNet, and the gluon interface. This project grew out of the MIT course 6.338 Modern Numerical Computing with Julia taught by professor Alan Edelman. Our main objectives are:

• Introduce the Julia language and its main packages in the context of deep learning
• Introduce Julia's package Knet: an alternative/complementary option to MXNet
• Leverage the strengths of Jupyter notebooks to present prose, graphics, equations, and code together in one place

Wherever possible we provide Julia's version of any particular code. However, in many instances Julia can achieve the same goal with fewer or different commands such that a direct (or one two one) translation is either impossible or cumbersome. Thus, at our discretion we may replace large blocks of code with a significanly different Julia version, though our objective is to stay true to the essense and goal of the original work.

In addition to offering a complementary Julia version, wherever possible we also replace or modify examples to experiment with different datasets (with a focus on medical data) for any particular topic (in which case the reader is referred to the original example for context), and we offer additional theoretical explanations whenever approapiate. Finally, with the same spirit of its predecessor, we welcome contributions from the community and hope to coauthor chapters and entire sections with experts and community members.

New Datasets

In this repo we implement new examples with different datasets, including:

• Simulated cardiac MRI CINE images
• Cardiac Electrocardiograms
• Medical appointment dataset

Implementation with Knet

Throughout this book, we rely upon Julia's package Knet. Knet relies on the AutoGrad package and the KnetArray data type for its functionality and performance. AutoGrad computes the gradient of Julia functions and KnetArray implements high performance GPU arrays with custom memory management.

We make heavy use of packages like JuliaDB, Plots and the plotting backend GR,HDF5, Distributions, and ofcourse Knet

Dependencies

To run these notebooks, you'll want to install Julia and add all the required packages (this is done automatically for you at the start of every notebook) that we will use throughout this tutorial. Fortunately, after installing Julia this is very easy with the command Pkg.add('PackageName'). You'll also want to install IJulia, a Julia kernel for Jupyter.

Table of contents

Part 1: Deep Learning Fundamentals

• Chapter 1: Crash course

  • Preface
  • Introduction
  • Manipulating data with KnetArray
  • Linear algebra
  • Probability and statistics
  • Automatic differentiation via autograd

• Chapter 2: Introduction to supervised learning

  • introduction
  • linear-regression
  • logistic-regression
  • multiclass-logistic-regression
  • regularization
  • perceptron
  • learning-environments

• Chapter 3: Deep neural networks (DNNs)

  • multilayer perceptrons
  • dropout

• Chapter 4: Convolutional neural networks (CNNs)

  • cnn

• Chapter 5: Recurrent neural networks (RNNs)

  • MLP
  • LSTM

Part 2: Applications

• Chapter 12: Time series

  • Exponential Smoothing and Innovation State Space Model

• Chapter 14: Generative adversarial networks

  • introduction
  • GANs
  • GANs in less than 50 lines of code with Julia Knet