MachineLearning.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#  Coursera: [Machine Learning](https://www.coursera.org/learn/machine-learning)\n",
    "\n",
    "Created by Andrew Ng / Stanford University\n",
    "\n",
    "## About This Course\n",
    "Machine learning is the science of getting computers to act without being explicitly programmed. In the past decade, machine learning has given us self-driving cars, practical speech recognition, effective web search, and a vastly improved understanding of the human genome. Machine learning is so pervasive today that you probably use it dozens of times a day without knowing it. Many researchers also think it is the best way to make progress towards human-level AI.\n",
    "\n",
    "In this class, you will learn about the most effective machine learning techniques, and gain practice implementing them and getting them to work for yourself. More importantly, you'll learn about not only the theoretical underpinnings of learning, but also gain the practical know-how needed to quickly and powerfully apply these techniques to new problems. Finally, you'll learn about some of Silicon Valley's best practices in innovation as it pertains to machine learning and AI.\n",
    "\n",
    "This course provides a broad introduction to machine learning, datamining, and statistical pattern recognition. Topics include:\n",
    "* (i) Supervised learning (parametric/non-parametric algorithms, support vector machines, kernels, neural networks).\n",
    "* (ii) Unsupervised learning (clustering, dimensionality reduction, recommender systems, deep learning).\n",
    "* (iii) Best practices in machine learning (bias/variance theory; innovation process in machine learning and AI).\n",
    "\n",
    "The course will also draw from numerous case studies and applications, so that you'll also learn how to apply learning algorithms to building smart robots (perception, control), text understanding (web search, anti-spam), computer vision, medical informatics, audio, database mining, and other areas."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## WEEK 1\n",
    "\n",
    "### [Introduction](MachineLearning-01.ipynb#Introduction)\n",
    "\n",
    "Welcome to Machine Learning! In this module, we introduce the core idea of teaching a computer to learn concepts using data—without being explicitly programmed.\n",
    "\n",
    "   * Introduction\n",
    "      * Welcome\n",
    "      * What is Machine Learning?\n",
    "      * Supervised Learning\n",
    "      * Unsupervised Learning\n",
    "   * Review\n",
    "\n",
    "### [Linear Regression with One Variable](MachineLearning-01.ipynb#Linear-Regression-with-One-Variable)\n",
    "Linear regression predicts a real-valued output based on an input value. We discuss the application of linear regression to housing price prediction, present the notion of a cost function, and introduce the gradient descent method for learning.\n",
    "\n",
    "   * Model and Cost Function\n",
    "      * Model Representation\n",
    "      * Cost Function\n",
    "      * Cost Function - Intuition I\n",
    "      * Cost Function - Intuition II\n",
    "   * Parameter Learning\n",
    "      * Gradient Descent\n",
    "      * Gradient Descent Intuition\n",
    "      * Gradient Descent For Linear Regression\n",
    "   * Review\n",
    "   \n",
    "### [Linear Algebra Review](MachineLearning-01.ipynb#Linear-Algebra-Review)\n",
    "This optional module provides a refresher on linear algebra concepts. Basic understanding of linear algebra is necessary for the rest of the course, especially as we begin to cover models with multiple variables.\n",
    "\n",
    "   * Linear Algebra Review\n",
    "      * Matrices and Vectors\n",
    "      * Addition and Scalar Multiplication\n",
    "      * Matrix Vector Multiplication\n",
    "      * Matrix Matrix Multiplication\n",
    "      * Matrix Multiplication Properties\n",
    "      * Inverse and Transpose\n",
    "   * Review"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## WEEK 2\n",
    "\n",
    "### [Linear Regression with Multiple Variables](MachineLearning-02.ipynb#Linear-Regression-with-Multiple-Variables)\n",
    "What if your input has more than one value? In this module, we show how linear regression can be extended to accommodate multiple input features. We also discuss best practices for implementing linear regression.\n",
    "\n",
    "   * Multivariate Linear Regression\n",
    "      * Multiple Features\n",
    "      * Gradient Descent for Multiple Variables\n",
    "      * Gradient Descent in Practice I - Feature Scaling\n",
    "      * Gradient Descent in Practice II - Learning Rate\n",
    "      * Features and Polynomial Regression\n",
    "   * Computing Parameters Analytically\n",
    "      * Normal Equation\n",
    "      * Normal Equation Noninvertibility\n",
    "   * Review\n",
    "\n",
    "### [Octave/Matlab Tutorial](MachineLearning-02.ipynb#Octave/Matlab-Tutorial)\n",
    "This course includes programming assignments designed to help you understand how to implement the learning algorithms in practice. To complete the programming assignments, you will need to use Octave or MATLAB. This module introduces Octave/Matlab and shows you how to submit an assignment.\n",
    "\n",
    "   * Review"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## WEEK 3\n",
    "\n",
    "### [Logistic Regression](MachineLearning-03.ipynb#Logistic-Regression)\n",
    "Logistic regression is a method for classifying data into discrete outcomes. For example, we might use logistic regression to classify an email as spam or not spam. In this module, we introduce the notion of classification, the cost function for logistic regression, and the application of logistic regression to multi-class classification.\n",
    "\n",
    "   * Classification and Representation\n",
    "      * Classification\n",
    "      * Hypothesis Representation\n",
    "      * Decision Boundary\n",
    "   * Logistic Regression Model\n",
    "      * Cost Function\n",
    "      * Simplified Cost Function and Gradient Descent\n",
    "      * Advanced Optimization\n",
    "   * Multiclass Classification\n",
    "      * Multiclass Classification: One-vs-all\n",
    "   * Review   \n",
    "\n",
    "### [Regularization](MachineLearning-03.ipynb#Regularization)\n",
    "Machine learning models need to generalize well to new examples that the model has not seen in practice. In this module, we introduce regularization, which helps prevent models from overfitting the training data.\n",
    "\n",
    "   * Solving the Problem of Overfitting\n",
    "      * Cost Function\n",
    "      * Regularized Linear Regression\n",
    "      * Regularized Logistic Regression\n",
    "   * Review"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## WEEK 4\n",
    "\n",
    "### [Neural Networks: Representation](MachineLearning-04.ipynb#Neural-Networks:-Representation)\n",
    "Neural networks is a model inspired by how the brain works. It is widely used today in many applications: when your phone interprets and understand your voice commands, it is likely that a neural network is helping to understand your speech; when you cash a check, the machines that automatically read the digits also use neural networks.\n",
    "\n",
    "   * Motivations\n",
    "      * Non-linear Hypotheses\n",
    "      * Neurons and the Brain\n",
    "   * Neural Networks\n",
    "      * Model Representation I\n",
    "      * Model Representation II\n",
    "   * Applications\n",
    "      * Examples and Intuitions I\n",
    "      * Examples and Intuitions II\n",
    "      * Multiclass Classification\n",
    "   * Review"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## WEEK 5\n",
    "\n",
    "### [Neural Networks: Learning](MachineLearning-05.ipynb#Neural-Networks:-Learning)\n",
    "In this module, we introduce the backpropagation algorithm that is used to help learn parameters for a neural network. At the end of this module, you will be implementing your own neural network for digit recognition.\n",
    "\n",
    "   * Cost Function and Backpropagation\n",
    "      * Cost Function\n",
    "      * Backpropagation Algorithm\n",
    "      * Backpropagation Intuition\n",
    "   * Backpropagation in Practice\n",
    "      * Implementation Note: Unrolling Parameters\n",
    "      * Gradient Checking\n",
    "      * Random Initialization\n",
    "      * Putting it Together\n",
    "   * Application of Neural Networks\n",
    "      * Autonomous Driving\n",
    "   * Review"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## WEEK 6\n",
    "\n",
    "### [Advice for Applying Machine Learning](MachineLearning-06.ipynb#Advice-for-Applying-Machine-Learning)\n",
    "Applying machine learning in practice is not always straightforward. In this module, we share best practices for applying machine learning in practice, and discuss the best ways to evaluate performance of the learned models.\n",
    "\n",
    "   * Evaluating a Learning Algorithm\n",
    "      * Evaluating a Hypothesis\n",
    "      * Model Selection and Train/Validation/Test Sets\n",
    "   * Bias vs. Variance\n",
    "      * Diagnosing Bias vs. Variance\n",
    "      * Regularization and Bias/Variance\n",
    "      * Learning Curves\n",
    "      * Deciding What to do Next Revisited\n",
    "   * Review\n",
    "   \n",
    "### [Machine Learning System Design](MachineLearning-06.ipynb#Machine-Learning-System-Design)\n",
    "To optimize a machine learning algorithm, you’ll need to first understand where the biggest improvements can be made. In this module, we discuss how to understand the performance of a machine learning system with multiple parts, and also how to deal with skewed data.\n",
    "\n",
    "   * Building a Spam Classifier\n",
    "      * Prioritizing What to Work On\n",
    "      * Error Analysis\n",
    "   * Handling Skewed Data\n",
    "      * Error Metrics for Skewed Classes\n",
    "      * Trading Off Precision and Recall\n",
    "   * Using Large Data Sets\n",
    "      * Data For Machine Learning\n",
    "   * Review"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## WEEK 7\n",
    "\n",
    "### [Support Vector Machines](MachineLearning-07.ipynb#Support-Vector-Machines)\n",
    "Support vector machines, or SVMs, is a machine learning algorithm for classification. We introduce the idea and intuitions behind SVMs and discuss how to use it in practice.\n",
    "\n",
    "   * Large Margin Classification\n",
    "      * Optimization Objective\n",
    "      * Large Margin Intuition\n",
    "      * Mathematics Behind Large Margin Classification\n",
    "   * Kernels\n",
    "      * Kernels I\n",
    "      * Kernels II\n",
    "   * SVMs in Practice\n",
    "      * Using An SVM\n",
    "   * Review"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## WEEK 8\n",
    "\n",
    "### [Unsupervised Learning](MachineLearning-08.ipynb#Unsupervised-Learning)\n",
    "We use unsupervised learning to build models that help us understand our data better. We discuss the k-Means algorithm for clustering that enable us to learn groupings of unlabeled data points.\n",
    "\n",
    "   * Clustering\n",
    "      * Unsupervised Learning: Introduction\n",
    "      * K-Means Algorithm\n",
    "      * Optimization Objective\n",
    "      * Random Initialization\n",
    "      * Choosing the Number of Clusters\n",
    "   * Review\n",
    "\n",
    "### [Dimensionality Reduction](MachineLearning-08.ipynb#Dimensionality-Reduction)\n",
    "In this module, we introduce Principal Components Analysis, and show how it can be used for data compression to speed up learning algorithms as well as for visualizations of complex datasets.\n",
    "\n",
    "   * Motivation\n",
    "      * Motivation I: Data Compression\n",
    "      * Motivation II: Visualization\n",
    "   * Principal Component Analysis\n",
    "      * Principal Component Analysis Problem Formulation\n",
    "      * Principal Component Analysis Algorithm\n",
    "   * Applying PCA\n",
    "      * Reconstruction from Compressed Representation\n",
    "      * Choosing the Number of Principal Components\n",
    "      * Advice for Applying PCA\n",
    "   * Review"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## WEEK 9\n",
    "\n",
    "### [Anomaly Detection](MachineLearning-09.ipynb#Anomaly-Detection)\n",
    "Given a large number of data points, we may sometimes want to figure out which ones vary significantly from the average. For example, in manufacturing, we may want to detect defects or anomalies. We show how a dataset can be modeled using a Gaussian distribution, and how the model can be used for anomaly detection.\n",
    "\n",
    "   * Density Estimation\n",
    "      * Problem Motivation\n",
    "      * Gaussian Distribution\n",
    "      * Algorithm\n",
    "   * Building an Anomaly Detection System\n",
    "      * Developing and Evaluating an Anomaly Detection System\n",
    "      * Anomaly Detection vs. Supervised Learning\n",
    "      * Choosing What Features to Use\n",
    "   * Multivariate Gaussian Distribution (Optional)\n",
    "      * Multivariate Gaussian Distribution\n",
    "      * Anomaly Detection using the Multivariate Gaussian Distribution\n",
    "   * Review\n",
    "\n",
    "### [Recommender Systems](MachineLearning-09.ipynb#Recommender-Systems)\n",
    "When you buy a product online, most websites automatically recommend other products that you may like. Recommender systems look at patterns of activities between different users and different products to produce these recommendations. In this module, we introduce recommender algorithms such as the collaborative filtering algorithm and low-rank matrix factorization.\n",
    "\n",
    "   * Predicting Movie Ratings\n",
    "      * Problem Formulation\n",
    "      * Content Based Recommendations\n",
    "   * Collaborative Filtering\n",
    "      * Collaborative Filtering\n",
    "      * Collaborative Filtering Algorithm\n",
    "   * Low Rank Matrix Factorization\n",
    "      * Vectorization: Low Rank Matrix Factorization\n",
    "      * Implementational Detail: Mean Normalization\n",
    "   * Review"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## WEEK 10\n",
    "\n",
    "### [Large Scale Machine Learning](MachineLearning-10.ipynb#Large-Scale-Machine-Learning)\n",
    "Machine learning works best when there is an abundance of data to leverage for training. In this module, we discuss how to apply the machine learning algorithms with large datasets.\n",
    "\n",
    "   * Gradient Descent with Large Datasets\n",
    "      * Learning With Large Datasets\n",
    "      * Stochastic Gradient Descent\n",
    "      * Mini-Batch Gradient Descent\n",
    "      * Stochastic Gradient Descent Convergence\n",
    "   * Advanced Topics\n",
    "      * Online Learning\n",
    "      * Map Reduce and Data Parallelism\n",
    "   * Review"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## WEEK 11\n",
    "\n",
    "### [Application Example: Photo OCR](MachineLearning-11.ipynb#Application-Example:-Photo-OCR)\n",
    "Identifying and recognizing objects, words, and digits in an image is a challenging task. We discuss how a pipeline can be built to tackle this problem and how to analyze and improve the performance of such a system.\n",
    "\n",
    "   * Photo OCR\n",
    "      * Problem Description and Pipeline\n",
    "      * Sliding Windows\n",
    "      * Getting Lots of Data and Artificial Data\n",
    "      * Ceiling Analysis: What Part of the Pipeline to Work on Next\n",
    "   * Review\n",
    "\n",
    "### [Conclusion](MachineLearning-11.ipynb#Conclusion)\n",
    "Conclusion and thank you.\n",
    "\n",
    "   * Summary and Thank You"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}