02_linear_regression.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "02-linear-regression.ipynb",
      "provenance": [],
      "authorship_tag": "ABX9TyO6E9oScoVv6wKKHDThPkYz",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/rename-z/Machine-Learning/blob/master/02_linear_regression.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "O8Ja0_NIiXBw",
        "colab_type": "text"
      },
      "source": [
        "## **Linear Regression with PyTorch**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7BqK63rMknv8",
        "colab_type": "text"
      },
      "source": [
        "*Linear regression*. We'll create a model that predicts crop yields for apples and oranges (*target variables*) by looking at the average temperature, rainfall and humidity (*input variables or features*) in a region. Here's the training data:\n",
        "\n",
        "![linear-regression-training-data](https://i.imgur.com/6Ujttb4.png)\n",
        "\n",
        "In a linear regression model, each target variable is estimated to be a weighted sum of the input variables, offset by some constant, known as a bias :\n",
        "\n",
        "```\n",
        "yield_apple  = w11 * temp + w12 * rainfall + w13 * humidity + b1\n",
        "yield_orange = w21 * temp + w22 * rainfall + w23 * humidity + b2\n",
        "```\n",
        "\n",
        "Visually, it means that the yield of apples is a linear or planar function of temperature, rainfall and humidity:\n",
        "\n",
        "![linear-regression-graph](https://i.imgur.com/4DJ9f8X.png)\n",
        "\n",
        "The *learning* part of linear regression is to figure out a set of weights `w11, w12,... w23, b1 & b2` by looking at the training data, to make accurate predictions for new data (i.e. to predict the yields for apples and oranges in a new region using the average temperature, rainfall and humidity). This is done by adjusting the weights slightly many times to make better predictions, using an optimization technique called *gradient descent*."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Fnt_o49LGFdw",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import numpy as np\n",
        "import torch"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "HLtKu6u1JAur",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Input (temp, rainfall, humidity)\n",
        "inputs = np.array([[73, 67, 43],\n",
        "                   [91, 88, 64],\n",
        "                   [87, 134, 58],\n",
        "                   [102, 43, 37],\n",
        "                   [69, 96, 70]], dtype='float32')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Ax8h2_llJw0F",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Target (apple, orange)\n",
        "targets = np.array([[56, 70],\n",
        "                    [81, 101],\n",
        "                    [119, 133],\n",
        "                    [22, 37],\n",
        "                    [103, 119]], dtype='float32')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "sg9DMu9OKUbj",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Converting inputs and targets to tensors\n",
        "inputs = torch.from_numpy(inputs)\n",
        "targets = torch.from_numpy(targets)\n",
        "print(inputs)\n",
        "print()\n",
        "print(targets)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VCrYKaegLSqX",
        "colab_type": "text"
      },
      "source": [
        "# ***Linear regression model from scratch***"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "oo7R3Dz2KzkY",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# weights and biases\n",
        "w = torch.randn(2, 3, requires_grad=True)\n",
        "b = torch.randn(2, requires_grad=True)\n",
        "print(w)\n",
        "print(b)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3Jl-OUBCL1um",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def model(x):\n",
        "    return x @ w.t() + b"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "MpfR8Z04MV-0",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "preds = model(inputs)\n",
        "print(preds)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "kmGXAndQMf-f",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "print(targets)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BLaO7HbYMkrv",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# MSE\n",
        "def mse(t1, t2):\n",
        "    diff = t1 - t2\n",
        "    return torch.sum(diff * diff) / diff.numel()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "710ZbsEANfn0",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "loss = mse(preds, targets)\n",
        "loss"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rNM7awyfNoJh",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Compute Gradients\n",
        "loss.backward()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BAOG1JF_OJXp",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Gradient for weights\n",
        "print(w)\n",
        "print(w.grad)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0CopOYLBOUm4",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "w.grad.zero_()\n",
        "b.grad.zero_()\n",
        "print(w.grad)\n",
        "print(b.grad)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "S9pjIyBLO2mj",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "preds = model(inputs)\n",
        "print(preds)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "QmCpP7fKPNbh",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "loss = mse(preds, targets)\n",
        "print(loss)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "OE5wdDkcPW2s",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "loss.backward()\n",
        "print(w.grad)\n",
        "print(b.grad)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_lHJEnunPlKw",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Adjust weight & reset gradients\n",
        "with torch.no_grad():\n",
        "    w -= w.grad * 1e-5\n",
        "    b -= b.grad * 1e-5\n",
        "    w.grad.zero_()\n",
        "    b.grad.zero_()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cmDC7aR2QS4H",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "print(w)\n",
        "print(b)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bA_-4v90QeW-",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "preds = model(inputs)\n",
        "loss = mse(preds, targets)\n",
        "print(loss)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gB-chj5RQu7C",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "for i in range(100):\n",
        "    preds = model(inputs)\n",
        "    loss = mse(preds, targets)\n",
        "    loss.backward()\n",
        "    with torch.no_grad():\n",
        "        w -= w.grad * 1e-5\n",
        "        b -= b.grad * 1e-5\n",
        "        w.grad.zero_()\n",
        "        b.grad.zero_()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "MBNVwrKnRgxI",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "preds = model(inputs)\n",
        "loss = mse(preds, targets)\n",
        "print(loss)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Gqx1Ffu3SIYw",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "preds"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "yv8MC7PrSZpa",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "targets"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "y4so0KQYSp1d",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    }
  ]
}