Deep-Learning/Part-I/Chap-02-Linear-Algebra/02-09-The-Moore-Penrose-Pseudoinverse.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2.9 Moore-Penrose 伪逆"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "对于非方阵，矩阵的逆是没有定义的。\n",
    "\n",
    "假设我们希望找到 $\\bf A$ 的一个左逆 $\\bf B$，这样我们可以通过两边左乘 $\\bf B$ 解方程：\n",
    "\n",
    "$$\n",
    "\\bf Ax=y, x = By\n",
    "$$\n",
    "\n",
    "我们可以定义 Moore-Penrose 伪逆如下：\n",
    "\n",
    "$$\n",
    "\\mathbf A^+ = \\lim_{\\alpha\\to 0}(\\mathbf{A^\\top A}+\\alpha\\mathbf I)^{-1}\\mathbf A^\\top\n",
    "$$\n",
    "\n",
    "计算上，我们有\n",
    "\n",
    "$$\n",
    "\\bf A^+ = VD^+U^\\top\n",
    "$$\n",
    "\n",
    "其中 $\\bf U, D, V$ 对应于 $\\bf A$ 的奇异值分解的成分，$D^+$ 对应是将非零奇异值取倒数之后得到的矩阵的转置。\n",
    "\n",
    "当矩阵 $\\bf A$ 的列比行多时，可能有很多解，使用 Moore-Penrose 伪逆求解得到的结果是 $\\|\\mathbf x\\|_2$ 最小的结果；当矩阵 $\\bf A$ 的行比列多时，可能没有解，使用 Moore-Penrose 伪逆求解得到的结果是 $\\|\\mathbf{Ax-y}\\|_2$ 最小的结果。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}