06-深度学习框架/PyTorch教程1-Tensor核心知识点.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "7764ee08",
   "metadata": {},
   "source": [
    "&emsp;&emsp;本小结主要是对PyTorch官网教程的翻译。原文链接为：\n",
    "\n",
    "1. https://pytorch.org/tutorials/beginner/basics/tensorqs_tutorial.html"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "126b6990",
   "metadata": {},
   "source": [
    "- PyTorch官网：[https://pytorch.org/](https://pytorch.org/)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "58650c8a",
   "metadata": {},
   "source": [
    "## Tensor"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ab2e31b2",
   "metadata": {},
   "source": [
    "&emsp;&emsp;`Tensor`是一种与`arrays`非常相似的数据结构，在`PyTorch`中采用这种数据结构来编码模型的输入输出和网络参数。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "61494c4a",
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e94bbfbe",
   "metadata": {},
   "source": [
    "### 初始化一个Tensor"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0f4439ba",
   "metadata": {},
   "source": [
    "#### 从列表中进行创建"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "b7cc6943",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "x_data格式如下:\n",
      " tensor([[1., 2.],\n",
      "        [3., 4.]])\n"
     ]
    }
   ],
   "source": [
    "data = [[1, 2], [3, 4.]]\n",
    "x_data = torch.tensor(data)\n",
    "print(\"x_data格式如下:\\n {}\".format(x_data))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f57de4ec",
   "metadata": {},
   "source": [
    "#### 从其它tensor中创建"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "c3ce726b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "t_ones is tensor([1., 1., 1., 1.])\n",
      "t_rand is tensor([0.0151, 0.0892, 0.5934, 0.2087])\n"
     ]
    }
   ],
   "source": [
    "t = torch.tensor([1, 2, 3, 4.])\n",
    "t_ones = torch.ones_like(t)\n",
    "t_rand = torch.rand_like(t)\n",
    "print(\"t_ones is {}\".format(t_ones))\n",
    "print(\"t_rand is {}\".format(t_rand))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "49339e41",
   "metadata": {},
   "source": [
    "#### tensor中的属性"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "50c6b671",
   "metadata": {},
   "source": [
    "&emsp;&emsp;`tensor`有个`dtype`属性，表示`tensor`里面数据的数据类型。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "e758b3bd",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "从list转变过来的元素类型为 : torch.float32\n",
      "从 Numpy array转变过来的元素类型为 :torch.float64\n"
     ]
    }
   ],
   "source": [
    "t = torch.tensor([1, 2, 3, 4.])\n",
    "print(\"从list转变过来的元素类型为 : {}\".format(x_data.dtype))\n",
    "\n",
    "np_array = np.array([1., 2, 3, 4])\n",
    "x_np = torch.from_numpy(np_array)\n",
    "print(\"从 Numpy array转变过来的元素类型为 :{}\".format(x_np.dtype))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f7461666",
   "metadata": {},
   "source": [
    "&emsp;&emsp;从上述运行结果可以看到，如果不指定`tensor`中的数据类型的话，`PyTorch`中默认从`NumPy`转化过来的数据类型其实是`float64`的。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fe6a1c75",
   "metadata": {},
   "source": [
    "&emsp;&emsp;除了数据类型需要注意以外，这里还有一些其它的属性需要注意："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "4934314a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor的shape为: torch.Size([4])\n",
      "tensor的存储属性为: cpu\n"
     ]
    }
   ],
   "source": [
    "t = torch.tensor([1, 2, 3, 4.])\n",
    "print(\"tensor的shape为: {}\".format(t.shape))\n",
    "print(\"tensor的存储属性为: {}\".format(t.device))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "894a87f4",
   "metadata": {},
   "source": [
    "## Tensor与NumPy共享内存\n",
    "\n",
    "&emsp;&emsp;`Tensors`和`NumPy ndarrys`可以共享同一块内存。如果我们在`PyTorch`中改变`tensor`数据的值的话，那么相对应`NumPy`的数组也会改变。反过来，如果去改变`Numpy`对应的`array`数组的值的话，这个`Tensor`本身也会发生改变。只要这个改变是原地操作的都可以。\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "268cd4b0",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "t : tensor([1, 2, 3, 4])\n",
      "n : [1 2 3 4]\n"
     ]
    }
   ],
   "source": [
    "t = torch.tensor([1, 2, 3, 4])\n",
    "n = t.numpy()\n",
    "print(\"t : {}\".format(t))\n",
    "print(\"n : {}\".format(n))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4762c17c",
   "metadata": {},
   "source": [
    "&emsp;&emsp;当对其中一个元素进行累加时，另一个元素也会发生变化。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "c31d3b8a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "t : tensor([2, 3, 4, 5])\n",
      "n : [2 3 4 5]\n"
     ]
    }
   ],
   "source": [
    "t.add_(1)\n",
    "print(\"t : {}\".format(t))\n",
    "print(\"n : {}\".format(n))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "533c2183",
   "metadata": {},
   "source": [
    "## Tensor操作运算符"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dd0fe044",
   "metadata": {},
   "source": [
    "&emsp;&emsp;`PyTorch`中提供了有超过`100`中`tensor`的运算符，包括转置、索引、切片、数学运算、线性代数、随机采样。\n",
    "\n",
    "&emsp;&emsp;在如下链接：[https://pytorch.org/docs/stable/torch.html](https://pytorch.org/docs/stable/torch.html)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c3088ae5",
   "metadata": {},
   "source": [
    "### 常用Tensors相关运算符"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "53c51794",
   "metadata": {},
   "source": [
    "#### is_tensor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "dfb9ad6f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "t是tensor: True\n"
     ]
    }
   ],
   "source": [
    "t = torch.tensor([1, 2, 3, 4.])\n",
    "print(\"t是tensor: {}\".format(torch.is_tensor(t)))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6621e535",
   "metadata": {},
   "source": [
    "&emsp;&emsp;对于这个`is_tensor`运算符来说，源码里面就是非常简单的`isinstance`判断的。\n",
    "\n",
    "```python\n",
    "def is_tensor(obj):\n",
    "    r\"\"\"Returns True if `obj` is a PyTorch tensor.\n",
    "\n",
    "    Note that this function is simply doing ``isinstance(obj, Tensor)``.\n",
    "    Using that ``isinstance`` check is better for typechecking with mypy,\n",
    "    and more explicit - so it's recommended to use that instead of\n",
    "    ``is_tensor``.\n",
    "\n",
    "    Args:\n",
    "        obj (Object): Object to test\n",
    "    Example::\n",
    "\n",
    "        >>> x=torch.tensor([1,2,3])\n",
    "        >>> torch.is_tensor(x)\n",
    "        True\n",
    "\n",
    "    \"\"\"\n",
    "    return isinstance(obj, torch.Tensor)\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dc76b04a",
   "metadata": {},
   "source": [
    "#### is_floating_point"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "37f351fe",
   "metadata": {},
   "source": [
    "&emsp;&emsp;`is_floating_point`操作符，判断是否是浮点类型。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "ccdfe1c4",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "True\n"
     ]
    }
   ],
   "source": [
    "print(torch.is_floating_point(t))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "47f894af",
   "metadata": {},
   "source": [
    "#### is_nonzero\n",
    "\n",
    "如果输入是一个单个元素的话，可以用来判断是否为0。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "be339da0",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.is_nonzero(torch.tensor([0.]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "31bb7813",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.is_nonzero(torch.tensor([0.5]))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "205d2b77",
   "metadata": {},
   "source": [
    "#### nonzero\n",
    "\n",
    "函数原型：\n",
    "\n",
    "```python\n",
    "torch.nonzero(input, *, out=None, as_tuple=False)\n",
    "```\n",
    "\n",
    "`as_tuple=False`时，输出结果的大小为$(z \\times n)$。`as_tuple=False`时，返回的是一个tuple。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "0e23dc8f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0],\n",
       "        [1],\n",
       "        [2],\n",
       "        [4]])"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.nonzero(torch.tensor([1, 1, 1, 0, 1]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "76af0f7b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0, 0],\n",
       "        [1, 1],\n",
       "        [2, 2],\n",
       "        [3, 3]])"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.nonzero(torch.tensor([[0.6, 0.0, 0.0, 0.0],\n",
    "                            [0.0, 0.4, 0.0, 0.0],\n",
    "                            [0.0, 0.0, 1.2, 0.0],\n",
    "                            [0.0, 0.0, 0.0, -0.4]]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "401b629e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(tensor([0, 1, 2, 4]),)"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.nonzero(torch.tensor([1, 1, 1, 0, 1]), as_tuple=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "b0c2c610",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(tensor([0, 1, 2, 3]), tensor([0, 1, 2, 3]))"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.nonzero(torch.tensor([[0.6, 0.0, 0.0, 0.0],\n",
    "                            [0.0, 0.4, 0.0, 0.0],\n",
    "                            [0.0, 0.0, 1.2, 0.0],\n",
    "                            [0.0, 0.0, 0.0, -0.4]]), as_tuple=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "35b35f97",
   "metadata": {},
   "source": [
    "#### set_default_tensor_type\n",
    "\n",
    "&emsp;&emsp;设置默认的`tensor`类型。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "5815c56f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.float32"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.tensor([1, 2.3]).dtype"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "52a1b538",
   "metadata": {},
   "outputs": [],
   "source": [
    "torch.set_default_tensor_type(torch.DoubleTensor)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "006d9a61",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.float64"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.tensor([1, 2.3]).dtype"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "748a2a48",
   "metadata": {},
   "source": [
    "#### numel"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "03eef8d3",
   "metadata": {},
   "source": [
    "&emsp;&emsp;返回输入`tensor`中元素的总个数。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "d250add0",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "24"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a = torch.randn(1, 2, 3, 4)\n",
    "torch.numel(a)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "503b8949",
   "metadata": {},
   "source": [
    "#### Unfold\n",
    "\n",
    "Unfold可以将卷积过程中的计算量拉直，比如如下示例，kernel的大小为2 x 3，输入数据的通道有5个，因此，每次卷积的计算量都是2 x 3 x 5 = 30个向量运算。把这样一次卷积的计算称为patch，每个patch包含30个值，假设整个卷积过程中padding=0，stride=1，那么对于输入为3 x 4的图像，kernel大小为2 x 3的话，需要滑动4次来做整个卷积过程，因此会有4个block，所以输出为[2, 30, 4]。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "79be7928",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([2, 30, 4])\n"
     ]
    }
   ],
   "source": [
    "import torch\n",
    "import torch.nn as nn\n",
    "\n",
    "unfold = nn.Unfold(kernel_size=(2, 3))\n",
    "input_data = torch.randn(2, 5, 3, 4)\n",
    "output = unfold(input_data)\n",
    "print(output.size())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "635c3464",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "7050e74c",
   "metadata": {},
   "source": [
    "### 创建tensor操作符"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6e646bff",
   "metadata": {},
   "source": [
    "#### zeros与zeros_like"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "37e1290b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0., 0., 0.],\n",
       "        [0., 0., 0.]])"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t = torch.zeros((2,3))\n",
    "t"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "cc92bee6",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0., 0., 0.],\n",
       "        [0., 0., 0.]])"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.zeros_like(t)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "56f46f8a",
   "metadata": {},
   "source": [
    "#### arange\n",
    "\n",
    "&emsp;&emsp;返回一个`1-D`的`tensor`，这个`tensor`的大小为$\\left\\lceil\\frac{\\text { end-start }}{\\text { step }}\\right\\rceil$，函数定义如下：\n",
    "\n",
    "```python\n",
    "torch.arange(start=0, end, step=1, *, out=None, dtype=None, layout=torch.strided, device=None,\n",
    "             requires_grad=False) \n",
    "```\n",
    "\n",
    "&emsp;&emsp;但是取不到`end`。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "62b56770",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([1.0000, 1.5000, 2.0000, 2.5000, 3.0000, 3.5000])"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.arange(1, 4, 0.5)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f40b3d25",
   "metadata": {},
   "source": [
    "#### range\n",
    "\n",
    "&emsp;&emsp;`range`相对于`arange`来说就是能够取到`end`这个元素。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "bd4c0fda",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/var/folders/x9/xgj8n5xd2bb0cv55w7c72bn40000gn/T/ipykernel_26931/3378922890.py:1: UserWarning: torch.range is deprecated and will be removed in a future release because its behavior is inconsistent with Python's range builtin. Instead, use torch.arange, which produces values in [start, end).\n",
      "  torch.range(1, 4, 0.5)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "tensor([1.0000, 1.5000, 2.0000, 2.5000, 3.0000, 3.5000, 4.0000])"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.range(1, 4, 0.5)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f671198c",
   "metadata": {},
   "source": [
    "#### linspace"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fdfb0afa",
   "metadata": {},
   "source": [
    "&emsp;&emsp;与`arange`和`range`不同，`linspace`是指定元素的个数:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "a7b1d9c8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([2.0000, 2.1667, 2.3333, 2.5000, 2.6667, 2.8333, 3.0000])"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.linspace(2, 3, 7)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "68030d38",
   "metadata": {},
   "source": [
    "#### eye\n",
    "\n",
    "&emsp;&emsp;`eye`的函数定义如下:\n",
    "\n",
    "```python\n",
    "torch.eye(n, m=None, *, out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False)\n",
    "```\n",
    "\n",
    "&emsp;&emsp;返回一个`2-D`的`tensor`，对角线上的元素是`1`，其它元素为`0`，需要给定的参数是行和列，如果列参数不给定的话，默认是一个方阵。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "fa46382e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[1., 0.],\n",
       "        [0., 1.]])"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.eye(2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "267d7ac2",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[1., 0., 0.],\n",
       "        [0., 1., 0.]])"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.eye(2, 3)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "84f3742e",
   "metadata": {},
   "source": [
    "#### full"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3b46c186",
   "metadata": {},
   "source": [
    "&emsp;&emsp;创建一个指定元素的`tensor`。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "233a45f8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[1.3000, 1.3000]])"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.full(size=(1, 2), fill_value=1.3)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eeb18695",
   "metadata": {},
   "source": [
    "#### tensor.masked_fill"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "2d05312f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[-1.9524, -0.3740, -1.3448, -0.2276,  0.7343],\n",
      "        [ 0.6656, -1.9865, -2.3492,  0.5301, -0.5501],\n",
      "        [-1.2254,  0.0061, -1.7755, -0.3869,  0.0285],\n",
      "        [ 1.1440,  0.0068,  0.6245,  0.5427,  0.5645],\n",
      "        [-0.3113, -0.9607, -0.0965,  1.3154,  0.9507]])\n"
     ]
    }
   ],
   "source": [
    "import torch\n",
    "from torch.autograd import Variable\n",
    "input_data = Variable(torch.randn(5, 5))\n",
    "print(input_data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "d960403b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[0., 0., 0., 0., 0.],\n",
      "        [0., 0., 0., 0., 0.],\n",
      "        [0., 0., 0., 0., 0.],\n",
      "        [0., 0., 0., 0., 0.],\n",
      "        [0., 0., 0., 0., 0.]])\n"
     ]
    }
   ],
   "source": [
    "mask = Variable(torch.zeros(5, 5))\n",
    "print(mask)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "d35e084a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[1.0000e-09, 1.0000e-09, 1.0000e-09, 1.0000e-09, 1.0000e-09],\n",
      "        [1.0000e-09, 1.0000e-09, 1.0000e-09, 1.0000e-09, 1.0000e-09],\n",
      "        [1.0000e-09, 1.0000e-09, 1.0000e-09, 1.0000e-09, 1.0000e-09],\n",
      "        [1.0000e-09, 1.0000e-09, 1.0000e-09, 1.0000e-09, 1.0000e-09],\n",
      "        [1.0000e-09, 1.0000e-09, 1.0000e-09, 1.0000e-09, 1.0000e-09]])\n"
     ]
    }
   ],
   "source": [
    "final_data = input_data.masked_fill(mask == 0, 1e-9)\n",
    "print(final_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bdcae31e",
   "metadata": {},
   "source": [
    "## 索引、切片、连接、转变操作符"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "457b049f",
   "metadata": {},
   "source": [
    "#### cat\n",
    "\n",
    "&emsp;&emsp;对给定的`tensor`序列的指定维度进行拼接，所有的`tensor`，除了待拼接的维度外，需要具有相同的`shape`。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "ad6c2f52",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 1.2783, -1.1765, -0.1997],\n",
       "        [-1.3430,  0.5486,  1.5526]])"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t = torch.randn(2, 3)\n",
    "t"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "95153c52",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 1.2783, -1.1765, -0.1997],\n",
       "        [-1.3430,  0.5486,  1.5526],\n",
       "        [ 1.2783, -1.1765, -0.1997],\n",
       "        [-1.3430,  0.5486,  1.5526]])"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.cat((t, t), dim=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "ac6db6fc",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 1.2783, -1.1765, -0.1997,  1.2783, -1.1765, -0.1997],\n",
       "        [-1.3430,  0.5486,  1.5526, -1.3430,  0.5486,  1.5526]])"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.cat((t, t), dim=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e6451a23",
   "metadata": {},
   "source": [
    "#### stack\n",
    "\n",
    "&emsp;&emsp;对`tensor`进行拼接，但是会具有新的维度。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "1ea8b62c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[[ 1.2783, -1.1765, -0.1997],\n",
       "         [-1.3430,  0.5486,  1.5526]],\n",
       "\n",
       "        [[ 1.2783, -1.1765, -0.1997],\n",
       "         [-1.3430,  0.5486,  1.5526]]])"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.stack((t, t), dim=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "b61be19d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([2, 2, 3])"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.stack((t, t), dim=0).size()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "a31f6b12",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[[ 1.2783, -1.1765, -0.1997],\n",
       "         [ 1.2783, -1.1765, -0.1997]],\n",
       "\n",
       "        [[-1.3430,  0.5486,  1.5526],\n",
       "         [-1.3430,  0.5486,  1.5526]]])"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.stack((t, t), dim=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "15505084",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([2, 2, 3])"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.stack((t, t), dim=1).size()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a342b5a4",
   "metadata": {},
   "source": [
    "#### chunk\n",
    "\n",
    "函数原型为：\n",
    "\n",
    "```python\n",
    "torch.chunk(input, chunks, dim=0)\n",
    "```\n",
    "\n",
    "如果输入的`tensor`在给定的`dim`维度能够被`chunks`整除的话，返回的所有的`chunks`将具备相同的大小。如果不能够被整除，那么最后一个将不具备之前的那些一样的大小。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "b7936ecf",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[-1.1992, -0.6293,  1.6096],\n",
       "        [ 1.6889,  0.4714,  0.3982]])"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t = torch.randn(2, 3)\n",
    "t"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "b0f0b9ab",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(tensor([[-1.1992, -0.6293,  1.6096]]), tensor([[1.6889, 0.4714, 0.3982]]))"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.chunk(t, 2, dim=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "43e6c54d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(tensor([[-1.1992, -0.6293],\n",
       "         [ 1.6889,  0.4714]]),\n",
       " tensor([[1.6096],\n",
       "         [0.3982]]))"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.chunk(t, 2, dim=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "17f299f4",
   "metadata": {},
   "source": [
    "#### gather\n",
    "\n",
    "函数原型为：\n",
    "\n",
    "```python\n",
    "torch.gather(input, dim, index, *, sparse_grad=False, out=None)\n",
    "```\n",
    "\n",
    "对于一个给定的`3-D`的`tensor`，输出可以表示为：\n",
    "\n",
    "```bash\n",
    "out[i][j][k] = input[index[i][j][k]][j][k]  # if dim == 0\n",
    "out[i][j][k] = input[i][index[i][j][k]][k]  # if dim == 1\n",
    "out[i][j][k] = input[i][j][index[i][j][k]]  # if dim == 2\n",
    "```\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "e9debccd",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[1, 2],\n",
       "        [3, 4]])"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t = torch.tensor([[1, 2], [3, 4]])\n",
    "t"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "028de834",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[1, 2],\n",
       "        [3, 2]])"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.gather(t, 0, torch.tensor([[0, 0], [1, 0]]))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cd08d6c8",
   "metadata": {},
   "source": [
    "&emsp;&emsp;解释一下上述的输出结果：给定的`index`是`[[0, 0], [1, 0]]`，`dim=0`，所以对应输出为：\n",
    "```python\n",
    "1. out[0][0] = input[index[0][0]][0] = input[0][0] = 1\n",
    "2. out[0][1] = input[index[0][1]][1] = input[0][1] = 2\n",
    "3. out[1][0] = input[index[1][0]][0] = input[1][0] = 3\n",
    "4. out[1][1] = input[index[1][1]][0] = input[0][1] = 2\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "e3b388ea",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[1, 1],\n",
       "        [4, 3]])"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.gather(t, 1, torch.tensor([[0, 0], [1, 0]]))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ba474028",
   "metadata": {},
   "source": [
    "&emsp;&emsp;解释一下上述的输出结果：给定的`index`是`[[0, 0], [1, 0]]`，`dim=1`，所以对应输出为：\n",
    "\n",
    "```python\n",
    "1. out[0][0] = input[0][index[0][0]] = input[0][0] = 1\n",
    "2. out[0][1] = input[0][index[0][1]] = input[0][0] = 1\n",
    "3. out[1][0] = input[1][index[1][0]] = input[1][1] = 4\n",
    "4. out[1][1] = input[1][index[1][1]] = input[1][0] = 3\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8777bd90",
   "metadata": {},
   "source": [
    "#### scatter_\n",
    "\n",
    "函数原型为：\n",
    "\n",
    "```python\n",
    "scatter_(dim, index, src, reduce=None) \n",
    "```\n",
    "\n",
    "从src中按照index索引写入到self中，self的更新公式为：\n",
    "\n",
    "\n",
    "```python\n",
    "self[index[i][j][k]][j][k] = src[i][j][k]  # if dim == 0\n",
    "self[i][index[i][j][k]][k] = src[i][j][k]  # if dim == 1\n",
    "self[i][j][index[i][j][k]] = src[i][j][k]  # if dim == 2\n",
    "```\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "2b3f474e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 1,  2,  3,  4,  5],\n",
       "        [ 6,  7,  8,  9, 10]])"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "src = torch.arange(1, 11).reshape((2, 5))\n",
    "src"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "id": "332e1dcd",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0, 1, 2, 0]])"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "index = torch.tensor([[0, 1, 2, 0]])\n",
    "index"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "id": "52fb277a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[1, 0, 0, 4],\n",
       "        [0, 2, 0, 0],\n",
       "        [0, 0, 3, 0]])"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.zeros(3, 4, dtype=src.dtype).scatter_(0, index, src)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1d0a5476",
   "metadata": {},
   "source": [
    "&emsp;&emsp;解释一下上述的输出结果：给定的`index`是`[[0, 1, 2, 0]]`，`dim=0`，所以对应输出为：\n",
    "\n",
    "```python\n",
    "self[index[0][0]][0] = self[0][0] = src[0][0] = 1\n",
    "self[index[0][1]][1] = self[1][1] = src[0][1] = 2\n",
    "self[index[0][2]][2] = self[2][2] = src[0][2] = 3\n",
    "self[index[0][3]][3] = self[0][3] = src[0][3] = 4\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "id": "fd7f93c6",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[4, 2, 3, 0],\n",
       "        [0, 0, 0, 0],\n",
       "        [0, 0, 0, 0]])"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.zeros(3, 4, dtype=src.dtype).scatter_(1, index, src)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "64869857",
   "metadata": {},
   "source": [
    "&emsp;&emsp;解释一下上述的输出结果：给定的`index`是`[[0, 1, 2, 0]]`，`dim=1`，所以对应输出为：\n",
    "\n",
    "```python\n",
    "self[0][index[0][0]] = self[0][0] = src[0][0] = 1\n",
    "self[0][index[0][1]] = self[0][1] = src[0][1] = 2\n",
    "self[0][index[0][2]] = self[0][2] = src[0][2] = 3\n",
    "self[0][index[0][3]] = self[0][0] = src[0][3] = 4\n",
    "```\n",
    "\n",
    "&emsp;&emsp;`self[0][0]`之所以会为`4`不是`1`，是因为，它之后会被`src[0][3] = 4`覆盖。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d7d87d3d",
   "metadata": {},
   "source": [
    "#### transpose\n",
    "\n",
    "对原`tensor`进行维度转化：\n",
    "\n",
    "```python\n",
    "torch.transpose(input, dim0, dim1)\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "id": "5eee6f1b",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[-0.3123, -1.7891, -0.9699],\n",
       "        [ 1.5250,  1.2797,  1.0574]])"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t = torch.randn(2, 3)\n",
    "t"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "id": "f764b047",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([2, 3])"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t.size()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "id": "30929983",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[-0.3123,  1.5250],\n",
       "        [-1.7891,  1.2797],\n",
       "        [-0.9699,  1.0574]])"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.transpose(t, 1, 0)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "52afad5d",
   "metadata": {},
   "source": [
    "#### reshape\n",
    "\n",
    "函数原型为：\n",
    "\n",
    "```python\n",
    "torch.reshape(input, shape)\n",
    "```\n",
    "\n",
    "对给定`input`的`tensor`，将其转变为指定的`shape`。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "id": "3883c049",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([0., 1., 2., 3.])"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t = torch.arange(4.)\n",
    "t"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "id": "be322e9b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0., 1.],\n",
       "        [2., 3.]])"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.reshape(t, (2, 2))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "id": "246c0f14",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([0., 1., 2., 3.])"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.reshape(t, (-1, ))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "24021223",
   "metadata": {},
   "source": [
    "#### squeeze与unsqueeze\n",
    "\n",
    "函数原型：\n",
    "\n",
    "```python\n",
    "torch.squeeze(input, dim=None, *, out=None)\n",
    "```\n",
    "\n",
    "返回一个`tensor`，所有为`1`的维度都会被抹除掉。\n",
    "\n",
    "比如输入的维度为$(A \\times 1 \\times B \\times C \\times 1 \\times D)$，输出的tensor的维度将会为$(A \\times B \\times C \\times D)$。\n",
    "\n",
    "unsqueeze则是在指定维度增加一个维度。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "id": "cb7ce59d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([2, 1, 2, 1, 2])"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t = torch.zeros(2, 1, 2, 1, 2)\n",
    "t.size()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "id": "a9f53455",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([2, 2, 2])"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t1 = torch.squeeze(t)\n",
    "t1.size()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "id": "f9b85bf3",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([2, 2, 1, 2])"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t2 = torch.squeeze(t, 1)\n",
    "t2.size()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "id": "f8e5e77a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([1, 2, 2, 1, 2])"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t3 = torch.unsqueeze(t2, 0)\n",
    "t3.size()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "40faa619",
   "metadata": {},
   "source": [
    "#### tack\n",
    "\n",
    "函数原型为：\n",
    "\n",
    "```python\n",
    "torch.take(input, index)\n",
    "```\n",
    "\n",
    "依据indices索引，返回一个新的tensor，给定的input tensor会被视为一维的tensor，然后再进行索引。最终返回的结果大小与indices的大小一致。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "id": "f7ca6239",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([4, 5, 8])"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "src = torch.tensor([[4, 3, 5],\n",
    "                    [6, 7, 8]])\n",
    "torch.take(src, torch.tensor([0, 2, 5]))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6eca8119",
   "metadata": {},
   "source": [
    "#### tile\n",
    "\n",
    "函数原型为：\n",
    "\n",
    "```python\n",
    "torch.tile(input, dims)\n",
    "```\n",
    "\n",
    "在指定的dims上，复制这个输入的input。如果dims的维度比输入input的维度小，那么会在dims的维度前面填充1。比如输入的input维度为(8, 6, 4, 2)，dims的维度为(2, 2)的话，dims的维度会被填充为(1, 1, 2, 2)。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "id": "515b37a7",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[1, 2, 1, 2],\n",
       "        [3, 4, 3, 4]])"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t = torch.tensor([[1, 2], [3, 4]])\n",
    "t1 = torch.tile(t, dims=(2,))\n",
    "t1"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "07dde1f6",
   "metadata": {},
   "source": [
    "上述的输出结果就是默认前面的那个维度为1得到的。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "id": "9b2fa952",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[1, 2, 1, 2],\n",
       "        [3, 4, 3, 4],\n",
       "        [1, 2, 1, 2],\n",
       "        [3, 4, 3, 4]])"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t2 = torch.tile(t, dims=(2, 2))\n",
    "t2"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f91b3ec3",
   "metadata": {},
   "source": [
    "#### unbind"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9feaff74",
   "metadata": {},
   "source": [
    "函数原型为：\n",
    "\n",
    "```python\n",
    "torch.unbind(input, dim=0) → seq\n",
    "```\n",
    "\n",
    "按照dim的指定维度，移除掉一个维度，使其能够变成一个seq。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "id": "88fa90b0",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(tensor([1, 2, 3]), tensor([4, 5, 6]), tensor([7, 8, 9]))"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.unbind(torch.tensor([[1, 2, 3],\n",
    "                          [4, 5, 6],\n",
    "                          [7, 8, 9]]), dim=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c33e6d7e",
   "metadata": {},
   "source": [
    "#### where\n",
    "\n",
    "函数原型为：\n",
    "\n",
    "```python\n",
    "torch.where(condition, x, y) \n",
    "```\n",
    "\n",
    "依据给定的`condition`，选择`x`或者`y`。上述操作可以被定义为：\n",
    "\n",
    "$$\n",
    "\\text { out }_{i}= \\begin{cases}\\mathrm{x}_{i} & \\text { if condition }_{i} \\\\ \\mathrm{y}_{i} & \\text { otherwise }\\end{cases}\n",
    "$$\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "id": "5940db56",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 0.9033, -0.9423],\n",
       "        [ 0.2850, -0.9643],\n",
       "        [-1.5870, -0.4876]])"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = torch.randn(3, 2)\n",
    "y = torch.ones(3, 2)\n",
    "x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "id": "8ebcb979",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0.9033, 1.0000],\n",
       "        [0.2850, 1.0000],\n",
       "        [1.0000, 1.0000]])"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.where(x > 0, x, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "28faf45d",
   "metadata": {},
   "source": [
    "上述结果中，将`x`元素中大于`0`的值都不变，小于`0`的值都赋值为`1`。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "29f4adcf",
   "metadata": {},
   "source": [
    "### 随机采样"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c0539447",
   "metadata": {},
   "source": [
    "#### manual_seed"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f49c938b",
   "metadata": {},
   "source": [
    "设置随机种子，可以看到其内部是通过调用cuda的设置随机种子来设置的。\n",
    "\n",
    "```python\n",
    "def manual_seed(seed) -> torch._C.Generator:\n",
    "    r\"\"\"Sets the seed for generating random numbers. Returns a\n",
    "    `torch.Generator` object.\n",
    "\n",
    "    Args:\n",
    "        seed (int): The desired seed. Value must be within the inclusive range\n",
    "            `[-0x8000_0000_0000_0000, 0xffff_ffff_ffff_ffff]`. Otherwise, a RuntimeError\n",
    "            is raised. Negative inputs are remapped to positive values with the formula\n",
    "            `0xffff_ffff_ffff_ffff + seed`.\n",
    "    \"\"\"\n",
    "    seed = int(seed)\n",
    "    import torch.cuda\n",
    "\n",
    "    if not torch.cuda._is_in_bad_fork():\n",
    "        torch.cuda.manual_seed_all(seed)\n",
    "\n",
    "    return default_generator.manual_seed(seed)\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "51c8369d",
   "metadata": {},
   "source": [
    "#### bernoulli"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f5c090c0",
   "metadata": {},
   "source": [
    "\n",
    "函数原型为：\n",
    "\n",
    "```python\n",
    "torch.bernoulli(input, *, generator=None, out=None)\n",
    "```\n",
    "\n",
    "直接依据概率选择0或者1。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "id": "802e46d3",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0.2132, 0.1981, 0.4533],\n",
       "        [0.3551, 0.9903, 0.0623],\n",
       "        [0.0401, 0.2300, 0.7234]])"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t = torch.empty(3, 3).uniform_(0, 1)\n",
    "t"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "id": "1defd458",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0., 1., 0.],\n",
       "        [0., 1., 0.],\n",
       "        [0., 0., 1.]])"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.bernoulli(t)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "id": "aeb3d484",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[1., 1., 1.],\n",
       "        [1., 1., 1.],\n",
       "        [1., 1., 1.]])"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t1 = torch.ones(3, 3)\n",
    "torch.bernoulli(t1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "id": "9d96875b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0., 0., 0.],\n",
       "        [0., 0., 0.],\n",
       "        [0., 0., 0.]])"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t2 = torch.zeros(3, 3)\n",
    "torch.bernoulli(t2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6ce1f701",
   "metadata": {},
   "source": [
    "#### normal\n",
    "\n",
    "生成高斯分布。\n",
    "\n",
    "函数原型为：\n",
    "\n",
    "```python\n",
    "torch.normal(mean, std, *, generator=None, out=None)\n",
    "```\n",
    "\n",
    "依据给定的`mean`和`std`，生成随机数。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "id": "7707bbeb",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([-0.5042,  3.8942,  1.0071,  4.1209,  3.5986,  5.9754,  6.6091,  7.7459,\n",
       "         8.8295, 10.0138])"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.normal(mean=torch.arange(1., 11.), std=torch.arange(1, 0, -0.1))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7d09d8eb",
   "metadata": {},
   "source": [
    "如果mean只给定一个标量的话，那么会将这个标量与多个方差共享。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "id": "22878f2c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([-1.2062,  3.6133, -2.6371, -1.1055,  5.0636])"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.normal(mean=0.0, std=torch.arange(1., 6.))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6612f899",
   "metadata": {},
   "source": [
    "如果没有给定方差的话，方差也会按照默认值1，共享。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "id": "18d00829",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([-0.3958,  2.0300,  4.3742,  3.3280,  4.5535])"
      ]
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.normal(mean=torch.arange(1, 6.))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "53b2e9e7",
   "metadata": {},
   "source": [
    "还可以指定均值和方差，以及需要生成的数据的大小。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "id": "5b52b7e3",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 0.9359,  4.5304,  4.3686,  0.1311],\n",
       "        [ 0.3274,  2.8057, -0.2907,  1.2547]])"
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.normal(2, 3, size=(2, 4))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "04ff6a14",
   "metadata": {},
   "source": [
    "#### rand\n",
    "\n",
    "从[0, 1)的区间里生成一个均匀分布。函数原型为：\n",
    "\n",
    "```python\n",
    "torch.rand(*size, *, out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False)\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "id": "2457355f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0.3630, 0.8106, 0.5972],\n",
       "        [0.9917, 0.9647, 0.0874]])"
      ]
     },
     "execution_count": 69,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.rand(2, 3)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f6299e27",
   "metadata": {},
   "source": [
    "#### randn\n",
    "\n",
    "从均值为`0`，方差为`1`的正太分布中返回数据。函数原型为：\n",
    "\n",
    "```python\n",
    "torch.randn(*size, *, out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False)\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "id": "3cb76722",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[-0.6924,  0.5837, -1.4388],\n",
       "        [ 0.7302, -0.7548,  1.2483]])"
      ]
     },
     "execution_count": 70,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.randn(2, 3)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a1d508a9",
   "metadata": {},
   "source": [
    "#### randint\n",
    "\n",
    "函数原型为：\n",
    "\n",
    "```python\n",
    "torch.randint(low=0, high, size, \\*, generator=None, out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False)\n",
    "```\n",
    "\n",
    "从整数low(包含)到high(不包含)范围内，选取大小为size大小的整数数据出来。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "id": "706df035",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[3, 3, 3],\n",
       "        [4, 3, 3]])"
      ]
     },
     "execution_count": 71,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.randint(3, 5, (2, 3))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "18f75fe6",
   "metadata": {},
   "source": [
    "#### randperm\n",
    "\n",
    "函数原型为：\n",
    "\n",
    "```python\n",
    "torch.randperm(n, *, generator=None, out=None, dtype=torch.int64, layout=torch.strided, device=None, requires_grad=False, pin_memory=False)\n",
    "```\n",
    "\n",
    "返回一个从`0～n-1`的随机整数。\n",
    "\n",
    "可以在构建数据集的时候用到，用它来产生随机的`index`。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "id": "caf5eed7",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([0, 1, 2, 3])"
      ]
     },
     "execution_count": 72,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.randperm(4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8eb9a342",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "01fe63fc",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7132b607",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.4"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {
    "height": "calc(100% - 180px)",
    "left": "10px",
    "top": "150px",
    "width": "288px"
   },
   "toc_section_display": true,
   "toc_window_display": true
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}