Chapter01-quickstart/quickstart.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Tensor\n",
    "在Pytorch中，Tensor(张量)属于一种数据结构，他可以是一个标量，一个向量，一个矩阵，甚至是更高维的数组，因此Pytorch中的Tensor和Numpy中的数组非常相似，但是Tensor可以在GPU上加速运算。\n",
    "\n",
    "同时应该注意的是，在Pytorc的0.4版本之前，Tensor是不能计算梯度的，因此需要Variable来包装Tensor，但是在0.4版本之后，Variable和Tensor合并了，Tensor可以直接计算梯度了。\n",
    "\n",
    "下面介绍一下Tensor的基本使用：\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 数据类型\n",
    "torch 在CPU和GPU中 Tensor分别有8中数据类型，主要包括以下：\n",
    "1. `torch.float32` 或 `torch.float`：32位浮点数\n",
    "2. `torch.float64` 或 `torch.double`：64位双精度浮点数\n",
    "3. `torch.float16` 或 `torch.half`：16位半精度浮点数\n",
    "4. `torch.int8`：8位整数（有符号）\n",
    "5. `torch.uint8`：8位整数（无符号）\n",
    "6. `torch.int16` 或 `torch.short`：16位整数（有符号）\n",
    "7. `torch.int32` 或 `torch.int`：32位整数（有符号）\n",
    "8. `torch.int64` 或 `torch.long`：64位整数（有符号）\n",
    "\n",
    "注意：在 GPU 上，不是所有的数据类型都被支持。例如，torch.int8 和 torch.uint8 在 GPU 上可能不被支持。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 导入需要的库\n",
    "import torch\n",
    "# 获得Tensor的数据类型\n",
    "torch.tensor([1.2,3.4]).dtype"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 更改默认数据类型\n",
    "torch.set_default_dtype(torch.float64)\n",
    "\n",
    "# 获得Tensor的数据类型\n",
    "torch.tensor([1.2,3.4]).dtype"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 生成Tensor\n",
    "\n",
    "在Pytorch中，有多种方式可以生成一个Tensor，下面使用具体代码介绍如何使用pytorch生成tensor:\n",
    "1. `torch.tensor()`: 将python的列表转换为tensor\n",
    "2. `torch.Tensor()`: 将python的列表转换为tensor;或者生成指定形状的tensor\n",
    "3. `torch.**_like()`: 生成与指定tensor维度相同、性质相似的tensor\n",
    "4. `torch.numpy() 和torch.from_numpy()`: 将numpy的ndarray转换为tensor\n",
    "5. `troch.normal()/torch.rand()/torch.randn()`: 生成具有特定分布的随机数\n",
    "6. `torch.randperm(n)`: 生成0-n之间整数进行随机排序的tensor\n",
    "7. `torch.arange(start, end, step)`: 生成指定范围内的整数tensor\n",
    "8. `torch.linspace(start, end, steps)`: 生成指定范围内等间隔的tensor\n",
    "9. `torch.logspace(start, end, steps)`: 生成指定范围内等比数的tensor\n",
    "10. `troch.ones()/torch.zeros()/torch.empty()/torch.eye()`: 生成全为1/0/随机数/单位矩阵的tensor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 使用torch.tensor()创建Tensor\n",
    "A = torch.tensor([1.2,3.4])\n",
    "\n",
    "# 使用dtype指定tensor的数据类型\n",
    "# 使用requires_grad指定是否需要梯度\n",
    "A = torch.tensor([1.2,3.4],dtype=torch.float32,requires_grad=True)\n",
    "\n",
    "#注意，只有浮点类型的tensor才能指定是否需要梯度\n",
    "A\n",
    "# 使用torch.Tensor() 创建Tensor\n",
    "B = torch.Tensor([1.2,3.4])\n",
    "\n",
    "# 创建一个 Tensor，形状为 [3, 3]\n",
    "B = torch.Tensor(3, 3)\n",
    "# 创建一个与B相同大小，全为1的tensor\n",
    "C = torch.ones_like(B)\n",
    "\n",
    "# 创建一个与B相同大小，全为0的tensor\n",
    "D = torch.zeros_like(B)\n",
    "\n",
    "# 创建一个与B相同大小的随机tensor\n",
    "E = torch.rand_like(B)\n",
    "\n",
    "# 利用numpy生成张量\n",
    "import numpy as np\n",
    "F = np.array([[1,2,3],[4,5,6]])\n",
    "G = torch.from_numpy(F)\n",
    "\n",
    "# 使用torch.numpy()转化Numpy数组\n",
    "H = G.numpy()\n",
    "\n",
    "# 使用指定均值和方差生成随机数\n",
    "torch.manual_seed(0)\n",
    "I = torch.normal(mean=torch.full([10],0),std=torch.arange(1,0,-0.1))\n",
    "\n",
    "# 使用torch.rand() 在区间[0, 1]内生成一服从均匀分布的随机数。\n",
    "torch.manual_seed(0)\n",
    "J = torch.rand(3,4)\n",
    "\n",
    "# 使用torch.rand_like() 生成与其他维度相同的随机数。\n",
    "torch.manual_seed(0)\n",
    "K = torch.rand_like(B)\n",
    "\n",
    "# 使用torch.randn()生成服从标准正态分布的随机数。\n",
    "torch.manual_seed(0)\n",
    "L = torch.randn(3,4)\n",
    "M= torch.randn_like(B)\n",
    "\n",
    "# torch.randperm(n) 生成一个从 0 到 n-1 的随机排列。\n",
    "torch.manual_seed(0)\n",
    "N = torch.randperm(10)\n",
    "\n",
    "# torch.arange() 生成一个给定范围内的连续整数张量。参数：起始值、结束值和步长\n",
    "O = torch.arange(0,10)\n",
    "\n",
    "# torch.linspace() 生成给定范围内的等间隔整数或浮点数张量\n",
    "P = torch.linspace(0,10,steps=5)\n",
    "\n",
    "# torch.logspace() 生成给定范围内的以对数间隔整数或浮点数张量\n",
    "Q = torch.logspace(0,-1,steps=10)\n",
    "\n",
    "# torch.eye() 生成单位矩阵\n",
    "R = torch.eye(3)\n",
    "\n",
    "# torch.zeros()   生成全0的张量\n",
    "S = torch.zeros(3,3)\n",
    "\n",
    "# torch.ones() 生成全1的张量\n",
    "T = torch.ones(3,3) \n",
    "\n",
    "# torch.full() 生成全为某个值的张量\n",
    "U = torch.full((3,3), 5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Tensor 操作\n",
    "对于Tensor的操作，主要是改变Tensor的形状、提取Tensor元素、统计Tensor信息等"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "改变tensor形状的操作主要包括以下几种：\n",
    "1. `reshape()`: 返回具有相同数据但大小不同的 tensor,原始tensor 和 新的tensor 共享内存\n",
    "2. `resize_()`: 返回相同数据的 tensor，但形状已经改变。与 `reshape()` 不同，此函数可以改变 tensor 的大小。\n",
    "3. `resize_as_()`: 返回相同数据的 tensor，形状与另一个 tensor 相同。\n",
    "4. `view()`: 返回具有相同数据但大小不同的 tensor,要求原始tensor必须连续，或者使用 `contiguous()` 函数将其转换为连续的 tensor。\n",
    "5. `permute()`: 用来交换 tensor 的维度。\n",
    "6. `squeeze()`: 返回一个具有相同数据但所有维度为1的维度都被移除的 tensor。\n",
    "7. `unsqueeze()`: 返回一个具有相同数据但在指定位置添加一个维度的 tensor。\n",
    "8. `expand()`: 将tensor广播到指定形状。\n",
    "9. `expand_as()`: 返回一个具有相同数据但形状与另一个 tensor 相同的 tensor。\n",
    "10. `repeat()`: 根据指定形状进行重复填充，接受整数元组，表示沿每个轴重复的次数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 创建一个形状为 [2, 2] 的 tensor\n",
    "A = torch.tensor([[1, 2], [3, 4]])\n",
    "\n",
    "# 使用 reshape 改变 tensor 的形状\n",
    "B = A.reshape(4, 1)\n",
    "\n",
    "# 使用 view 改变 tensor 的形状\n",
    "C = A.view(4, 1)\n",
    "\n",
    "# 使用 permute 改变 tensor 的维度\n",
    "D = A.permute(1, 0)\n",
    "\n",
    "# 使用 squeeze 移除 tensor 的所有维度为1的维度\n",
    "E = A.squeeze()\n",
    "\n",
    "# 使用 unsqueeze 在指定位置添加一个维度\n",
    "F = A.unsqueeze(0)\n",
    "\n",
    "# 使用 expand 在指定位置添加一个维度\n",
    "G = A.expand(3, 2, 2)\n",
    "\n",
    "# 使用 repeat 重复 tensor 中的元素\n",
    "H = A.repeat(3, 2)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "获取tensor中元素的操作主要包括以下几种：\n",
    "1. 切片\n",
    "2. 索引\n",
    "3. `torch.tril()、torch.triu()、torch.diag()` 分别获取tensor的下三角元素、上三角元素、对角线元素"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 创建一个形状为 [2, 2] 的 tensor\n",
    "A = torch.tensor([[1, 2], [3, 4]])  \n",
    "\n",
    "# 使用切片获得A中元素\n",
    "B = A[0, 0]\n",
    "\n",
    "# 使用索引获取A中元素\n",
    "C = A[A>2] \n",
    "\n",
    "# 使用 torch.where() 获取A中元素\n",
    "D = torch.where(A>2, A, torch.full_like(A, 0))\n",
    "\n",
    "# 使用 torch.tril() 获取A的下三角矩阵\n",
    "E = torch.tril(A)\n",
    "\n",
    "# 使用 torch.triu() 获取A的上三角矩阵\n",
    "F = torch.triu(A)   \n",
    "\n",
    "# 使用 torch.diag() 获取A的对角线元素\n",
    "G = torch.diag(A) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "tensor的拼接和拆分包括以下几种操作\n",
    "1. `torch.cat()` 将多个tensor在指定维度上进行拼接，不会增加维度\n",
    "2. `torch.stack()` 将多个tensor在指定维度拼接，但是会增加维度\n",
    "3. `torch.chunk()` 将tensor 分割为特定数量的块\n",
    "4. `torch.split()` 将tensor分割成多个子tensor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 创建一个tensor\n",
    "A = torch.tensor([[1, 2], [3, 4]])\n",
    "B = torch.tensor([[5, 6], [7, 8]])\n",
    "\n",
    "# 使用cat() 连接A，B\n",
    "C = torch.cat([A, B], dim=1)\n",
    "D = torch.cat([A, B], dim=0)\n",
    "\n",
    "# 使用stack() 在新维度上连接A，B\n",
    "E = torch.stack([A, B], dim=0)\n",
    "F = torch.stack([A, B], dim=1)\n",
    "\n",
    "# 使用chunk() 将C分割成两个张量\n",
    "G = torch.chunk(C, 2, dim=1)\n",
    "\n",
    "# 使split() 将C分割成两个张量\n",
    "H = torch.split(C, [1,1,2], dim=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "tensor 的计算主要包括大小比较、元素的基本运算、排序与最值等方法。\n",
    "\n",
    "**大小比较**\n",
    "1. `torch.eq()` : 逐元素比较是否相等\n",
    "2. `torch.ge()` : 逐元素比较是否大于等于\n",
    "3. `torch.gt()` : 逐元素比较是否大于\n",
    "4. `torch.le()` : 逐元素比较是否小于等于\n",
    "5. `troch.lt()` : 逐元素比较是否小于\n",
    "6. `torch.ne()` : 逐元素比较是否不相等\n",
    "7. `torch.equal()` : 比较两个tensor是否具有相同的大小和元素"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 创建一个tensor\n",
    "A = torch.tensor([1, 2,3, 4])\n",
    "B = torch.arange(1, 5)\n",
    "\n",
    "# 返回值为mask\n",
    "torch.eq(A, B)\n",
    "\n",
    "# 返回值为True 或者 False\n",
    "torch.equal(A, B)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**基本运算**\n",
    "1. `torch.add()` 矩阵逐元素相加\n",
    "2. `torch.sub()` 矩阵逐元素相减\n",
    "3. `torch.mul()` 矩阵逐元素相乘\n",
    "4. `torch.div()` 矩阵逐元素相除\n",
    "5. `torch.pow()` 矩阵的幂\n",
    "6. `torch.sqrt()` 矩阵的平方根\n",
    "7. `troch.exp()` 矩阵的指数\n",
    "8. `torch.log()` 矩阵的对数\n",
    "9. `torch.log10()` 矩阵的常用对数\n",
    "10. `torch.clamp` 限制矩阵元素在指定范围内\n",
    "11. `torch.clamp_max()` 根据最大值裁切\n",
    "12. `torch.clamp_min()` 根据最小值裁切\n",
    "13. `torch.t()` 矩阵转置\n",
    "14. `torch.mm()` 矩阵乘法\n",
    "15. `torch.inverse()` 求逆\n",
    "16. `torch.trace() ` 求迹"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "A = torch.arange(6.0).reshape(2,3)\n",
    "B = torch.linspace(10, 20, 6).reshape(2,3)\n",
    "\n",
    "# 矩阵逐个元素相乘\n",
    "C = A * B\n",
    "\n",
    "# 矩阵逐个元素相除\n",
    "D = A / B\n",
    "\n",
    "# 矩阵逐个元素相加\n",
    "E = A + B\n",
    "\n",
    "# 矩阵逐个元素相减\n",
    "F = A - B\n",
    "\n",
    "# 矩阵逐个元素求幂\n",
    "G = A ** 2\n",
    "\n",
    "# 矩阵逐个元素求指数\n",
    "H = torch.exp(A)\n",
    "\n",
    "# 矩阵逐个元素求对数\n",
    "I = torch.log(A)\n",
    "\n",
    "# 矩阵逐个元素求平方根\n",
    "J = torch.sqrt(A)\n",
    "\n",
    "# 矩阵裁切\n",
    "K = A.clamp(0, 2)\n",
    "\n",
    "# 矩阵最小值裁切\n",
    "L = A.clamp_min(2)\n",
    "\n",
    "# 矩阵最大值裁切\n",
    "M = A.clamp_max(2)\n",
    "\n",
    "# 矩阵转置\n",
    "N = A.t()\n",
    "\n",
    "# 矩阵相乘\n",
    "O = A @ B.t()\n",
    "\n",
    "P = torch.mm(A, B.t())\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "统计tensor中元素主要包括以下方法：\n",
    "\n",
    "1. `torch.sum()`: 计算 Tensor 中所有元素的和。\n",
    "2. `torch.mean()`: 计算 Tensor 中所有元素的平均值。\n",
    "3. `torch.max()`: 返回 Tensor 中的最大值。\n",
    "4. `torch.min()`: 返回 Tensor 中的最小值。\n",
    "5. `torch.prod()`: 计算 Tensor 中所有元素的乘积。\n",
    "6. `torch.std()`: 计算 Tensor 中所有元素的标准差。\n",
    "7. `torch.var()`: 计算 Tensor 中所有元素的方差。\n",
    "8. `torch.median()`: 返回 Tensor 中的中位数。\n",
    "9. `torch.mode()`: 返回 Tensor 中的众数。\n",
    "10. `torch.norm()`: 计算 Tensor 的范数。\n",
    "11. `torch.argmax()`: 返回 Tensor 中最大值的索引。\n",
    "12. `torch.argmin()`: 返回 Tensor 中最小值的索引。\n",
    "13. `torch.sort()`: 对 Tensor 进行排序并返回排序后的 Tensor以及索引index。\n",
    "14. `torch.topk()`: 返回 Tensor 中 k 个最大值的元素及其索引。\n",
    "15. `torch.kthvalue()`: 返回 Tensor 中第 k 个最小值的元素及其索引。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "A = torch.tensor([18,34,66,3,34,54,23,46,56,90], dtype=torch.float32)\n",
    "\n",
    "# 求和\n",
    "B = torch.sum(A)\n",
    "\n",
    "# 求均值\n",
    "C = torch.mean(A)\n",
    "\n",
    "# 求最大值\n",
    "D = torch.max(A)\n",
    "\n",
    "# 求最小值\n",
    "E = torch.min(A)\n",
    "\n",
    "# 求标准差\n",
    "F = torch.std(A)\n",
    "\n",
    "# 求乘积\n",
    "G = torch.prod(A)\n",
    "\n",
    "# 求范数\n",
    "H = torch.norm(A)\n",
    "\n",
    "# 最大值索引\n",
    "I = torch.argmax(A)\n",
    "\n",
    "# 最小值索引\n",
    "J = torch.argmin(A) \n",
    "\n",
    "# 排序\n",
    "K = torch.sort(A)\n",
    "\n",
    "# topk\n",
    "L = torch.topk(A, 3)\n",
    "\n",
    "# kthvalue\n",
    "M = torch.kthvalue(A, 3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 自动微分\n",
    "\n",
    "针对tensor，设置其`requires_grad = True`，可以通过相关计算输出其在传播过程中的梯度（导数）信息。\n",
    "其中注意一点：tensor的更新方向，如果需要求得函数的最小值，应该使用减号，即`x -= learning_rate * x.grad`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 使用pytorch的自动微分求函数最小值：\n",
    "\n",
    "# 初始化变量x\n",
    "x = torch.tensor([9.], requires_grad=True)\n",
    "\n",
    "# 定义函数f(x)\n",
    "def f(x):\n",
    "    return x ** 2 + 2 * x + 1\n",
    "\n",
    "# 梯度下降\n",
    "learning_rate = 0.2\n",
    "for i in range(10):\n",
    "    # 计算函数值和梯度\n",
    "    y = f(x)\n",
    "    y.backward()\n",
    "    \n",
    "    # 使用梯度来更新x\n",
    "    with torch.no_grad(): # 使用torch.no_grad()来防止跟踪历史记录\n",
    "        x -= learning_rate * x.grad\n",
    "    # 清空梯度\n",
    "    x.grad.zero_()\n",
    "\n",
    "# 输出最小值\n",
    "print(f'函数的最小值为：{y.item()}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## torch.nn 模块\n",
    "torch.nn模块中主要包括卷积层、池化层、激活函数层、循环层、全连接层等相关使用方法"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 卷积层\n",
    "使用卷积可以达到稀疏连接、参数共享、平移不变性等特点。\n",
    "\n",
    "常见的卷积操作对应类如下：（以2d为例）\n",
    "1. `torch.nn.Conv2d`\n",
    "2. `torch.nn.ConvTranspose2d()`\n",
    "\n",
    "`torch.nn.Conv2d`其参数说明如下：\n",
    "~~~python\n",
    "torch.nn.Conv2d(in_channels, out_channels, kernel_size, stride=1, padding=0, dilation=1, groups=1, bias=True, padding_mode='zeros')\n",
    "~~~\n",
    "\n",
    "`in_channels`（int）：输入图像的通道数。\n",
    "`out_channels`（int）：卷积产生的通道数。\n",
    "`kernel_size`（int or tuple）：卷积核的尺寸，可以是一个int（例如，3表示3x3）或者一个tuple（例如，(3, 3)）。\n",
    "`stride`（int or tuple，optional）：卷积步长，即卷积核移动的步数。默认为1。\n",
    "`padding`（int or tuple，optional）：输入的每一条边补充0的层数。默认为0。\n",
    "`dilation`（int or tuple，optional）：卷积核元素之间的间距。默认为1。\n",
    "`groups`（int，optional）：从输入通道到输出通道的阻塞连接的数目。默认为1。\n",
    "`bias`（bool，optional）：如果设置为False，那么这个层不会学习附加的偏置参数。默认为True。\n",
    "`padding_mode`（string，optional）：指定填充模式。可以是'zeros'、'reflect'、'replicate'或'circular'。默认为'zeros'。\n",
    "\n",
    "卷积前后尺寸变化如下：\n",
    "output_height = (input_height + 2 * padding_height - dilation_height * (kernel_height - 1) - 1) / stride_height + 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "import torch.nn as nn\n",
    "import numpy as np\n",
    "from PIL import Image\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# 载入图像\n",
    "image_path = \"imgs\\LenaRGB.bmp\"\n",
    "image = Image.open(image_path)\n",
    "imgray = np.array(image.convert('L'),dtype=np.float32)\n",
    "\n",
    "# 将array转换为tensor\n",
    "imh,imw = imgray.shape\n",
    "imgray_t = torch.from_numpy(imgray.reshape((1,1,imh,imw)))\n",
    "\n",
    "## 对灰度图像进行卷积提取图像轮廓\n",
    "# 定义边缘检测卷积核，[batch_size, in_channels, height, width]\n",
    "kernel = torch.tensor([[-1,-1,-1],[-1,8,-1],[-1,-1,-1]],dtype=torch.float32).reshape((1,1,3,3))\n",
    "\n",
    "# 定义卷积层\n",
    "conv2d = nn.Conv2d(in_channels=1, out_channels=2, kernel_size=3, bias=False)\n",
    "# 卷积的第一个核使用边缘检测核，第二个核随机初始化\n",
    "conv2d.weight.data[0] = kernel\n",
    "\n",
    "imout = conv2d(imgray_t)\n",
    "# 对卷积输出进行维度压缩\n",
    "imout_data = imout.data.squeeze()\n",
    "plt.figure(figsize=(10,10))\n",
    "plt.subplot(121)\n",
    "plt.imshow(imout_data[0],cmap='gray')\n",
    "plt.axis('off')\n",
    "plt.subplot(122)\n",
    "plt.imshow(imout_data[1],cmap='gray')\n",
    "plt.axis('off')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 池化层\n",
    "\n",
    "池化操作主要是对卷积后的图像进行降低维度、增大感受野的效果，通过平均池化或者全局池化等操作可以保留数据的主要特征。\n",
    "常见的池化操作对应类如下：(以2d为例)\n",
    "1. `torch.nn.MaxPool2d`\n",
    "2. `torch.nn.MaxUnpool2d`\n",
    "3. `torch.nn.AvgPool2d`\n",
    "4. `torch.nn.AdaptiveMaxPool2d()`\n",
    "5. `torch.nn.AdaptiveAvgPool2d()`\n",
    "6. `torch.nn.LPPoool2d()`\n",
    "\n",
    "torch.nn.MaxPool2d()输入为(N，Ci，Hi，Wi)的tensor，输出为(N,Co,Ho,Wo),其中\n",
    "Ho = (Hi-kernel_size)/stride + 1\n",
    "Wo = (Wi-kernel_size)/stride + 1\n",
    "\n",
    "`torch.nn.MaxPool2d()`函数的参数如下：\n",
    "\n",
    "```python\n",
    "torch.nn.MaxPool2d(kernel_size, stride=None, padding=0, dilation=1, return_indices=False, ceil_mode=False)\n",
    "```\n",
    "\n",
    "`kernel_size`（int或tuple）：最大池化操作的窗口大小。可以是一个整数（例如，2表示2x2的窗口大小）或者一个元组（例如，(2, 2)）。\n",
    "`stride`（int或tuple，可选）：窗口移动的步长。默认值是`kernel_size`。\n",
    "`padding`（int或tuple，可选）：在输入的每一条边补充0的层数。默认为0。\n",
    "`dilation`（int或tuple，可选）：控制窗口中元素的间距。默认为1。\n",
    "`return_indices`（bool，可选）：如果为True，会返回输出最大值的索引，通常用于`torch.nn.MaxUnpool2d`。默认为False。\n",
    "`ceil_mode`（bool，可选）：如果为True，会使用向上取整的方式计算输出的形状。默认为False。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 定义最大池化层\n",
    "maxpool = nn.MaxPool2d(kernel_size=2, stride=2)\n",
    "\n",
    "# 定义平均池化层\n",
    "avgpool = nn.AvgPool2d(kernel_size=2, stride=2)\n",
    "\n",
    "# 定义自适应池化层 output_size为输出的大小\n",
    "adapool = nn.AdaptiveAvgPool2d(output_size=(100,100))\n",
    "\n",
    "poolout = adapool(imout)\n",
    "poolout_data = poolout.squeeze().detach()\n",
    "plt.figure(figsize=(10,10))\n",
    "plt.subplot(121)\n",
    "plt.imshow(poolout_data[0],cmap='gray')\n",
    "plt.axis('off')\n",
    "plt.subplot(122)\n",
    "plt.imshow(poolout_data[1],cmap='gray')\n",
    "plt.axis('off')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 激活函数\n",
    "激活函数主要包括以下几类：\n",
    "\n",
    "1. `torch.nn.ReLU()`:   $f(x) = max(0,x)$\n",
    "2. `torch.nn.Sigmoid()`: $ f(x) = \\frac{1}{1+e^{-x}}$\n",
    "3. `torch.nn.Tanh()`: $ f(x) = \\tanh(x) = \\frac{\\sinh x}{\\cosh x} = \\frac{e^x - e^{-x}}{e^x + e^{-x}} = \\frac{1 - \\exp(-2x)}{1 + \\exp(-2x)}$\n",
    "4. `torch.nn.Softmax()`: $ f(x) = \\frac{\\exp(x_i)}{\\sum_j \\exp(x_j)} $\n",
    "5. `torch.nn.LeakyReLU()`: $ f(x) = \\max(0.01x,x) $"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = torch.linspace(-5, 5, 100)\n",
    "\n",
    "sigmoid = nn.Sigmoid()\n",
    "y_sigmoid = sigmoid(x)\n",
    "\n",
    "tanh = nn.Tanh()\n",
    "y_tanh = tanh(x)\n",
    "\n",
    "relu = nn.ReLU()\n",
    "y_relu = relu(x)\n",
    "\n",
    "softmax = nn.Softmax(dim=0)\n",
    "y_softmax = softmax(x)\n",
    "\n",
    "lrelu = nn.LeakyReLU()\n",
    "y_lrelu = lrelu(x)\n",
    "\n",
    "plt.figure(figsize=(25,5))\n",
    "plt.subplot(151)\n",
    "plt.plot(x.data.numpy(), y_sigmoid.data.numpy(),\"r-\")\n",
    "plt.title(\"Sigmoid\")\n",
    "plt.grid()\n",
    "plt.subplot(152)\n",
    "plt.plot(x.data.numpy(), y_tanh.data.numpy(),\"r-\")\n",
    "plt.title(\"Tanh\")\n",
    "plt.grid()\n",
    "plt.subplot(153)\n",
    "plt.plot(x.data.numpy(), y_relu.data.numpy(),\"r-\")\n",
    "plt.title(\"ReLU\")\n",
    "plt.grid()\n",
    "plt.subplot(154)\n",
    "plt.plot(x.data.numpy(), y_softmax.data.numpy(),\"r-\")\n",
    "plt.title(\"Softmax\")\n",
    "plt.grid()\n",
    "plt.subplot(155)\n",
    "plt.plot(x.data.numpy(), y_lrelu.data.numpy(),\"r-\")\n",
    "plt.title(\"LeakyReLU\")\n",
    "plt.grid()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 循环层\n",
    "pytorch 中主要实现了以下三种循环层，\n",
    "1. `nn.RNN()` 多层RNN单元\n",
    "2. `nn.LSTM()` 多层LSTM单元\n",
    "3. `nn.GRU()` 多层GRU单元\n",
    "\n",
    "后续将详细介绍。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 全连接层\n",
    "通常所说的全连接层指的是由多个神经网络组成的层，其所有输出和该层的所有输入都有连接，即每个输入都会影响到每一个输出。在Pytorch中，`nn.Linear()`函数表示线性变换，加上激活函数就可以构成全连接层。\n",
    "\n",
    "`nn.Linear(in_features, out_features，biased=True)`\n",
    "\n",
    "参数：\n",
    "`in_features`: 输入的维度\n",
    "`out_features`: 输出的维度\n",
    "`biased`: 是否使用偏置项，默认为True\n",
    "\n",
    "全连接层的应用范围非常广泛，只有全连接层组成的网络是全连接神经网络，可以用来数据分类或者回归预测;CNN或者RNN的末端通常也会由全连接层组成"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 数据操作与预处理\n",
    "在pytorch中，需要对于文本、图像或者高维数据进行操作和预处理，其中`torch.utils.data`提供了基本的数据操作和预处理方法。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 高维数组\n",
    "\n",
    "一般情况下，我们需要从文本中读取高维的数据，这种数据主要由特征（多个预测变量）和 标签（被预测变量）组成。特征一般是数值变量或者离散变量，标签如果是连续的数值，则对应回归问题；如果是离散的数值，则对应分类问题。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 回归数据准备"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "import torch.utils.data as Data\n",
    "from sklearn.datasets import load_iris,fetch_openml\n",
    "import numpy as np\n",
    "\n",
    "# 读取Boston数据集,读取进来的可能是numpy数组或者DataFrame\n",
    "boston = fetch_openml(name='boston', version=1, as_frame=True)\n",
    "boston_X, boston_y = boston.data, boston.target\n",
    "\n",
    "# 将数据集转换为torch 的32位浮点数\n",
    "train_X = torch.from_numpy(boston_X.values.astype(np.float32))\n",
    "train_y = torch.from_numpy(boston_y.values.astype(np.float32))\n",
    "\n",
    "# 将数据集包装成TensorDataset，以方便后续操作\n",
    "dataset = Data.TensorDataset(train_X, train_y)\n",
    "\n",
    "# 定义一个数据加载器，将训练数据进行批量处理\n",
    "train_loader = Data.DataLoader(\n",
    "    dataset=dataset,      # torch TensorDataset format\n",
    "    batch_size=32,      # 批处理样本大小\n",
    "    shuffle=True,       # 是否随机打乱数据\n",
    "    num_workers=0,      # 读取数据的线程数量,默认为0\n",
    ")\n",
    "\n",
    "# 检查数据集的一个batch样本维度是否一致\n",
    "for step, (batch_x, batch_y) in enumerate(train_loader):\n",
    "    print(f\"step:{step},batch_x:{batch_x.shape},batch_y:{batch_y.shape}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 分类数据集准备\n",
    "与上述不同的是，分类数据集标签默认为64位有符号的整形数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 读取Iris数据集\n",
    "iris_X, iris_y = load_iris(return_X_y=True)\n",
    "\n",
    "# 将读取数据转换为张量\n",
    "train_X = torch.from_numpy(iris_X.astype(np.float32))\n",
    "train_y = torch.from_numpy(iris_y.astype(np.int64))\n",
    "\n",
    "# 将数据集包装成TensorDataset，以方便后续操作\n",
    "dataset = Data.TensorDataset(train_X, train_y)\n",
    "\n",
    "# 定义一个数据加载器，将训练数据进行批量处理\n",
    "train_loader = Data.DataLoader(\n",
    "    dataset=dataset,      # torch TensorDataset format\n",
    "    batch_size=32,      # 批处理样本大小\n",
    "    shuffle=True,       # 是否随机打乱数据\n",
    "    num_workers=0,      # 读取数据的线程数量,默认为0\n",
    ")\n",
    "\n",
    "# 检查数据集的一个batch样本维度是否一致\n",
    "for step, (batch_x, batch_y) in enumerate(train_loader):\n",
    "    print(f\"step:{step},batch_x:{batch_x.shape,batch_x.dtype},batch_y:{batch_y.shape,batch_y.dtype}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 图像数据\n",
    "\n",
    "图像数据中，torchvision 中的transform 模块提供了对每张图片的预处理操作，包括以下：\n",
    "`torchvision.transforms`模块提供了许多预处理图像的方法，包括：\n",
    "\n",
    "1. `Resize()` 调整图像大小\n",
    "2. `CenterCrop()` 从图像中心位置裁剪图像\n",
    "3. `RandomCrop()` 从图像中随机裁剪指定大小的图像\n",
    "4. `RandomHorizontalFlip()` 随机水平翻转图像\n",
    "5. `RandomVerticalFlip()` 随机垂直翻转图像\n",
    "6. `RandomRotation()` 随机旋转图像\n",
    "7. `RandomAffine()` 随机仿射变换\n",
    "8. `ColorJitter()` 随机调整图像亮度、对比度、饱和度和色相\n",
    "9. `RandomGrayscale()` 随机将图像转换为灰度图\n",
    "10. `Normalize()` 对图像进行归一化\n",
    "11. `ToTensor()` 将图像转换为Tensor。具体来讲，将取值为[0,255]的图像转换为取值范围为[0,1]的Tensor,并转换通道顺序从HWC到CHW。\n",
    "12. `Compose()` 将多个预处理操作组合在一起，按顺序执行。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 从datasets模块中导入数据集并且预处理\n",
    "\n",
    "以导入FashionMNIST数据集为例，该数据集包括60000张28X28的图像作为训练集，10000张28X28的灰度图像作为测试集，数据共有10类。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "import torch.utils.data as Data\n",
    "from torchvision.datasets import FashionMNIST\n",
    "import torchvision.transforms as transforms\n",
    "from torchvision.datasets import ImageFolder\n",
    "\n",
    "# FashionMNIST 准备训练集\n",
    "train_data = FashionMNIST(\n",
    "    root='./data/FashionMNIST',  # 数据集存放路径\n",
    "    train=True,  # 是否作为训练集数据\n",
    "    download=True,  # 是否从网络下载数据集\n",
    "    transform=transforms.ToTensor()  # 数据集是否需要转换\n",
    ")\n",
    "\n",
    "# 定义一个数据加载器\n",
    "train_loader = Data.DataLoader(\n",
    "    dataset=train_data,  # 数据集\n",
    "    batch_size=32,  # 批处理样本大小\n",
    "    shuffle=True,  # 是否随机打乱数据\n",
    "    num_workers=0  # 读取数据的线程数量,默认为0\n",
    ")\n",
    "print(len(train_loader))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 对测试集进行相同的操作\n",
    "test_data = FashionMNIST(\n",
    "    root='./data/FashionMNIST', # 数据路径\n",
    "    train=False,\n",
    "    download=False, # 已经下载过了，不需要再下载\n",
    "    transform=transforms.ToTensor()\n",
    ")\n",
    "\n",
    "# 获得每个图像的数据\n",
    "test_x = test_data.data.type(torch.FloatTensor)/255.\n",
    "\n",
    "# 获得每个图像的标签\n",
    "test_y = test_data.targets\n",
    "\n",
    "# 检查数据和标签的维度\n",
    "print(test_x.shape, test_y.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 文件夹导入数据并预处理\n",
    "\n",
    "猫狗数据集下载地址：https://www.kaggle.com/datasets/tongpython/cat-and-dog?resource=download\n",
    "\n",
    "文件夹结构如下：\n",
    "\n",
    "```\n",
    "data/\n",
    "  mydata/\n",
    "    training_set/\n",
    "      cat/\n",
    "        cat01.jpg\n",
    "        cat02.jpg\n",
    "        ...\n",
    "      dog/\n",
    "        dog01.jpg\n",
    "        dog02.jpg\n",
    "        ...\n",
    "    test_set/\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 定义预处理方式\n",
    "transform = transforms.Compose([\n",
    "    transforms.Resize([224,224]), # 缩放到224像素大小\n",
    "    transforms.ToTensor(), # 转换为Tensor\n",
    "    transforms.Normalize(mean=[0.5], std=[0.5]) # 标准化\n",
    "])\n",
    "\n",
    "# 读取数据集,\n",
    "#`ImageFolder`会自动根据子目录的名称生成类别标签，子目录的名称就是类别的名称，子目录中的所有图像都被认为是该类别的。\n",
    "# 例如，如果你有一个名为'cats'的子目录和一个名为'dogs'的子目录，那么`ImageFolder`会生成两个类别：'cats'和'dogs'，\n",
    "# 并且将'cats'目录中的所有图像标记为类别'cats'，将'dogs'目录中的所有图像标记为类别'dogs'。\n",
    "train_data = ImageFolder(root='./data/mydata/training_set', transform=transform)\n",
    "\n",
    "# 定义一个数据加载器\n",
    "train_data_loader = Data.DataLoader(\n",
    "    dataset=train_data,  # 数据集\n",
    "    batch_size=4,  # 批处理样本大小\n",
    "    shuffle=True,  # 是否随机打乱数据\n",
    "    num_workers=0  # 读取数据的线程数量,默认为0\n",
    ")\n",
    "\n",
    "# 获得一个batch的数据\n",
    "for step, (batch_x, batch_y) in enumerate(train_data_loader):\n",
    "    print(f\"step:{step},batch_x:{batch_x.shape},batch_y:{batch_y.shape}\")\n",
    "    if step > 1:\n",
    "        break\n",
    "\n",
    "# 输出训练图像的尺寸和标签\n",
    "print(batch_x.shape, batch_y.shape)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "pytorch",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}