chap5-Pytorch基础/5.2 张量与自动求导.ipynb · 飞城/ai_tutorial_book

Gitee 2025 年度开源项目评选中

加入 Gitee

与超过 1200万开发者一起发现、参与优秀开源项目，私有仓库也完全免费：）

文件

克隆/下载

5.2 张量与自动求导.ipynb 17.89 KB

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "94757370",
   "metadata": {},
   "source": [
    "# 5.2 张量与自动求导\n",
    "\n",
    "PyTorch使用中有几个比较重要的概念，如张量（Tensor）、自动求导（Autograd）等，需要初学者理解，并灵活使用。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "49b30fce",
   "metadata": {},
   "source": [
    "## 5.2.1 张量\n",
    "\n",
    "（1）初始化张量\n",
    "\n",
    "张量可以用Python的list数据进行初始化，并自动判断其数据类型，也可以使用NumPy数组进行初始化。张量和NumPy数组也可以互相转化。例子如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "83195bcc",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[1, 2],\n",
      "        [3, 4]]) \n",
      " [[1 2]\n",
      " [3 4]] \n",
      " [[1, 2], [3, 4]]\n"
     ]
    }
   ],
   "source": [
    "import torch  # 导入PyTorch\n",
    "import numpy as np  # 导入NumPy\n",
    "\n",
    "data = [[1, 2],[3, 4]]\n",
    "np_array = np.array(data)  # 用list数组创建Numpy对象\n",
    "t = torch.tensor(data)  # 用list数组创建Tensor对象\n",
    "tensor_from_np = torch.from_numpy(np_array)  # 用NumPy数组创建Tensor对象\n",
    "\n",
    "np_from_tensor = t.numpy()  # 将Tensor对象转换为NumPy数组\n",
    "listdata = tensor_from_np.tolist()  # 将Tensor对象转换为list数组\n",
    "print(t, '\\n', np_from_tensor, '\\n', listdata)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4b9f6015",
   "metadata": {},
   "source": [
    "也可以从另一个张量对象复制数据来初始化新的张量，例如："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "e51082a8",
   "metadata": {},
   "outputs": [],
   "source": [
    "tensor_data = t.clone()  # 返回与t值相同的tensor,新对象存储在新的内存中\n",
    "new_data = t.detach()  # 返回与t完全相同的tensor,新对象与旧对象t共享内存\n",
    "ones_data = torch.ones_like(t)  # 和t形状一致的全1张量\n",
    "zeros_data = torch.zeros_like(t)  # 和t形状一致的全0张量\n",
    "rand_data = torch.rand_like(t, dtype=torch.float)  # 和t形状一致的随机浮点数张量"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "368f8d82",
   "metadata": {},
   "source": [
    "（2）张量的属性\n",
    "\n",
    "张量的属性描述了其形状、数据类型、存储设备及在内存中的存储形式等信息。张量的形状表示各个维度的大小。张量支持的数据类型主要是不同精度的浮点数和整数，如32bit的float类型、64bit的double类型等。存储设备是指张量在CPU上计算，还是在GPU上计算。张量在内存中的存储形式表示是按稠密矩阵存储，还是按稀疏矩阵存储。例如："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "8005ba7a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "张量形状: torch.Size([3, 4])\n",
      "张量数据类型: torch.float32\n",
      "张量存储设备: cpu\n"
     ]
    }
   ],
   "source": [
    "tensor = torch.rand(3,4)  # 3x4的随机tensor\n",
    "print(f\"张量形状: {tensor.shape}\")\n",
    "print(f\"张量数据类型: {tensor.dtype}\")\n",
    "print(f\"张量存储设备: {tensor.device}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "da9a6f75",
   "metadata": {},
   "source": [
    "Tensor也可存储的GPU上进行加速运算，例如："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "02c16416",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cpu\n",
      "张量存储设备: cpu\n"
     ]
    }
   ],
   "source": [
    "device = \"cuda\" if torch.cuda.is_available() else \"cpu\" #判断是否可用CUDA加速运算\n",
    "print(device)\n",
    "tensor = tensor.to(device) #将Tensor放到GPU上\n",
    "print(f\"张量存储设备: {tensor.device}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c4f114db",
   "metadata": {},
   "source": [
    "（3）索引和切片\n",
    "\n",
    "张量的索引和切片类似NumPy的索引和切片。示例如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "3cf87b11",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "第一行: tensor([51, 83, 82, 20])\n",
      "第一列: tensor([51, 55, 44, 66])\n",
      "最后一列: tensor([20, 23, 28, 27])\n",
      "tensor([[51,  0, 82, 20],\n",
      "        [55,  0, 16, 23],\n",
      "        [44,  0, 40, 28],\n",
      "        [66,  0, 55, 27]])\n"
     ]
    }
   ],
   "source": [
    "tensor = torch.randint(1,100,(4, 4))  #4行4列，各元素值为[1，100)区间的随机数\n",
    "print(f\"第一行: {tensor[0]}\")\n",
    "print(f\"第一列: {tensor[:, 0]}\")\n",
    "print(f\"最后一列: {tensor[..., -1]}\")  # ...和:是一样的\n",
    "tensor[:, 1] = 0  # 第二列置为0\n",
    "print(tensor)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fabc1915",
   "metadata": {},
   "source": [
    "（4）张量连接\n",
    "\n",
    "多个张量可以在某个维度上进行连接，除了要连接的维度值不同，其他维度的值应保持一致，否则不可以进行连接。例如两个张量的形状分别为[2, 4]和[3, 4]，则可以在第一个维度，即纵向上进行连接。示例如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "31f6357d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([4, 4]) torch.Size([3, 4]) torch.Size([4, 2]) torch.Size([7, 4]) torch.Size([4, 6])\n"
     ]
    }
   ],
   "source": [
    "tensor1 = torch.randint(1,100,(4, 4))\n",
    "tensor2 = torch.rand(3,4) \n",
    "tensor3 = torch.rand(4,2) \n",
    "t1 = torch.cat([tensor1, tensor2], dim=0)  # 纵向连接\n",
    "t2 = torch.cat((tensor1, tensor3), dim=1)  # 横向连接,用()和[]均可\n",
    "print(tensor1.shape, tensor2.shape, tensor3.shape, t1.shape, t2.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1f2a084d",
   "metadata": {},
   "source": [
    "torch.cat函数返回张量维度不会变，只是连接维度上的值变为各个被链接的张量在该维度的值之和。除了torch.cat函数外，torch.stack函数也可以进行连接，不过各个被链接的张量的各个维度值都应一致，返回的新张量维度会多增加一维。示例如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "7837646c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([2, 4]) torch.Size([2, 4]) torch.Size([2, 4]) torch.Size([3, 2, 4]) torch.Size([2, 3, 4]) torch.Size([2, 4, 3])\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "(tensor([[[ 5., 79., 23., 43.],\n",
       "          [30., 27., 23., 83.]],\n",
       " \n",
       "         [[ 0.,  0.,  0.,  0.],\n",
       "          [ 0.,  0.,  0.,  0.]],\n",
       " \n",
       "         [[ 1.,  1.,  1.,  1.],\n",
       "          [ 1.,  1.,  1.,  1.]]]),\n",
       " tensor([[[ 5., 79., 23., 43.],\n",
       "          [ 0.,  0.,  0.,  0.],\n",
       "          [ 1.,  1.,  1.,  1.]],\n",
       " \n",
       "         [[30., 27., 23., 83.],\n",
       "          [ 0.,  0.,  0.,  0.],\n",
       "          [ 1.,  1.,  1.,  1.]]]),\n",
       " tensor([[[ 5.,  0.,  1.],\n",
       "          [79.,  0.,  1.],\n",
       "          [23.,  0.,  1.],\n",
       "          [43.,  0.,  1.]],\n",
       " \n",
       "         [[30.,  0.,  1.],\n",
       "          [27.,  0.,  1.],\n",
       "          [23.,  0.,  1.],\n",
       "          [83.,  0.,  1.]]]))"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tensor1 = torch.randint(1,100,(2, 4))\n",
    "tensor2 = torch.zeros(2,4) \n",
    "tensor3 = torch.ones(2,4) \n",
    "t1 = torch.stack([tensor1, tensor2, tensor3],dim=0)\n",
    "t2 = torch.stack((tensor1, tensor2, tensor3),dim=1)\n",
    "t3 = torch.stack((tensor1, tensor2, tensor3),dim=2)\n",
    "print(tensor1.shape, tensor2.shape, tensor3.shape, t1.shape, t2.shape, t3.shape)\n",
    "t1,t2,t3"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dbe0fd68",
   "metadata": {},
   "source": [
    "（5）数学运算\n",
    "\n",
    "张量作为矩阵，可以进行加、减、乘、转置、按元素乘、按元素除等操作。示例如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "357fb969",
   "metadata": {},
   "outputs": [],
   "source": [
    "tensor1 = torch.randint(1,100,(2, 4))\n",
    "tensor2 = torch.ones(2,4) \n",
    "t_add1 = tensor1 + tensor2  # 张量加法，本质上是按元素相加\n",
    "t_add2 = tensor1.add(tensor2)  # 与上面的操作是一致的\n",
    "t_add3 = torch.add(tensor1, tensor2)  # 与上面的操作是一致的\n",
    "\n",
    "t_add4 = tensor1 + 3  # 张量所有元素都加3，得到新的张量，原张量未改变\n",
    "t_add5 = tensor1.add(3)  # 与上面的操作是一致的\n",
    "t_add6 = torch.add(tensor1, 3)  # 与上面的操作是一致的\n",
    "\n",
    "t_add7 = tensor1.add_(3)  # 张量所有元素都加3，原张量tensor1也被修改。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "96edb383",
   "metadata": {},
   "source": [
    "上面的例子中实现了张量的加操作。张量的各种基本运算和例中的加操作类型，都可以有多种操作方法。通常操作函数，如add，后加“_”表示执行完运算后，原张量也被修改。下面给出一些示例："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "054fd68b",
   "metadata": {},
   "outputs": [],
   "source": [
    "t_sub1 = tensor1 - tensor2  # 张量减法\n",
    "t_sub2 = tensor1.sub(tensor2)  # 与上面的操作是一致的\n",
    "t_sub4 = 1 - tensor2  # 张量所有元素都被1减\n",
    "t_sub6 = torch.sub(tensor1, 1)  # 张量所有元素都减1\n",
    "\n",
    "#张量乘法，参与运算的张量类型需一致\n",
    "t_matmul1 = tensor1.float() @ tensor2.T  # T属性表示张量转置,按单精度浮点数(32bit)进行相乘\n",
    "t_matmul2 = tensor1.matmul(tensor2.T.long())  # 与上面的操作不同，两张量按长整型(64bit)进行相乘\n",
    "\n",
    "#按元素相乘，参加运算的张量形状应一致\n",
    "t_mul1 = tensor1 * tensor2  \n",
    "t_mul3 = torch.mul(tensor1, tensor2)  # 与上面的操作是一致的\n",
    "t_mul4 = tensor1 * 3  # 张量每元素都乘以3\n",
    "t_mul5 = tensor1.mul(3)  # 与上面的操作是一致的\n",
    "\n",
    "#按元素相除，参加运算的张量形状应一致\n",
    "t_div1 = tensor1 / tensor1  \n",
    "t_div2 = tensor1.div(tensor2) \n",
    "t_div3 = torch.div(tensor1, tensor2)  # 与上面的操作是一致的"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7d06a196",
   "metadata": {},
   "source": [
    "我们在实际应用中，会经常使用一维或二维张量的乘法操作，PyTorch也提供了一些常用的特殊运算函数，如dot函数、mm函数等，示例如下。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "37201b61",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(True)\n",
      "tensor([[True, True],\n",
      "        [True, True]])\n",
      "torch.Size([2])\n",
      "tensor([[True, True]])\n"
     ]
    }
   ],
   "source": [
    "t1=torch.randn((5))  # 一维张量\n",
    "t2=torch.ones((5))  # 一维张量\n",
    "t3=torch.randn((2,5))  # 二维张量\n",
    "t4=torch.ones((5,2))  # 二维张量\n",
    "t_d1 = torch.dot(t1, t2)  # dot函数仅支持两个一维向量的点集\n",
    "t_d2 = torch.matmul(t1, t2)  # 与上面的操作是一致的\n",
    "print(t_d1 == t_d2)  # 观察两个结果是否相同\n",
    "\n",
    "t_m1 = torch.mm(t3, t4)  # mm函数仅支持两个二维张量的相乘\n",
    "t_m2 = torch.matmul(t3, t4)  # 与上面的操作一致\n",
    "print(t_m1 == t_m2)  # 观察两个结果是否相同\n",
    "\n",
    "t_n1 = torch.matmul(t3, t1) # 二维张量和一维张量相乘，一维张量自动维度扩展, 结果会删掉扩展维度\n",
    "print(t_n1.shape)  # 打印计算结果形状\n",
    "t_n2 = torch.matmul(t3, t1.view(5,1)).T # 手动扩展进行计算，与上面的操作结果一致\n",
    "print(t_n1 == t_n2)  # 观察两个结果是否相同"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "da3d3f1b",
   "metadata": {},
   "source": [
    "若要取张量中的某个元素，变为普通的数据类型进行运算，可使用item()函数。例如："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "cf90b2fd",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([7, 9, 3, 0, 1, 8, 6, 2, 4, 5])\n",
      "tensor(45) <class 'torch.Tensor'>\n",
      "45 <class 'int'>\n"
     ]
    }
   ],
   "source": [
    "tensor = torch.randperm(10)\n",
    "sum = tensor.sum()\n",
    "sum_item = sum.item()\n",
    "print(tensor)\n",
    "print(sum, type(sum))\n",
    "print(sum_item, type(sum_item))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7f6e0f63",
   "metadata": {},
   "source": [
    "（6）张量的其他运算\n",
    "\n",
    "PyTorch提供的张量运算函数是非常丰富的，如幂运算、指数运算（以e为底数）、对数运算、近似运算（如取整）、统计运算（如取平均值）等等，在此不再给出示例，只列举一些常用的函数，读者可自行练习这些函数的使用方法。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fa723e3f",
   "metadata": {},
   "source": [
    "## 5.2.2 Autograd自动求导\n",
    "\n",
    "神经网络模型训练时，最常用的是BP算法，即反向传播算法。模型参数根据梯度进行学习，为了计算梯度，PyTorch提供了Autograd支持自动求导来计算梯度。\n",
    "下面以最简单的一层神经网络为例，给出Autograd进行反向传播优化参数的示例。例子中，设x和y是真实数据，z为模型预测值，设z = w * x + b，则w和b是模型需要优化的参数，通过loss损失可根据梯度下降法来优化w和b。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "bc90e36a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[0.3267, 0.3003, 0.0754],\n",
      "        [0.3267, 0.3003, 0.0754],\n",
      "        [0.3267, 0.3003, 0.0754],\n",
      "        [0.3267, 0.3003, 0.0754],\n",
      "        [0.3267, 0.3003, 0.0754]])\n",
      "tensor([0.3267, 0.3003, 0.0754])\n"
     ]
    }
   ],
   "source": [
    "import torch\n",
    "from torch.nn.functional import binary_cross_entropy_with_logits\n",
    "x = torch.ones(5)  # 输入x设为[1.,1.,1.,1.,1.]\n",
    "y = torch.zeros(3)  # 输出y设为[0.,0.,0.]\n",
    "w = torch.randn(5, 3, requires_grad=True)  # w形状为[5, 3]，梯度计算设为True\n",
    "b = torch.randn(3, requires_grad=True)  # b形状为[3, 1]，梯度计算设为True\n",
    "z = torch.matmul(x, w) + b  # z = w * x + b\n",
    "loss = binary_cross_entropy_with_logits(z, y)  # 计算损失根据loss来优化网络参数\n",
    "loss.backward()  # 损失反向传播进行自动求导，得到参数梯度\n",
    "print(w.grad)  # 输出w的梯度，存储在w的grad属性中，grad属性也是张量\n",
    "print(b.grad)  # 输出b的梯度"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f1f1fb97",
   "metadata": {},
   "source": [
    "上例中，得到了w和b的梯度后，就可以通过设置学习率来修正w和b的值了，实际上修正的方法PyTorch也封装好了，后续的章节中将给出具体的示例。\n",
    "\n",
    "默认情况下，所有张量的requires_grad属性被设置为True，表示都在跟踪梯度历史，但是有些情况下并不需要跟踪梯度，比如测试集上的评估就不需要反向传播来跟踪梯度，可用以下代码来停止对计算结果的梯度跟踪\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "6f32f3a4",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "True\n",
      "False\n"
     ]
    }
   ],
   "source": [
    "z = torch.matmul(x, w)+b  # z = w * x + b\n",
    "print(z.requires_grad)\n",
    "with torch.no_grad():  # 禁用梯度跟踪\n",
    "    z = torch.matmul(x, w)+b\n",
    "print(z.requires_grad)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2cea39a0",
   "metadata": {},
   "source": [
    "禁用梯度跟踪也可以使用下面的方法："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "322dba39",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "False\n"
     ]
    }
   ],
   "source": [
    "z = torch.matmul(x, w)+b\n",
    "z_det = z.detach()\n",
    "print(z_det.requires_grad)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "589743cc",
   "metadata": {},
   "source": [
    "后续的实战中，读者将会发现，在进行深度模型训练时，一般都需要调用损失张量的backward()函数进行自动求导。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}

一键复制编辑原始数据按行查看历史

提交于 2022-09-30 16:03 +08:00 . add chap5 codes and readme.md

马建仓 AI 助手

尝试更多

代码解读

代码找茬

代码优化

Python

https://gitee.com/flycity/ai_tutorial_book.git

git@gitee.com:flycity/ai_tutorial_book.git

flycity

ai_tutorial_book

master

飞城/ai_tutorial_book

简介

发行版

贡献者

语言

近期动态

飞城/ai_tutorial_book .gitee-modal { width: 500px !important; }

简介

发行版

开源评估指数源自 OSS-Compass 评估体系，评估体系围绕以下三个维度对项目展开评估：

贡献者

语言

近期动态

搜索帮助

飞城/ai_tutorial_book