tutorials/source_zh_cn/beginner/model.ipynb · MindSpore/docs

加入 Gitee

与超过 1200万开发者一起发现、参与优秀开源项目，私有仓库也完全免费：）

克隆/下载

model.ipynb 15.77 KB

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[![下载Notebook](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/master/resource/_static/logo_notebook.svg)](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/notebook/master/tutorials/zh_cn/beginner/mindspore_model.ipynb)&emsp;[![下载样例代码](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/master/resource/_static/logo_download_code.svg)](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/notebook/master/tutorials/zh_cn/beginner/mindspore_model.py)&emsp;[![查看源文件](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/master/resource/_static/logo_source.svg)](https://gitee.com/mindspore/docs/blob/master/tutorials/source_zh_cn/beginner/model.ipynb)\n",
    "\n",
    "[基本介绍](https://www.mindspore.cn/tutorials/zh-CN/master/beginner/introduction.html) || [快速入门](https://www.mindspore.cn/tutorials/zh-CN/master/beginner/quick_start.html) || [张量 Tensor](https://www.mindspore.cn/tutorials/zh-CN/master/beginner/tensor.html) || [数据集 Dataset](https://www.mindspore.cn/tutorials/zh-CN/master/beginner/dataset.html) || [数据变换 Transforms](https://www.mindspore.cn/tutorials/zh-CN/master/beginner/transforms.html) || **网络构建** || [函数式自动微分](https://www.mindspore.cn/tutorials/zh-CN/master/beginner/autograd.html) || [模型训练](https://www.mindspore.cn/tutorials/zh-CN/master/beginner/train.html) || [保存与加载](https://www.mindspore.cn/tutorials/zh-CN/master/beginner/save_load.html) || [使用静态图加速](https://www.mindspore.cn/tutorials/zh-CN/master/beginner/accelerate_with_static_graph.html)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 网络构建"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "神经网络模型是由神经网络层和Tensor操作构成的，[mindspore.nn](https://www.mindspore.cn/docs/zh-CN/master/api_python/mindspore.nn.html)提供了常见神经网络层的实现，在MindSpore中，[Cell](https://www.mindspore.cn/docs/zh-CN/master/api_python/nn/mindspore.nn.Cell.html)类是构建所有网络的基类，也是网络的基本单元。一个神经网络模型表示为一个`Cell`，它由不同的子`Cell`构成。使用这样的嵌套结构，可以简单地使用面向对象编程的思维，对神经网络结构进行构建和管理。\n",
    "\n",
    "下面我们将构建一个用于Mnist数据集分类的神经网络模型。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import mindspore\n",
    "from mindspore import nn, ops"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## 定义模型类\n",
    "\n",
    "当我们定义神经网络时，可以继承`nn.Cell`类，在`__init__`方法中进行子Cell的实例化和状态管理，在`construct`方法中实现Tensor操作。\n",
    "\n",
    "> `construct`意为神经网络（计算图）构建，相关内容详见[使用静态图加速](https://www.mindspore.cn/tutorials/zh-CN/master/beginner/accelerate_with_static_graph.html)。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "pycharm": {
     "is_executing": true,
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "class Network(nn.Cell):\n",
    "    def __init__(self):\n",
    "        super().__init__()\n",
    "        self.flatten = nn.Flatten()\n",
    "        self.dense_relu_sequential = nn.SequentialCell(\n",
    "            nn.Dense(28*28, 512, weight_init=\"normal\", bias_init=\"zeros\"),\n",
    "            nn.ReLU(),\n",
    "            nn.Dense(512, 512, weight_init=\"normal\", bias_init=\"zeros\"),\n",
    "            nn.ReLU(),\n",
    "            nn.Dense(512, 10, weight_init=\"normal\", bias_init=\"zeros\")\n",
    "        )\n",
    "\n",
    "    def construct(self, x):\n",
    "        x = self.flatten(x)\n",
    "        logits = self.dense_relu_sequential(x)\n",
    "        return logits"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "构建完成后，实例化`Network`对象，并查看其结构。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Network<\n",
       "  (flatten): Flatten<>\n",
       "  (dense_relu_sequential): SequentialCell<\n",
       "    (0): Dense<input_channels=784, output_channels=512, has_bias=True>\n",
       "    (1): ReLU<>\n",
       "    (2): Dense<input_channels=512, output_channels=512, has_bias=True>\n",
       "    (3): ReLU<>\n",
       "    (4): Dense<input_channels=512, output_channels=10, has_bias=True>\n",
       "    >\n",
       "  >"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model = Network()\n",
    "print(model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们构造一个输入数据，直接调用模型，可以获得一个十维的Tensor输出，其包含每个类别的原始预测值。\n",
    "\n",
    "> `model.construct()`方法不可直接调用。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Tensor(shape=[1, 10], dtype=Float32, value=\n",
       "[[-5.08734025e-04,  3.39190010e-04,  4.62840870e-03 ... -1.20305456e-03, -5.05689112e-03,  3.99264274e-03]])"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X = ops.ones((1, 28, 28), mindspore.float32)\n",
    "logits = model(X)\n",
    "# print logits\n",
    "logits"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在此基础上，我们通过一个`nn.Softmax`层实例来获得预测概率。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Predicted class: [4]\n"
     ]
    }
   ],
   "source": [
    "pred_probab = nn.Softmax(axis=1)(logits)\n",
    "y_pred = pred_probab.argmax(1)\n",
    "print(f\"Predicted class: {y_pred}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## 模型层\n",
    "\n",
    "本节中我们分解上节构造的神经网络模型中的每一层。首先我们构造一个shape为(3, 28, 28)的随机数据（3个28x28的图像），依次通过每一个神经网络层来观察其效果。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(3, 28, 28)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "input_image = ops.ones((3, 28, 28), mindspore.float32)\n",
    "print(input_image.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### nn.Flatten\n",
    "\n",
    "实例化[nn.Flatten](https://www.mindspore.cn/docs/zh-CN/master/api_python/nn/mindspore.nn.Flatten.html)层，将28x28的2D张量转换为784大小的连续数组。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(3, 784)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "flatten = nn.Flatten()\n",
    "flat_image = flatten(input_image)\n",
    "print(flat_image.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "### nn.Dense\n",
    "\n",
    "[nn.Dense](https://www.mindspore.cn/docs/zh-CN/master/api_python/nn/mindspore.nn.Dense.html)为全连接层，其使用权重和偏差对输入进行线性变换。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(3, 20)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "layer1 = nn.Dense(in_channels=28*28, out_channels=20)\n",
    "hidden1 = layer1(flat_image)\n",
    "print(hidden1.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### nn.ReLU\n",
    "\n",
    "[nn.ReLU](https://www.mindspore.cn/docs/zh-CN/master/api_python/nn/mindspore.nn.ReLU.html)层给网络中加入非线性的激活函数，帮助神经网络学习各种复杂的特征。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Before ReLU: [[-0.04736331  0.2939465  -0.02713677 -0.30988005 -0.11504349 -0.11661264\n",
      "   0.18007928  0.43213072  0.12091967 -0.17465964  0.53133243  0.12605792\n",
      "   0.01825903  0.01287796  0.17238477 -0.1621131  -0.0080034  -0.24523425\n",
      "  -0.10083733  0.05171938]\n",
      " [-0.04736331  0.2939465  -0.02713677 -0.30988005 -0.11504349 -0.11661264\n",
      "   0.18007928  0.43213072  0.12091967 -0.17465964  0.53133243  0.12605792\n",
      "   0.01825903  0.01287796  0.17238477 -0.1621131  -0.0080034  -0.24523425\n",
      "  -0.10083733  0.05171938]\n",
      " [-0.04736331  0.2939465  -0.02713677 -0.30988005 -0.11504349 -0.11661264\n",
      "   0.18007928  0.43213072  0.12091967 -0.17465964  0.53133243  0.12605792\n",
      "   0.01825903  0.01287796  0.17238477 -0.1621131  -0.0080034  -0.24523425\n",
      "  -0.10083733  0.05171938]]\n",
      "\n",
      "\n",
      "After ReLU: [[0.         0.2939465  0.         0.         0.         0.\n",
      "  0.18007928 0.43213072 0.12091967 0.         0.53133243 0.12605792\n",
      "  0.01825903 0.01287796 0.17238477 0.         0.         0.\n",
      "  0.         0.05171938]\n",
      " [0.         0.2939465  0.         0.         0.         0.\n",
      "  0.18007928 0.43213072 0.12091967 0.         0.53133243 0.12605792\n",
      "  0.01825903 0.01287796 0.17238477 0.         0.         0.\n",
      "  0.         0.05171938]\n",
      " [0.         0.2939465  0.         0.         0.         0.\n",
      "  0.18007928 0.43213072 0.12091967 0.         0.53133243 0.12605792\n",
      "  0.01825903 0.01287796 0.17238477 0.         0.         0.\n",
      "  0.         0.05171938]]\n"
     ]
    }
   ],
   "source": [
    "print(f\"Before ReLU: {hidden1}\\n\\n\")\n",
    "hidden1 = nn.ReLU()(hidden1)\n",
    "print(f\"After ReLU: {hidden1}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "### nn.SequentialCell\n",
    "\n",
    "[nn.SequentialCell](https://www.mindspore.cn/docs/zh-CN/master/api_python/nn/mindspore.nn.SequentialCell.html)是一个有序的Cell容器。输入Tensor将按照定义的顺序通过所有Cell。我们可以使用`SequentialCell`来快速组合构造一个神经网络模型。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(3, 10)"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "seq_modules = nn.SequentialCell(\n",
    "    flatten,\n",
    "    layer1,\n",
    "    nn.ReLU(),\n",
    "    nn.Dense(20, 10)\n",
    ")\n",
    "\n",
    "logits = seq_modules(input_image)\n",
    "print(logits.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### nn.Softmax\n",
    "\n",
    "最后使用[nn.Softmax](https://www.mindspore.cn/docs/zh-CN/master/api_python/nn/mindspore.nn.Softmax.html)将神经网络最后一个全连接层返回的logits的值缩放为\\[0, 1\\]，表示每个类别的预测概率。`axis`指定的维度数值和为1。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "softmax = nn.Softmax(axis=1)\n",
    "pred_probab = softmax(logits)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## 模型参数\n",
    "\n",
    "网络内部神经网络层具有权重参数和偏置参数（如`nn.Dense`），这些参数会在训练过程中不断进行优化，可通过 `model.parameters_and_names()` 来获取参数名及对应的参数详情。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model structure: Network<\n",
      "  (flatten): Flatten<>\n",
      "  (dense_relu_sequential): SequentialCell<\n",
      "    (0): Dense<input_channels=784, output_channels=512, has_bias=True>\n",
      "    (1): ReLU<>\n",
      "    (2): Dense<input_channels=512, output_channels=512, has_bias=True>\n",
      "    (3): ReLU<>\n",
      "    (4): Dense<input_channels=512, output_channels=10, has_bias=True>\n",
      "    >\n",
      "  >\n",
      "\n",
      "\n",
      "Layer: dense_relu_sequential.0.weight\n",
      "Size: (512, 784)\n",
      "Values : [[-0.01491369  0.00353318 -0.00694948 ...  0.01226766 -0.00014423\n",
      "   0.00544263]\n",
      " [ 0.00212971  0.0019974  -0.00624789 ... -0.01214037  0.00118004\n",
      "  -0.01594325]] \n",
      "\n",
      "Layer: dense_relu_sequential.0.bias\n",
      "Size: (512,)\n",
      "Values : [0. 0.] \n",
      "\n",
      "Layer: dense_relu_sequential.2.weight\n",
      "Size: (512, 512)\n",
      "Values : [[ 0.00565423  0.00354313  0.00637383 ... -0.00352688  0.00262949\n",
      "   0.01157355]\n",
      " [-0.01284141  0.00657666 -0.01217057 ...  0.00318963  0.00319115\n",
      "  -0.00186801]] \n",
      "\n",
      "Layer: dense_relu_sequential.2.bias\n",
      "Size: (512,)\n",
      "Values : [0. 0.] \n",
      "\n",
      "Layer: dense_relu_sequential.4.weight\n",
      "Size: (10, 512)\n",
      "Values : [[ 0.0087168  -0.00381866 -0.00865665 ... -0.00273731 -0.00391623\n",
      "   0.00612853]\n",
      " [-0.00593031  0.0008721  -0.0060081  ... -0.00271535 -0.00850481\n",
      "  -0.00820513]] \n",
      "\n",
      "Layer: dense_relu_sequential.4.bias\n",
      "Size: (10,)\n",
      "Values : [0. 0.] \n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(f\"Model structure: {model}\\n\\n\")\n",
    "\n",
    "for name, param in model.parameters_and_names():\n",
    "    print(f\"Layer: {name}\\nSize: {param.shape}\\nValues : {param[:2]} \\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "更多内置神经网络层详见[mindspore.nn API](https://www.mindspore.cn/docs/zh-CN/master/api_python/mindspore.nn.html)。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "MindSpore",
   "language": "python",
   "name": "mindspore"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}

一键复制编辑原始数据按行查看历史

提交于 2023-11-08 11:38 . add links in files in 2.2

https://gitee.com/mindspore/docs.git

git@gitee.com:mindspore/docs.git

mindspore

docs

master

MindSpore / docs

简介

发行版

贡献者

近期动态

MindSpore / docs .gitee-modal { width: 500px !important; }

简介

发行版

开源评估指数源自 OSS-Compass 评估体系，评估体系围绕以下三个维度对项目展开评估：

贡献者

近期动态

搜索帮助

MindSpore / docs