diff --git a/MindFlow/applications/data_driven/burgers/fno1d/FNO1D.ipynb b/MindFlow/applications/data_driven/burgers/fno1d/FNO1D.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..95aa9d383d48bb47ff8b53c02713216d2e02c905 --- /dev/null +++ b/MindFlow/applications/data_driven/burgers/fno1d/FNO1D.ipynb @@ -0,0 +1,600 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "pycharm": { + "name": "#%% md\n" + } + }, + "source": [ + "# Solve Burgers' equation based on Fourier Neural Operator\n", + "\n", + "[![DownloadNotebook](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/master/resource/_static/logo_notebook_en.png)](https://obs.dualstack.cn-north-4.myhuaweicloud.com/mindspore-website/notebook/master/mindflow/en/data_driven/mindspore_burgers_FNO1D.ipynb) [![DownloadCode](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/master/resource/_static/logo_download_code_en.png)](https://obs.dualstack.cn-north-4.myhuaweicloud.com/mindspore-website/notebook/master/mindflow/en/data_driven/mindspore_burgers_FNO1D.py) [![ViewSource](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/master/resource/_static/logo_source_en.png)](https://gitee.com/mindspore/docs/blob/master/docs/mindflow/docs/source_en/data_driven/burgers_FNO1D.ipynb)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "pycharm": { + "name": "#%% md\n" + } + }, + "source": [ + "## Overview\n", + "\n", + "Computational fluid dynamics is one of the most important techniques in the field of fluid mechanics in the 21st century. The flow analysis, prediction and control can be realized by solving the governing equations of fluid mechanics by numerical method. Traditional finite element method (FEM) and finite difference method (FDM) are inefficient because of the complex simulation process (physical modeling, meshing, numerical discretization, iterative solution, etc.) and high computing costs. Therefore, it is necessary to improve the efficiency of fluid simulation with AI.\n", + "\n", + "Machine learning methods provide a new paradigm for scientific computing by providing a fast solver similar to traditional methods. Classical neural networks learn mappings between finite dimensional spaces and can only learn solutions related to a specific discretization. Different from traditional neural networks, Fourier Neural Operator (FNO) is a new deep learning architecture that can learn mappings between infinite-dimensional function spaces. It directly learns mappings from arbitrary function parameters to solutions to solve a class of partial differential equations. Therefore, it has a stronger generalization capability. More information can be found in the paper, [Fourier Neural Operator for Parametric Partial Differential Equations](https://arxiv.org/abs/2010.08895).\n", + "\n", + "This tutorial describes how to solve the 1-d Burgers' equation using Fourier neural operator." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "pycharm": { + "name": "#%% md\n" + } + }, + "source": [ + "## Burgers' equation\n", + "\n", + "The 1-d Burgers’ equation is a non-linear PDE with various applications including modeling the one\n", + "dimensional flow of a viscous fluid. It takes the form\n", + "\n", + "$$\n", + "\\partial_t u(x, t)+\\partial_x (u^2(x, t)/2)=\\nu \\partial_{xx} u(x, t), \\quad x \\in(0,1), t \\in(0, 1]\n", + "$$\n", + "\n", + "$$\n", + "u(x, 0)=u_0(x), \\quad x \\in(0,1)\n", + "$$\n", + "\n", + "where $u$ is the velocity field, $u_0$ is the initial condition and $\\nu$ is the viscosity coefficient.\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Problem Description\n", + "\n", + "We aim to learn the operator mapping the initial condition to the solution at time one:\n", + "\n", + "$$\n", + "u_0 \\mapsto u(\\cdot, 1)\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Technology Path\n", + "\n", + "MindFlow solves the problem as follows:\n", + "\n", + "1. Training Dataset Construction.\n", + "2. Model Construction.\n", + "3. Optimizer and Loss Function.\n", + "4. Model Training." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Fourier Neural Operator\n", + "\n", + "The Fourier Neural Operator consists of the Lifting Layer, Fourier Layers, and the Decoding Layer.\n", + "\n", + "![Fourier Neural Operator model structure](images/FNO.png)\n", + "\n", + "Fourier layers: Start from input V. On top: apply the Fourier transform $\\mathcal{F}$; a linear transform R on the lower Fourier modes and filters out the higher modes; then apply the inverse Fourier transform $\\mathcal{F}^{-1}$. On the bottom: apply a local linear transform W. Finally, the Fourier Layer output vector is obtained through the activation function.\n", + "\n", + "![Fourier Layer structure](images/FNO-2.png)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "#add search path\n", + "import sys\n", + "import os\n", + "project_root = os.path.abspath(\"../../../../..\")\n", + "sys.path.append(project_root)" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "pycharm": { + "name": "#%%\n" + } + }, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "/home/mindspore/miniconda/envs/jupyter/lib/python3.9/site-packages/numpy/core/getlimits.py:549: UserWarning: The value of the smallest subnormal for type is zero.\n", + " setattr(self, word, getattr(machar, word).flat[0])\n", + "/home/mindspore/miniconda/envs/jupyter/lib/python3.9/site-packages/numpy/core/getlimits.py:89: UserWarning: The value of the smallest subnormal for type is zero.\n", + " return self._float_to_str(self.smallest_subnormal)\n", + "/home/mindspore/miniconda/envs/jupyter/lib/python3.9/site-packages/numpy/core/getlimits.py:549: UserWarning: The value of the smallest subnormal for type is zero.\n", + " setattr(self, word, getattr(machar, word).flat[0])\n", + "/home/mindspore/miniconda/envs/jupyter/lib/python3.9/site-packages/numpy/core/getlimits.py:89: UserWarning: The value of the smallest subnormal for type is zero.\n", + " return self._float_to_str(self.smallest_subnormal)\n" + ] + } + ], + "source": [ + "import sys\n", + "sys.path.append(\"/home/mindspore/work/mindscience_newest\")\n", + "\n", + "import os\n", + "import time\n", + "import numpy as np\n", + "\n", + "from mindspore.amp import DynamicLossScaler, auto_mixed_precision, all_finite\n", + "from mindspore import context, nn, Tensor, set_seed, ops, data_sink, jit, save_checkpoint\n", + "from mindspore import dtype as mstype\n", + "from mindscience import FNO1D, RelativeRMSELoss, load_yaml_config, get_warmup_cosine_annealing_lr\n", + "from mindscience.pde import UnsteadyFlowWithLoss" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The following `src` pacakage can be downloaded in [applications/data_driven/burgers/fno1d/src](https://gitee.com/mindspore/mindscience/tree/master/MindFlow/applications/data_driven/burgers/fno1d/src)." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "pycharm": { + "name": "#%%\n" + } + }, + "outputs": [], + "source": [ + "from src import create_training_dataset\n", + "\n", + "set_seed(0)\n", + "np.random.seed(0)\n", + "\n", + "context.set_context(mode=context.GRAPH_MODE, device_target=\"Ascend\", device_id=0,max_device_memory=\"32GB\")\n", + "use_ascend = context.get_context(attr_key='device_target') == \"Ascend\"" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "You can get parameters of model, data and optimizer from `config`." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": { + "pycharm": { + "name": "#%%\n" + } + }, + "outputs": [], + "source": [ + "config = load_yaml_config('./configs/fno1d.yaml')\n", + "data_params = config[\"data\"]\n", + "model_params = config[\"model\"]\n", + "optimizer_params = config[\"optimizer\"]" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "pycharm": { + "name": "#%% md\n" + } + }, + "source": [ + "## Training Dataset Construction\n", + "\n", + "Download the training and test dataset: [data_driven/burgers/dataset](https://download.mindspore.cn/mindscience/mindflow/dataset/applications/data_driven/burgers/dataset/) .\n", + "\n", + "In this case, training datasets and test datasets are generated according to Zongyi Li's dataset in [Fourier Neural Operator for Parametric Partial Differential Equations](https://arxiv.org/pdf/2010.08895.pdf) . The settings are as follows:\n", + "\n", + "the initial condition $u_0(x)$ is generated according to periodic boundary conditions:\n", + "\n", + "$$\n", + "u_0 \\sim \\mu, \\mu=\\mathcal{N}\\left(0,625(-\\Delta+25 I)^{-2}\\right)\n", + "$$\n", + "\n", + "We set the viscosity to $\\nu=0.1$ and solve the equation using a split step method where the heat equation part is solved exactly in Fourier space then the non-linear part is advanced, again in Fourier space, using a very fine forward Euler method. The number of samples in the training set is 1000, and the number of samples in the test set is 200.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": { + "pycharm": { + "name": "#%%\n" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Data preparation finished\n", + "input_path: (1000, 1024, 1)\n", + "label_path: (1000, 1024)\n" + ] + } + ], + "source": [ + "# create training dataset\n", + "train_dataset = create_training_dataset(data_params, model_params, shuffle=True)\n", + "\n", + "# create test dataset\n", + "test_input, test_label = np.load(os.path.join(data_params[\"root_dir\"], \"test/inputs.npy\")), \\\n", + " np.load(os.path.join(data_params[\"root_dir\"], \"test/label.npy\"))\n", + "test_input = Tensor(np.expand_dims(test_input, -2), mstype.float32)\n", + "test_label = Tensor(np.expand_dims(test_label, -2), mstype.float32)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "pycharm": { + "name": "#%% md\n" + } + }, + "source": [ + "## Model Construction\n", + "\n", + "The network is composed of 1 lifting layer, multiple Fourier layers and 1 decoding layer:\n", + "\n", + "- The Lifting layer corresponds to the `FNO1D.fc0` in the case, and maps the output data $x$ to the high dimension;\n", + "\n", + "- Multi-layer Fourier Layer corresponds to the `FNO1D.fno_seq` in the case. Discrete Fourier transform is used to realize the conversion between time domain and frequency domain;\n", + "\n", + "- The Decoding layer corresponds to `FNO1D.fc1` and `FNO1D.fc2` in the case to obtain the final predictive value.\n", + "\n", + "The initialization of the model based on the network above, parameters can be modified in [configuration file](https://gitee.com/mindspore/mindscience/blob/master/MindFlow/applications/data_driven/burgers/fno1d/configs/fno1d.yaml)." + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": { + "pycharm": { + "name": "#%%\n" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "name:FNO1D_in_channels:1_out_channels:1_modes:16_resolutions:1024_hidden_channels:64_depths:4\n" + ] + } + ], + "source": [ + "model = FNO1D(in_channels=model_params[\"in_channels\"],\n", + " out_channels=model_params[\"out_channels\"],\n", + " n_modes=model_params[\"modes\"],\n", + " resolutions=model_params[\"resolutions\"],\n", + " hidden_channels=model_params[\"hidden_channels\"],\n", + " n_layers=model_params[\"depths\"],\n", + " projection_channels=4*model_params[\"hidden_channels\"],\n", + " )\n", + "\n", + "model_params_list = []\n", + "for k, v in model_params.items():\n", + " model_params_list.append(f\"{k}:{v}\")\n", + "model_name = \"_\".join(model_params_list)\n", + "print(model_name)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "pycharm": { + "name": "#%% md\n" + } + }, + "source": [ + "## Optimizer and Loss Function" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": { + "pycharm": { + "name": "#%%\n" + } + }, + "outputs": [], + "source": [ + "steps_per_epoch = train_dataset.get_dataset_size()\n", + "lr = get_warmup_cosine_annealing_lr(lr_init=optimizer_params[\"learning_rate\"],\n", + " last_epoch=optimizer_params[\"epochs\"],\n", + " steps_per_epoch=steps_per_epoch,\n", + " warmup_epochs=1)\n", + "optimizer = nn.Adam(model.trainable_params(), learning_rate=Tensor(lr))\n", + "\n", + "if use_ascend:\n", + " from mindspore.amp import DynamicLossScaler, auto_mixed_precision, all_finite\n", + " loss_scaler = DynamicLossScaler(1024, 2, 100)\n", + " auto_mixed_precision(model, 'O3')\n", + "else:\n", + " loss_scaler = None" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "pycharm": { + "name": "#%% md\n" + } + }, + "source": [ + "## Model Training\n", + "\n", + "With **MindSpore version >= 2.0.0**, we can use the functional programming for training neural networks. `MindFlow` provide a training interface for unsteady problems `UnsteadyFlowWithLoss` for model training and evaluation." + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": { + "pycharm": { + "name": "#%%\n" + }, + "tags": [] + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "./summary/name:FNO1D_in_channels:1_out_channels:1_modes:16_resolutions:1024_hidden_channels:64_depths:4\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "[ERROR] CORE(31531,ffffb57ca020,python3.9):2025-11-15-09:43:56.448.210 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_31531/2341691899.py]\n", + "[ERROR] CORE(31531,ffffb57ca020,python3.9):2025-11-15-09:43:56.450.846 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_31531/2341691899.py]\n", + "[ERROR] CORE(31531,ffffb57ca020,python3.9):2025-11-15-09:43:56.450.902 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_31531/2341691899.py]\n", + "[ERROR] CORE(31531,ffffb57ca020,python3.9):2025-11-15-09:43:56.451.086 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_31531/2341691899.py]\n", + "[ERROR] CORE(31531,ffffb57ca020,python3.9):2025-11-15-09:43:56.451.135 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_31531/2341691899.py]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "..epoch: 1 train loss: 0.23465186 epoch time: 19.78s step time: 0.1582s\n", + "epoch: 2 train loss: 0.28419363 epoch time: 2.12s step time: 0.0170s\n", + "epoch: 3 train loss: 0.31122661 epoch time: 2.07s step time: 0.0165s\n", + "epoch: 4 train loss: 0.28644967 epoch time: 2.07s step time: 0.0166s\n", + "epoch: 5 train loss: 0.20785393 epoch time: 2.07s step time: 0.0165s\n", + "epoch: 6 train loss: 0.19288626 epoch time: 2.06s step time: 0.0165s\n", + "epoch: 7 train loss: 0.23312779 epoch time: 2.08s step time: 0.0167s\n", + "epoch: 8 train loss: 0.18624401 epoch time: 2.20s step time: 0.0176s\n", + "epoch: 9 train loss: 0.10345152 epoch time: 2.13s step time: 0.0170s\n", + "epoch: 10 train loss: 0.18798406 epoch time: 2.18s step time: 0.0174s\n", + "================================Start Evaluation================================\n", + "mean rms_error: 0.021879202\n", + "=================================End Evaluation=================================\n", + "epoch: 11 train loss: 0.17524227 epoch time: 2.25s step time: 0.0180s\n", + "epoch: 12 train loss: 0.16727453 epoch time: 2.26s step time: 0.0181s\n", + "epoch: 13 train loss: 0.10675637 epoch time: 2.22s step time: 0.0178s\n", + "epoch: 14 train loss: 0.13700883 epoch time: 2.16s step time: 0.0173s\n", + "epoch: 15 train loss: 0.14147925 epoch time: 2.11s step time: 0.0169s\n", + "epoch: 16 train loss: 0.10691701 epoch time: 2.08s step time: 0.0167s\n", + "epoch: 17 train loss: 0.16218747 epoch time: 2.04s step time: 0.0163s\n", + "epoch: 18 train loss: 0.20996606 epoch time: 2.09s step time: 0.0167s\n", + "epoch: 19 train loss: 0.15155949 epoch time: 2.02s step time: 0.0162s\n", + "epoch: 20 train loss: 0.18981066 epoch time: 2.04s step time: 0.0163s\n", + "================================Start Evaluation================================\n", + "mean rms_error: 0.019013295\n", + "=================================End Evaluation=================================\n", + "epoch: 21 train loss: 0.07383593 epoch time: 2.12s step time: 0.0170s\n", + "epoch: 22 train loss: 0.11531584 epoch time: 2.06s step time: 0.0165s\n", + "epoch: 23 train loss: 0.17910039 epoch time: 2.08s step time: 0.0167s\n", + "epoch: 24 train loss: 0.09044659 epoch time: 2.15s step time: 0.0172s\n", + "epoch: 25 train loss: 0.15973650 epoch time: 2.10s step time: 0.0168s\n", + "epoch: 26 train loss: 0.08857839 epoch time: 2.08s step time: 0.0166s\n", + "epoch: 27 train loss: 0.06426008 epoch time: 2.13s step time: 0.0170s\n", + "epoch: 28 train loss: 0.09480067 epoch time: 2.08s step time: 0.0166s\n", + "epoch: 29 train loss: 0.22773507 epoch time: 2.13s step time: 0.0170s\n", + "epoch: 30 train loss: 0.12277951 epoch time: 2.08s step time: 0.0166s\n", + "================================Start Evaluation================================\n", + "mean rms_error: 0.013496651\n", + "=================================End Evaluation=================================\n", + "epoch: 31 train loss: 0.09083001 epoch time: 2.07s step time: 0.0166s\n", + "epoch: 32 train loss: 0.10202033 epoch time: 2.10s step time: 0.0168s\n", + "epoch: 33 train loss: 0.12928744 epoch time: 2.10s step time: 0.0168s\n", + "epoch: 34 train loss: 0.06271134 epoch time: 2.07s step time: 0.0166s\n", + "epoch: 35 train loss: 0.07041215 epoch time: 2.08s step time: 0.0166s\n", + "epoch: 36 train loss: 0.12557518 epoch time: 2.06s step time: 0.0165s\n", + "epoch: 37 train loss: 0.10148985 epoch time: 2.04s step time: 0.0163s\n", + "epoch: 38 train loss: 0.08175293 epoch time: 2.11s step time: 0.0169s\n", + "epoch: 39 train loss: 0.11296615 epoch time: 2.09s step time: 0.0167s\n", + "epoch: 40 train loss: 0.08916582 epoch time: 2.01s step time: 0.0161s\n", + "================================Start Evaluation================================\n", + "mean rms_error: 0.014983387\n", + "=================================End Evaluation=================================\n", + "epoch: 41 train loss: 0.06424745 epoch time: 2.09s step time: 0.0167s\n", + "epoch: 42 train loss: 0.07220607 epoch time: 2.10s step time: 0.0168s\n", + "epoch: 43 train loss: 0.09584412 epoch time: 2.06s step time: 0.0164s\n", + "epoch: 44 train loss: 0.08036990 epoch time: 2.08s step time: 0.0167s\n", + "epoch: 45 train loss: 0.05654721 epoch time: 2.09s step time: 0.0167s\n", + "epoch: 46 train loss: 0.04830655 epoch time: 2.04s step time: 0.0163s\n", + "epoch: 47 train loss: 0.10450873 epoch time: 2.13s step time: 0.0171s\n", + "epoch: 48 train loss: 0.06309631 epoch time: 2.12s step time: 0.0170s\n", + "epoch: 49 train loss: 0.05044140 epoch time: 2.08s step time: 0.0166s\n", + "epoch: 50 train loss: 0.08654185 epoch time: 2.11s step time: 0.0169s\n", + "================================Start Evaluation================================\n", + "mean rms_error: 0.009487706\n", + "=================================End Evaluation=================================\n", + "epoch: 51 train loss: 0.04118668 epoch time: 2.11s step time: 0.0169s\n", + "epoch: 52 train loss: 0.08913346 epoch time: 2.06s step time: 0.0165s\n", + "epoch: 53 train loss: 0.04456577 epoch time: 2.08s step time: 0.0167s\n", + "epoch: 54 train loss: 0.05342043 epoch time: 2.09s step time: 0.0168s\n", + "epoch: 55 train loss: 0.06059274 epoch time: 2.06s step time: 0.0165s\n", + "epoch: 56 train loss: 0.05875401 epoch time: 2.09s step time: 0.0168s\n", + "epoch: 57 train loss: 0.05666578 epoch time: 2.11s step time: 0.0169s\n", + "epoch: 58 train loss: 0.04027025 epoch time: 2.07s step time: 0.0166s\n", + "epoch: 59 train loss: 0.07292267 epoch time: 2.14s step time: 0.0171s\n", + "epoch: 60 train loss: 0.04893249 epoch time: 2.06s step time: 0.0165s\n", + "================================Start Evaluation================================\n", + "mean rms_error: 0.006333221\n", + "=================================End Evaluation=================================\n", + "epoch: 61 train loss: 0.03615757 epoch time: 2.07s step time: 0.0166s\n", + "epoch: 62 train loss: 0.06545272 epoch time: 2.16s step time: 0.0173s\n", + "epoch: 63 train loss: 0.03222132 epoch time: 2.11s step time: 0.0169s\n", + "epoch: 64 train loss: 0.04665821 epoch time: 2.07s step time: 0.0166s\n", + "epoch: 65 train loss: 0.04956944 epoch time: 2.17s step time: 0.0173s\n", + "epoch: 66 train loss: 0.03793489 epoch time: 2.12s step time: 0.0169s\n", + "epoch: 67 train loss: 0.04018638 epoch time: 2.12s step time: 0.0170s\n", + "epoch: 68 train loss: 0.02908281 epoch time: 2.11s step time: 0.0169s\n", + "epoch: 69 train loss: 0.04153252 epoch time: 2.07s step time: 0.0166s\n", + "epoch: 70 train loss: 0.03012481 epoch time: 2.09s step time: 0.0167s\n", + "================================Start Evaluation================================\n", + "mean rms_error: 0.0041090883\n", + "=================================End Evaluation=================================\n", + "epoch: 71 train loss: 0.02333428 epoch time: 2.11s step time: 0.0169s\n", + "epoch: 72 train loss: 0.02513113 epoch time: 2.03s step time: 0.0162s\n", + "epoch: 73 train loss: 0.02411744 epoch time: 2.03s step time: 0.0162s\n", + "epoch: 74 train loss: 0.02313637 epoch time: 2.02s step time: 0.0162s\n", + "epoch: 75 train loss: 0.03571169 epoch time: 1.99s step time: 0.0159s\n", + "epoch: 76 train loss: 0.01535346 epoch time: 2.03s step time: 0.0162s\n", + "epoch: 77 train loss: 0.02628051 epoch time: 2.03s step time: 0.0162s\n", + "epoch: 78 train loss: 0.02273233 epoch time: 2.00s step time: 0.0160s\n", + "epoch: 79 train loss: 0.02078306 epoch time: 2.03s step time: 0.0162s\n", + "epoch: 80 train loss: 0.01021176 epoch time: 2.03s step time: 0.0162s\n", + "================================Start Evaluation================================\n", + "mean rms_error: 0.0017531833\n", + "=================================End Evaluation=================================\n", + "epoch: 81 train loss: 0.01558476 epoch time: 2.01s step time: 0.0161s\n", + "epoch: 82 train loss: 0.01213225 epoch time: 2.03s step time: 0.0162s\n", + "epoch: 83 train loss: 0.00979056 epoch time: 2.06s step time: 0.0165s\n", + "epoch: 84 train loss: 0.01600446 epoch time: 2.13s step time: 0.0171s\n", + "epoch: 85 train loss: 0.00978592 epoch time: 2.07s step time: 0.0166s\n", + "epoch: 86 train loss: 0.01050989 epoch time: 2.07s step time: 0.0166s\n", + "epoch: 87 train loss: 0.01243659 epoch time: 2.02s step time: 0.0161s\n", + "epoch: 88 train loss: 0.00832084 epoch time: 2.04s step time: 0.0163s\n", + "epoch: 89 train loss: 0.01216878 epoch time: 2.03s step time: 0.0163s\n", + "epoch: 90 train loss: 0.00763223 epoch time: 2.02s step time: 0.0162s\n", + "================================Start Evaluation================================\n", + "mean rms_error: 0.0011959259\n", + "=================================End Evaluation=================================\n", + "epoch: 91 train loss: 0.00671979 epoch time: 2.13s step time: 0.0170s\n", + "epoch: 92 train loss: 0.00736447 epoch time: 2.05s step time: 0.0164s\n", + "epoch: 93 train loss: 0.00652055 epoch time: 2.02s step time: 0.0162s\n", + "epoch: 94 train loss: 0.00615171 epoch time: 2.07s step time: 0.0165s\n", + "epoch: 95 train loss: 0.00544963 epoch time: 2.08s step time: 0.0166s\n", + "epoch: 96 train loss: 0.00478581 epoch time: 2.10s step time: 0.0168s\n", + "epoch: 97 train loss: 0.00580032 epoch time: 2.09s step time: 0.0167s\n", + "epoch: 98 train loss: 0.00527284 epoch time: 2.08s step time: 0.0166s\n", + "epoch: 99 train loss: 0.00610725 epoch time: 2.11s step time: 0.0169s\n", + "epoch: 100 train loss: 0.00590835 epoch time: 2.12s step time: 0.0170s\n", + "================================Start Evaluation================================\n", + "mean rms_error: 0.001089375\n", + "=================================End Evaluation=================================\n" + ] + } + ], + "source": [ + "problem = UnsteadyFlowWithLoss(model, loss_fn=RelativeRMSELoss(), data_format=\"NHWTC\")\n", + "\n", + "summary_dir = os.path.join(config[\"summary\"][\"summary_dir\"], model_name)\n", + "print(summary_dir)\n", + "\n", + "def forward_fn(data, label):\n", + " loss = problem.get_loss(data, label)\n", + " if use_ascend:\n", + " loss = loss_scaler.scale(loss)\n", + " return loss\n", + "\n", + "grad_fn = ops.value_and_grad(forward_fn, None, optimizer.parameters, has_aux=False)\n", + "\n", + "@jit\n", + "def train_step(data, label):\n", + " loss, grads = grad_fn(data, label)\n", + " if use_ascend:\n", + " loss = loss_scaler.unscale(loss)\n", + " if all_finite(grads):\n", + " grads = loss_scaler.unscale(grads)\n", + " loss = ops.depend(loss, optimizer(grads))\n", + " return loss\n", + "\n", + "sink_process = data_sink(train_step, train_dataset, 1)\n", + "#summary_dir = os.path.join(config[\"summary_dir\"], model_name)\n", + "ckpt_dir = os.path.join(summary_dir, \"ckpt\")\n", + "if not os.path.exists(ckpt_dir):\n", + " os.makedirs(ckpt_dir)\n", + "\n", + "for epoch in range(1, optimizer_params[\"epochs\"] + 1):\n", + " model.set_train()\n", + " local_time_beg = time.time()\n", + " for _ in range(steps_per_epoch):\n", + " cur_loss = sink_process()\n", + " print(\n", + " f\"epoch: {epoch} train loss: {cur_loss.asnumpy():.8f}\"\\\n", + " f\" epoch time: {time.time() - local_time_beg:.2f}s\"\\\n", + " f\" step time: {(time.time() - local_time_beg)/steps_per_epoch:.4f}s\")\n", + "\n", + " if epoch % config['summary']['test_interval'] == 0:\n", + " model.set_train(False)\n", + " print(\"================================Start Evaluation================================\")\n", + " rms_error = problem.get_loss(test_input, test_label)/test_input.shape[0]\n", + " print(f\"mean rms_error: {rms_error}\")\n", + " print(\"=================================End Evaluation=================================\")\n", + " save_checkpoint(model, os.path.join(ckpt_dir, model_params[\"name\"] + '_epoch' + str(epoch)))" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.23" + }, + "vscode": { + "interpreter": { + "hash": "916dbcbb3f70747c44a77c7bcd40155683ae19c65e1c03b4aa3499c5328201f1" + } + } + }, + "nbformat": 4, + "nbformat_minor": 4 +} diff --git a/MindFlow/applications/data_driven/burgers/fno1d/FNO1D_CN.ipynb b/MindFlow/applications/data_driven/burgers/fno1d/FNO1D_CN.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..ba9ffc5d9b0f661eba4af14c2afb446464008981 --- /dev/null +++ b/MindFlow/applications/data_driven/burgers/fno1d/FNO1D_CN.ipynb @@ -0,0 +1,598 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "pycharm": { + "name": "#%% md\n" + } + }, + "source": [ + "# 基于Fourier Neural Operator的Burgers' equation求解\n", + "\n", + "[![下载Notebook](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/master/resource/_static/logo_notebook.png)](https://obs.dualstack.cn-north-4.myhuaweicloud.com/mindspore-website/notebook/master/mindflow/zh_cn/data_driven/mindspore_burgers_FNO1D.ipynb) [![下载样例代码](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/master/resource/_static/logo_download_code.png)](https://obs.dualstack.cn-north-4.myhuaweicloud.com/mindspore-website/notebook/master/mindflow/zh_cn/data_driven/mindspore_burgers_FNO1D.py) [![查看源文件](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/master/resource/_static/logo_source.png)](https://gitee.com/mindspore/docs/blob/master/docs/mindflow/docs/source_zh_cn/data_driven/burgers_FNO1D.ipynb)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "pycharm": { + "name": "#%% md\n" + } + }, + "source": [ + "## 概述\n", + "\n", + "计算流体力学是21世纪流体力学领域的重要技术之一,其通过使用数值方法在计算机中对流体力学的控制方程进行求解,从而实现流动的分析、预测和控制。传统的有限元法(finite element method,FEM)和有限差分法(finite difference method,FDM)常用于复杂的仿真流程(物理建模、网格划分、数值离散、迭代求解等)和较高的计算成本,往往效率低下。因此,借助AI提升流体仿真效率是十分必要的。\n", + "\n", + "近年来,随着神经网络的迅猛发展,为科学计算提供了新的范式。经典的神经网络是在有限维度的空间进行映射,只能学习与特定离散化相关的解。与经典神经网络不同,傅里叶神经算子(Fourier Neural Operator,FNO)是一种能够学习无限维函数空间映射的新型深度学习架构。该架构可直接学习从任意函数参数到解的映射,用于解决一类偏微分方程的求解问题,具有更强的泛化能力。更多信息可参考[Fourier Neural Operator for Parametric Partial Differential Equations](https://arxiv.org/abs/2010.08895)。\n", + "\n", + "本案例教程介绍利用傅里叶神经算子的1-d Burgers方程求解方法。" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "pycharm": { + "name": "#%% md\n" + } + }, + "source": [ + "## 伯格斯方程(Burgers' equation)\n", + "\n", + "一维伯格斯方程(1-d Burgers' equation)是一个非线性偏微分方程,具有广泛应用,包括一维粘性流体流动建模。它的形式如下:\n", + "\n", + "$$\n", + "\\partial_t u(x, t)+\\partial_x (u^2(x, t)/2)=\\nu \\partial_{xx} u(x, t), \\quad x \\in(0,1), t \\in(0, 1]\n", + "$$\n", + "\n", + "$$\n", + "u(x, 0)=u_0(x), \\quad x \\in(0,1)\n", + "$$\n", + "\n", + "其中$u$表示速度场,$u_0$表示初始条件,$\\nu$表示粘度系数。\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 问题描述\n", + "\n", + "本案例利用Fourier Neural Operator学习初始状态到下一时刻状态的映射,实现一维Burgers'方程的求解:\n", + "\n", + "$$\n", + "u_0 \\mapsto u(\\cdot, 1)\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 技术路径\n", + "\n", + "MindFlow求解该问题的具体流程如下:\n", + "\n", + "1. 创建数据集。\n", + "2. 构建模型。\n", + "3. 优化器与损失函数。\n", + "4. 模型训练。" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Fourier Neural Operator\n", + "\n", + "Fourier Neural Operator模型构架如下图所示。图中$w_0(x)$表示初始涡度,通过Lifting Layer实现输入向量的高维映射,然后将映射结果作为Fourier Layer的输入,进行频域信息的非线性变换,最后由Decoding Layer将变换结果映射至最终的预测结果$w_1(x)$。\n", + "\n", + "Lifting Layer、Fourier Layer以及Decoding Layer共同组成了Fourier Neural Operator。\n", + "\n", + "![Fourier Neural Operator模型构架](images/FNO.png)\n", + "\n", + "Fourier Layer网络结构如下图所示。图中V表示输入向量,上框表示向量经过傅里叶变换后,经过线性变换R,过滤高频信息,然后进行傅里叶逆变换;另一分支经过线性变换W,最后通过激活函数,得到Fourier Layer输出向量。\n", + "\n", + "![Fourier Layer网络结构](images/FNO-2.png)" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [], + "source": [ + "#添加搜索路径\n", + "import sys\n", + "import os\n", + "project_root = os.path.abspath(\"../../../../..\")\n", + "sys.path.append(project_root)" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "pycharm": { + "name": "#%%\n" + }, + "tags": [] + }, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "/home/mindspore/miniconda/envs/jupyter/lib/python3.9/site-packages/numpy/core/getlimits.py:549: UserWarning: The value of the smallest subnormal for type is zero.\n", + " setattr(self, word, getattr(machar, word).flat[0])\n", + "/home/mindspore/miniconda/envs/jupyter/lib/python3.9/site-packages/numpy/core/getlimits.py:89: UserWarning: The value of the smallest subnormal for type is zero.\n", + " return self._float_to_str(self.smallest_subnormal)\n", + "/home/mindspore/miniconda/envs/jupyter/lib/python3.9/site-packages/numpy/core/getlimits.py:549: UserWarning: The value of the smallest subnormal for type is zero.\n", + " setattr(self, word, getattr(machar, word).flat[0])\n", + "/home/mindspore/miniconda/envs/jupyter/lib/python3.9/site-packages/numpy/core/getlimits.py:89: UserWarning: The value of the smallest subnormal for type is zero.\n", + " return self._float_to_str(self.smallest_subnormal)\n" + ] + } + ], + "source": [ + "import os\n", + "import time\n", + "import numpy as np\n", + "\n", + "from mindspore.amp import DynamicLossScaler, auto_mixed_precision, all_finite\n", + "from mindspore import context, nn, Tensor, set_seed, ops, data_sink, jit, save_checkpoint\n", + "from mindspore import dtype as mstype\n", + "from mindscience import FNO1D, RelativeRMSELoss, load_yaml_config, get_warmup_cosine_annealing_lr\n", + "from mindscience.pde import UnsteadyFlowWithLoss" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "下述`src`包可以在[applications/data_driven/burgers/fno1d/src](https://gitee.com/mindspore/mindscience/tree/master/MindFlow/applications/data_driven/burgers/fno1d/src)下载。" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "pycharm": { + "name": "#%%\n" + }, + "tags": [] + }, + "outputs": [], + "source": [ + "from src import create_training_dataset\n", + "\n", + "set_seed(0)\n", + "np.random.seed(0)\n", + "\n", + "context.set_context(mode=context.GRAPH_MODE, device_target=\"Ascend\", device_id=0,max_device_memory=\"32GB\")\n", + "use_ascend = context.get_context(attr_key='device_target') == \"Ascend\"" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "从`config`中获得模型、数据、优化器的参数。" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": { + "pycharm": { + "name": "#%%\n" + } + }, + "outputs": [], + "source": [ + "config = load_yaml_config('./configs/fno1d.yaml')\n", + "data_params = config[\"data\"]\n", + "model_params = config[\"model\"]\n", + "optimizer_params = config[\"optimizer\"]" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "pycharm": { + "name": "#%% md\n" + } + }, + "source": [ + "## 创建数据集\n", + "\n", + "下载训练与测试数据集: [data_driven/burgers/fno1d/dataset](https://download.mindspore.cn/mindscience/mindflow/dataset/applications/data_driven/burgers/dataset/)。\n", + "\n", + "本案例根据Zongyi Li在 [Fourier Neural Operator for Parametric Partial Differential Equations](https://arxiv.org/pdf/2010.08895.pdf) 一文中对数据集的设置生成训练数据集与测试数据集。具体设置如下:\n", + "基于周期性边界,生成满足如下分布的初始条件$u_0(x)$:\n", + "\n", + "$$\n", + "u_0 \\sim \\mu, \\mu=\\mathcal{N}\\left(0,625(-\\Delta+25 I)^{-2}\\right)\n", + "$$\n", + "\n", + "本案例选取粘度系数$\\nu=0.1$,并使用分步法求解方程,其中热方程部分在傅里叶空间中精确求解,然后使用前向欧拉方法求解非线性部分。训练集样本量为1000个,测试集样本量为200个。\n" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": { + "pycharm": { + "name": "#%%\n" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Data preparation finished\n", + "input_path: (1000, 1024, 1)\n", + "label_path: (1000, 1024)\n" + ] + } + ], + "source": [ + "# create training dataset\n", + "train_dataset = create_training_dataset(data_params, model_params, shuffle=True)\n", + "\n", + "# create test dataset\n", + "test_input, test_label = np.load(os.path.join(data_params[\"root_dir\"], \"test/inputs.npy\")), \\\n", + " np.load(os.path.join(data_params[\"root_dir\"], \"test/label.npy\"))\n", + "test_input = Tensor(np.expand_dims(test_input, -2), mstype.float32)\n", + "test_label = Tensor(np.expand_dims(test_label, -2), mstype.float32)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "pycharm": { + "name": "#%% md\n" + } + }, + "source": [ + "## 构建模型\n", + "\n", + "网络由1层Lifting layer、1层Decoding layer以及多层Fourier Layer叠加组成:\n", + "\n", + "- Lifting layer对应样例代码中`FNO1D.fc0`,将输出数据$x$映射至高维;\n", + "\n", + "- 多层Fourier Layer的叠加对应样例代码中`FNO1D.fno_seq`,本案例采用离散傅里叶变换实现时域与频域的转换;\n", + "\n", + "- Decoding layer对应代码中`FNO1D.fc1`与`FNO1D.fc2`,获得最终的预测值。\n", + "\n", + "基于上述网络结构,进行模型初始化,其中模型参数可在[配置文件](https://gitee.com/mindspore/mindscience/blob/master/MindFlow/applications/data_driven/burgers/fno1d/configs/fno1d.yaml)中修改。" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": { + "pycharm": { + "name": "#%%\n" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "name:FNO1D_in_channels:1_out_channels:1_modes:16_resolutions:1024_hidden_channels:64_depths:4\n" + ] + } + ], + "source": [ + "model = FNO1D(in_channels=model_params[\"in_channels\"],\n", + " out_channels=model_params[\"out_channels\"],\n", + " n_modes=model_params[\"modes\"],\n", + " resolutions=model_params[\"resolutions\"],\n", + " hidden_channels=model_params[\"hidden_channels\"],\n", + " n_layers=model_params[\"depths\"],\n", + " projection_channels=4*model_params[\"hidden_channels\"],\n", + " )\n", + "model_params_list = []\n", + "for k, v in model_params.items():\n", + " model_params_list.append(f\"{k}:{v}\")\n", + "model_name = \"_\".join(model_params_list)\n", + "print(model_name)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "pycharm": { + "name": "#%% md\n" + } + }, + "source": [ + "## 优化器与损失函数\n", + "\n", + "使用相对均方根误差作为网络训练损失函数:" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": { + "pycharm": { + "name": "#%%\n" + } + }, + "outputs": [], + "source": [ + "steps_per_epoch = train_dataset.get_dataset_size()\n", + "lr = get_warmup_cosine_annealing_lr(lr_init=optimizer_params[\"learning_rate\"],\n", + " last_epoch=optimizer_params[\"epochs\"],\n", + " steps_per_epoch=steps_per_epoch,\n", + " warmup_epochs=1)\n", + "optimizer = nn.Adam(model.trainable_params(), learning_rate=Tensor(lr))\n", + "\n", + "if use_ascend:\n", + " from mindspore.amp import DynamicLossScaler, auto_mixed_precision, all_finite\n", + " loss_scaler = DynamicLossScaler(1024, 2, 100)\n", + " auto_mixed_precision(model, 'O3')\n", + "else:\n", + " loss_scaler = None" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "pycharm": { + "name": "#%% md\n" + } + }, + "source": [ + "## 模型训练\n", + "\n", + "使用 **MindSpore version >= 2.0.0**, 我们可以使用函数式编程来训练神经网络。 `MindFlow` 为非稳态问题 `UnsteadyFlowWithLoss` 提供了一个训练接口,用于模型训练和评估." + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": { + "pycharm": { + "name": "#%%\n" + }, + "tags": [] + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "./summary/name:FNO1D_in_channels:1_out_channels:1_modes:16_resolutions:1024_hidden_channels:64_depths:4\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "[ERROR] CORE(31549,ffffbd982020,python):2025-11-15-09:43:58.445.122 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_31549/2341691899.py]\n", + "[ERROR] CORE(31549,ffffbd982020,python):2025-11-15-09:43:58.447.639 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_31549/2341691899.py]\n", + "[ERROR] CORE(31549,ffffbd982020,python):2025-11-15-09:43:58.447.695 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_31549/2341691899.py]\n", + "[ERROR] CORE(31549,ffffbd982020,python):2025-11-15-09:43:58.447.875 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_31549/2341691899.py]\n", + "[ERROR] CORE(31549,ffffbd982020,python):2025-11-15-09:43:58.447.922 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_31549/2341691899.py]\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "..epoch: 1 train loss: 0.23476513 epoch time: 19.20s step time: 0.1536s\n", + "epoch: 2 train loss: 0.25118420 epoch time: 2.14s step time: 0.0171s\n", + "epoch: 3 train loss: 0.21490771 epoch time: 2.20s step time: 0.0176s\n", + "epoch: 4 train loss: 0.22294529 epoch time: 2.28s step time: 0.0183s\n", + "epoch: 5 train loss: 0.15385811 epoch time: 2.19s step time: 0.0176s\n", + "epoch: 6 train loss: 0.11222684 epoch time: 2.16s step time: 0.0173s\n", + "epoch: 7 train loss: 0.26857409 epoch time: 2.21s step time: 0.0177s\n", + "epoch: 8 train loss: 0.23994932 epoch time: 2.20s step time: 0.0176s\n", + "epoch: 9 train loss: 0.12968877 epoch time: 2.17s step time: 0.0174s\n", + "epoch: 10 train loss: 0.14602631 epoch time: 2.30s step time: 0.0184s\n", + "================================Start Evaluation================================\n", + "mean rms_error: 0.020715814\n", + "=================================End Evaluation=================================\n", + "epoch: 11 train loss: 0.12445749 epoch time: 2.21s step time: 0.0176s\n", + "epoch: 12 train loss: 0.17673939 epoch time: 2.05s step time: 0.0164s\n", + "epoch: 13 train loss: 0.09387001 epoch time: 2.03s step time: 0.0162s\n", + "epoch: 14 train loss: 0.20918289 epoch time: 2.07s step time: 0.0165s\n", + "epoch: 15 train loss: 0.20073509 epoch time: 2.11s step time: 0.0169s\n", + "epoch: 16 train loss: 0.19236282 epoch time: 2.07s step time: 0.0166s\n", + "epoch: 17 train loss: 0.12148518 epoch time: 2.12s step time: 0.0170s\n", + "epoch: 18 train loss: 0.13007648 epoch time: 2.13s step time: 0.0170s\n", + "epoch: 19 train loss: 0.14730214 epoch time: 2.10s step time: 0.0168s\n", + "epoch: 20 train loss: 0.11193341 epoch time: 2.15s step time: 0.0172s\n", + "================================Start Evaluation================================\n", + "mean rms_error: 0.009921882\n", + "=================================End Evaluation=================================\n", + "epoch: 21 train loss: 0.13001360 epoch time: 2.20s step time: 0.0176s\n", + "epoch: 22 train loss: 0.13757694 epoch time: 2.17s step time: 0.0174s\n", + "epoch: 23 train loss: 0.12695193 epoch time: 2.20s step time: 0.0176s\n", + "epoch: 24 train loss: 0.13907336 epoch time: 2.20s step time: 0.0176s\n", + "epoch: 25 train loss: 0.10276581 epoch time: 2.18s step time: 0.0175s\n", + "epoch: 26 train loss: 0.11725406 epoch time: 2.27s step time: 0.0182s\n", + "epoch: 27 train loss: 0.09574451 epoch time: 2.25s step time: 0.0180s\n", + "epoch: 28 train loss: 0.15954311 epoch time: 2.25s step time: 0.0180s\n", + "epoch: 29 train loss: 0.08458930 epoch time: 2.25s step time: 0.0180s\n", + "epoch: 30 train loss: 0.16492826 epoch time: 2.18s step time: 0.0174s\n", + "================================Start Evaluation================================\n", + "mean rms_error: 0.015584474\n", + "=================================End Evaluation=================================\n", + "epoch: 31 train loss: 0.06738295 epoch time: 2.23s step time: 0.0178s\n", + "epoch: 32 train loss: 0.11054665 epoch time: 2.19s step time: 0.0175s\n", + "epoch: 33 train loss: 0.09714314 epoch time: 2.34s step time: 0.0187s\n", + "epoch: 34 train loss: 0.05955748 epoch time: 2.40s step time: 0.0192s\n", + "epoch: 35 train loss: 0.12066956 epoch time: 2.28s step time: 0.0183s\n", + "epoch: 36 train loss: 0.10293169 epoch time: 2.36s step time: 0.0189s\n", + "epoch: 37 train loss: 0.13184263 epoch time: 2.31s step time: 0.0185s\n", + "epoch: 38 train loss: 0.09791068 epoch time: 2.16s step time: 0.0173s\n", + "epoch: 39 train loss: 0.09240112 epoch time: 2.23s step time: 0.0178s\n", + "epoch: 40 train loss: 0.12446348 epoch time: 2.25s step time: 0.0180s\n", + "================================Start Evaluation================================\n", + "mean rms_error: 0.012187392\n", + "=================================End Evaluation=================================\n", + "epoch: 41 train loss: 0.06516904 epoch time: 2.17s step time: 0.0174s\n", + "epoch: 42 train loss: 0.13848309 epoch time: 2.23s step time: 0.0178s\n", + "epoch: 43 train loss: 0.10850596 epoch time: 2.42s step time: 0.0194s\n", + "epoch: 44 train loss: 0.10974278 epoch time: 2.32s step time: 0.0186s\n", + "epoch: 45 train loss: 0.05560862 epoch time: 2.22s step time: 0.0178s\n", + "epoch: 46 train loss: 0.06424462 epoch time: 2.18s step time: 0.0174s\n", + "epoch: 47 train loss: 0.07734193 epoch time: 2.17s step time: 0.0174s\n", + "epoch: 48 train loss: 0.05293225 epoch time: 2.17s step time: 0.0174s\n", + "epoch: 49 train loss: 0.11110708 epoch time: 2.20s step time: 0.0176s\n", + "epoch: 50 train loss: 0.07038455 epoch time: 2.20s step time: 0.0176s\n", + "================================Start Evaluation================================\n", + "mean rms_error: 0.008971626\n", + "=================================End Evaluation=================================\n", + "epoch: 51 train loss: 0.12441821 epoch time: 2.19s step time: 0.0175s\n", + "epoch: 52 train loss: 0.06804045 epoch time: 2.18s step time: 0.0174s\n", + "epoch: 53 train loss: 0.05506612 epoch time: 2.49s step time: 0.0199s\n", + "epoch: 54 train loss: 0.05211698 epoch time: 2.21s step time: 0.0177s\n", + "epoch: 55 train loss: 0.05002218 epoch time: 2.20s step time: 0.0176s\n", + "epoch: 56 train loss: 0.07349440 epoch time: 2.19s step time: 0.0175s\n", + "epoch: 57 train loss: 0.07091989 epoch time: 2.13s step time: 0.0170s\n", + "epoch: 58 train loss: 0.05169376 epoch time: 2.28s step time: 0.0182s\n", + "epoch: 59 train loss: 0.05403522 epoch time: 2.20s step time: 0.0176s\n", + "epoch: 60 train loss: 0.04980489 epoch time: 2.14s step time: 0.0171s\n", + "================================Start Evaluation================================\n", + "mean rms_error: 0.006142739\n", + "=================================End Evaluation=================================\n", + "epoch: 61 train loss: 0.05079499 epoch time: 2.12s step time: 0.0169s\n", + "epoch: 62 train loss: 0.06567945 epoch time: 2.17s step time: 0.0173s\n", + "epoch: 63 train loss: 0.05983985 epoch time: 2.12s step time: 0.0170s\n", + "epoch: 64 train loss: 0.03497960 epoch time: 2.20s step time: 0.0176s\n", + "epoch: 65 train loss: 0.04502716 epoch time: 2.18s step time: 0.0174s\n", + "epoch: 66 train loss: 0.04706389 epoch time: 2.15s step time: 0.0172s\n", + "epoch: 67 train loss: 0.04403052 epoch time: 2.20s step time: 0.0176s\n", + "epoch: 68 train loss: 0.02547505 epoch time: 2.16s step time: 0.0173s\n", + "epoch: 69 train loss: 0.03348733 epoch time: 2.13s step time: 0.0170s\n", + "epoch: 70 train loss: 0.02987053 epoch time: 2.16s step time: 0.0173s\n", + "================================Start Evaluation================================\n", + "mean rms_error: 0.0036494005\n", + "=================================End Evaluation=================================\n", + "epoch: 71 train loss: 0.02783433 epoch time: 2.12s step time: 0.0170s\n", + "epoch: 72 train loss: 0.02211184 epoch time: 2.12s step time: 0.0169s\n", + "epoch: 73 train loss: 0.01878185 epoch time: 2.10s step time: 0.0168s\n", + "epoch: 74 train loss: 0.01822488 epoch time: 2.14s step time: 0.0172s\n", + "epoch: 75 train loss: 0.02666300 epoch time: 2.15s step time: 0.0172s\n", + "epoch: 76 train loss: 0.02968729 epoch time: 2.12s step time: 0.0170s\n", + "epoch: 77 train loss: 0.01373592 epoch time: 2.13s step time: 0.0171s\n", + "epoch: 78 train loss: 0.01685985 epoch time: 2.09s step time: 0.0167s\n", + "epoch: 79 train loss: 0.01720855 epoch time: 2.11s step time: 0.0169s\n", + "epoch: 80 train loss: 0.00984283 epoch time: 2.18s step time: 0.0174s\n", + "================================Start Evaluation================================\n", + "mean rms_error: 0.0022107356\n", + "=================================End Evaluation=================================\n", + "epoch: 81 train loss: 0.01453377 epoch time: 2.13s step time: 0.0170s\n", + "epoch: 82 train loss: 0.01104419 epoch time: 2.22s step time: 0.0177s\n", + "epoch: 83 train loss: 0.00970412 epoch time: 2.17s step time: 0.0174s\n", + "epoch: 84 train loss: 0.00941654 epoch time: 2.12s step time: 0.0169s\n", + "epoch: 85 train loss: 0.00973161 epoch time: 2.17s step time: 0.0173s\n", + "epoch: 86 train loss: 0.01063135 epoch time: 2.22s step time: 0.0178s\n", + "epoch: 87 train loss: 0.01009477 epoch time: 2.29s step time: 0.0183s\n", + "epoch: 88 train loss: 0.01052440 epoch time: 2.11s step time: 0.0169s\n", + "epoch: 89 train loss: 0.00695687 epoch time: 2.15s step time: 0.0172s\n", + "epoch: 90 train loss: 0.00711782 epoch time: 2.13s step time: 0.0171s\n", + "================================Start Evaluation================================\n", + "mean rms_error: 0.0012141274\n", + "=================================End Evaluation=================================\n", + "epoch: 91 train loss: 0.00646544 epoch time: 2.12s step time: 0.0170s\n", + "epoch: 92 train loss: 0.00788212 epoch time: 2.18s step time: 0.0174s\n", + "epoch: 93 train loss: 0.00627555 epoch time: 2.17s step time: 0.0173s\n", + "epoch: 94 train loss: 0.00621355 epoch time: 2.08s step time: 0.0166s\n", + "epoch: 95 train loss: 0.00569737 epoch time: 2.15s step time: 0.0172s\n", + "epoch: 96 train loss: 0.00505786 epoch time: 2.09s step time: 0.0167s\n", + "epoch: 97 train loss: 0.00565789 epoch time: 2.07s step time: 0.0166s\n", + "epoch: 98 train loss: 0.00527059 epoch time: 2.07s step time: 0.0166s\n", + "epoch: 99 train loss: 0.00606439 epoch time: 2.07s step time: 0.0165s\n", + "epoch: 100 train loss: 0.00620594 epoch time: 2.08s step time: 0.0166s\n", + "================================Start Evaluation================================\n", + "mean rms_error: 0.0011061892\n", + "=================================End Evaluation=================================\n" + ] + } + ], + "source": [ + "problem = UnsteadyFlowWithLoss(model, loss_fn=RelativeRMSELoss(), data_format=\"NHWTC\")\n", + "\n", + "summary_dir = os.path.join(config[\"summary\"][\"summary_dir\"], model_name)\n", + "print(summary_dir)\n", + "\n", + "def forward_fn(data, label):\n", + " loss = problem.get_loss(data, label)\n", + " if use_ascend:\n", + " loss = loss_scaler.scale(loss)\n", + " return loss\n", + "\n", + "grad_fn = ops.value_and_grad(forward_fn, None, optimizer.parameters, has_aux=False)\n", + "\n", + "@jit\n", + "def train_step(data, label):\n", + " loss, grads = grad_fn(data, label)\n", + " if use_ascend:\n", + " loss = loss_scaler.unscale(loss)\n", + " if all_finite(grads):\n", + " grads = loss_scaler.unscale(grads)\n", + " loss = ops.depend(loss, optimizer(grads))\n", + " return loss\n", + "\n", + "sink_process = data_sink(train_step, train_dataset, 1)\n", + "#summary_dir = os.path.join(config[\"summary_dir\"], model_name)\n", + "ckpt_dir = os.path.join(summary_dir, \"ckpt\")\n", + "if not os.path.exists(ckpt_dir):\n", + " os.makedirs(ckpt_dir)\n", + "\n", + "for epoch in range(1, optimizer_params[\"epochs\"] + 1):\n", + " model.set_train()\n", + " local_time_beg = time.time()\n", + " for _ in range(steps_per_epoch):\n", + " cur_loss = sink_process()\n", + " print(\n", + " f\"epoch: {epoch} train loss: {cur_loss.asnumpy():.8f}\"\\\n", + " f\" epoch time: {time.time() - local_time_beg:.2f}s\"\\\n", + " f\" step time: {(time.time() - local_time_beg)/steps_per_epoch:.4f}s\")\n", + "\n", + " if epoch % config['summary']['test_interval'] == 0:\n", + " model.set_train(False)\n", + " print(\"================================Start Evaluation================================\")\n", + " rms_error = problem.get_loss(test_input, test_label)/test_input.shape[0]\n", + " print(f\"mean rms_error: {rms_error}\")\n", + " print(\"=================================End Evaluation=================================\")\n", + " save_checkpoint(model, os.path.join(ckpt_dir, model_params[\"name\"] + '_epoch' + str(epoch)))" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "jupyter", + "language": "python", + "name": "jupyter" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.23" + }, + "vscode": { + "interpreter": { + "hash": "916dbcbb3f70747c44a77c7bcd40155683ae19c65e1c03b4aa3499c5328201f1" + } + } + }, + "nbformat": 4, + "nbformat_minor": 4 +} diff --git a/MindFlow/applications/data_driven/burgers/fno1d/README.MD b/MindFlow/applications/data_driven/burgers/fno1d/README.MD new file mode 100644 index 0000000000000000000000000000000000000000..8ec1db7ea172870a816a91b2d6537f89acfbb296 --- /dev/null +++ b/MindFlow/applications/data_driven/burgers/fno1d/README.MD @@ -0,0 +1,98 @@ +# FNO operator solves Burgers equation + +## Introduction + +### Background + +Computational fluid dynamics is one of the most important techniques in the field of fluid mechanics in the 21st century. The flow analysis, prediction and control can be realized by solving the governing equations of fluid mechanics by numerical method. Traditional finite element method (FEM) and finite difference method (FDM) are inefficient because of the complex simulation process (physical modeling, meshing, numerical discretization, iterative solution, etc.) and high computing costs. Therefore, it is necessary to improve the efficiency of fluid simulation with AI. + +Machine learning methods provide a new paradigm for scientific computing by providing a fast solver similar to traditional methods. Classical neural networks learn mappings between finite dimensional spaces and can only learn solutions related to a specific discretization. Different from traditional neural networks, Fourier Neural Operator (FNO) is a new deep learning architecture that can learn mappings between infinite-dimensional function spaces. It directly learns mappings from arbitrary function parameters to solutions to solve a class of partial differential equations. Therefore, it has a stronger generalization capability. More information can be found in the paper, [Fourier Neural Operator for Parametric Partial Differential Equations](https://arxiv.org/abs/2010.08895). + +This tutorial describes how to solve the 1-d Burgers' equation using Fourier neural operator. + +### 问题描述 + +## Burgers' equation + +The 1-d Burgers’ equation is a non-linear PDE with various applications including modeling the one +dimensional flow of a viscous fluid. It takes the form + +$$ +\partial_t u(x, t)+\partial_x (u^2(x, t)/2)=\nu \partial_{xx} u(x, t), \quad x \in(0,1), t \in(0, 1] +$$ + +$$ +u(x, 0)=u_0(x), \quad x \in(0,1) +$$ + +where $u$ is the velocity field, $u_0$ is the initial condition and $\nu$ is the viscosity coefficient. + +We aim to learn the operator mapping the initial condition to the solution at time one: + +$$ +u_0 \mapsto u(\cdot, 1) +$$ + +### Technical path + +The Fourier Neural Operator consists of the Lifting Layer, Fourier Layers, and the Decoding Layer. + +![Fourier Neural Operator model structure](images/FNO.png) + +Fourier layers: Start from input V. On top: apply the Fourier transform $\mathcal{F}$; a linear transform R on the lower Fourier modes and filters out the higher modes; then apply the inverse Fourier transform $\mathcal{F}^{-1}$. On the bottom: apply a local linear transform W. Finally, the Fourier Layer output vector is obtained through the activation function. + +![Fourier Layer structure](images/FNO-2.png) + +## QuickStart + +You can download dataset from [data_driven/burgers/](https://download.mindspore.cn/mindscience/mindflow/dataset/applications/data_driven/burgers/). Save these dataset at `./dataset`. + +### Run Method 1: Call `train.py` from command line + +```shell +export PYTHONPATH=$(cd ../../../../../ && pwd):$PYTHONPATH +python train.py --config_file_path ./configs/fno1d.yaml --mode GRAPH --device_target Ascend --device_id 0 +``` + +where: + +`--config_file_path` indicates the path of the parameter file. Default './configs/fno1d.yaml'; + +`--device_target` indicates the computing platform. You can choose 'Ascend' or 'GPU'. Default 'Ascend'. + +`--device_id` indicates the index of NPU or GPU. Default 0. + +`--mode` is the running mode. 'GRAPH' indicates static graph mode. 'PYNATIVE' indicates dynamic graph mode. + +### Run Method 2: Run Jupyter Notebook + +You can run the training and validation code line by line using the Chinese or English version of the Jupyter Notebook [Chinese Version](./FNO1D_CN.ipynb) and [English Version](./FNO1D.ipynb). + +## Results Display + +![FNO1D Solves Burgers Equation](images/result.jpg) + +## Performance + +| Parameter | Ascend | GPU | +|:----------------------:|:--------------------------:|:---------------:| +| Hardware | Ascend, 32G | NVIDIA V100, 32G | +| MindSpore版本 | 2.7.0 | >=2.1.0 | +| Dataset | [1D Burgers Equation Resolution Dataset](https://download-mindspore.osinfra.cn/mindscience/mindflow/dataset/applications/data_driven/burgers/) | [1D Burgers Equation Resolution Dataset](https://download-mindspore.osinfra.cn/mindscience/mindflow/dataset/applications/data_driven/burgers/) | +| Parameters | 5.57e5 | 5.5e5 | +| Train Config | resolution=1024, modes=16, hidden_channels=64, depth=4, batch_size=8, epoch=100 | resolution=256, modes=16, hidden_channels=64, depth=10, batch_size=64, epoch=1000 | +| Evaluation Config | batch_size=8 | batch_size=64 | +| Optimizer | Adam | Adam | +| Train Loss(MSE) | 0.004872 | 0.011212 | +| Evaluation Error(RMSE) | 0.001088 | 0.000279 | +| Speed(ms/step) | 15 | 17 | + +The datasets at different resolutions are taken for testing and according to the following results it can be concluded that the dataset resolution has no effect on the training results. + +![FNO Solves Burgers Equation](images/resolution_test.jpg) + +## Contributor + +gitee id:[liulei277](https://gitee.com/liulei277), [yezhenghao2023](https://gitee.com/yezhenghao2023) + +email: liulei2770919@163.com, yezhenghao@isrc.iscas.ac.cn \ No newline at end of file diff --git a/MindFlow/applications/data_driven/burgers/fno1d/README_CN.md b/MindFlow/applications/data_driven/burgers/fno1d/README_CN.md new file mode 100644 index 0000000000000000000000000000000000000000..ad4b6c780678218f75e50cdaabcfc5955746735a --- /dev/null +++ b/MindFlow/applications/data_driven/burgers/fno1d/README_CN.md @@ -0,0 +1,97 @@ +# FNO算子求解Burgers方程 + +## 概述 + +### 背景 + +计算流体力学是21世纪流体力学领域的重要技术之一,其通过使用数值方法在计算机中对流体力学的控制方程进行求解,从而实现流动的分析、预测和控制。传统的有限元法(finite element method,FEM)和有限差分法(finite difference method,FDM)常用于复杂的仿真流程(物理建模、网格划分、数值离散、迭代求解等)和较高的计算成本,往往效率低下。因此,借助AI提升流体仿真效率是十分必要的。 + +近年来,随着神经网络的迅猛发展,为科学计算提供了新的范式。经典的神经网络是在有限维度的空间进行映射,只能学习与特定离散化相关的解。与经典神经网络不同,傅里叶神经算子(Fourier Neural Operator,FNO)是一种能够学习无限维函数空间映射的新型深度学习架构。该架构可直接学习从任意函数参数到解的映射,用于解决一类偏微分方程的求解问题,具有更强的泛化能力。更多信息可参考[Fourier Neural Operator for Parametric Partial Differential Equations](https://arxiv.org/abs/2010.08895)。 + +本案例教程介绍利用傅里叶神经算子的1-d Burgers方程求解方法。 + +### 问题描述 + +一维伯格斯方程(1-d Burgers' equation)是一个非线性偏微分方程,具有广泛应用,包括一维粘性流体流动建模。它的形式如下: + +$$ +\partial_t u(x, t)+\partial_x (u^2(x, t)/2)=\nu \partial_{xx} u(x, t), \quad x \in(0,1), t \in(0, 1] +$$ + +$$ +u(x, 0)=u_0(x), \quad x \in(0,1) +$$ + +其中$u$表示速度场,$u_0$表示初始条件,$\nu$表示粘度系数。 + +本案例利用Fourier Neural Operator学习初始状态到下一时刻状态的映射,实现一维Burgers'方程的求解: + +$$ +u_0 \mapsto u(\cdot, 1) +$$ + +### 技术路径 + +Fourier Neural Operator模型构架如下图所示。图中$w_0(x)$表示初始涡度,通过Lifting Layer实现输入向量的高维映射,然后将映射结果作为Fourier Layer的输入,进行频域信息的非线性变换,最后由Decoding Layer将变换结果映射至最终的预测结果$w_1(x)$。 + +Lifting Layer、Fourier Layer以及Decoding Layer共同组成了Fourier Neural Operator。 + +![Fourier Neural Operator模型构架](images/FNO.png) + +Fourier Layer网络结构如下图所示。图中V表示输入向量,上框表示向量经过傅里叶变换后,经过线性变换R,过滤高频信息,然后进行傅里叶逆变换;另一分支经过线性变换W,最后通过激活函数,得到Fourier Layer输出向量。 + +![Fourier Layer网络结构](images/FNO-2.png) + +## 快速开始 + +数据集下载地址:[data_driven/burgers/fno1d/dataset](https://download.mindspore.cn/mindscience/mindflow/dataset/applications/data_driven/burgers/dataset/). 将数据集保存在`./dataset`路径下. + +### 训练方式一:在命令行中调用`train.py`脚本 + +```shell +export PYTHONPATH=$(cd ../../../../../ && pwd):$PYTHONPATH +python train.py --config_file_path ./configs/fno1d.yaml --device_target Ascend --device_id 0 --mode GRAPH +``` + +其中, + +`--config_file_path`表示配置文件的路径,默认值'./configs/fno1d.yaml'; + +`--device_target`表示使用的计算平台类型,可以选择'Ascend'或'GPU',默认值'GPU'; + +`--device_id`表示使用的计算卡编号,可按照实际情况填写,默认值 0; + +`--mode`表示运行的模式,'GRAPH'表示静态图模式, 'PYNATIVE'表示动态图模式。 + +### 训练方式二:运行 Jupyter Notebook + +您可以使用[中文版](./FNO1D_CN.ipynb)和[英文版](./FNO1D.ipynb)Jupyter Notebook 逐行运行训练和验证代码。 + +## 结果展示 + +![FNO1D Solves Burgers](images/result.jpg) + +## 性能 + +| 参数 | Ascend | GPU | +|:----------------------:|:--------------------------:|:---------------:| +| 硬件资源 | Ascend, 显存32G | NVIDIA V100, 显存32G | +| MindSpore版本 | 2.7.0 | >=2.1.0 | +| 数据集 | [一维Burgers方程分辨率数据集](https://download-mindspore.osinfra.cn/mindscience/mindflow/dataset/applications/data_driven/burgers/) | [一维Burgers方程分辨率数据集](https://download-mindspore.osinfra.cn/mindscience/mindflow/dataset/applications/data_driven/burgers/) | +| 参数量 | 5.57e5 | 5.5e5 | +| 训练参数 | resolution=1024, modes=16, hidden_channels=64, depth=4, batch_size=8, epoch=100 | resolution=256, modes=16, hidden_channels=64, depth=10, batch_size=64, epoch=1000 | +| 测试参数 | batch_size=8 | batch_size=64 | +| 优化器 | Adam | Adam | +| 训练损失(MSE) | 0.004872 | 0.011212 | +| 验证损失(RMSE) | 0.001088 | 0.000279 | +| 速度(ms/step) | 15 | 17 | + +取不同分辨率下的数据集进行测试,根据以下结果可得出数据集分辨率对训练结果没有影响。 + +![FNO求解burgers方程](images/resolution_test.jpg) + +## Contributor + +gitee id:[liulei277](https://gitee.com/liulei277), [yezhenghao2023](https://gitee.com/yezhenghao2023) + +email: liulei2770919@163.com, yezhenghao@isrc.iscas.ac.cn \ No newline at end of file diff --git a/MindFlow/applications/data_driven/burgers/fno1d/__init__.py b/MindFlow/applications/data_driven/burgers/fno1d/__init__.py new file mode 100644 index 0000000000000000000000000000000000000000..a5ebd344215bb697784d3ebd6d27c9c3e5ca165a --- /dev/null +++ b/MindFlow/applications/data_driven/burgers/fno1d/__init__.py @@ -0,0 +1,20 @@ +# Copyright 2023 Huawei Technologies Co., Ltd +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# ============================================================================ +"""init""" +from train import train + +__all__ = [ + "train", +] diff --git a/MindFlow/applications/data_driven/burgers/fno1d/configs/fno1d.yaml b/MindFlow/applications/data_driven/burgers/fno1d/configs/fno1d.yaml new file mode 100644 index 0000000000000000000000000000000000000000..af8295056a83da71ebc60a3c3fbd8eb00d80ea43 --- /dev/null +++ b/MindFlow/applications/data_driven/burgers/fno1d/configs/fno1d.yaml @@ -0,0 +1,27 @@ +model: + name: FNO1D + in_channels: 1 + out_channels: 1 + modes: 16 + resolutions: 1024 + hidden_channels: 64 + depths: 4 +data: + name: "burgers1d" + root_dir: "./dataset" + train: + num_samples: 1000 + test: + num_samples: 200 + batch_size: 8 + resolution: 1024 + t_in: 1 + t_out: 1 + step: 8 +optimizer: + learning_rate: 0.001 + epochs: 100 +summary: + test_interval: 10 + summary_dir: "./summary" + ckpt_dir: "./checkpoints" \ No newline at end of file diff --git a/MindFlow/applications/data_driven/burgers/fno1d/images/FNO-2.png b/MindFlow/applications/data_driven/burgers/fno1d/images/FNO-2.png new file mode 100644 index 0000000000000000000000000000000000000000..1556f366ae893a855b37ebe1666fc4fc0e59ef99 Binary files /dev/null and b/MindFlow/applications/data_driven/burgers/fno1d/images/FNO-2.png differ diff --git a/MindFlow/applications/data_driven/burgers/fno1d/images/FNO.png b/MindFlow/applications/data_driven/burgers/fno1d/images/FNO.png new file mode 100644 index 0000000000000000000000000000000000000000..63c860f85f9a34644202175c2e4da55aaf96ba80 Binary files /dev/null and b/MindFlow/applications/data_driven/burgers/fno1d/images/FNO.png differ diff --git a/MindFlow/applications/data_driven/burgers/fno1d/images/resolution_test.jpg b/MindFlow/applications/data_driven/burgers/fno1d/images/resolution_test.jpg new file mode 100644 index 0000000000000000000000000000000000000000..bc06d7598596b9d169fdb3564851689914eb38a5 Binary files /dev/null and b/MindFlow/applications/data_driven/burgers/fno1d/images/resolution_test.jpg differ diff --git a/MindFlow/applications/data_driven/burgers/fno1d/images/result.jpg b/MindFlow/applications/data_driven/burgers/fno1d/images/result.jpg new file mode 100644 index 0000000000000000000000000000000000000000..8fdfb1b9af70d2cd15ba5161288b6fd554eb8a6e Binary files /dev/null and b/MindFlow/applications/data_driven/burgers/fno1d/images/result.jpg differ diff --git a/MindFlow/applications/data_driven/burgers/fno1d/src/__init__.py b/MindFlow/applications/data_driven/burgers/fno1d/src/__init__.py new file mode 100644 index 0000000000000000000000000000000000000000..d1638cb044e298a1fd884608ee92d68dc5260f84 --- /dev/null +++ b/MindFlow/applications/data_driven/burgers/fno1d/src/__init__.py @@ -0,0 +1,20 @@ +# Copyright 2023 Huawei Technologies Co., Ltd +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# ============================================================================ +"""init""" +from .dataset import create_training_dataset + +__all__ = [ + "create_training_dataset", +] diff --git a/MindFlow/applications/data_driven/burgers/fno1d/src/dataset.py b/MindFlow/applications/data_driven/burgers/fno1d/src/dataset.py new file mode 100644 index 0000000000000000000000000000000000000000..e8366d39c795b35a6e5f109b10fee57f126d9496 --- /dev/null +++ b/MindFlow/applications/data_driven/burgers/fno1d/src/dataset.py @@ -0,0 +1,94 @@ +# Copyright 2023 Huawei Technologies Co., Ltd +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# ============================================================================== +"""dataset""" +import os +import numpy as np +from MindFlow.data import Dataset, ExistedDataConfig +from mindscience.utils import print_log +EPS = 1e-8 + + +def create_npy(data_params, step=8): + '''create inputs and label data for trainset and testset''' + data_path = data_params["root_dir"] + train_size = data_params["train"]["num_samples"] + test_size = data_params["test"]["num_samples"] + s = 2 ** 13 // data_params["step"] + + train_path = os.path.join(data_path, "train") + test_path = os.path.join(data_path, "test") + + if not os.path.exists(train_path): + os.makedirs(train_path) + else: + print_log("Data preparation finished") + return + if not os.path.exists(test_path): + os.makedirs(test_path) + + inputs = np.load(os.path.join(train_path, "inputs.npy")) + label = np.load(os.path.join(train_path, "label.npy")) + + x_train = inputs[:train_size, :][:, ::step] + y_train = label[:train_size, :][:, ::step] + x_test = inputs[-test_size:, :][:, ::step] + y_test = label[-test_size:, :][:, ::step] + x_train = x_train.reshape(train_size, s, 1) + x_test = x_test.reshape(test_size, s, 1) + + train_input_path = os.path.join(train_path, "inputs.npy") + train_label_path = os.path.join(train_path, "label.npy") + test_input_path = os.path.join(test_path, "inputs.npy") + test_label_path = os.path.join(test_path, "label.npy") + + with os.fdopen(train_input_path, "wb") as f: + np.save(f, x_train) + with os.fdopen(train_label_path, "wb") as f: + np.save(f, y_train) + with os.fdopen(test_input_path, "wb") as f: + np.save(f, x_test) + with os.fdopen(test_label_path, "wb") as f: + np.save(f, y_test) + + +def create_training_dataset(data_params, model_params, shuffle=True, drop_remainder=True, is_train=True): + """create dataset""" + create_npy(data_params) + data_path = data_params["root_dir"] + if is_train: + train_path = os.path.abspath(os.path.join(data_path, "train")) + input_path = os.path.join(train_path, "inputs.npy") + print_log('input_path: ', np.load(input_path).shape) + label_path = os.path.join(train_path, "label.npy") + print_log('label_path: ', np.load(label_path).shape) + else: + test_path = os.path.abspath(os.path.join(data_path, "test")) + input_path = os.path.join(test_path, "inputs.npy") + label_path = os.path.join(test_path, "label.npy") + burgers_1d_data = ExistedDataConfig(name=data_params["name"], + data_dir=[input_path, label_path], + columns_list=["inputs", "label"], + data_format="npy") + dataset = Dataset(existed_data_list=[burgers_1d_data]) + data_loader = dataset.create_dataset(batch_size=data_params["batch_size"], + shuffle=shuffle, + drop_remainder=drop_remainder) + + operations = [lambda x, y: (x.reshape(-1, data_params["resolution"], + data_params["t_in"], model_params["out_channels"]), + y.reshape(-1, data_params["resolution"], + data_params["t_out"], model_params["out_channels"]))] + data_loader = data_loader.map(operations, input_columns=["burgers1d_inputs", "burgers1d_label"]) + return data_loader diff --git a/MindFlow/applications/data_driven/burgers/fno1d/train.py b/MindFlow/applications/data_driven/burgers/fno1d/train.py new file mode 100644 index 0000000000000000000000000000000000000000..f28417d9850c7cd42b159026c3748dbaea0cd39b --- /dev/null +++ b/MindFlow/applications/data_driven/burgers/fno1d/train.py @@ -0,0 +1,154 @@ +"""train""" +import os +import time +import argparse +import datetime +import numpy as np + +from mindspore import context, nn, Tensor, set_seed, ops, data_sink, jit, save_checkpoint +from mindspore import dtype as mstype +from mindspore.amp import DynamicLossScaler, auto_mixed_precision, all_finite +from src import create_training_dataset +from mindscience import FNO1D, RelativeRMSELoss, load_yaml_config, get_warmup_cosine_annealing_lr +from mindscience.pde import UnsteadyFlowWithLoss +from mindscience.utils import log_config, print_log + + + +set_seed(0) +np.random.seed(0) + + +def parse_args(): + '''Parse input args''' + parser = argparse.ArgumentParser(description='Burgers 1D problem') + parser.add_argument("--mode", type=str, default="GRAPH", choices=["GRAPH", "PYNATIVE"], + help="Context mode, support 'GRAPH', 'PYNATIVE'") + parser.add_argument("--save_graphs", type=bool, default=False, choices=[True, False], + help="Whether to save intermediate compilation graphs") + parser.add_argument("--save_graphs_path", type=str, default="./graphs") + parser.add_argument("--device_target", type=str, default="GPU", choices=["GPU", "Ascend"], + help="The target device to run, support 'Ascend', 'GPU'") + parser.add_argument("--device_id", type=int, default=0, + help="ID of the target device") + parser.add_argument("--config_file_path", type=str, + default="./configs/fno1d.yaml") + input_args = parser.parse_args() + return input_args + + +def train(input_args): + '''Train and evaluate the network''' + use_ascend = context.get_context(attr_key='device_target') == "Ascend" + print_log(f"use_ascend: {use_ascend}") + + config = load_yaml_config(input_args.config_file_path) + data_params = config["data"] + model_params = config["model"] + optimizer_params = config["optimizer"] + + # create training dataset + train_dataset = create_training_dataset(data_params, model_params, shuffle=True) + + # create test dataset + test_input, test_label = np.load(os.path.join(data_params["root_dir"], "test/inputs.npy")), \ + np.load(os.path.join(data_params["root_dir"], "test/label.npy")) + test_input = Tensor(np.expand_dims(test_input, -2), mstype.float32) + test_label = Tensor(np.expand_dims(test_label, -2), mstype.float32) + + model = FNO1D(in_channels=model_params["in_channels"], + out_channels=model_params["out_channels"], + n_modes=model_params["modes"], + resolutions=model_params["resolutions"], + hidden_channels=model_params["hidden_channels"], + n_layers=model_params["depths"], + projection_channels=4*model_params["hidden_channels"], + ) + + steps_per_epoch = train_dataset.get_dataset_size() + lr = get_warmup_cosine_annealing_lr(lr_init=optimizer_params["learning_rate"], + last_epoch=optimizer_params["epochs"], + steps_per_epoch=steps_per_epoch, + warmup_epochs=1) + optimizer = nn.Adam(model.trainable_params(), learning_rate=Tensor(lr)) + + if use_ascend: + loss_scaler = DynamicLossScaler(1024, 2, 100) + auto_mixed_precision(model, 'O3') + else: + loss_scaler = None + + problem = UnsteadyFlowWithLoss( + model, loss_fn=RelativeRMSELoss(), data_format="NHWTC") + + summary_dir = config["summary"]["summary_dir"] + print_log(summary_dir) + + def forward_fn(data, label): + loss = problem.get_loss(data, label) + if use_ascend: + loss = loss_scaler.scale(loss) + return loss + + grad_fn = ops.value_and_grad( + forward_fn, None, optimizer.parameters, has_aux=False) + + @jit + def train_step(data, label): + loss, grads = grad_fn(data, label) + if use_ascend: + loss = loss_scaler.unscale(loss) + if all_finite(grads): + grads = loss_scaler.unscale(grads) + loss = ops.depend(loss, optimizer(grads)) + return loss + + sink_process = data_sink(train_step, train_dataset, 1) + ckpt_dir = "./checkpoints" + if not os.path.exists(ckpt_dir): + os.makedirs(ckpt_dir) + min_train_loss = float('inf') + min_eval_loss = float('inf') + min_step_time = float('inf') + for epoch in range(1, optimizer_params["epochs"] + 1): + model.set_train() + local_time_beg = time.time() + for _ in range(steps_per_epoch): + cur_loss = sink_process() + print_log( + f"epoch: {epoch} train loss: {cur_loss.asnumpy():.8f}"\ + f" epoch time: {time.time() - local_time_beg:.2f}s"\ + f" step time: {(time.time() - local_time_beg)/steps_per_epoch:.4f}s") + min_train_loss = min(min_train_loss, cur_loss) + min_step_time = min(min_step_time, (time.time() - local_time_beg) / steps_per_epoch) + if epoch % config['summary']['test_interval'] == 0: + eval_time_start = time.time() + model.set_train(False) + print_log( + "================================Start Evaluation================================") + rms_error = problem.get_loss( + test_input, test_label)/test_input.shape[0] + print_log(f"mean rms_error: {rms_error}") + min_eval_loss = min(min_eval_loss, rms_error) + print_log( + "=================================End Evaluation=================================") + print_log(f'evaluation time: {time.time() - eval_time_start}s') + save_checkpoint(model, os.path.join( + ckpt_dir, f"{model_params['name']}_epoch{epoch}")) + print_log( + f"the minimum train mse is {min_train_loss}\n", + f"the minimum step time is {min_step_time}\n", + f"the minimum eval rmse is {min_eval_loss}\n", + ) +if __name__ == '__main__': + log_config('./logs', 'fno1d') + print_log(f"pid: {os.getpid()}") + print_log(datetime.datetime.now()) + args = parse_args() + context.set_context(mode=context.GRAPH_MODE if args.mode.upper().startswith("GRAPH") else context.PYNATIVE_MODE, + save_graphs=args.save_graphs, save_graphs_path=args.save_graphs_path, + device_target=args.device_target, device_id=args.device_id,max_device_memory="32GB") + set_max_mem = context.get_context(attr_key='max_device_memory') + print(f"设置的最大显存:{set_max_mem } GB") + print_log(f"device_id: {context.get_context(attr_key='device_id')}") + train(args) diff --git a/MindFlow/data/__init__.py b/MindFlow/data/__init__.py new file mode 100644 index 0000000000000000000000000000000000000000..2a15f218ffc33ae7e13d4a5822381c088b12261d --- /dev/null +++ b/MindFlow/data/__init__.py @@ -0,0 +1,30 @@ +# Copyright 2021 Huawei Technologies Co., Ltd +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# ============================================================================ +"""init""" +from .dataset import Dataset +#from .mind_dataset import MindDataset +from .boundary import BoundaryBC, BoundaryIC, Boundary +from .equation import Equation +from .existed_data import ExistedDataset +from .data_base import ExistedDataConfig, CONSTRAINT_TYPES + +__all__ = ["Dataset", + #"MindDataset", + "Equation", + "BoundaryBC", + "BoundaryIC", + "ExistedDataset", + "ExistedDataConfig", + "CONSTRAINT_TYPES"] diff --git a/MindFlow/data/boundary.py b/MindFlow/data/boundary.py new file mode 100644 index 0000000000000000000000000000000000000000..3033ff7a565df0af6f270a447b7914da05cc7172 --- /dev/null +++ b/MindFlow/data/boundary.py @@ -0,0 +1,174 @@ +# Copyright 2021 Huawei Technologies Co., Ltd +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# ============================================================================== +#pylint: disable=W0223 +#pylint: disable=W0221 +""" +Sampling data of boundary condition and initial condition. +""" +from __future__ import absolute_import + +import numpy as np +from mindspore import log as logger + + +from mindscience.data.flow.geometry.geometry_base import Geometry, SamplingConfig +from mindscience.utils.check_func import check_param_type + +from .data_base import Data + + +class Boundary(Data): + """ + Base class of boundary condition and initial condition. + + Args: + geometry (Geometry): specifies geometry information of boundary condition and initial condition. + + Raises: + ValueError: if `sampling_config` of geometry is ``None``. + TypeError: if `geometry` is not an instance of Geometry. + """ + + def __init__(self, geometry): + check_param_type(geometry, "geometry", data_type=Geometry) + self.geometry = geometry + check_param_type(geometry.sampling_config, geometry.name, data_type=SamplingConfig) + + self.data = None + self.data_size = None + self.batch_size = 1 + self.shuffle = False + self.batched_data_size = None + self.columns_list = None + self._bound_index = None + self._bound_index_num = 0 + + super().__init__() + + def _get_sampling_data(self, geom_type="BC"): + """get sampling data""" + sample_data = self.geometry.sampling(geom_type=geom_type) + if not isinstance(sample_data, tuple): + sample_data = (sample_data,) + logger.info(f"Get {geom_type} sample_data size: {len(sample_data[0])}") + return sample_data, self.geometry.columns_dict[geom_type] + + def _initialize(self, batch_size=1, shuffle=False, geom_type="BC"): + """initialization: sampling and set attrs.""" + data, self.columns_list = self._get_sampling_data(geom_type=geom_type) + if not isinstance(data, tuple): + data = (data,) + self.data = data + self.data_size = len(self.data[0]) + self.batch_size = batch_size + + if batch_size > self.data_size: + raise ValueError( + f"If prebatch data, batch_size: {batch_size} " + f"should not be larger than data size: {self.data_size}." + ) + + self.batched_data_size = self.data_size // batch_size + self.shuffle = shuffle + self._bound_index = np.arange(self.data_size) + + logger.info( + f"Get {geom_type} dataset: {self.name}, columns: {self.columns_list}, " + f"size: {self.data_size}, batched_size: {self.batched_data_size}, shuffle: {self.shuffle}" + ) + return data + + def _get_index_when_sample_iter(self, index, geom_type="BC"): + if self._bound_index_num == self.batched_data_size: + self.data = self._initialize(self.batch_size, self.shuffle, geom_type=geom_type) + self._bound_index_num = 0 + index = self._bound_index_num + self._bound_index_num += 1 + return index + + def _get_index_when_sample_all(self, index): + data_size = len(self) + if (self._random_merge or self.shuffle) and index % data_size == 0: + self._bound_index = np.random.permutation(self.data_size) + index = index % data_size if index >= data_size else index + return index + + def __len__(self): + return self.batched_data_size + + def _get_item(self, index): + col_data = None + for i in range(len(self.columns_list)): + temp_data = ( + self.data[i][self._bound_index[index]] + if self.batch_size == 1 + else self.data[i][index * self.batch_size : (index + 1) * self.batch_size] + ) + col_data = (temp_data,) if col_data is None else col_data + (temp_data,) + return col_data + + +class BoundaryBC(Boundary): + """Sampling data of boundary condition.""" + + def __init__(self, geometry): + super().__init__(geometry) + self.sampling_config = self.geometry.sampling_config.bc + if not self.sampling_config: + raise ValueError( + f"BC info for the current geometry: {geometry.name} should not be None" + ) + self.name = geometry.name + "_BC" + self.constraint_type = "BC" + self._random_merge = self.sampling_config.random_merge + + def __getitem__(self, bc_index): + if not self.data: + self._initialization() + if self.sampling_config.random_sampling: + bc_index = self._get_index_when_sample_iter(bc_index, geom_type="BC") + else: + bc_index = self._get_index_when_sample_all(bc_index) + return self._get_item(bc_index) + + def _initialization(self, batch_size=1, shuffle=False): + return self._initialize(batch_size=batch_size, shuffle=shuffle, geom_type="BC") + + +class BoundaryIC(Boundary): + """Sampling data of initial condition.""" + + def __init__(self, geometry): + super().__init__(geometry) + self.sampling_config = self.geometry.sampling_config.ic + if not self.sampling_config: + raise ValueError( + f"IC info for the current geometry: {geometry.name} should not be None" + ) + self.name = geometry.name + "_IC" + self.constraint_type = "IC" + self._random_merge = self.sampling_config.random_merge + + def __getitem__(self, ic_index): + if not self.data: + self._initialization() + if self.sampling_config.random_sampling: + ic_index = self._get_index_when_sample_iter(ic_index, geom_type="IC") + else: + ic_index = self._get_index_when_sample_all(ic_index) + return self._get_item(ic_index) + + def _initialization(self, batch_size=1, shuffle=False): + return self._initialize(batch_size=batch_size, shuffle=shuffle, geom_type="IC") diff --git a/MindFlow/data/data_base.py b/MindFlow/data/data_base.py new file mode 100644 index 0000000000000000000000000000000000000000..6ca591c53bfd210835c19bcbcad21e2f9cce1a3f --- /dev/null +++ b/MindFlow/data/data_base.py @@ -0,0 +1,145 @@ +# Copyright 2021 Huawei Technologies Co., Ltd +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# ============================================================================== +""" +This dataset module supports various type of datasets, including .... Some of the operations that are +provided to users to preprocess data include shuffle, batch, repeat, map, and zip. +""" +from __future__ import absolute_import + +import os +import abc + +from mindscience.utils.check_func import ( + check_param_type, + check_param_value, + check_param_type_value +) + +CONSTRAINT_TYPES = ["equation", "bc", "ic", "label", "function", "custom"] +DATA_FORMATS = ["npy"] + + +class Data: + """ + Base class of Dataset, Equation, Boundary and ExistedDataset. + + Args: + name (str): Dataset name. + columns_list (list/tuple): Column names. + constraint_type (str): Type of constraint. + + Supported Platforms: + ``Ascend`` + """ + + def __init__(self, name=None, columns_list=None, constraint_type=None): + none_type = type(None) + + # type checks + check_param_type(name, "name", data_type=[str, none_type]) + check_param_type(columns_list, "columns_list", data_type=[list, tuple, none_type]) + check_param_type(constraint_type, "constraint_type", data_type=[str, none_type]) + + if constraint_type: + check_param_value(constraint_type.lower(), "constraint_type", CONSTRAINT_TYPES) + + self.name = name + self.columns_list = columns_list + self.constraint_type = constraint_type + self.dataset_type = type(self).__name__ + + def set_constraint_type(self, constraint_type="Equation"): + """Set dataset constraint type.""" + check_param_type(constraint_type, "constraint_type", data_type=str) + check_param_value(constraint_type.lower(), "constraint_type", CONSTRAINT_TYPES) + self.constraint_type = constraint_type + + @abc.abstractmethod + def create_dataset(self): + """Return a dataset (abstract).""" + raise NotImplementedError(f"{self.dataset_type}.create_dataset not implemented") + + @abc.abstractmethod + def _initialization(self): + """Initialize dataset (abstract).""" + raise NotImplementedError(f"{self.dataset_type}._initialization not implemented") + + @abc.abstractmethod + def __getitem__(self, index): + """Return item by index (abstract).""" + raise NotImplementedError(f"{self.dataset_type}.__getitem__ not implemented") + + @abc.abstractmethod + def __len__(self): + """Return dataset length (abstract).""" + raise NotImplementedError(f"{self.dataset_type}.__len__ not implemented") + + +class ExistedDataConfig: + """ + Configuration of ExistedDataset. + + Args: + name (str): Dataset name. + data_dir (str/list): Path(s) to existing data files. + columns_list (str/list): Column names. + data_format (str): File format (supports 'npy'). + constraint_type (str): Constraint type. + random_merge (bool): Whether to randomly merge datasets. + """ + + def __init__( + self, + name, + data_dir, + columns_list, + data_format="npy", + constraint_type="Label", + random_merge=True + ): + # name + check_param_type(name, "name", data_type=str) + self.name = name + + # data_dir + if isinstance(data_dir, str): + data_dir = [data_dir] + + check_param_type(data_dir, "data_dir", data_type=[str, list, tuple]) + for path in data_dir: + if not os.path.exists(path): + raise ValueError(f"ExistedDataset file: {path} does not exist") + + self.data_dir = data_dir + + # columns_list + if isinstance(columns_list, str): + columns_list = [columns_list] + + check_param_type(columns_list, "columns_list", data_type=[str, tuple, list]) + self.columns_list = columns_list + + # constraint_type + check_param_type(constraint_type, "constraint_type", data_type=str) + check_param_value(constraint_type.lower(), "constraint_type", CONSTRAINT_TYPES) + self.constraint_type = constraint_type + + # data_format + check_param_type_value(data_format, "data_format", DATA_FORMATS, data_type=str) + self.data_format = data_format + + # random_merge + check_param_type(random_merge, "random_merge", data_type=bool) + self.random_merge = random_merge diff --git a/MindFlow/data/dataset.py b/MindFlow/data/dataset.py new file mode 100644 index 0000000000000000000000000000000000000000..0bcf5903a4451f8bb9a05a1fabf27f474dbf56f0 --- /dev/null +++ b/MindFlow/data/dataset.py @@ -0,0 +1,340 @@ +# Copyright 2021 Huawei Technologies Co., Ltd +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# ============================================================================== +#pylint: disable=W0223 +#pylint: disable=W0221 +#pylint: disable=W0212 +""" +Combine pde/ic/bc datasets together +""" +from __future__ import absolute_import + +import copy + +import mindspore.dataset as ds +from mindspore import log as logger + + +from mindscience.data.flow.geometry import Geometry +from mindscience.utils.check_func import check_param_type, check_dict_type_value + +from .data_base import Data, ExistedDataConfig +from .existed_data import ExistedDataset +from .equation import Equation +from .boundary import BoundaryIC, BoundaryBC + +_geomdata_dict = { + "domain": Equation, + "IC": BoundaryIC, + "BC": BoundaryBC, +} + + +class Dataset(Data): + r""" + Combine datasets together. + """ + + def __init__(self, geometry_dict=None, existed_data_list=None, dataset_list=None): + super().__init__() + if all((geometry_dict is None, existed_data_list is None, dataset_list is None)): + raise ValueError("Dataset should have at least one sub-dataset, but got None") + + if geometry_dict is not None: + check_param_type(geometry_dict, "geometry_dict", data_type=dict) + check_dict_type_value( + geometry_dict, + "geometry_dict", + key_type=Geometry, + value_type=str, + value_value=list(_geomdata_dict.keys()) + ) + + if existed_data_list is not None: + if isinstance(existed_data_list, ExistedDataConfig): + existed_data_list = [existed_data_list] + check_param_type(existed_data_list, "existed_data_list", (list, tuple)) + + for data_config in existed_data_list: + check_param_type(data_config, "element in existed_data_list", ExistedDataConfig) + + if dataset_list is not None: + if isinstance(dataset_list, Data): + dataset_list = [dataset_list] + check_param_type(dataset_list, "dataset_list", (list, tuple)) + for dataset in dataset_list: + check_param_type(dataset, "element in dataset_list", Data) + + self.existed_data_list = existed_data_list + self.geometry_dict = geometry_dict + self.dataset_list = dataset_list + self.all_datasets = dataset_list if dataset_list else [] + self.columns_list = None + self._iterable_datasets = None + + self.num_dataset = len(dataset_list) if dataset_list else 0 + if existed_data_list: + self.num_dataset += len(existed_data_list) + if geometry_dict: + for geom in geometry_dict: + self.num_dataset += len(geometry_dict[geom]) + + logger.info(f"Total datasets number: {self.num_dataset}") + self.dataset_columns_map = {} + self.column_index_map = {} + self.dataset_constraint_map = {} + + def _create_dataset_from_geometry(self, geometry, geom_type="domain"): + """create dataset from geometry.""" + dataset_instance = _geomdata_dict.get(geom_type)(geometry) + return dataset_instance + + def _get_all_datasets(self): + """get all datasets""" + if self.geometry_dict: + for geom, types in self.geometry_dict.items(): + for geom_type in types: + dataset = self._create_dataset_from_geometry(geom, geom_type) + self.all_datasets.append(dataset) + + if self.existed_data_list: + for data_config in self.existed_data_list: + dataset = ExistedDataset(data_config=data_config) + self.all_datasets.append(dataset) + + def create_dataset(self, + batch_size=1, + preprocess_fn=None, + input_output_columns_map=None, + shuffle=True, + drop_remainder=True, + prebatched_data=False, + num_parallel_workers=1, + num_shards=None, + shard_id=None, + python_multiprocessing=False, + sampler=None): + """ + create the final mindspore type dataset to merge all the sub-datasets. + """ + self._get_all_datasets() + check_param_type(prebatched_data, "prebatched_data", data_type=bool) + check_param_type(drop_remainder, "drop_remainder", data_type=bool) + check_param_type(shuffle, "shuffle", data_type=bool) + check_param_type(batch_size, "batch_size", data_type=int, exclude_type=bool) + + if prebatched_data and not drop_remainder: + raise ValueError( + "prebatched_data is not supported when drop_remainder is set to be False" + ) + + for dataset in self.all_datasets: + prebatch_size = batch_size if prebatched_data else 1 + prebatch_shuffle = shuffle if prebatched_data else False + dataset._initialization(batch_size=prebatch_size, shuffle=prebatch_shuffle) + self.columns_list = ( + dataset.columns_list if not self.columns_list + else self.columns_list + dataset.columns_list + ) + logger.info( + f"Check initial all dataset, dataset: {dataset.name}, " + f"columns_list: {dataset.columns_list}, data_size: {len(dataset)}" + ) + + dataset = self._merge_all_datasets( + shuffle=False if prebatched_data else shuffle, + num_parallel_workers=num_parallel_workers, + num_shards=num_shards, + shard_id=shard_id, + python_multiprocessing=python_multiprocessing + ) + logger.info(f"Initial dataset size: {dataset.get_dataset_size()}") + logger.info(f"Get all dataset columns names: {self.columns_list}") + + self.dataset_columns_map, self.dataset_constraint_map, self.column_index_map = \ + self._create_trace_maps() + logger.info(f"Dataset columns map: {self.dataset_columns_map}") + logger.info(f"Dataset column index map: {self.column_index_map}") + logger.info(f"Dataset constraints map: {self.dataset_constraint_map}") + + if sampler: + logger.info("Dataset uses sampler") + dataset.use_sampler(sampler) + + if preprocess_fn: + input_columns = copy.deepcopy(self.columns_list) + check_param_type( + input_output_columns_map, + "input_output_columns_map", + (type(None), dict) + ) + if input_output_columns_map: + new_columns_list, new_dataset_columns_map = self._update_columns_list( + input_output_columns_map + ) + self.columns_list = new_columns_list + self.dataset_columns_map = new_dataset_columns_map + self.column_index_map = {} + for idx, name in enumerate(self.columns_list): + self.column_index_map[name] = idx + logger.info( + f"Dataset columns map after preprocess: {self.dataset_columns_map}" + ) + logger.info( + f"Dataset column index after preprocess: {self.column_index_map}" + ) + logger.info( + f"Dataset constraints after preprocess: {self.dataset_constraint_map}" + ) + output_columns = self.columns_list + + dataset = dataset.map( + operations=preprocess_fn, + input_columns=input_columns, + output_columns=output_columns, + num_parallel_workers=num_parallel_workers, + python_multiprocessing=python_multiprocessing + ) + dataset = dataset.project(output_columns) + logger.info( + f"Get all dataset columns names after preprocess: {self.columns_list}" + ) + + if not prebatched_data: + dataset = dataset.batch( + batch_size=batch_size, + drop_remainder=drop_remainder, + num_parallel_workers=num_parallel_workers + ) + logger.info(f"Final dataset size: {dataset.get_dataset_size()}") + return dataset + + def _merge_all_datasets(self, shuffle=True, num_parallel_workers=1, num_shards=1, + shard_id=0, python_multiprocessing=False): + """merge all datasets""" + self._iterable_datasets = _IterableDatasets(self.all_datasets) + dataset = ds.GeneratorDataset( + source=self._iterable_datasets, + column_names=self.columns_list, + shuffle=shuffle, + num_parallel_workers=num_parallel_workers, + num_shards=num_shards, + shard_id=shard_id, + python_multiprocessing=python_multiprocessing + ) + return dataset + + def _update_columns_list(self, input_output_columns_map): + """update columns list""" + new_dataset_columns_map = {} + for dataset in self.all_datasets: + columns_list = dataset.columns_list + new_dataset_columns_map[dataset.name] = [] + for column in columns_list: + if column in input_output_columns_map.keys(): + new_column = input_output_columns_map[column] + if isinstance(new_column, list): + new_dataset_columns_map[dataset.name] += new_column + else: + new_dataset_columns_map.get(dataset.name).append(new_column) + else: + new_dataset_columns_map.get(dataset.name).append(column) + + new_columns_list = [] + for _, columns in new_dataset_columns_map.items(): + new_columns_list += columns + return new_columns_list, new_dataset_columns_map + + def get_columns_list(self): + """ + get columns list + + Returns: + list[str]. column names list of the final unified dataset. + """ + if not self.columns_list: + raise ValueError( + "Please call create_dataset() first before get final columns list to avoid unexpected error" + ) + return self.columns_list + + def _create_trace_maps(self): + """create trace maps""" + dataset_columns_map = {} + dataset_constraint_map = {} + column_index_map = {} + for dataset in self.all_datasets: + name = dataset.name + dataset_columns_map[name] = dataset.columns_list + dataset_constraint_map[name] = dataset.constraint_type + + for i, column in enumerate(self.columns_list): + column_index_map[column] = i + return dataset_columns_map, dataset_constraint_map, column_index_map + + def __getitem__(self, index): + if not self._iterable_datasets: + raise ValueError( + "Call create_dataset() before getting item by index to avoid unexpected error" + ) + return self._iterable_datasets[index] + + def set_constraint_type(self, constraint_type="Equation"): + """set constraint type of dataset""" + if isinstance(constraint_type, str): + logger.warning( + f"Argument constraint_type: {constraint_type} is str, " + "the same type will be set for all of the sub-datasets" + ) + for datasets in self.all_datasets: + datasets.set_constraint_type(constraint_type) + elif isinstance(constraint_type, dict): + for dataset in constraint_type.keys(): + if dataset not in self.all_datasets: + raise ValueError( + f"Unknown dataset: {dataset}. All sub-dataset are: " + f"{[data.name for data in self.all_datasets]}" + ) + dataset.set_constraint_type(constraint_type[dataset]) + else: + raise TypeError( + f"the type of constraint_type should be dict or str but got {type(constraint_type)}" + ) + + def __len__(self): + if not self._iterable_datasets: + raise ValueError( + "Call create_dataset() before getting item by index to avoid unexpected error" + ) + return len(self._iterable_datasets) + + +class _IterableDatasets: + """get data iteratively""" + + def __init__(self, dataset_list): + self.dataset_list = dataset_list + dataset_size = [len(dataset) for dataset in dataset_list] + logger.info(f"Get all dataset sizes: {dataset_size}") + self.longest = max(dataset_size) + + def __getitem__(self, index): + col_data = None + for dataset_instance in self.dataset_list: + item = dataset_instance[index] + col_data = col_data + item if col_data else item + return col_data + + def __len__(self): + return self.longest diff --git a/MindFlow/data/equation.py b/MindFlow/data/equation.py new file mode 100644 index 0000000000000000000000000000000000000000..134a5dc6beaa382880a4cfa76e41a51720e0bb75 --- /dev/null +++ b/MindFlow/data/equation.py @@ -0,0 +1,142 @@ +# Copyright 2021 Huawei Technologies Co., Ltd +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# ============================================================================== +#pylint: disable=W0223 +#pylint: disable=W0221 +""" +Sampling data of equation domain. +""" +from __future__ import absolute_import + +import numpy as np +from mindspore import log as logger + +from mindscience.data.flow.geometry import Geometry, SamplingConfig +from mindscience.utils.check_func import check_param_type +from .data_base import Data + + +_SPACE = "" + + +class Equation(Data): + """ + Sampling data of equation domain. + """ + + def __init__(self, geometry): + check_param_type(geometry, "geometry", data_type=Geometry) + check_param_type( + geometry.sampling_config, + f"sampling_config of geometry:{geometry.name}", + data_type=SamplingConfig, + ) + + self.sampling_config = geometry.sampling_config.domain + if not self.sampling_config: + raise ValueError(f"domain info of geometry: {geometry.name} should not be None") + + self.geometry = geometry + + # dataset internal fields + self.data = None + self.data_size = None + self.batch_size = 1 + self.shuffle = False + self.batched_data_size = None + self.columns_list = None + + self._domain_index = None + self._domain_index_num = 0 + self._random_merge = self.sampling_config.random_merge + + name = f"{geometry.name}_domain" + columns_list = [f"{geometry.name}_domain_points"] + constraint_type = "Equation" + + super().__init__(name=name, columns_list=columns_list, constraint_type=constraint_type) + + def _get_sampling_data(self): + """Sampling domain points.""" + sample_data = self.geometry.sampling(geom_type="domain") + return sample_data, self.geometry.columns_dict["domain"] + + def _initialization(self, batch_size=1, shuffle=False): + """initialization: sampling and set attrs.""" + data, self.columns_list = self._get_sampling_data() + if not isinstance(data, tuple): + data = (data,) + + self.data = data + self.data_size = len(self.data[0]) + self.batch_size = batch_size + + if batch_size > self.data_size: + raise ValueError( + f"If prebatch data, batch_size: {batch_size} should not be larger than " + f"data size: {self.data_size}." + ) + + self.batched_data_size = self.data_size // batch_size + self.shuffle = shuffle + self._domain_index = np.arange(self.data_size) + + logger.info( + f"Get domain dataset: {self.name}, columns: {self.columns_list}, " + f"size: {self.data_size}, batched_size: {self.batched_data_size}, shuffle: {self.shuffle}" + ) + + def _get_index_when_sample_iter(self, index): + """Index logic for random_sampling=True.""" + if self._domain_index_num == self.batched_data_size: + self._initialization(self.batch_size, self.shuffle) + self._domain_index_num = 0 + index = self._domain_index_num + self._domain_index_num += 1 + return index + + def _get_index_when_sample_all(self, index): + """Index logic for random_sampling=False.""" + data_size = len(self) + if (self._random_merge or self.shuffle) and index % data_size == 0: + self._domain_index = np.random.permutation(self.data_size) + return index % data_size + + def __getitem__(self, index): + if self.data is None: + self._initialization() + + if self.sampling_config.random_sampling: + index = self._get_index_when_sample_iter(index) + else: + index = self._get_index_when_sample_all(index) + + col_data = None + for i in range(len(self.columns_list)): + if self.batch_size == 1: + idx = self._domain_index[index] + else: + idx = self._domain_index[ + index * self.batch_size : (index + 1) * self.batch_size + ] + + temp_data = self.data[i][idx] + col_data = (temp_data,) if col_data is None else col_data + (temp_data,) + + return col_data + + def __len__(self): + if self.data is None: + self._initialization() + return self.batched_data_size diff --git a/MindFlow/data/existed_data.py b/MindFlow/data/existed_data.py new file mode 100644 index 0000000000000000000000000000000000000000..8d77576f29014846d7f76f533ad1d38384e6f23c --- /dev/null +++ b/MindFlow/data/existed_data.py @@ -0,0 +1,166 @@ +# Copyright 2021 Huawei Technologies Co., Ltd +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# ============================================================================== +#pylint: disable=W0223 +#pylint: disable=W0221 +""" +This dataset module supports npy type of datasets. Some of the operations that are +provided to users to preprocess data include shuffle, batch, repeat, map, and zip. +""" +from __future__ import absolute_import + +import numpy as np +from mindspore import log as logger + +from mindscience.utils.check_func import check_param_type +from .data_base import Data, ExistedDataConfig + + + +class ExistedDataset(Data): + """ + Load existing dataset (currently supports npy format only). + """ + + def __init__( + self, + name=None, + data_dir=None, + columns_list=None, + data_format="npy", + constraint_type="Label", + random_merge=True, + data_config=None + ): + # Handle configuration source + if data_config is None: + if not name or not data_dir or not columns_list: + raise ValueError( + f"If data_config is None, name/data_dir/columns_list must not be None, " + f"but got name={name}, data_dir={data_dir}, columns_list={columns_list}" + ) + data_config = ExistedDataConfig( + name=name, + data_dir=data_dir, + columns_list=columns_list, + data_format=data_format, + constraint_type=constraint_type, + random_merge=random_merge + ) + + check_param_type(data_config, "data_config", data_type=ExistedDataConfig) + + # Assign from config + name = data_config.name + columns_list = [f"{name}_{col}" for col in data_config.columns_list] + constraint_type = data_config.constraint_type + + self.data_dir = data_config.data_dir + self._data_format = data_config.data_format + self._random_merge = data_config.random_merge + + # Internal dataset storage + self.data = None + self.data_size = None + self.batch_size = 1 + self.shuffle = False + self.batched_data_size = None + self._index = None + + # Data loader mapping + self.load_data = { + "npy": self._load_npy_data + } + + super().__init__(name=name, columns_list=columns_list, constraint_type=constraint_type) + + def _initialization(self, batch_size=1, shuffle=False): + """Load data once before training starts.""" + loader = self.load_data.get(self._data_format.lower()) + if loader is None: + raise ValueError(f"Unsupported data format: {self._data_format}") + + data = loader() + if not isinstance(data, tuple): + data = (data,) + + self.data = data + self.data_size = len(data[0]) + self.batch_size = batch_size + + if batch_size > self.data_size: + raise ValueError( + f"If prebatch data, batch_size={batch_size} cannot exceed data_size={self.data_size}" + ) + + self.batched_data_size = self.data_size // batch_size + self.shuffle = shuffle + self._index = np.arange(self.data_size) + + logger.info( + f"Loaded existed dataset: {self.name}, columns={self.columns_list}, " + f"size={self.data_size}, batched_size={self.batched_data_size}, shuffle={self.shuffle}" + ) + + def __getitem__(self, index): + if self.data is None: + self._initialization() + + if self._random_merge: + # Select random batch index + index = ( + np.random.randint(0, self.batched_data_size) + if index >= self.batched_data_size + else index + ) + else: + index = index % self.batched_data_size + + if self.shuffle and index % self.batched_data_size == 0: + self._index = np.random.permutation(self.data_size) + + col_data = None + for i in range(len(self.columns_list)): + + if self.batch_size == 1: + idx = self._index[index] + else: + idx = self._index[ + index * self.batch_size : (index + 1) * self.batch_size + ] + + temp = self.data[i][idx] + col_data = (temp,) if col_data is None else col_data + (temp,) + + return col_data + + def _load_npy_data(self): + """Load data from npy files.""" + results = [] + for path in self.data_dir: + logger.info(f"Loading npy data from: {path}") + arr = np.load(path).astype(np.float32) + + if arr.ndim < 1: + raise ValueError(f"Loaded npy file must have at least 1 dimension: {path}") + + results.append(arr) + + logger.info(f"Loaded npy dataset size: {len(results[0])}") + return tuple(results) + + def __len__(self): + if self.data is None: + self._initialization() + return self.batched_data_size diff --git a/mindscience/models/neural_operator/fno.py b/mindscience/models/neural_operator/fno.py index edd214bb5a8e349567d73234f44d49aceb0fed90..800a1872d8dec2f8ab80339de72ff3e7a459ab1a 100644 --- a/mindscience/models/neural_operator/fno.py +++ b/mindscience/models/neural_operator/fno.py @@ -94,9 +94,10 @@ class FNOBlocks(nn.Cell): self.resolutions = resolutions if len(self.n_modes) != len(self.resolutions): raise ValueError( - "The dimension of n_modes should be equal to that of resolutions\ - but got dimension of n_modes {} and dimension of resolutions {}".format(len(self.n_modes), - len(self.resolutions))) + f"The dimension of n_modes should be equal to that of resolutions " + f"but got dimension of n_modes {len(self.n_modes)} " + f"and dimension of resolutions {len(self.resolutions)}" + ) self.act = get_activation(act) if isinstance(act, str) else act self.add_residual = add_residual self.dft_compute_dtype = dft_compute_dtype @@ -136,8 +137,10 @@ class FNOBlocks(nn.Cell): has_bias=False, weight_init="HeUniform" ).to_float(self.fno_compute_dtype) else: - raise ValueError("The length of input resolutions dimensions should be in [1, 2, 3], but got: {}".format( - len(self.resolutions))) + raise ValueError( + f"The length of input resolutions dimensions should be in [1, 2, 3], " + f"but got: {len(self.resolutions)}" + ) def construct(self, x: Tensor): if self.add_residual: @@ -247,9 +250,10 @@ class FNO(nn.Cell): self.resolutions = resolutions if len(self.n_modes) != len(self.resolutions): raise ValueError( - "The dimension of n_modes should be equal to that of resolutions\ - but got dimension of n_modes {} and dimension of resolutions {}".format(len(self.n_modes), - len(self.resolutions))) + f"The dimension of n_modes should be equal to that of resolutions " + f"but got dimension of n_modes {len(self.n_modes)} " + f"and dimension of resolutions {len(self.resolutions)}" + ) self.n_layers = n_layers self.data_format = data_format if fnoblock_act == "identity": @@ -324,7 +328,8 @@ class FNO(nn.Cell): output_perm = (0, 2, 3, 4, 1) else: raise ValueError( - "The length of input resolutions dimensions should be in [1, 2, 3], but got: {}".format(n_dim)) + f"The length of input resolutions dimensions should be in [1, 2, 3], but got: {n_dim}" + ) return positional_embedding, input_perm, output_perm @@ -519,13 +524,13 @@ class FNO2D(FNO): resolutions = [resolutions, resolutions] if len(n_modes) != 2: raise ValueError( - "The dimension of n_modes should be equal to 2 when using FNO2D " - "but got dimension of n_modes {}".format(len(n_modes)) + f"The dimension of n_modes should be equal to 2 when using FNO2D " + f"but got dimension of n_modes {len(n_modes)}" ) if len(resolutions) != 2: raise ValueError( - "The dimension of resolutions should be equal to 2 when using FNO2D " - "but got dimension of resolutions {}".format(len(resolutions)) + f"The dimension of resolutions should be equal to 2 when using FNO2D " + f"but got dimension of resolutions {len(resolutions)}" ) super().__init__( in_channels, @@ -637,13 +642,13 @@ class FNO3D(FNO): resolutions = [resolutions, resolutions, resolutions] if len(n_modes) != 3: raise ValueError( - "The dimension of n_modes should be equal to 3 when using FNO3D " - "but got dimension of n_modes {}".format(len(n_modes)) + f"The dimension of n_modes should be equal to 3 when using FNO3D " + f"but got dimension of n_modes {len(n_modes)}" ) if len(resolutions) != 3: raise ValueError( - "The dimension of resolutions should be equal to 3 when using FNO3D " - "but got dimension of resolutions {}".format(len(resolutions)) + f"The dimension of resolutions should be equal to 3 when using FNO3D " + f"but got dimension of resolutions {len(resolutions)}" ) super().__init__( in_channels,