mindspore Releases

Latest

v2.4.10

8e2ae93

2025-01-07 10:06

MindSpore 2.4.10

Last committed message: !79454 【Bugfix】增大stress detect的内存使用

v2.4.1

0184782

2024-12-04 14:45

MindSpore 2.4.1 Release Notes

## MindSpore 2.4.1 Release Notes

### Major Features and Improvements

#### AutoParallel

- [STABLE] Split/concat branch communication computation parallel is supported. Users split input data to form parallelizable branches. Automatic communication computing parallelism is performed between branches, reducing communication overhead.
- [STABLE] Sequence pipelines are supported. The LLama series models for the dev branch of MindFormers reduces the Bubble as well as the memory overhead of pipeline parallelism by introducing Sequence dimension splitting.

#### PyNative

- [STABLE] In PyNative mode, communication operators are assigned streams by default based on the communication domain. They support concurrent execution of communication operators, optimize collaborative parallel strategies, provide fine-grained communication masking, and enhance model performance.

### Bug Fixes

- [IB0R4N](https://gitee.com/mindspore/mindspore/issues/IB0R4N): Fixed the problem of loading distributed weights with inaccurate accuracy under certain splitting strategies.

### Contributors

bantao;caifubi;candanzg;chaijinwei;changzherui;chengbin;chujinjin;DeshiChen;dingjinshan;fary86;fuhouyu;gaoyong10;GuoZhibin;halo;haozhang;hedongdong;huangbingjian;hujiahui8;huoxinyou;jiangshanfeng;jiaorui;jiaxueyu;jshawjc;kisnwang;lichen;limingqi107;liubuyu;looop5;luochao60;luoyang;machenggui;MengXiangyu;Mrtutu;NaCN;panzhihui;qiuzhongya;shenhaojing;shilishan;tanghuikang;TuDouNi;wang_ziqi;weiyang;wujueying;XianglongZeng;xuxinglei;yang guodong;yanghaoran;yao_yf;yide12;yihangchen;YijieChen;YingtongHu;yuchaojie;YuJianfeng;zhangdanyang;ZhangZGC;zhengzuohe;zong_shuai;ZPaC;冯一航;胡彬;宦晓玲;李林杰;刘崇鸣;刘勇琪;任新;王禹程;王振邦;熊攀;俞涵;张栩浩;周一航;

Last committed message: !77790 fix PagedAttention Resize cost time, update ms_kernels_internal

v2.4.0

8c86f33

2024-10-29 10:37

MindSpore 2.4.0 Release Notes

## MindSpore 2.4.0 Release Notes

### Major Features and Improvements

#### Dataset

- [STABLE] Modify the default value of the `max_rowsize` parameter in the interface [mindspore.dataset.GeneratorDataset](https://www.mindspore.cn/docs/en/r2.4.0/api_python/dataset/mindspore.dataset.GeneratorDataset.html), [mindspore.dataset.Dataset.map](https://www.mindspore.cn/docs/en/r2.4.0/api_python/dataset/dataset_method/operation/mindspore.dataset.Dataset.map.html), and [mindspore.dataset.Dataset.batch](https://www.mindspore.cn/docs/en/r2.4.0/api_python/dataset/dataset_method/batch/mindspore.dataset.Dataset.batch.html) to None to enable dynamic allocation of shared memory by default, in which case the shared memory will be requested in real time with the input data and accelerate the data processing, so the user does not need to adjust the size of this parameter in advance.
- [BETA] Data processing supports the independent process mode, which will reduce the GIL lock conflict between the training process and the data reading process to improve the performance in dynamic graph mode. This mode can be enabled or disabled via the environment variable `MS_INDEPENDENT_DATASET`.

#### Ascend

- [STABLE] Customized operators support the Ascend Dynamics graph scenario Pyboost execution mode, which reduces operator call overhead.
- [STABLE] The Ascend Print operator supports scenarios where the output is oversized tensor or print calls are intensive, and users can specify the slice size and timeout time to support different scenarios via the `MS_DUMP_SLICE_SIZE` and `MS_DUMP_WAIT_TIME` environment variables.
- [STABLE] Unified deterministic computation settings. Users can enable ascending deterministic computation by only setting `mindspore.set_context(deterministic="ON")`.
- [STABLE] Supports aggregate communication anomaly monitoring, quickly exits training after monitoring communication anomalies to avoid timeout waiting.
- [STABLE] Supports the [graceful exit function for sub-healthy devices](https://www.mindspore.cn/docs/en/r2.4.0/model_train/train_availability/graceful_exit.html). When the training framework detects the presence of sub-healthy device configuration information in the cluster, it saves the CKPT and uniformly ends the cluster training process.

#### Runtime

- [STABLE] Backend compilation cache is supported in O0/O1 mode and is turned on by default when frontend compilation cache is turned on.
- [STABLE] The aclnnAllGatherMatmul, aclnnMatmulReduceScatter, and aclnnMatmulAllReduce algorithms are supported in O0/O1 modes to improve performance.
- [STABLE] O0/O1 modes support to disable cluster heartbeat configuration by export MS_DISABLE_HEARTBEAT=1 to reduce scheduler load.
- [STABLE] O0/O1 modes support communication arithmetic fusion.
- [STABLE] Virtual memory support in O2 mode, defragmentation support, which is enabled in Ascend backend by default.
- [STABLE] Dynamic request for device memory occupation, support single card multi-user use, which is enabled in Ascend backend by default.
- [STABLE] Optimize graph fusion compilation performance in O1 mode, enabled by default.
- [STABLE] Support kernel packet fusion optimization in O1 mode to improve the performance of dynamic shape network execution, enabled by default.
- [BETA] Epilogue fusion between the MatMul and Elementwise operator is supported in O1 mode. Enable via `mindspore.set_context(graph_kernel_flags="--enable_cluster_ops=MatMul")`.
- [BETA] O1 mode supports user-controlled graph fusion optimization scope, user can control to turn on or off the corresponding fusion operator via the enable_pass/disable_pass option of graph_kernel_flags.
- [BETA] The GPTO execution order optimization module is supported in O0 mode and is enabled through mindspore.set_context(exec_order="gpto").

#### PyNative

- [STABLE] Parameter cell_id of Hook function corresponding to [mindspore.nn.Cell.register_backward_hook](https://www.mindspore.cn/docs/en/r2.4.0/api_python/nn/mindspore.nn.Cell.html#mindspore.nn.Cell.register_backward_hook) and [mindspore.nn.Cell.register_forward_hook](https://www.mindspore.cn/docs/en/r2.4.0/api_python/nn/mindspore.nn.Cell.html#mindspore.nn.Cell.register_forward_hook) is changed to cell's python object.
- [STABLE] Added [Cell.register_backward_pre_hook](https://www.mindspore.cn/docs/en/r2.4.0/api_python/nn/mindspore.nn.Cell.html#mindspore.nn.Cell.register_backward_pre_hook) interface, this API registers the backward propagation hook function on a Cell, which is called each time the gradient of that Cell is computed.
- [STABLE] Optimize the PyNative process AICPU class operator downstream cache to improve API execution performance.
- [STABLE] Added the function of converting the device memory occupied by a group of Tensor to a contiguous piece of memory under dynamic graph.

#### FrontEnd

- [STABLE] Weight de-redundancy saving and loading is supported in fault recovery scenarios.
- [STABLE] Mixed precision training with support for [auto mode](https://www.mindspore.cn/docs/en/r2.4.0/api_python/amp/mindspore.amp.auto_mixed_precision.html#mindspore.amp.auto_mixed_precision).
- [STABLE] Support saving and loading of safetensors format, as well as offline aggregation and distributed loading based on safetensors in parallel scenarios.
- [BETA] Added new cyclic arithmetic interface [mindspore.ops.WhileLoop](https://www.mindspore.cn/docs/en/r2.4.0/api_python/ops/mindspore.ops.WhileLoop.html), [mindspore.ops.ForiLoop](https://www.mindspore.cn/docs/en/r2.4.0/api_python/ops/mindspore.ops.ForiLoop.html), and [mindspore.ops.Scan](https://www.mindspore.cn/docs/en/r2.4.0/api_python/ops/mindspore.ops.Scan.html), optimizing loop compilation time.
- [BETA] The graph mode supports the operator passing keyword arguments.

#### Parallel

- [STABLE] [mindspore.ops.TensorDump](https://www.mindspore.cn/docs/en/r2.4.0/api_python/ops/mindspore.ops.TensorDump.html) operator supports distributed parallel scenarios, and users can decide to print input/output slices by configuring the TensorDump operator's `input_output` attribute; add new interface [mindspore.ops.tensordump](https://www.mindspore.cn/docs/en/r2.4.0/api_python/ops/mindspore.ops.tensordump.html).
- [STABLE] msrun supports customizing the rank id based on the passing rank table file, and supports rearranging the rank id via the `--rank_table_file` passing json file.
- [STABLE] Supports LCCL, a high-performance communication library in Ascend stand-alone. Users can enable LCCL in Ascend back-end training scenarios via the `MS_ENABLE_LCCL` environment variable.
- [STABLE] The strategy propagation algorithm is adapted to LLaMA/Mixtral networks, which reduces the workload of users in configuring the sharding strategy for LLaMA/Mixtral networks.
- [STABLE] Support high dimensional tensor parallelism, user can configure [mindspore.ops.MatMul](https://www.mindspore.cn/docs/en/r2.4.0/api_python/ops/mindspore.ops.MatMul.html) and [mindspore.oops.BatchMatMul](https://www.mindspore.cn/docs/en/r2.4.0/api_python/ops/mindspore.ops.BatchMatMul.html) input_layout switching 1D/2D/3D tensor slice mode.
- [STABLE] Simulation compilation does not consume hardware resources when SIMULATION_LEVEL=0 and SIMULATION_LEVEL=1 runtime jit_level is O0/O1.
- [STABLE] Allreduce introduced in parallel by the BatchMatMul model is automatically converted to a ReduceScatter to reduce communication according to the matching rules if enable_allreduce_slice_to_reducescatter is turned on in parallel_speed_up_json when it follows the slice operation.
- [STABLE] [mindspore.nn.Cell.shard](https://www.mindspore.cn/docs/en/master/api_python/nn/mindspore.nn.Cell.html#mindspore.nn.Cell.shard) and [mindspore.shard](https://www.mindspore.cn/docs/en/r2.4.0/api_python/mindspore/mindspore.shard.html) support user-configurable policies of type mindspore.Layout and sharding strategy for each parameter parameter_plan.
- [BETA] SAPP supports fully automatic generation of residual arithmetic policies after manual preconfiguration of arithmetic parallel sharding strategy. The user activates the `.shard()` preconfigured parallel sharding strategy by turning on the `MS_INTERFERED_SAPP` environment variable.
- The [BETA] [mindspore.ops.Custom](https://www.mindspore.cn/docs/en/r2.4.0/api_python/ops/mindspore.ops.Custom.html) operator supports configuring the sharding strategy.

#### Inference

- [STABLE] New Qwen2 and LLaMA3.1 series of large models support training and inference architecture, realize the unification of script, distributed policy and runtime, reduce the inference delay by fusing large operators, and effectively improve the network throughput.
- [STABLE] Support parallel decoding service-oriented deployment to realize LookAhead speculative inference for large models of LLaMA series.
- [BETA] Support SLoRA service-oriented deployment, realizing multi-trimming weight scheduling inference for large models.

#### Dump

- [STABLE] Optimize [Dump](https://www.mindspore.cn/docs/en/r2.4.0/model_train/debug/dump.html) for use by device type and optimization level.
- [STABLE] Asynchronous Dump support in Ascend O0/O1 mode, including asynchronous Tensor, overflow, and statistics (host and device modes).
- [STABLE] Overflow Dump supports configuring the maximum number of overflows.
- [STABLE] Ascend O2 mode supports set dump.
- [STABLE] Support qint4 x 2 quantization type Dump.

### API Change

#### New API

- [STABLE] [mindspore.mint](https://www.mindspore.cn/docs/en/r2.4.0/api_python/mindspore.mint.html) APIs add a large number of functional, nn interfaces. mint interfaces are currently experimental interfaces, performance is better than ops in graph compilation mode O0 and PyNative mode. Currently does not support graph sink mode and CPU, GPU backend. It will be gradually improved.

#### Interface Changes

- Interface name: mindspore.dataset.GeneratorDataset

Changed: The default value of parameter `max_rowsize` is changed from `6` to `None` to enable dynamic allocation of shared memory by default.

<table>
  <tr>
  <td style="text-align:center"> Original interface </td> <td style="text-align:center"> v2.4.0 interface </td>
  </tr>
  <tr>
  <td><pre>
  class GeneratorDataset(source,
                         column_names=None,
                         column_types=None,
                         schema=None,
                         num_samples=None,
                         num_parallel_workers=1,
                         shuffle=None,
                         sampler=None,
                         num_shards=None,
                         shard_id=None,
                         python_multiprocessing=True,
                         max_rowsize=6)
  </pre>
  </td>
  <td><pre>
  class GeneratorDataset(source,
                         column_names=None,
                         column_types=None,
                         schema=None,
                         num_samples=None,
                         num_parallel_workers=1,
                         shuffle=None,
                         sampler=None,
                         num_shards=None,
                         shard_id=None,
                         python_multiprocessing=True,
                         max_rowsize=None)
  </pre>
  </td>
  </tr>
  </table>

- Interface name: mindspore.dataset.Dataset.batch

Changed: The default value of parameter `max_rowsize` is changed from `16` to `None` to enable dynamic allocation of shared memory by default.

<table>
  <tr>
  <td style="text-align:center"> Original interface </td> <td style="text-align:center"> v2.4.0 interface </td>
  </tr>
  <tr>
  <td><pre>
  def batch(input_dataset,
            batch_size,
            drop_remainder=False,
            num_parallel_workers=None,
            per_batch_map=None,
            input_columns=None,
            output_columns=None,
            python_multiprocessing=False,
            max_rowsize=16)
  </pre>
  </td>
  <td><pre>
  def batch(input_dataset,
            batch_size,
            drop_remainder=False,
            num_parallel_workers=None,
            per_batch_map=None,
            input_columns=None,
            output_columns=None,
            python_multiprocessing=False,
            max_rowsize=None)
  </pre>
  </td>
  </tr>
  </table>

- Interface name: mindspore.dataset.Dataset.map

Changed: The default value of parameter `max_rowsize` is changed from `16` to `None` to enable dynamic allocation of shared memory by default.

<table>
  <tr>
  <td style="text-align:center"> Original interface </td> <td style="text-align:center"> v2.4.0 interface </td>
  </tr>
  <tr>
  <td><pre>
  def map(input_dataset,
          operations=None,
          input_columns=None,
          output_columns=None,
          num_parallel_workers=None,
          python_multiprocessing=False,
          cache=None,
          callbacks=None,
          max_rowsize=16, offload=None)
  </pre>
  </td>
  <td><pre>
  def map(input_dataset,
          operations=None,
          input_columns=None,
          output_columns=None,
          num_parallel_workers=None,
          python_multiprocessing=False,
          cache=None,
          callbacks=None,
          max_rowsize=None, offload=None)
  </pre>
  </td>
  </tr>
  </table>

- Interface name: mindspore.ops.TensorDump

Changed: New parameter `input_output` to control printing behavior.

<table>
  <tr>
  <td style="text-align:center"> Original interface </td> <td style="text-align:center"> v2.4.0 interface </td>
  </tr>
  <tr>
  <td><pre>
  class TensorDump()
  </pre>
  </td>
  <td><pre>
  class TensorDump(input_output='out')
  </pre>
  </td>
  </tr>
  </table>

- Interface name: File formats saved by MindSpore Dump Tensor

Changed: The npy file obtained by Dump adds the dtype information of the original Tensor to the filename.

<table>
  <tr>
  <td style="text-align:center"> Original interface </td> <td style="text-align:center"> v2.4.0 interface </td>
  </tr>
  <tr>
  <td><pre>
  {op_type}.{op_name}.{task_id}.{stream_id}.
  {timestamp}.{input_output_index}.{slot}.
  {format}.npy
  </pre>
  </td>
  <td><pre>
  {op_type}.{op_name}.{task_id}.{stream_id}.
  {timestamp}.{input_output_index}.{slot}.
  {format}.{dtype}.npy
  </pre>
  </td>
  </tr>
  </table>

#### Non-compatible Interface Changes

- Interface name: mindspore.nn.Cell.register_backward_hook(hook_fn)

Changed: The input parameter of hook_fn is changed from cell_id to cell object.

Descriptions: For the original hook, you can get the original cell_id by id(cell) in hook_fn.

<table>
  <tr>
  <td style="text-align:center"> Original interface </td> <td style="text-align:center"> v2.4.0 interface </td>
  </tr>
  <tr>
  <td><pre>
  def register_backward_hook(hook_fn)
  Parameter hook_fn(cell_id,
             grad_input, grad_output)
             -> New grad_output or None
  </pre>
  </td>
  <td><pre>
  def register_backward_hook(hook_fn)
  Parameter hook_fn(cell,
             grad_input, grad_output)
             -> New grad_input or None
  </pre>
  </td>
  </tr>
  </table>

- Interface name: mindspore.nn.Cell.register_forward_hook(hook_fn)

Changed: The input parameter of hook_fn is changed from cell_id to cell object.

Descriptions: For the original hook, you can get the original cell_id by id(cell) in hook_fn.

<table>
  <tr>
  <td style="text-align:center"> Original interface </td> <td style="text-align:center"> v2.4.0 interface </td>
  </tr>
  <tr>
  <td><pre>
  def register_forward_hook(hook_fn)
  Parameter hook_fn(cell_id, inputs, outputs)-> New outputs or None
  </pre>
  </td>
  <td><pre>
  def register_forward_hook(hook_fn)
  Parameter hook_fn(cell, inputs, outputs)-> New outputs or None
  </pre>
  </td>
  </tr>
  </table>

- Interface name: mindspore.communication.comm_func.all_reduce

Changed: all_reduce adds a new parameter async_op, and the return value is changed from Tensor to a tuple consisting of Tensor and CommHandle.

Descriptions: async_op indicates whether all_reduce has multi-stream parallelism turned on, and the default value is False.

<table>
  <tr>
  <td style="text-align:center"> Original interface </td> <td style="text-align:center"> v2.4.0 interface </td>
  </tr>
  <tr>
  <td><pre>
  def all_reduce(tensor,
                 op=ReduceOp.SUM,
                 group=GlobalComm.WORLD_COMM_GROUP)->Tensor
  </pre>
  </td>
  <td><pre>
  def all_reduce(tensor,
                 op=ReduceOp.SUM,
                 group=GlobalComm.WORLD_COMM_GROUP,
                 async_op=False)
                 ->tuple(Tensor, CommHandle)
  </pre>
  </td>
  </tr>
  </table>

- Interface name: mindspore.communication.comm_func.all_gather_into_tensor

Changed: all_reduce adds a new parameter async_op, and the return value is changed from Tensor to a tuple consisting of Tensor and CommHandle.

Descriptions: async_op indicates whether all_gather_into_tensor has multi-stream parallelism turned on, and the default value is False.

<table>
  <tr>
  <td style="text-align:center"> Original interface </td> <td style="text-align:center"> v2.4.0 interface </td>
  </tr>
  <tr>
  <td><pre>
  def all_gather_into_tensor(tensor,
                             group=GlobalComm.
                             WORLD_COMM_GROUP)->Tensor
  </pre>
  </td>
  <td><pre>
  def all_gather_into_tensor(tensor,
                             group=GlobalComm.
                             WORLD_COMM_GROUP,
                             async_op=False)->
                             tuple(Tensor, CommHandle)
  </pre>
  </td>
  </tr>
  </table>

- Interface name: mindspore.communication.comm_func.reduce_scatter_tensor

Changed: all_reduce adds a new parameter async_op, and the return value is changed from Tensor to a tuple consisting of Tensor and CommHandle.

Descriptions: async_op indicates whether reduce_scatter_tensor has multi-stream parallelism turned on, and the default value is False.

<table>
  <tr>
  <td style="text-align:center"> Original interface </td> <td style="text-align:center"> v2.4.0 interface </td>
  </tr>
  <tr>
  <td><pre>
  def reduce_scatter_tensor(tensor,
                            op=ReduceOp.SUM,
                            group=GlobalComm.
                            WORLD_COMM_GROUP)->Tensor
  </pre>
  </td>
  <td><pre>
  def reduce_scatter_tensor(tensor,
                            op=ReduceOp.SUM,
                            group=GlobalComm.WORLD_COMM_GROUP,
                            async_op=False)->
                            tuple(Tensor, CommHandle)
  </pre>
  </td>
  </tr>
  </table>

- Interface name: mindspore.communication.comm_func.isend

Changed: The return value is changed from Tensor to Handle.

Descriptions: isend enables multi-stream parallelism by default.

<table>
  <tr>
  <td style="text-align:center"> Original interface </td> <td style="text-align:center"> v2.4.0 interface </td>
  </tr>
  <tr>
  <td><pre>
  def isend(tensor,
            dst=0,group=GlobalComm.
            WORLD_COMM_GROUP, tag=0)->Tensor
  </pre>
  </td>
  <td><pre>
  def isend(tensor,
            dst=0,group=GlobalComm.
            WORLD_COMM_GROUP, tag=0)->CommHandle
  </pre>
  </td>
  </tr>
  </table>

- Interface name: mindspore.communication.comm_func.irecv

Changed: The return value is changed from Tensor to Handle.

Descriptions: irecv enables multi-stream parallelism by default.

<table>
  <tr>
  <td style="text-align:center"> Original interface </td> <td style="text-align:center"> v2.4.0 interface </td>
  </tr>
  <tr>
  <td><pre>
  def irecv(tensor,
            src=0, group=GlobalComm.
            WORLD_COMM_GROUP, tag=0)->Tensor
  </pre>
  </td>
  <td><pre>
  def irecv(tensor,
            src=0,
            group=GlobalComm.
            WORLD_COMM_GROUP, tag=0)->CommHandle
  </pre>
  </td>
  </tr>
  </table>

- Interface name: mindspore.communication.comm_func.all_to_all_with_output_shape

Changed: all_to_all_with_output_shape adds a new parameter async_op, and the return value is changed from Tensor to a tuple consisting of Tensor and CommHandle.

Descriptions: async_op indicates whether all_to_all_with_output_shape enables multi-stream parallelism, the default value is False.

<table>
  <tr>
  <td style="text-align:center"> Original interface </td> <td style="text-align:center"> v2.4.0 interface </td>
  </tr>
  <tr>
  <td><pre>
  def all_to_all_with_output_shape(output_shape_list,
                                   input_tensor_list,
                                   group=None)->tuple(Tensor)
  </pre>
  </td>
  <td><pre>
  def all_to_all_with_output_shape(output_shape_list,
                                   input_tensor_list,
                                   group=None,
                                   async_op=False)->
                                   tuple(tuple(Tensor),
                                   CommHandle)
  </pre>
  </td>
  </tr>
  </table>

- Interface name: mindspore.communication.comm_func.all_to_all_single_with_output_shape

Changed: all_to_all_single_with_output_shape adds a new parameter async_op, and the return value is changed from Tensor to a tuple consisting of Tensor and CommHandle.

Descriptions: async_op indicates whether all_to_all_single_with_output_shape enables multi-stream parallelism, the default value is False.

<table>
  <tr>
  <td style="text-align:center"> Original interface </td> <td style="text-align:center"> v2.4.0 interface </td>
  </tr>
  <tr>
  <td><pre>
  def all_to_all_single_with_output_shape(output_shape,
                                          tensor, output_split_sizes=None,
                                          input_split_sizes=None,
                                          group=None)->Tensor
  </pre>
  </td>
  <td><pre>
  def all_to_all_single_with_output_shape(output_shape,
                                          tensor, output_split_sizes=None,
                                          input_split_sizes=None,
                                          group=None, async_op=False)->
                                          tuple(Tensor, CommHandle)
  </pre>
  </td>
  </tr>
  </table>

### Contributors

anyrenwei,bantao,baochong,Bellatan,BJ-WANG,caifubi,candanzg,candyhong,Carey,cccc1111,ccsszz,changzherui,chengbin,chengfeng27,chengxb7532,chenjianping,chenweifeng,chujinjin,dairenjie,DavidFFFan,DeshiChen,dingjinshan,emmmmtang,fanyi20,fary86,fengyixing,fix-dryrun,fuchao,fuhouyu,gaoyong10,gengdongjie,gent1e,GuoZhibin,guozhijian,halo,hangq,haozhang,hedongdong,Henry Shi,HighCloud,Hongxing,huandong1,huangbingjian,HuangLe02,huangziling,huda,huiliang166,hujiahui8,huoxinyou,jiangchenglin3,jianghui58,jiangshanfeng,jiaorui,jiaxueyu,jijiarong,jjfeing,JoeyLin,jshawjc,jxl,kairui_kou,kisnwang,kk,lanzhineng,LiangZhibo,lichen,limingqi107,lionelchang,liubuyu,liujunzhu,liuluobin,liyejun,LLLRT,looop5,luochao60,luoxuewei,luoyang,machenggui,maning202007,maoyuanpeng1,Margaret_wangrui,MengXiangyu,mengyuanli,moran,Mrtutu,mylinchi,NaCN,nomindcarry,panzhihui,paolopoggi,pengqi,pierreleca,qiuleilei,qiuyufeng,qiuzhongya,r1chardf1d0,shaoshengqi,shen_haochen,shenhaojing,shenwei41,shihlCST,shilishan,shiro-zzz,shiziyang,shop-pin,shunyuanhan,shuqian0,stavewu,superxf,suteng,tanghuikang,tangmengcheng,tan-wei-cheng,tan-wei-cheng-3260,tianxiaodong,TronZhang,TuDouNi,VectorSL,vincen45,wang_ziqi,wanghenchang,wangjie,wangshaocong,weiyang,wtobill,wudawei,wujueying,wwwbby,xfan233,XianglongZeng,xiaotianci,xiaoxin_zhang,xiaoxiongzhu,xiaoxuanKL,xiaoyao,XinDu,xuxinglei,xuzhubin,yanghaoran,yanglong,yangzhenzhang,yanx,Yanzhi_YI,yao_yf,yefeng,yide12,yihangchen,YijieChen,YingLai Lin,ylw,yuanpeng2024,yuanqi,yuchaojie,Yuheng Wang,YuJianfeng,YukioZzz,yyuse,zangqx,ZeyuHan,zhangbuxue,zhanghaibo,zhangminli,zhangqinghua,zhangyanhui,ZhangZGC,zhangzhen,zhanzhan,zhengzuohe,zhouyaqiang0,zhuguodong,zichun_ye,zjun,zong_shuai,ZPaC,zuochuanyong,zyli2020,程超,蛋蛋de忧桑,狄新凯,范吉斌,冯一航,付国华,胡彬,宦晓玲,黄勇,黄卓,康伟,李良灿,李林杰,李寅杰3,刘崇鸣,刘思铭,刘涛Liu,刘勇琪,刘子涵,吕浩宇,吕昱峰（Nate.River）,钱丹,十一雷,孙昊辰,王禹程,王振邦,王梓润,吴大维,熊攀,徐安越,许子豪,俞涵,云骑士,张峻源,张王泽,张栩浩,赵文璇,周莉莉,朱家兴,邹文祥

Last committed message: !76565 310p上 kv shape 是3维 & rms norm 走 asd

v2.3.1

9cdc932

2024-08-12 10:50

MindSpore 2.3.1 Release Notes

# MindSpore Release Notes

[查看中文](./RELEASE_CN.md)

## MindSpore 2.3.1 Release Notes

### Major Features and Improvements

- [STABLE] Remove the restriction that the value of device_matrix must be 2 correspongding to interleaved_parallel when using [Layout](https://www.mindspore.cn/docs/en/r2.3.1/api_python/mindspore/mindspore.Layout.html) to construct the parallel strategy.
- [STABLE] Add user-defined control edges environment [MS_CUSTOM_DEPEND_CONFIG_PATH](https://www.mindspore.cn/docs/en/r2.3.1/note/env_var_list.html) support to achieve better overlapping of communication and computation.

### API Change

#### New API

- [STABLE] Add new API [mindspore.mint.repeat_interleave](https://www.mindspore.cn/docs/en/r2.3.1/api_python/mint/mindspore.mint.repeat_interleave.html).

### Contributors

ccsszz;dairenjie;DeshiChen;fuhouyu;gaoshuanglong;gaoyong10;GuoZhibin;halo;huoxinyou;jiangchao_j;jiaorui;jiaxueyu;jijiarong;JuiceZ;lichen;liujunzhu;liuluobin;LLLRT;looop5;luoyang ;Margaret_wangrui;mengyuanli;panzhihui;pengqi;PingqiLi;Renyuan Zhang;tanghuikang;tianxiaodong;TuDouNi;wudawei;XianglongZeng;xiaosh;xiaoxin_zhang;XinDu;yanghaoran;yanglong;yangruoqi713;Yanzhi_YI;yao_yf;YijieChen;yuchaojie;YuJianfeng;zangqx;zhengzuohe;zhouyaqiang0;ZPaC;zyli2020;胡彬;宦晓玲;康伟;李林杰;刘崇鸣;王禹程;俞涵;周莉莉;邹文祥

Contributions of any kind are welcome!

## MindSpore Lite 2.3.1 Release Notes

### Major Features and Improvements

When converting Ascend backend models, the [input_shape](https://www.mindspore.cn/lite/docs/en/r2.3.1/use/cloud_infer/converter_tool_ascend.html) parameter in the configuration file is supported to specify the input size.

### API Change

- [ModelGroup](https://www.mindspore.cn/lite/docs/en/r2.3.1/use/cloud_infer/runtime_cpp.html) interface adds model weight sharing support to save video memory.

- [Model.get_model_info](https://www.mindspore.cn/lite/docs/en/r2.3.1/use/converter_tool.html?highlight=get_model_info) interface adds support for obtaining the input size of the model.

### Contributors

熊攀;ZhangZGC;jxl;zhangyanhui;emmmmtang;huandong1;yefeng

Last committed message: !73146 fix dvm msprof bug

v2.3.0

a4230c7

2024-07-16 21:51

MindSpore 2.3.0 Release Notes

## MindSpore 2.3.0 Release Notes

### Major Features and Improvements

#### AutoParallel

- [STABLE] Extend functional parallelism. [mindspore.shard](https://www.mindspore.cn/docs/en/r2.3.0/api_python/mindspore/mindspore.shard.html) supports now the Graph mode. In Graph mode, the parallel sharding strategy of input and weight can be set for nn.Cell/function. For other operators, the parallel strategy can be automatically configured through "sharding_propagation". Add [mindspore.reshard](https://www.mindspore.cn/docs/en/r2.3.0/api_python/mindspore/mindspore.reshard.html) interface that supports manual rearranging and set up a precise sharding strategy ([mindspore.Layout](https://www.mindspore.cn/docs/en/r2.3.0/api_python/mindspore/mindspore.Layout.html)) for tensors.
- [STABLE] Added Callback interface [mindspore.train.FlopsUtilizationCollector](https://www.mindspore.cn/docs/en/r2.3.0/api_python/train/mindspore.train.FlopsUtilizationCollector.html) statistical model flops utilization information MFU and hardware flops utilization information HFU.
- [STABLE] Add functional communication API [mindspore.communication.comm_func](https://www.mindspore.cn/docs/en/r2.3.0/api_python/mindspore.communication.comm_func.html).
- [BETA] Optimize the memory usage of interleaved pipeline in O0 and O1 mode.
- [BETA] AutoParallel supports automatic pipeline strategy generation in multi-nodes scenarios (not supported in single-node scenario). Need to set `parallel_mode` to ``auto_parallel`` and `search_mode` to ``recursive_programming``.

#### PyNative

- [STABLE] Optimize the basic data structure of PyNative and improve operator API performance.
- [STABLE] Tensor supports [register_hook](https://www.mindspore.cn/docs/en/r2.3.0/api_python/mindspore/Tensor/mindspore.Tensor.register_hook.html) so that users can print or modify the gradient with respect to the tensor.
- [STABLE] The PyNative mode supports the recompute function. You can use the recompute interface to reduce the peak device memory of the network.

#### FrontEnd

- [STABLE] Optimize Checkpoint saving and loading basic processes to improve performance by 20%.
- [STABLE] Support CRC verification of Checkpoint files during saving and loading processes to enhance security.

#### Dataset

- [STABLE] Support Ascend processing backend for the following transforms: Equalize, Rotate, AutoContrast, Posterize, AdjustSharpness, Invert, Solarize, ConvertColor, Erase.
- [STABLE] Support video files reading and parsing function. For more detailed information, see APIs: [mindspore.dataset.vision.DecodeVideo](https://www.mindspore.cn/docs/en/r2.3.0/api_python/dataset_vision/mindspore.dataset.vision.DecodeVideo.html), [mindspore.dataset.vision.read_video](https://www.mindspore.cn/docs/en/r2.3.0/api_python/dataset_vision/mindspore.dataset.vision.read_video.html#mindspore.dataset.vision.read_video), and [mindspore.dataset.vision.read_video_timestamps](https://www.mindspore.cn/docs/en/r2.3.0/api_python/dataset_vision/mindspore.dataset.vision.read_video_timestamps.html#mindspore.dataset.vision.read_video_timestamps).
- [STABLE] Support specifying the `max_rowsize` parameter as -1 in `mindspore.dataset.GeneratorDataset`, `mindspore.dataset.Dataset.map` and `mindspore.dataset.Dataset.batch` interfaces. The size of shared memory used by the dataset multiprocessing will be dynamically allocated according to the size of the data. The `max_rowsize` parameter does not need to be adjusted manually.

#### Inference

- [STABLE] 14 large models such as LLaMa2, LLaMa3, and Qwen1.5 are added to support the integrated training and inference architecture to unify scripts, distributed strategies, and runtime. The period from training to inference deployment of typical large models is reduced to days. Large operators are integrated to reduce the inference latency and effectively improve the network throughput.

#### PIJIT

- [BETA] Support bytecode parsing for Python 3.8 and Python 3.10 to expand the supporting version of Python.
- [BETA] Support dynamic shape and symbolic shape as input to enable the dynamic input scenarios.
- [BETA] Enable single-step composition capability to optimize compile time
- [BETA] Support bytecode capture with side effects (STORE_ATTR, STORE_GLOBAL, LIST_APPEND, dict.pop) by bytecode tuning, enabling auto-mixed precision, reduction of cleavage diagrams, and improved performance.

#### Profiler

- [STABLE] Provides a hierarchical Profiler function, controls different levels of performance data collection through the profiler_level parameter.
- [STABLE] Profiler analyse adds a new mode parameter to configure asynchronous parsing mode to parallelize performance data parsing and training.
- [STABLE] The Profiler adds a new data_simplification parameter, which allows users to control whether to delete redundant data after parsing the performance data to save hard disk space.
- [STABLE] The Profiler enhances the memory analysis function. Users can collect the memory application and release information of the framework, CANN and hardware through the profile_memory parameter, and visualize and analyze the information through the [MindStudio tool](https://www.hiascend.com/forum/thread-0230130822583032044-1-1.html).
- [BETA] In Pynative mode, Timeline integrates host profiling information, including task time and user side stack information.

#### Dump

- [STABLE] Enhanced synchronous & asynchronous dump functionality and adds L2Norm information to statistics dumps, and the statistic_category field to allow users to customize which statistics to save, improving dump usability. For details about the support for synchronous/asynchronous dump, see [Dump Introduction](https://www.mindspore.cn/tutorials/experts/en/r2.3.0/debug/dump.html#dump-introduction).
- [STABLE] Improved synchronous dump functionality: Enables overflow and exception dumps through the op_debug_mode field.
- [STABLE] Enhanced synchronous dump functionality: The stat_calc_mode field enables device-side computation of statistics (default is host-side), and the sample_mode field is configured to perform sample-based dumps, improving dump performance.
- [STABLE] Enhanced asynchronous dump functionality: Now supports saving in complex64 and complex128 formats.

#### Runtime

- [Stable] Supports multi-level compilation of the staic graph by setting [mindspore.set_context(jit_config={"jit_level": "O0/O1/O2"})](https://www.mindspore.cn/docs/en/r2.3.0/api_python/mindspore/mindspore.set_context.html). The default value is empty, the framework automatically selects the optimization level according to the product category, O2 for Altas training products and O0 for the rest of the products.
- [Stable] Staic graph supports multi-stream concurrent execution of communication calculations in O0/O1.
- [STABLE] Add memory management API [mindspore.hal.memory](https://www.mindspore.cn/docs/en/r2.3.0/api_python/mindspore.hal.html#memory).
- [Beta] The memory pool supports virtual memory defragmentation, and virtual memory is enabled by default under graph O0/O1.

#### Ascend

- [STABLE] Provide an operator memory out of bounds access detection switch on the Ascend platform, where users can detect internal memory out of bounds issues of operators on the Ascend platform by setting `mindspore.set_context (Ascend_configuration={"op_debug_option": "oom"})`.
- [BETA] The environment variable [MS_SIMULATION_LEVEL](https://www.mindspore.cn/docs/en/r2.3.0/note/env_var_list.html) supports graph compilation O0 execution mode on the Ascend platform, which can support compilation performance and runtime memory analysis
- [BETA] Ascend platform supports [AscendC custom operators](https://www.mindspore.cn/tutorials/experts/en/r2.3.0/operation/op_custom_ascendc.html) through AOT.

### API Change

#### New APIs

- [STABLE] Adds [mindspore.mint](https://www.mindspore.cn/docs/en/r2.3.0/api_python/mindspore.mint.html) API, provides a lot of functional, nn, optimizer interfaces. The API usage and functions are consistent with the mainstream usage in the industry, which is convenient for users to refer to and use. The mint interface is currently an experimental interface and performs better than ops in `jit_level="O0"` and pynative mode. Currently, the graph sinking mode and CPU/GPU backend are not supported, and it will be gradually improved in the future.

| mindspore.mint  |  |   | |
  |:----|:----|:----|:----|
  | mindspore.mint.eye |mindspore.mint.rand_like|mindspore.mint.isfinite|mindspore.mint.any|
  | mindspore.mint.ones |mindspore.mint.rand|mindspore.mint.log|mindspore.mint.greater_equal|
  | mindspore.mint.ones_like |mindspore.mint.gather|mindspore.mint.logical_and|mindspore.mint.all|
  | mindspore.mint.zeros |mindspore.mint.permute|mindspore.mint.logical_not|mindspore.mint.mean|
  | mindspore.mint.zeros_like |mindspore.mint.repeat_interleave|mindspore.mint.logical_or|mindspore.mint.prod|
  | mindspore.mint.arange |mindspore.mint.abs|mindspore.mint.mul|mindspore.mint.sum|
  | mindspore.mint.broadcast_to |mindspore.mint.add|mindspore.mint.neg|mindspore.mint.eq|
  | mindspore.mint.cat |mindspore.mint.clamp|mindspore.mint.negative|mindspore.mint.ne|
  | mindspore.mint.index_select |mindspore.mint.cumsum|mindspore.mint.pow|mindspore.mint.greater|
  | mindspore.mint.max |mindspore.mint.atan2|mindspore.mint.reciprocal|mindspore.mint.gt|
  | mindspore.mint.min |mindspore.mint.arctan2|mindspore.mint.rsqrt|mindspore.mint.isclose|
  | mindspore.mint.scatter_add |mindspore.mint.ceil|mindspore.mint.sigmoid|mindspore.mint.le|
  | mindspore.mint.narrow |mindspore.mint.unique|mindspore.mint.sin|mindspore.mint.less_equal|
  | mindspore.mint.nonzero |mindspore.mint.div|mindspore.mint.sqrt|mindspore.mint.lt|
  | mindspore.mint.normal |mindspore.mint.divide|mindspore.mint.square|mindspore.mint.maximum|
  | mindspore.mint.tile |mindspore.mint.erf|mindspore.mint.sub|mindspore.mint.minimum|
  | mindspore.mint.topk |mindspore.mint.erfinv|mindspore.mint.tanh|mindspore.mint.inverse|
  | mindspore.mint.sort |mindspore.mint.exp|mindspore.mint.bmm|mindspore.mint.searchsorted|
  | mindspore.mint.stack |mindspore.mint.floor|mindspore.mint.matmul|mindspore.mint.argmax|
  | mindspore.mint.where |mindspore.mint.flip|mindspore.mint.split|mindspore.mint.cos|
  | mindspore.mint.less |||

| mindspore.mint.nn|
  |:----|
  | mindspore.mint.nn.Dropout  |
  | mindspore.mint.nn.Unfold |
  | mindspore.mint.nn.Fold |
  | mindspore.mint.nn.Linear|
  | mindspore.mint.nn.BCEWithLogitsLoss |

| mindspore.mint.nn.functional||
  |:----|:----|
  |mindspore.mint.nn.functional.batch_norm |mindspore.mint.nn.functional.group_norm|
  |mindspore.mint.nn.functional.fold |mindspore.mint.nn.functional.layer_norm|
  |mindspore.mint.nn.functional.max_pool2d |mindspore.mint.nn.functional.linear|
  |mindspore.mint.nn.functional.binary_cross_entropy |mindspore.mint.nn.functional.unfold|
  |mindspore.mint.nn.functional.sigmoid |mindspore.mint.nn.functional.one_hot|
  |mindspore.mint.nn.functional.tanh |mindspore.mint.nn.functional.elu|
  |mindspore.mint.nn.functional.binary_cross_entropy_with_logits |mindspore.mint.nn.functional.gelu|
  |mindspore.mint.nn.functional.dropout|mindspore.mint.nn.functional.leaky_relu|
  |mindspore.mint.nn.functional.embedding  |mindspore.mint.nn.functional.silu|
  |mindspore.mint.nn.functional.grid_sample|mindspore.mint.nn.functional.softplus|
  |mindspore.mint.nn.functional.relu|mindspore.mint.nn.functional.softmax|
  |mindspore.mint.nn.functional.pad||

| mindspore.mint.optim |
  |:----|
  | mindspore.mint.optim.AdamW |

| mindspore.mint.linalg |
  |:----|
  | mindspore.mint.linalg.inv |

### Non-compatible Interface Changes

- Interface name: `Profiler`

Changes: The performance data file generated by parsing is streamlined to save space. Delete the FRAMEWORK directory data and other redundant data after exporting the performance data. Retain only the deliverables of the profiler and the original performance data in the PROF_XXX directory to save space. Data simplification mode can be turned off by configuring the `data_simplification` parameter to `False`, which will be consistent with the performance data files generated by the historical version.
- Interface name: The `saved_data` field in the configuration file of the dump function is `"tensor"`.

Changes: The name of the file to be dumped to disks is changed. `"/"` is replaced with `"_"`, and the operator name is changed to the global name of the operator.

<table>
  <tr>
  <td style="text-align:center"> Original interface </td> <td style="text-align:center"> v2.1 interface </td>
  </tr>
  <tr>
  <td><pre>
  File name format:
  {op_type}.{op_name}.{task_id}.{stream_id}.
  {timestamp}.{input_output_index}.{slot}.{format}.npy
  </br>
  Example:
  Conv2D.Conv2D-op12.0.0.1623124369613540.
  output.0.DefaultFormat.npy
  </pre>
  </td>
  <td><pre>
  File name format:
  {op_type}.{op_name}.{task_id}.{stream_id}.
  {timestamp}.{input_output_index}.{slot}.{format}.npy
  </br>
  Example:
  Conv2D.Default_network-WithLossCell__backbone-AlexNet_conv3
  -Conv2d_Conv2D-op12.0.0.1623124369613540.output.0.DefaultFormat.npy
  </pre>
  </td>
  </tr>
  </table>
- Interface name: The `saved_data` field in the Dump function configuration file is `"statistic"`.

Changes: By default, `'max'`, `'min'`, `'avg'`, `'count'`, `'negative zero count'`, `'positive zero count'`, `'nan count'`,  `'negative inf count'` ,`'positive inf count'`,`'zero count'` and `'md5'`. In the 2.3 version, the `'max'`, `'min'`, and `'l2norm'` statistical items are saved by default. You can customize statistical items by configuring `'statistic_category'`.

### Contributors

caifubi;candanzg;ccsszz;chaiyouheng;changzherui;chenfei_mindspore;chengbin;chengfeng27;Chong;dairenjie;DavidFFFan;DeshiChen;dingjinshan;douzhixing;emmmmtang;Erpim;fary86;fengyixing;fuhouyu;gaoyong10;GuoZhibin;guozhijian;halo;haozhang;hejianheng;Henry Shi;horcham;huandong1;huangbingjian;Jackson_Wong;jiangchenglin3;jiangshanfeng;jiangzhenguang;jiaorui;bantao;jiaxueyu;jijiarong;JuiceZ;jxl;kairui_kou;lanzhineng;LiangZhibo;lichen;limingqi107;linqingke;liubuyu;liujunzhu;liuluobin;liyan2022;liyejun;LLLRT;looop5;lujiale;luochao60;luoyang;lvxudong;machenggui;maning202007;Margaret_wangrui;master_2;mengyuanli;moran;Mrtutu;NaCN;nomindcarry;panzhihui;pengqi;qiuyufeng;qiuzhongya;Renyuan Zhang;shaoshengqi;Shawny;shen_haochen;shenhaojing;shenwei41;shij1anhan;shilishan;shiziyang;shunyuanhan;shuqian0;TAJh;tanghuikang;tan-wei-cheng;Thibaut;tianxiaodong;TronZhang;TuDouNi;VectorSL;wang_ziqi;wanghenchang;wangjie;weiyang;wudawei;wujiangming;wujueying;XianglongZeng;xiaotianci;xiaoxin_zhang;xiaoxiongzhu;xiaoyao;XinDu;xuxinglei;yangchen;yanghaoran;yanglong;yangruoqi713;yangzhenzhang;yangzishuo;Yanzhi_YI;yao_yf;yefeng;yide12;YijieChen;YingLai Lin;yuchaojie;YuJianfeng;zangqx;zhaiyukun;zhangminli;zhangqinghua;ZhangZGC;zhengxinQian;zhengzuohe;zhouyaqiang0;zhuguodong;zhupuxu;zichun_ye;zjun;zlq2020;ZPaC;zuochuanyong;zyli2020;阿琛;狄新凯;范吉斌;冯一航;胡彬;宦晓玲;黄勇;康伟;雷仪婧;李良灿;李林杰;刘崇鸣;刘力力;刘勇琪;刘子涵;吕浩宇;王禹程;熊攀;徐安越;徐永飞;俞涵;张王泽;张栩浩;郑裔;周莉莉;周先琪;朱家兴;邹文祥

Contributions of any kind are welcome!

Last committed message: !72171 modify the release note files

v2.3.0-rc2

9c3cf20

2024-05-13 10:36

MindSpore 2.3.0-rc2 Release Notes

## MindSpore 2.3.0-rc2 Release Notes

### Major Features and Improvements

#### AutoParallel

- [STABLE] Transpose/Sub/Add/Mul/Div/ReLU/Softmax/Sigmoid supports layout configuration.
- [STABLE] The collective communication precision will affect network convergence. The configuration item [force_fp32_communication](https://www.mindspore.cn/docs/en/r2.3.0rc2/api_python/mindspore/mindspore.set_auto_parallel_context.html) is provided in the interface mindspore.set_auto_parallel_context. When set to True, the communication type of the reduce communication operator can be forced to be converted to float32.
- [BETA] Pipeline parallel support Interleave. Optimize the performance when micro batch is limited.
- [BETA] Optimize checkpoint transformation speed when using pipeline parallel, support single stage transform.

#### PyNative

- [BETA] Support [recompute](https://www.mindspore.cn/docs/en/r2.3.0rc2/api_python/mindspore/mindspore.recompute.html) on PyNative mode.
- [STABLE] Support [register_hook](https://www.mindspore.cn/docs/en/r2.3.0rc2/api_python/mindspore/Tensor/mindspore.Tensor.register_hook.html#mindspore.Tensor.register_hook) on PyNative mode.

### API Change

Add timeout environment variables in [dynamic networking](https://www.mindspore.cn/tutorials/experts/en/r2.3.0rc2/parallel/dynamic_cluster.html) scenarios:

- `MS_TOPO_TIMEOUT`: Cluster networking phase timeout time in seconds.
- `MS_NODE_TIMEOUT`: Node heartbeat timeout in seconds.
- `MS_RECEIVE_MSG_TIMEOUT`: Node timeout for receiving messages in seconds.

Added new environment variable `MS_ENABLE_LCCL` to support the use of LCCL communication library.

### Bug Fixes

- [#I9CR96] Fix the issue of insufficient timeout time causing failure for dynamic networking startup in large-scale clusters.
- [#I94AQQ] Fixed the problem of incorrect output shape of ops.Addcdiv operator in graph mode.

### Contributors

Thanks goes to these wonderful people:

bantao,caifubi,changzherui,chenfei_mindspore,chenweifeng,dairenjie,dingjinshan,fangzehua,fanyi20,fary86,GuoZhibin,hanhuifeng,haozhang,hedongdong,Henry Shi,huandong1,huangbingjian,huoxinyou,jiangchenglin3,jiangshanfeng,jiaorui,jiaxueyu,jxl,kairui_kou,lichen,limingqi107,liuluobin,LLLRT,looop5,luochao60,luojianing,maning202007,NaCN,niyuxin94520,nomindcarry,shiziyang,tanghuikang,TronZhang,TuDouNi,VectorSL,wang_ziqi,wanghenchang,wudawei,XianglongZeng,xiaoxiongzhu,xiaoyao,yanghaoran,Yanzhi_YI,yao_yf,yide12,YijieChen,YingLai Lin,yuchaojie,YuJianfeng,zangqx,zhanghanLeo,ZhangZGC,zhengzuohe,zhouyaqiang0,zichun_ye,zjun,ZPaC,zyli2020,冯一航,李林杰,刘力力,王禹程,俞涵,张栩浩,朱家兴,邹文祥

Contributions of any kind are welcome!

## MindSpore Lite 2.3.0-rc2 Release Notes

### Major Features and Improvements

- [STABLE] Support the configuration of FlashAttention related properties in the configuration file used by the cloud-side conversion tool.
- [STABLE] Support multi-devices memory sharing.

### Contributors

Thanks goes to these wonderful people:

emmmmtang,熊攀

Contributions of any kind are welcome!

Last committed message: !69311 fix release notes for 2.3.0rc2

v2.2.14

4f30cd5

2024-04-23 12:05

MindSpore 2.2.14 Release Notes

## MindSpore 2.2.14 Release Notes

### Major Features and Improvements

#### Parallel

- [STABLE] Changed the communication group of the send/receive operator to the world group in the parallel pipeline to avoid creating redundant communication groups and reduce the memory required for communication.
- [STABLE] Optimize the compilation cache to reduce the graph conversion process of the loading cache and improve the compilation cache performance.
- [BETA] Pipeline parallel supports Interleave. Optimize the performance when mciro batch is small.
- [BETA] Optimize checkpoint transformation speed when using pipeline parallel, support single stage transform.

#### Profiler

- [BETA] Dynamically start and stop profiling. Users can collect profiling data in real time according to the training situation, reducing the amount of data collected.
- [BETA] Profiling the communication operator time-consuming matrix. Users can find cluster communication performance bottlenecks by analyzing the communication operator time-consuming matrix.

#### Dump

- [BETA] The statistical information saved by Dump records MD5 values, and users can determine small differences in tensor values through MD5 values.
- [BETA] Dump supports the float16 data type and supports users to locate float16 type operator accuracy issues.

### Bug Fixes

- [I962EV] Fixed issue on CPU and GPU environment with cond input dimension of 4d, 5d, 6d, 7d and 8d.
- [I96E5R] Fixed the issue in the PyNative that the input of the Mul operator is NCHW format on the Ascend platform.
- [I96I5D] Fixed the issue of incorrect input type when calculating Scalar type in dynamic shape scenario.
- [I99QAB] Fixed the issue where asnumpy cannot correctly identify the bfloat16 tensor in some scenarios.
- [I9ADZS] Fixed the data timeout issue in network training due to inefficient dataset recovery in the fault recovery scenario.
- [I8Y9JT] Fixed the issue that some network training does not converge due to the incorrect execution sequence of the optimizer in some specific scenarios where the nn.SGD optimizer has a large loss_scale and a small weight_decay.

### Contributors

Thanks goes to these wonderful people:

fary86, wanghenchang, haozhang, mengyuanli, emmmmtang, luoyang, zhupuxu, zhangyongxian, liuluobin, LLLRT, TuDouNi, hujiahui8, wangtongyu6, ligan, zhuguodong, yanghaoran, YingtongHu, liyejun, zjun, 徐永飞, chuht, 张树仁, 徐安越, DeshiChen, shenyaxin, liujunzhu, shunyuanhan, yuchaojie, yao_yf, 没有窗户的小巷, yeyunpeng2020, weiyang, KevinYi, hedongdong, zhouyaqiang0, Margaret_wangrui, zhanghaibo, moran, huangziling, 朱家兴, GuoZhibin, 李良灿, jiaxueyu, gaoyong10, Greatpan, 宦晓玲, melody, 俞涵, jiangshanfeng, XinDu, ling, caifubi, zhangyinxia, gengdongjie, Erpim, XianglongZeng, zhangminli, fengyixing, 冯一航, 黄勇, panzhihui, 胡彬, linqingke, wangshaocong

Contributions of any kind are welcome!

## MindSpore Lite 2.2.14 Release Notes

### Bug Fixes

- [I96PJC] An error is reported when the CLIP model in MS format is loaded through the MindSpore Lite Python API.

### Contributors

Thanks goes to these wonderful people:

wangtongyu6, zhuguodong, 徐永飞, 徐安越, yeyunpeng2020, moran, XinDu, gengdongjie.

Contributions of any kind are welcome!

Last committed message: !67827 r2.2.14 - releasenote update

v2.2.13

b672006

2024-04-23 12:04

MindSpore 2.2.13 Release Notes

Last committed message: !67585 Json bugfix

v2.2.12

d74bc4e

2024-04-23 12:02

MindSpore 2.2.12 Release Notes

Last committed message: !66627 ckpt_type_convert_add_bf16

v2.3.0-rc1

2c24bcc

2024-04-22 17:20

MindSpore 2.3.0-rc1 Release Notes

## MindSpore 2.3.0-rc1 Release Notes

### Major Features and Improvements

#### DataSet

- [STABLE] Support integrity check, encryption and decryption check for MindRecord to protect the integrity and security of user data.
- [STABLE] MindRecord api changes: FileWriter.open_and_set_header is deprecated since it has been integrated into FilterWriter, if the old version code reports an error, delete this call; Add type checking for data in FileWriter to ensure that the data type defined by the Schema matches the real data type; The return value of all methods under Mindrecord are removed, replaced by an exception when processing error is occurred.
- [STABLE] Support Ascend processing backend for the following transforms: ResizedCrop, HorizontalFlip, VerticalFlip, Perspective, Crop, Pad, GaussianBlur, Affine.
- [STABLE] Optimized the content of data processing part in model migration guide, providing more examples to compare with third-party frameworks.
- [STABLE] Optimized the parsing efficiency of TFRecordDataset in multiple data columns scenario, improving the parsing performance by 20%.

#### PIJIT

- [BETA]PIJit analyzes and adjusts the Python bytecode and performs graph capture and graph optimization on the execution flow. Supported Python codes are executed in static graph mode, and unsupported ones are divided into subgraphs and executed in dynamic graph mode, automatically achieving dynamic and static unification. Users can enable the PIJit function by decorating the function with @jit(mode="PIJit", jit_config={options:value}).

#### Inference

- [DEMO] The integrated architecture of large model inference, upgrade, training, and promotion unifies scripts, distributed policies, and runtime. The period from training to inference deployment of typical large models is reduced to days. Large operators are integrated to reduce the inference latency and effectively improve the network throughput.

#### AutoParallel

- [STABLE] Add msrun startup method to launch distributed job with single instruction.
- [STABLE] Add to be deprecated hint for RankTable startup method.
- [STABLE] Eliminate redundant constants in graph mode to improve compilation performance and memory overhead.
- [STABLE] The subgraph scenario optimizer parallelizes the first subgraph inline, allowing some computation and communication masking under pipeline parallelism to be performed.
- [STABLE] Communication information export: export model communication information (communication domain, communication volume) during compilation, and input it to the cluster as the basis for communication scheduling.
- [STABLE] Pipeline parallel inference is optimized, eliminates shared weights forwarding between stages, improving execution performance. Supports automatic broadcast of pipeline inference results, improving the usability of autoregressive inference.
- [STABLE] Operator-level parallel sharding supports the configuration of the mapping between the device layout and tensor layout during MatMul/Add/LayerNorm/GeLU/BiasAdd operator sharding.
- [STABLE] Supports gradient communication and backward calculation overlapping in the data parallel dimension.
- [STABLE] Single device simulation compilation, used to simulate the compilation process of a certain device in multi device distributed training, assisting in analyzing the compilation processes and memory usage on the front and back ends.
- [STABLE] Implement ops.Tril sharding to reduce the memory and performance requirements on a single device.
- [BETA] Supports the fusion between communication operators and computing operators, in order to overlap communication overheads with computation and improve network performance.
- [BETA] Load checkpoints and compile graphs in parallel to accelerate fault recovery.

#### Runtime

- [BETA] Support O0/O1/O2 multi-level compilation to improve static graph debugging and tuning capabilities.

#### FrontEnd

- [STABLE] The framework supports the bfloat16 data type. dtype=mindspore.bfloat16 can be specified when a tensor is created.
- [STABLE] The syntax support capability of the rewrite component is optimized, syntaxs such as class variables, functions, and control flows can be parsed.
- [STABLE] New context setting: debug_level. User can use mindspore.set_context(debug_level=mindspore.DEBUG) to get more debug information.

#### Profiler

- [BETA] Dynamically start and stop profiling. Users can collect profiling data in real time according to the training situation, reducing the amount of data collected.
- [BETA] Profiling the communication operator time-consuming matrix. Users can find cluster communication performance bottlenecks by analyzing the communication operator time-consuming matrix.
- [BETA] Improve the performance of Ascend environment in parsing profiling data.
- [BETA] Supports offline analysis of data generated by Profiling. Users can collect data first and then parse the data as needed.
- [BETA] Supports collecting performance data of HBM, PCIe, and l2_cache to enrich performance analysis indicators.

#### Dump

- [BETA] The statistical information saved by Dump records MD5 values, and users can determine small differences in tensor values through MD5 values.
- [BETA] Dump supports the float16 data type and supports users to locate float16 type operator accuracy issues.

#### PyNative

- [STABLE] Reconstruct the single operator calling process for dynamic graphs to improve the performance of dynamic graphs.

#### Ascend

- [BETA] Support set configuration options of CANN, which are divided into two categories: global and session. Users can configure them through mindspore.set_context(Ascend_configuration={"ge_options": {"global": {"global_option": "option_value"}, "session": {"session option": "option_value"}}).

#### API Change

- Add mindspore.hal API to support stream, event, and device management capabilities.
- Add mindspore.multiprocessing API to provide the capability of creating multiple processes.

#### Operators

- [BETA] mindspore.ops.TopK now supports the second input k as an int32 type tensor.

### Bug Fixes

- [I92H93] Fixed the issue of 'Launch kernel failed' when using the Print operator to print string objects on the Ascend platform.
- [I8S6LY] Fixed RuntimeError: Attribute dyn_input_sizes of Default/AddN-op1 is [const vector]{}, of which size is less than 0 error of variable-length input operator, such as AddN or Concat, for dynamic shape process in graph mode on the Ascend platform.
- [I9ADZS] Fixed the data timeout issue in network training due to inefficient dataset recovery in the fault recovery scenario.

### Contributors

Thanks goes to these wonderful people:

AlanCheng511，AlanCheng712，bantao，Bingliang，BJ-WANG，Bokai Li，Brian-K，caifubi，cao1zhg，CaoWenbin，ccsszz，chaiyouheng，changzherui，chenfei_mindspore，chengbin，chengfeng27，chengxb7532，chenjianping，chenkang，chenweifeng，Chong，chuht，chujinjin，Cynthia叶，dairenjie，DavidFFFan，DeshiChen，douzhixing，emmmmtang，Erpim，fangzhou0329，fary86，fengxun，fengyixing，fuhouyu，gaoshuanglong，gaoyong10，GaoZhenlong，gengdongjie，gent1e，Greatpan，GTT，guoqi，guoxiaokang1，GuoZhibin，guozhijian，hangq，hanhuifeng，haozhang，hedongdong，hejianheng，Henry Shi，heyingjiao，HighCloud，Hongxing，huandong1，huangbingjian，HuangLe02，huangxinjing，huangziling，hujiahui8，huoxinyou，jiangchenglin3，jianghui58，jiangshanfeng，jiaorui，jiaxueyu，JichenZhao，jijiarong，jjfeing，JoeyLin，JuiceZ，jxl，kairui_kou，kate，KevinYi，kisnwang，lanzhineng，liangchenghui，LiangZhibo，lianliguang，lichen，ligan，lihao，limingqi107，ling，linqingke，liruyu，liubuyu，liuchao，liuchengji，liujunzhu，liuluobin，liutongtong9，liuzhuoran2333，liyan2022，liyejun，LLLRT，looop5，luochao60，luojianing，luoyang，LV，machenggui，maning202007，Margaret_wangrui，MaZhiming，mengyuanli，MooYeh，moran，Mrtutu，NaCN，nomindcarry，panshaowu，panzhihui，PingqiLi，qinzheng，qiuzhongya，Rice，shaojunsong，Shawny，shenwei41，shenyaxin，shunyuanhan，silver，Songyuanwei，tangdezhi_123，tanghuikang，tan-wei-cheng，TingWang，TronZhang，TuDouNi，VectorSL，WANG Cong，wang_ziqi，wanghenchang，wangpingan，wangshaocong，wangtongyu6，weiyang，WinXPQAQ，wtcheng，wudawei，wujiangming，wujueying，wuweikang，wwwbby，XianglongZeng，xiaosh，xiaotianci，xiaoxin_zhang，xiaoxiongzhu，xiaoyao，XinDu，xingzhongfan，yanghaoran，yangluhang，yangruoqi713，yangzhenzhang，yangzishuo，yanjiaming，Yanzhi_YI，yao_yf，yefeng，yeyunpeng2020，yide12，YijieChen，YingLai Lin，YingtongHu，youshu，yuchaojie，YuJianfeng，zangqx，zby，zhaiyukun，zhangdanyang，zhanghaibo，zhanghanLeo，zhangminli，zhangqinghua，zhangyanhui，zhangyifan，zhangyinxia，zhangyongxian，ZhangZGC，zhanzhan，zhaoting，zhengyafei，zhengzuohe，ZhihaoLi，zhouyaqiang0，zhuguodong，zhumingming，zhupuxu，zichun_ye，zjun，zlq2020，ZPaC，zuochuanyong，zyli2020，陈宇，代宇鑫，狄新凯，范吉斌，冯一航，胡彬，宦晓玲，黄勇，康伟，李良灿，李林杰，刘崇鸣，刘力力，刘勇琪，吕浩宇，没有窗户的小巷，王禹程，吴蕴溥，熊攀，徐安越，徐永飞，许哲纶，俞涵，张峻源，张树仁，张王泽，张栩浩，郑裔，周莉莉，周先琪，朱家兴，邹文祥

Contributions of any kind are welcome!

Last committed message: !68435 r2.3.q1 - releasenote update

v2.2.11

0afa4b2

2024-01-23 21:10

MindSpore 2.2.11 Release Notes

## MindSpore 2.2.11 Release Notes

### Major Features and Improvements

#### scipy

- [Stable] Add new API mindspore.scipy.optimize.linear_sum_assignment in scipy module to solve the linear sum assignment problem. It can find the least-cost assignment based on a given cost matrix.

### Bug Fixes

- [I8JVRU] Fixed the problem where the results of the bernoulli random operator running twice on the GPU are probabilistically consistent.
- [I8OC32] Fixed the segmentation fault error because the MatrixSetDiagV3 operator does not verify abnormal input.

### Contributors

Thanks goes to these wonderful people:

fary86, wanghenchang, haozhang, mengyuanli, emmmmtang, luoyang, zhupuxu, zhangyongxian, liuluobin, LLLRT, TuDouNi, hujiahui8, wangtongyu6, ligan, zhuguodong, yanghaoran, YingtongHu, liyejun, zjun, 徐永飞, chuht, 张树仁, 徐安越, DeshiChen, shenyaxin, liujunzhu, shunyuanhan, yuchaojie, yao_yf, 没有窗户的小巷, yeyunpeng2020, weiyang, KevinYi, hedongdong, zhouyaqiang0, Margaret_wangrui, zhanghaibo, moran, huangziling, 朱家兴, GuoZhibin, 李良灿, jiaxueyu, gaoyong10, Greatpan, 宦晓玲, melody, 俞涵, jiangshanfeng, XinDu, ling, caifubi, zhangyinxia, gengdongjie, Erpim, XianglongZeng, zhangminli, fengyixing, 冯一航, 黄勇, panzhihui, 胡彬, linqingke, wangshaocong

Contributions of any kind are welcome!

## MindSpore Lite 2.2.11 Release Notes

### Bug Fixes

- [I8TPLY] Fixed SSD MobileNetV2 FPN network inference error on Atlas inference series products(configured with Ascend 310P AI processor).

### Contributors

Thanks goes to these wonderful people:

wangtongyu6, zhuguodong, 徐永飞, 徐安越, yeyunpeng2020, moran, XinDu, gengdongjie.

Contributions of any kind are welcome!

Last committed message: !64311 fix axis int32 case for reduce op

v2.2.1

aa172d2

2023-12-25 21:39

MindSpore 2.2.1 Release Notes

## MindSpore 2.2.1 Release Notes

### Bug Fixes

- [I7R3R5] Fixed the problem that the network precision of the ResNet-50 on the Ascend platform deteriorates.
- [I8A9RH] Fixed an issue where the DBNet(ResNet-50) network precision on the Ascend platform deteriorates.
- [I8B8IW] Fixed the segment error caused by out-of-bounds multi-dimensional tensor assignment.
- [I8J0F4] Fixed an issue where the multidimensional Tensor extension dimension fails to be executed in the dynamic graph.
- [I87P3P] Fixed an issue where the compilation cache fails to be loaded during secondary training on the Ascend platform.
- [I86GP9] Fixed an issue where the UNet3D network inference precision deteriorates on the Ascend platform.
- [I89B4K] Fixed an issue where the dynamic rank execution of dynamic graphs on the Windows platform is suspended.
- [I8CX0C] Fixed an issue where dynamic images occasionally fail in mixed precision mode on the Ascend platform.
- [I8BGCF] Fixed an issue where a segment error occurs when the command is executed in dynamic diagram mode of the AirNet network on the Ascend platform.
- [I8L5DS] Fixed an issue where the ResNet-50 image segmentation network dynamic image is executed slowly on the Ascend platform.

### Contributors

Thanks goes to these wonderful people:

yufan, dingcheng, lvzhangcheng, zhunaipan, fangwenyi, weiyang, changzherui, chujinjin, zangqingxiang, yuchaojie, wuweikang, tanghuikang, xiaoyao, huangbinjian, zhoupeichen, chenfei_mindspore, hedongdong, wangnan, zhengzuohe, yanghaoran, zouliqin, luoyang, liuchongmin, lujiale, machenggui, wangcong, lixiangyi, wangting, huangyong

Contributions of any kind are welcome!

## MindSpore Lite 2.2.1 Release Notes

### Bug Fixes

- [I88055] Fixed a function issue caused by incorrect format setting of the gridsample operator in MindSpore Lite inference.
- [I8D80Y] The MindSpore Lite inference single-operator invoking process resources are not released and exits abnormally.

### Contributors

Thanks goes to these wonderful people:

zhanghaibo, wangsiyuan, wangshaocong, chenjianping

Contributions of any kind are welcome!

Last committed message: !62770 Fix randomchoicewithmask

v2.2.0

9390851

2023-12-25 21:32

MindSpore 2.2.0 Release Notes

## MindSpore 2.2.0 Release Notes

### Major Features and Improvements

#### DataSet

- [STABLE] The `row_size` parameter of data operation map/batch is extended to support passing list, which stands for [Input Shared Memory, Output Shared Memory], so as to flexibly control the size of shared memory in multi-process mode.
- [STABLE] Provide 100% mindspore.dataset and mindspore.dataset.transforms samples for reference.
- [STABLE] ConcatDataset supports global sampling. After combining data from multiple sources using concat operation, data can be globally sampled randomly to enhance data diversity.
- [STABLE] When the model.train API is used for training, TimeMonitor(.., data_time=True) can be used to monitor data processing performance in real time.
- [STABLE] Introduced the jemalloc library to solve the problem of slow memory rise due to untimely memory debris recovery in extreme scenarios.

#### FrontEnd

- [STABLE] Support adding decorator @lazy_inline to make a graph generated from cell being inlined lazily, which can improve the compilation performance effectively.
- [STABLE] Optimize the function of mixed precision training, support automatic rewriting of Python scripts through rewrite to achieve mixed precision strategies, and support automatic parsing of functions, branch statements, and other syntax.
- [STABLE] Mixed precision function optimization, ReWrite supports syntax parsing of class functions and branch statements, and extends O1 functionality.
- [STABLE] Optimize the dynamic learning rate function and add APIs such as MultiStepLR; function get_lr and global_step decoupling, extending optimizer module functionality.
- [STABLE] Optimize API code samples, API difference tables, and tutorials for using higher-order functions.

#### Operator

- [STABLE] Add new operator primitive `mindspore.ops.Dense`.
- [STABLE] Add the random number operator state management feature, which allows the random number operator to save the state of the random number, and can be stably reproduced in scenarios such as model parallelism and recalculation. Currently, it only supports CPU/GPU platforms, and the involved random number operators include: `mindspore.ops.Multinomial`, `mindspore.ops.MultinomialWithReplacement`, `mindspore.ops.ParameterizedTruncatedNormal`, `mindspore.ops.StandardLaplace`, `mindspore.ops.StandardLaplace`, `mindspore.ops.Uniform`, `mindspore.ops.UniformInt`, `mindspore.ops.UniformReal`, `mindspore.ops.UniformInt`, `mindspore.ops.Dropout`, `mindspore.ops.RandomChoiceWithMask`, `mindspore.ops.RandomCategorical`, `mindspore.ops.RandomShuffle`, `mindspore.ops.RandamGamma`, `mindspore.ops.RandomPoisson` and `mindspore.ops.TruncatedNormal`.
- [STABLE] When a GPU operator encounters an illegal input scenario, it supports asynchronously printing error logs in the CUDA kernel of the operator to the Host side and interrupting the execution of the current CUDA Stream, improving the efficiency of user operator problem positioning.

#### PyNative

- [STABLE] Support viewing mechanism in PyNative mode.
- [STABLE] Function enhancement in PyNative mode: sens supports dict input type.

#### Ascend

- [STABLE] Supports user configurable operator high-precision/high-performance mode, users can use `context.set_context(ascend_config={"op_precision_mode": "/path/to/op_precision_config_file"})` to configure high-precision/high-performance modes for some TBE operators.
- [BETA] Supports user configurable operators for fp16-in and fp32-out, users can use `context.set_context(ascend_config={"precision_mode": "force_fp32"})` to configure fp16-in and fp32-out for the TBE Cube operators.
- [BETA] Remove the strong binding between `jit_level="O3"` and GE processes, so users no longer need to set `jit_level="O3"` when executing GE processes.

#### Parallel

- [STABLE] Support the gradient accumulation feature in non-pipeline parallel scenarios in semi-automatic/fully automatic mode. Users can enable gradient accumulation by writing `net = GradAccumulationCell(net, micro_size)`. The gradient accumulation feature is compatible with the  lazy_inline feature.

#### Inference

Since version 2.2, the MindSpore main release package does not provide the inference interface enabling for the Ascend 310. If you need to use the inference interface, install the MindSpore Lite release package or download the MindSpore version earlier than 2.0. For details about how to install and use MindSpore Lite, see <https://www.mindspore.cn/lite/en>. HUAWEI Ascend 310 (Ascend) is an energy-efficient and highly integrated AI processor for edge scenarios. It supports inference on MindIR models. In the earlier version, MindSpore provides two methods for enabling inference on the Ascend 310 hardware:

1. The MindSpore main release package provides the matching Ascend 310 version that supports C++ inference interfaces.
2. The MindSpore Lite release package provides the matching Ascend version and supports C++ and Java inference.

The C++ APIs provided by the two solutions are basically the same. In the future, MindSpore Lite is used instead of building and maintaining two sets of interfaces. The original 310 inference service built based on the MindSpore main release package can be switched to MindSpore Lite with a few modifications. For details, see <https://www.mindspore.cn/docs/en/r2.2/faq/inference.html>.

### Bug fixes

- [I7SDA0] Fixed an issue where the accuracy of the CRNN network deteriorates on the NES platform.
- [I7T4QK] Fixed an issue where the inference precision of the WGAN network deteriorates on the OptiX OSN 8800 platform.
- [I7TJ8Z] Fixed an issue where the inference precision of the LGTM network deteriorates on the OptiX OSN 8800 platform.
- [I7M58O] Fixed ASR-dynamic network training core dump issue on Ascend platform.
- [I7L6B6] Fixed an issue where child processes do not exit in some scenarios when dataset is in multi-process mode.
- [I7L7AE] Fixed an issue where dataset pipeline contains repeat operations and dynamic batchinfo.get_epoch_num() is incorrectly used in dataset.batch.
- [I7UY7G] Rectify the file permission modification error in OBSMindDataset.

### Contributors

Thanks goes to these wonderful people:
bantao, Bingliang, BJ-WANG, Brian-K, caifubi, ccsszz, changzherui, chenfei_mindspore, chengfeng27, chenhaozhe, chenjianping, chenkang, chenweifeng, chuht, chujinjin, CShu0507, Cynthia叶, DeshiChen, douzhixing, Erpim, Etienne, fary86, fengxun, fengyixing, gaoshuanglong, Gaoxiong, gaoyong10, GaoZhenlong, Greatpan, GuoZhibin, guozhijian, hangq, hanhuifeng, haozhang, hedongdong, Henry Shi, HighCloud, Hongxing, huangbingjian, huanghui, huangxinjing, huangziling, hujiahui8, huoxinyou, HWalkingMan, jianghui58, jiangshanfeng, jiaorui, jijiarong, jjfeing, JuiceZ, jxl, KevinYi, kisnwang, KXiong, lanzhineng, Li Qingguo, LiangZhibo, lianliguang, ligan, lihao, Lihoon, limingqi107, ling, linqingke, liruyu, liubuyu, liuchao, liujunzhu, liuluobin, liupeng303, liutongtong9, liyan2022, liyejun, looop5, luochao60, luojianing, luoyang, machenggui, maning202007, Margaret_wangrui, MaZhiming, mengyuanli, moran, NaCN, nomindcarry, panshaowu, panzhihui, qinzheng, qiuzhongya, r1chardf1d0, shaojunsong, shenwei41, shenyaxin, shenzhangyi, Shira Zaloshinski, shunyuanhan, tangdezhi_123, tanghuikang, tan-wei-cheng, tan-wei-cheng-3260, TronZhang, TuDouNi, VectorSL, wang_ziqi, wanghenchang, wangpingan, wangshaocong, wangtongyu6, wtcheng, wujueying, XianglongZeng, xiaotianci, xiaoxin_zhang, xiaoxiongzhu, xiaoyao, xiaoyuanyuan, XinDu, xujinliang, xupan, yanghaoran, yangluhang, yangruoqi713, yangsijia, yangzhenzhang, yangzishuo, yanjiaming, Yanzhi_YI, yao_yf, yefeng, yeyunpeng2020, yide12, YijieChen, YingLai Lin, YingtongHu, yonibaehr, youshu, yuchaojie, YuJianfeng, zangqx, zhaizhiqiang, zhangbuxue, zhangchunlei, zhangdanyang, zhangdong, zhanghaibo, zhangminli, zhangqi, zhangqinghua, zhangyanhui, zhangyifan, zhangyongxian, zhangzhen, zhangzheng, zhanzhan, zhengzuohe, ZhihaoLi, zhoufeng, zhouyaqiang0, zhuguodong, zhupuxu, zichun_ye, zjun, ZPaC, zuochuanyong, zyli2020, 陈宇, 程超, 范吉斌, 冯浩, 冯一航, 胡彬, 宦晓玲, 黄勇, 雷元哲, 黎冠新, 李良灿, 李林杰, 刘崇鸣, 刘力力, 刘思铭, 刘勇琪, 吕浩宇, 没有窗户的小巷, 沈竞兴, 王禹程, 王振邦, 徐安越, 徐永飞, 俞涵, 张澍坤, 周超, 朱家兴

Contributions of any kind are welcome!

## MindSpore Lite 2.2.0 Release Notes

### Major Features and Improvements

#### FlashAttention Operator Fusion

- [STABLE] The OptiX OSN Ascend 910 series supports the FlashAttention large operator fusion of the LLAMA and stable diffusion models.

## MindSpore 2.1.1 Release Notes

### Bug fixes

- [I7Q9RX] The Ascend platform supports adaptive identification of different hardware types.
- [I7SDA0] Fixed an issue where the accuracy of the CRNN network deteriorates on the NES platform.
- [I7T4QK] Fixed an issue where the inference precision of the WGAN network deteriorates on the OptiX OSN 8800 platform.
- [I7TJ8Z] Fixed an issue where the inference precision of the LGTM network deteriorates on the OptiX OSN 8800 platform.

### Contributors

Thanks goes to these wonderful people:

changzherui, chenfei_mindspore, chenjianping, chenkang, chenweifeng, chujinjin, fangwenyi, GuoZhibin, guozhijian, hangq, hanhuifeng, haozhang, hedongdong, You Shu, Zhou Feng, Dai Yuxin

Contributions of any kind are welcome!

Last committed message: !60257 [bugfix] shall return null if tensor name not in model in ge...

v2.2.10

c93cb49

2023-12-25 20:42

MindSpore 2.2.10 Release Notes

## MindSpore 2.2.10 Release Notes

### Major Features and Improvements

#### Operators

- [STABLE] FastGelu, BatchMatMul, AllReduce, AllGather, Broadcast, ReduceScatter support bfloat16 data type
- [STABLE] AllGather support uint8 data type

### Bug Fixes

- [I8ALW3] Fixed networks including Faster R-CNN, DeepText, MaskRCNN-ResNet50, which had errors while training RandomChoiceWithMask operator in Ascend 910 8P scenario.
- [I8LKG7] Fixed graph compilation error of UNet-2D in Ascend 910 1P/8P scenario.
- [I8KU3X] Fixed CRNN-ResNet34 network, which stuck in training phase in Ascend 910 1P/8P PyNative mode.
- [I8KTHH] Fixed BERT network error when training without allreduce grouped fusion with enable_parallel_optimizer=True, in Ascend 910 8P scenario.

### Contributors

Thanks goes to these wonderful people:

李林杰, TuDouNi, chengxb7532, Henry Shi, rms-infer-type, 朱家兴, zhouyaqiang0, tanghuikang, gaoyong10, gengdongjie, yao_yf, hujiahui8, hanhuifeng, shenyaxin, KevinYi, 冯一航, chengfeng27, JuiceZ, zhangyanhui, jijiarong, xiaoxiongzhu, 没有窗户的小巷, ling, liyan2022, haozhang, zangqx, xiaoyao, liujunzhu, 胡彬, panzhihui, wangshaocong, linqingke, jianghui58, qiuzhongya, yangruoqi713, zhangminli, moran, 王禹程, shaojunsong, wangtongyu6, zhupuxu, luoyang, 徐安越, qinzheng, caifubi, 徐永飞, chenkang, youshu, XinDu, liubuyu, jxl, yeyunpeng2020, huoxinyou, yefeng, jiaorui, wangpingan, cao1zhg, zjun, zyli2020, yanjiaming, Cynthia叶, 胡安东, 李良灿, liruyu, liuluobin, lihao, huangbingjian, YijieChen, jjfeing, looop5, 刘力力, xiaoxin_zhang, yangluhang, chenweifeng, jiangshanfeng, zichun_ye, 陈宇, NaCN, ligan, YingLai Lin, huangziling, chenjianping, DeshiChen, chengbin, kairui_kou, ccsszz, yanghaoran, zhangdanyang, Yanzhi_YI, zhengzuohe, hangq, TronZhang, wanghenchang, HighCloud, 吕浩宇, VectorSL, ZPaC, mengyuanli, maning202007, 刘勇琪, r1chardf1d0, fary86, 刘崇鸣, yuchaojie, douzhixing, fengyixing

Contributions of any kind are welcome!

## MindSpore Lite 2.2.10 Release Notes

### Bug Fixes

- [I8K7CC] Optimize error message when non-string segments are passed to get_model_info.

### Contributors

Thanks goes to these wonderful people:

gengdongjie, zhangyanhui, xiaoxiongzhu, wangshaocong, jianghui58, moran, wangtongyu6, 徐安越, qinzheng, 徐永飞, youshu, XinDu, yeyunpeng2020, yefeng, wangpingan, zjun, 胡安东, 刘力力, 陈宇, chenjianping, kairui_kou, zhangdanyang, hangq, mengyuanli, 刘崇鸣

Contributions of any kind are welcome!

Last committed message: !63260 fix synax format

v2.1.1

1e4403e

2023-09-21 17:19

MindSpore 2.1.1 Release Notes

Last committed message: !59050 fix fa

v2.1.0

5822529

2023-09-02 10:20

MindSpore 2.1.0 Release Notes

## MindSpore 2.1.0 Release Notes

### Major Features and Improvements

#### FrontEnd

- [BETA] JIT Fallback supports variable scenarios. In static graph mode, JIT Fallback supports return of Dict type and Scalar type, supports property setting of non-Parameter type objects, supports partial in-place modification operations of List, and supports third-party libraries such as NumPy. Moreover, it supports related operations of user-defined classes and supports Python basic operators and built-in functions to use more data types. It is compatible with features like control flow, side effects, automatic differentiation. For more details, please refer to [Static Graph Syntax Support](https://www.mindspore.cn/docs/en/r2.1/note/static_graph_syntax_support.html).

- [BETA] In static graph mode, the error message of using undefined variables in the control flow scene is optimized. When using variables defined in if, while, and for control flow branches, the variables need to be initialized and defined before the control flow.

- [STABLE] Add module ReWrite, support the ability to modify multiple network in batches based on customized rules.

- [BETA] Add optim_ex module for optimizers, extend the current functionality, support parameter grouping for every parameter in the optimizer, and support parameter modification by assignment while training.

- [STABLE] Optimize PyTorch and MindSpore API Mapping Table, specify the differences between APIs among functionality, parameter, input, output and specialized cases.

#### PyNative

- Optimize the performance of dynamic shape scenes in PyNative mode.

#### DataSet

- [STABLE] Optimize the memory structure of MindRecord data files. Memory consumption can be reduced 60% when loading 100TB+ data for training.
- [STABLE] Support single-thread execution of data processing pipeline, and users can add code in the data pipeline for debugging.
- [STABLE] Optimize the performance of TFRecordDataset to improve the performance of dataset loading by 60%+. Optimize the performance of batch to improve the performance by 30% for the scenarios with large number of batch.
- [STABLE] Optimize API documentation of [mindspore.dataset](https://www.mindspore.cn/docs/en/r2.1/api_python/mindspore.dataset.html) and [mindspore.dataset.transforms](https://www.mindspore.cn/docs/en/r2.1/api_python/mindspore.dataset.transforms.html). Four new sample libraries have been added to show the effect of data enhancement, namely: [Load & Process Datasets Using Data Pipeline](https://www.mindspore.cn/docs/en/r2.1/api_python/mindspore.dataset.html#quick-start-of-dataset-pipeline), [Visual Transformation Sample Library](https://www.mindspore.cn/docs/en/r2.1/api_python/mindspore.dataset.transforms.html#module-mindspore.dataset.vision), [Text Transform Sample Library](https://www.mindspore.cn/docs/en/r2.1/api_python/mindspore.dataset.transforms.html#module-mindspore.dataset.text), [Audio Transform Sample Library](https://www.mindspore.cn/docs/en/r2.1/api_python/mindspore.dataset.transforms.html#module-mindspore.dataset.audio)

#### AutoParallel

- [STABLE] Support offload parameters or intermediate activations to the CPU or NVMe storage during training process. Users can enable this offload feature by configuring context to scale up the trainable model size.

- [STABLE] Enhanced automatic parallel capability including:

1. Performance of automatic strategy for typical networks is no less than 90% of default configuration.

2. Support 3D hybrid parallel training: automatic operator-level strategy generation combined with manual configured pipeline partition.

#### Runtime

- [STABLE] Upgrade OpenMPI version to 4.1.4.
- [STABLE] Upgrade NCCL version to 2.16.5.
- [STABLE] Assign rank id continuously in same node when using dynamic cluster to launch distributed jobs.
- [STABLE] No adaptation code is required for Scheduler node. The script of Scheduler could be identical to that of Worker.

#### Ascend

- [STABLE] Support dump assisted debug information for operator AIC Error scenario. The information includes the operator task name, stream ID, input/output/workspace address and so on.
- [STABLE] Provide default processing mechanism, which skips its execution,  for CANN operators for empty Tensor output scenarios.
- [STABLE] Supplement debug information when network model fails to execute in graph mode. The debug information will saved in a CSV file in rank_${id}/exec_order/, recording the task ID and stream ID of each task.

#### Profiler

- [STABLE] The Profiler supports the collection of time-consuming data from all phases on the Host side.
- [BETA] The Profiler supports the collection of memory data from all phases on the Host side.
- [BETA] The Profiler supports the collection of data processing operator time consumption.

### API Change

- `mindspore.dataset.GraphData`, `mindspore.dataset.Graph`, `mindspore.dataset.InMemoryGraphDataset`, `mindspore.dataset. ArgoverseDataset` are no longer evolved and are deprecated. Use [MindSpore Graph Learning](https://gitee.com/mindspore/graphlearning) for related functional replacements. When replacing networks in Model repositories that use this API, please refer to [GCN](https://gitee.com/mindspore/graphlearning/tree/master/model_zoo/gcn) for GCN and [GAT](https://gitee.com/mindspore/graphlearning/tree/master/model_zoo/gat).
- `mindspore.set_context` adds `jit_syntax_level` option, which is used to set JIT syntax support level. For more details, please refer to [set_context](https://www.mindspore.cn/docs/en/r2.1/api_python/mindspore/mindspore.set_context.html).
- Change the `model.infer_predict_layout` interface, which has a new parameter skip_backend_compile with a default value of False. Set to True when the user wants to skip the backend compilation process to get the parameter slicing strategy.

#### Operators

- Add operator primitive for `mindspore.ops.ApplyAdamWithAmsgradV2`. It is recommended to call this operator through API `mindspore.nn.Adam`.
- Add operator primitive for `mindspore.ops.UpsampleTrilinear3D`. It is recommended to call this operator through API `mindspore.ops.interpolate`.
- Add operator primitive for `mindspore.ops.UpsampleNearest3D`. It is recommended to call this operator through API `mindspore.ops.interpolate`.

#### API Deprecation

- Deprecate operator primitive `mindspore.ops.ScatterNonAliasingAdd`. It is recommended to use operator primitive `mindspore.ops.TensorScatterAdd` as a replacement.

#### Backwards Incompatible Change

- Interface name: `mindspore.nn.Dense`, `mindspore.nn.Conv1d`, `mindspore.nn.Conv1dTranspose`, `mindspore.nn.Conv2d`, `mindspore.nn.Conv2dTranspose`, `mindspore.nn.Conv3d`, `mindspore.nn.Conv3dTranspose`

Changes: Change initialization parameter strategy. The default value of weight_init is changed from "normal" to None, and the default value of bias_init is changed from "zeros" to None.

Description: The default initialization method for weights has been changed from "normal" to internal HeUniform initialization. The default initialization method of bias is changed from "zeros" to internal Uniform initialization.

<table>
  <tr>
  <td style="text-align:center"> Original interface </td> <td style="text-align:center"> v2.1 interface </td>
  </tr>
  <tr>
  <td><pre>
  mindspore.nn.Dense(in_channels,
                     out_channels,
                     weight_init='normal',
                     bias_init='zeros',
                     has_bias=True,
                     activation=None)
  </pre>
  </td>
  <td><pre>
  mindspore.nn.Dense(in_channels,
                     out_channels,
                     weight_init=None,
                     bias_init=None,
                     has_bias=True,
                     activation=None)
  </pre>
  </td>
  </tr>
  <tr>
  <td><pre>
  mindspore.nn.Conv1d(in_channels,
                      out_channels,
                      kernel_size,
                      stride=1,
                      pad_mode='same',
                      padding=0,
                      dilation=1,
                      group=1,
                      has_bias=False,
                      weight_init='normal',
                      bias_init='zeros')
  </pre>
  </td>
  <td><pre>
  mindspore.nn.Conv1d(in_channels,
                      out_channels,
                      kernel_size,
                      stride=1,
                      pad_mode='same',
                      padding=0,
                      dilation=1,
                      group=1,
                      has_bias=False,
                      weight_init=None,
                      bias_init=None)
  </pre>
  </td>
  </tr>
  <tr>
  <td><pre>
  mindspore.nn.Conv1dTranspose(in_channels,
                               out_channels,
                               kernel_size,
                               stride=1,
                               pad_mode='same',
                               padding=0,
                               dilation=1,
                               group=1,
                               has_bias=False,
                               weight_init='normal',
                               bias_init='zeros')
  </pre>
  </td>
  <td><pre>
  mindspore.nn.Conv1dTranspose(in_channels,
                               out_channels,
                               kernel_size,
                               stride=1,
                               pad_mode='same',
                               padding=0,
                               dilation=1,
                               group=1,
                               has_bias=False,
                               weight_init=None,
                               bias_init=None)
  </pre>
  </td>
  </tr>
  <tr>
  <td><pre>
  mindspore.nn.Conv2d(in_channels,
                      out_channels, kernel_size,
                      stride=1,
                      pad_mode='same',
                      padding=0,
                      dilation=1,
                      group=1,
                      has_bias=False,
                      weight_init='normal',
                      bias_init='zeros',
                      data_format='NCHW')
  </pre>
  </td>
  <td><pre>
  mindspore.nn.Conv2d(in_channels,
                      out_channels,
                      kernel_size,
                      stride=1,
                      pad_mode='same',
                      padding=0,
                      dilation=1,
                      group=1,
                      has_bias=False,
                      weight_init=None,
                      bias_init=None,
                      data_format='NCHW')
  </pre>
  </td>
  </tr>
  <tr>
  <td><pre>
  mindspore.nn.Conv2dTranspose(in_channels,
                               out_channels,
                               kernel_size,
                               stride=1,
                               pad_mode='same',
                               padding=0,
                               output_padding=0,
                               dilation=1,
                               group=1,
                               has_bias=False,
                               weight_init='normal',
                               bias_init='zeros')
  </pre>
  </td>
  <td><pre>
  mindspore.nn.Conv2dTranspose(in_channels,
                               out_channels,
                               kernel_size,
                               stride=1,
                               pad_mode='same',
                               padding=0,
                               output_padding=0,
                               dilation=1,
                               group=1,
                               has_bias=False,
                               weight_init=None,
                               bias_init=None)
  </pre>
  </td>
  </tr>
  <tr>
  <td><pre>
  mindspore.nn.Conv3d(in_channels,
                      out_channels,
                      kernel_size,
                      stride=1,
                      pad_mode='same',
                      padding=0,
                      dilation=1,
                      group=1,
                      has_bias=False,
                      weight_init='normal',
                      bias_init='zeros',
                      data_format='NCDHW')
  </pre>
  </td>
  <td><pre>
  mindspore.nn.Conv3d(in_channels,
                      out_channels,
                      kernel_size,
                      stride=1,
                      pad_mode='same',
                      padding=0,
                      dilation=1,
                      group=1,
                      has_bias=False,
                      weight_init=None,
                      bias_init=None,
                      data_format='NCDHW')
  </pre>
  </td>
  </tr>
  <tr>
  <td><pre>
  mindspore.nn.Conv3dTranspose(in_channels,
                               out_channels,
                               kernel_size,
                               stride=1,
                               pad_mode='same',
                               padding=0,
                               dilation=1,
                               group=1,
                               output_padding=0,
                               has_bias=False,
                               weight_init='normal',
                               bias_init='zeros',
                               data_format='NCDHW')
  </pre>
  </td>
  <td><pre>
  mindspore.nn.Conv3dTranspose(in_channels,
                               out_channels,
                               kernel_size,
                               stride=1,
                               pad_mode='same',
                               padding=0,
                               dilation=1,
                               group=1,
                               output_padding=0,
                               has_bias=False,
                               weight_init=None,
                               bias_init=None,
                               data_format='NCDHW')
  </pre>
  </td>
  </tr>
  </table>

### Bug Fixes

- [I6TKLW] Fix the issue of MobileNetV2 network performance degradation on the Ascend platform.
- [I7CP5H] Fix the issue where ASR network training failed on the Ascend platform.
- [I6QYCD] Fix the issue where the BERT-Large-Boost network fails to train in pynative mode on the Ascend platform.
- [I7I3EZ] Fix the issue that caused run_check() failure due to changes to the enumeration interface in Pillow version 10.0.0. If encountered in a lower version of MindSpore, install versions of Pillow below 10.0.0 to avoid this issue.
- [I7IZ8K] Fix accuracy issues with the assignsub interface in PyNative mode.
- [I7HGY0] Fix the issue that the loss of the functional programming does not converge in the PyNative data_sink mode.
- [I7J4N3] Fix the issue that the generation of Step Trace failed in Profiler dynamic Shape mode
- [I7J4N3] Fix the issue that there is no data displayed in the MindInsight parallel strategy view.
- [I79YY4] Fix SiLU operator error when high-order differential in PyNative mode.
- [I6NQJQ] Fix the issue of probabilistic failure in dynamic shape scenarios of the ScatterUpdate operator in PyNative mode.
- [I6Y4G5] Fix the issue of failure in dynamic Shape scenarios of the Conv3D operator in Graph mode.

Last committed message: !57287 Fix bug in slice_activateion in cell reuse.

v2.0.0

fde9ba7

2023-07-29 17:33

MindSpore 2.0.0 Release Notes

## MindSpore 2.0.0 Release Notes

### Major Features and Improvements

#### PyNative

- [STABLE] Dynamic shape is fully supported on framework. For detailed operator support, refer to [Dynamic Shape Support Status of nn Interface](https://www.mindspore.cn/docs/en/master/note/dynamic_shape_nn.html), [Dynamic Shape Support Status of ops Interface](https://www.mindspore.cn/docs/en/master/note/dynamic_shape_func.html), and [Dynamic Shape Support Status of primitive Interface](https://www.mindspore.cn/docs/en/master/note/dynamic_shape_primitive.html).

#### AutoParallel

- [STABLE] Build new MindFormers independent repositpry, providing distributed parallel suite, replacing mindspore.nn.transformer module.
- [DEMO] Distributed parallel operator Gather supports the BatchDim attribute.
- [DEMO] Streamline parallel supports specifying any dimension of the input data as the Batch dimension.

### API Change

#### operator

- Add operator primitive for `mindspore.ops.AdaptiveAvgPool2D` .
- Add operator primitive for `mindspore.ops.BatchToSpaceNDV2` .
- Add operator primitive for `mindspore.ops.CeLU` .
- Add operator primitive for `mindspore.ops.ExtractVolumePatches` .
- Add operator primitive for `mindspore.ops.FFTWithSize` .
- Add operator primitive for `mindspore.ops.FillDiagonal` .
- Add operator primitive for `mindspore.ops.FractionalMaxPool3DWithFixedKsize` .
- Add operator primitive for `mindspore.ops.Im2Col` .
- Add operator primitive for `mindspore.ops.MaskedScatter` .
- Add operator primitive for `mindspore.ops.MatrixBandPart` .
- Add operator primitive for `mindspore.ops.MatrixInverse` .
- Add operator primitive for `mindspore.ops.MaxPoolWithArgmaxV2` .
- Add operator primitive for `mindspore.ops.Ormqr` .
- Add operator primitive for `mindspore.ops.RandpermV2` .
- Add operator primitive for `mindspore.ops.ResizeBicubic` .
- Add operator primitive for `mindspore.ops.Triu` .
- Add operator primitive for `mindspore.ops.Zeta` .

#### Backwards Incompatible Change

- Interface: mindspore.ops.MultitypeFuncGraph

Change: The interface parameter doc_url is used as a test feature in MindSpore 2.0.0.rc1 version. After the optimization of MindSpore 2.0.0 version, users do not need to configure this parameter, so this parameter is deleted in MindSpore 2.0.0 version.

<table>
  <tr>
  <td style="text-align:center"> Original Interface </td> <td style="text-align:center"> Interface v2.0.0 </td>
  </tr>
  <tr>
  <td><pre>
  mindspore.ops.MultitypeFuncGraph（name, read_value=False, doc_url=""）
  </pre>
  </td>
  <td><pre>
  mindspore.ops.MultitypeFuncGraph（name, read_value=False）
  </pre>
  </td>
  </tr>
  </table>

- Interface: mindspore.set_context(auto_tune_mode="GA,RL")

Change: The AutoTune tool has been deprecated, delete auto_tune_mode option, new tuning tools will be planned in the future.

- Interface: mindspore.set_context(mode=PYNATIVE_MODE)

Change: The default value is changed from GRAPH_MODE to PYNATIVE_MODE.

Description: If the running mode is not set and the diagram mode needs to be set, use the following method:
  mindspore.set_context(mode=GRAPH_MODE).

<table>
  <tr>
  <td style="text-align:center"> Original Interface </td> <td style="text-align:center"> Interface v2.0.0-rc1 </td>
  </tr>
  <tr>
  <td><pre>
  mindspore.set_context(mode=GRAPH_MODE)
  </pre>
  </td>
  <td><pre>
  mindspore.set_context(mode=PYNATIVE_MODE)
  </pre>
  </td>
  </tr>
  </table>

- Interface: mindspore.train.Model.train

Change: The default value of dataset_sink_mode is changed from True to False.

Description: If dataset_sink_mode is not set and the data sinking mode needs to be set, use the following method:
  Model.train(dataset_sink_mode=True).

<table>
  <tr>
  <td style="text-align:center"> Original Interface </td> <td style="text-align:center"> Interface v2.0.0-rc1 </td>
  </tr>
  <tr>
  <td><pre>
  Model.train(dataset_sink_mode=True)
  </pre>
  </td>
  <td><pre>
  Model.train(dataset_sink_mode=False)
  </pre>
  </td>
  </tr>
  </table>

- Interface: mindspore.export

Change: The file_format parameter is changed from AIR to no default value.

Description: If file_format is not set in the original mode, you need to set file_format additionally. In this case, use the following method:
  mindspore.export(net, *inputs, file_name, file_format="AIR", **kwargs).

<table>
  <tr>
  <td style="text-align:center"> Original Interface </td> <td style="text-align:center"> Interface v2.0.0-rc1 </td>
  </tr>
  <tr>
  <td><pre>
  mindspore.export(net, *inputs, file_name,
                   file_format="AIR", **kwargs)
  </pre>
  </td>
  <td><pre>
  mindspore.export(net, *inputs, file_name,
                   file_format, **kwargs)
  </pre>
  </td>
  </tr>
  </table>

- Interface: mindspore.ops.norm

Change: The ord parameter function is extended to support multiple forms.

<table>
  <tr>
  <td style="text-align:center"> Original Interface </td> <td style="text-align:center"> Interface v2.0.0-rc1 </td>
  </tr>
  <tr>
  <td><pre>
  ops.norm(input_x, axis, p=2, keep_dims=False, epsilon=1e-12)
  >>> # Example:
  >>> input = Tensor(np.array([[[1.0, 2.0], [3.0, 4.0]],
  ...                          [[5.0, 6.0], [7.0, 8.0]]]).astype(np.float32))
  >>> output = ops.norm(input, [0, 1], p=2)
  </pre></td>
  <td><pre>
  ops.norm(A, ord=None, dim=None, keepdim=False, *, dtype=None)
  >>> # Example:
  >>> input = Tensor(np.array([[[1.0, 2.0], [3.0, 4.0]],
  ...                          [[5.0, 6.0], [7.0, 8.0]]]).astype(np.float32))
  >>> output = ops.norm(input, ord=2, dim=(0, 1))
  </pre>
  </td>
  </tr>
  </table>

- Interface: mindspore.Tensor.norm

Change: The ord parameter function is extended to support multiple forms.

Description: For details, see the example of ops.norm.

<table>
  <tr>
  <td style="text-align:center"> Original Interface </td> <td style="text-align:center"> Interface v2.0.0-rc1 </td>
  </tr>
  <tr>
  <td><pre>
  Tensor.norm(axis, p=2, keep_dims=False, epsilon=1e-12)
  </pre>
  </td>
  <td><pre>
  Tensor.norm(ord=None, dim=None, keepdim=False, *, dtype=None)
  </pre>
  </td>
  </tr>
  </table>

- Interface: mindspore.ops.dropout

Change: The seed0 and seed1 parameters are deleted and seed=None parameter is added. Instead of returning Tensors and masks, only Tensors are returned. The input parameter training=True is added.

<table>
  <tr>
  <td style="text-align:center"> Original Interface </td> <td style="text-align:center"> Interface v2.0.0-rc1 </td>
  </tr>
  <tr>
  <td><pre>
  ops.dropout(x, p=0.5, seed0=0, seed1=0)
  >>> # Example:
  >>> input = Tensor(((20, 16), (50, 50)),
  ...                mindspore.float32)
  >>> output, mask = dropout(x, p=0.5)
  </pre>
  </td>
  <td><pre>
  ops.dropout(input, p=0.5, training=True, seed=None)
  >>> # Example:
  >>> input = Tensor(((20, 16), (50, 50)),
  ...                mindspore.float32)
  >>> output = ops.dropout(input, p=0.5，training=True)
  </pre>
  </td>
  </tr>
  </table>

- Interface: mindspore.ops.dropout2d

Change: Return value is changed from Tensor and mask to Tensor only. The input parameter training=True is added.

<table>
  <tr>
  <td style="text-align:center"> Original Interface </td> <td style="text-align:center"> Interface v2.0.0-rc1 </td>
  </tr>
  <tr>
  <td><pre>
  ops.dropout2d(x, p=0.5)
  >>> # Example:
  >>> input = Tensor(np.ones([2, 1, 2, 3]),
  ...                mindspore.float32)
  >>> output, mask = dropout2d(input, 0.5)
  </pre>
  </td>
  <td><pre>
  ops.dropout2d(input, p=0.5, training=True)
  >>> # Example:
  >>> input = Tensor(np.ones([2, 1, 2, 3]),
  ...                mindspore.float32)
  >>> output = ops.dropout2d(input, 0.5, training=True)
  </pre>
  </td>
  </tr>
  </table>

- Interface: mindspore.ops.dropout3d

Change: Return value is changed from Tensor and mask to Tensor only. The input parameter training=True is added.

<table>
  <tr>
  <td style="text-align:center"> Original Interface </td> <td style="text-align:center"> Interface v2.0.0-rc1 </td>
  </tr>
  <tr>
  <td><pre>
  ops.dropout3d(x, p=0.5)
  >>> # Example:
  >>> input = Tensor(np.ones([2, 1, 2, 3]),
  ...                mindspore.float32)
  >>> output, mask = dropout3d(input, 0.5)
  </pre>
  </td>
  <td><pre>
  ops.dropout3d(input, p=0.5, training=True)
  >>> # Example:
  >>> input = Tensor(np.ones([2, 1, 2, 3]),
  ...                mindspore.float32)
  >>> output = ops.dropout3d(input, 0.5, training=True)
  </pre>
  </td>
  </tr>
  </table>

- Interface: mindspore.ops.std

Change: The interface is reconstructed, and the interface usage mode is more consistent with user habits.

Description: If parameter `unbiased` has been set, use the following alternative: `unbiased=False` -> `ddof=0`, `unbiased=True` -> `ddof=1`.

<table>
  <tr>
  <td style="text-align:center"> Original Interface </td> <td style="text-align:center"> Interface v2.0.0-rc1 </td>
  </tr>
  <tr>
  <td><pre>
  ops.std(input_x, axis=(), unbiased=True, keep_dims=False)
  </pre>
  </td>
  <td><pre>
  ops.std(input, axis=None, ddof=0, keepdims=False)
  </pre>
  </td>
  </tr>
  </table>

- Interface: mindspore.load_param_into_net

Change: Parameters that are not loaded in the ckpt are added as return values.

<table>
  <tr>
  <td style="text-align:center"> Original Interface </td> <td style="text-align:center"> Interface v2.0.0-rc1 </td>
  </tr>
  <tr>
  <td><pre>
  net_param = load_param_into_net()
  </pre>
  </td>
  <td><pre>
  net_param, ckpt_param = load_param_into_net()
  </pre>
  </td>
  </tr>
  </table>

- Interface: mindspore.nn.BCELoss

Change: The default value of `reduction` is changed from 'none' to 'mean'.

<table>
  <tr>
  <td style="text-align:center"> Original Interface </td> <td style="text-align:center"> Interface v2.0.0-rc1 </td>
  </tr>
  <tr>
  <td><pre>
  BCELoss(weight=None, reduction='none')
  >>> # Example:
  >>> weight = Tensor(np.array([[1.0, 2.0, 3.0],
  ...                           [4.0, 3.3, 2.2]]),
  ...                 mindspore.float32)
  >>> loss = nn.BCELoss(weight=weight, reduction='mean')
  >>> logits = Tensor(np.array([[0.1, 0.2, 0.3],
  ...                           [0.5, 0.7, 0.9]]),
  ...                 mindspore.float32)
  >>> labels = Tensor(np.array([[0, 1, 0], [0, 0, 1]]),
  ...                 mindspore.float32)
  >>> output = loss(logits, labels)
  >>> print(output)
  >>> 1.8952923
  </pre>
  </td>
  <td><pre>
  BCELoss(weight=None, reduction='mean')
  >>> # Example:
  >>> weight = Tensor(np.array([[1.0, 2.0, 3.0],
  ...                           [4.0, 3.3, 2.2]]),
  ...                 mindspore.float32)
  >>> loss = nn.BCELoss(weight=weight)
  >>> logits = Tensor(np.array([[0.1, 0.2, 0.3],
  ...                           [0.5, 0.7, 0.9]]),
  ...                 mindspore.float32)
  >>> labels = Tensor(np.array([[0, 1, 0], [0, 0, 1]]),
  ...                 mindspore.float32)
  >>> output = loss(logits, labels)
  >>> print(output)
  >>> 1.8952923
  </pre>
  </td>
  </tr>
  </table>

- Interface: mindspore.ops.split

Change: The interface is reconstructed. The interface usage mode is more suitable for users. The sequence of the second and third parameters is adjusted, and the split_size_or_sections function is modified and extended.

<table>
  <tr>
  <td style="text-align:center"> Original Interface </td> <td style="text-align:center"> Interface v2.0.0-rc1 </td>
  </tr>
  <tr>
  <td><pre>
  ops.split(input_x, axis=0, output_num=1)
  >>> # Example:
  >>> input = Tensor(np.array([[1, 1, 1, 1], [2, 2, 2, 2]]),
  ...                mindspore.int32)
  >>> output = ops.split(input, axis=1, output_num=4)
  </pre>
  </td>
  <td><pre>
  ops.split(tensor, split_size_or_sections, axis=0)
  >>> # Example:
  >>> input = Tensor(np.array([[1, 1, 1, 1], [2, 2, 2, 2]]),
  ...                mindspore.int32)
  >>> output = ops.split(input, split_size_or_sections=1, axis=1)
  </pre>
  </td>
  </tr>
  </table>

- Interface: mindspore.Tensor.split

Change: The interface is reconstructed. The interface usage mode is more suitable for users. The positions of the two parameters is adjusted, and the split_size_or_sections function is modified and extended.

Description: For details, see the example of ops.split.

<table>
  <tr>
  <td style="text-align:center"> Original Interface </td> <td style="text-align:center"> Interface v2.0.0-rc1 </td>
  </tr>
  <tr>
  <td><pre>
  Tensor.split(axis=0, output_num=1)
  </pre>
  </td>
  <td><pre>
  Tensor.split(split_size_or_sections, axis=0)
  </pre>
  </td>
  </tr>
  </table>

- Interface: mindspore.ops.pad

Change: Modify the parameter name paddings to padding, and the mode and value functions are added.

<table>
  <tr>
  <td style="text-align:center"> Original Interface </td> <td style="text-align:center"> Interface v2.0.0-rc1 </td>
  </tr>
  <tr>
  <td><pre>
  ops.pad(input_x, paddings)
  >>> # Example:
  >>> input_x = Tensor(np.array([[-0.1, 0.3, 3.6],
  ...                            [0.4, 0.5, -3.2]]),
  ...                  mindspore.float32)
  >>> paddings = ((1, 2), (2, 1))
  >>> output = ops.pad(input_x, paddings)
  </pre>
  </td>
  <td><pre>
  ops.pad(input_x, padding, mode='constant', value=None)
  >>> # Example:
  >>> input_x = Tensor(np.array([[-0.1, 0.3, 3.6],
  ...                            [0.4, 0.5, -3.2]]),
  ...                  mindspore.float32)
  >>> paddings = (2, 1, 1, 2)
  >>> output = ops.pad(input_x, paddings)
  </pre>
  </td>
  </tr>
  </table>

- Interface: mindspore.ops.meshgrid

Change: The input parameter is changed from `inputs` to `*input`.

<table>
  <tr>
  <td style="text-align:center"> Original Interface </td> <td style="text-align:center"> Interface v2.0.0-rc1 </td>
  </tr>
  <tr>
  <td><pre>
  ops.meshgrid(inputs, indexing='xy')
  >>> # Example:
  >>> x = Tensor(np.array([1, 2, 3, 4]).astype(np.int32))
  >>> y = Tensor(np.array([5, 6, 7]).astype(np.int32))
  >>> z = Tensor(np.array([8, 9, 0, 1, 2]).astype(np.int32))
  output = ops.meshgrid((x, y, z), indexing='xy')
  </pre>
  </td>
  <td><pre>
  ops.meshgrid(*inputs, indexing='xy')
  >>> # Example:
  >>> x = Tensor(np.array([1, 2, 3, 4]).astype(np.int32))
  >>> y = Tensor(np.array([5, 6, 7]).astype(np.int32))
  >>> z = Tensor(np.array([8, 9, 0, 1, 2]).astype(np.int32))
  output = ops.meshgrid(x, y, z, indexing='xy')
  </pre>
  </td>
  </tr>
  </table>

- Interface: mindspore.ops.max

Change: Return value exchange sequence. The value is changed from "index, value" to "value, index".

<table>
  <tr>
  <td style="text-align:center"> Original Interface </td> <td style="text-align:center"> Interface v2.0.0-rc1 </td>
  </tr>
  <tr>
  <td><pre>
  ops.max(x, axis=0, keep_dims=False)
  >>> # Example:
  >>> input = Tensor(np.array([0.0, 0.4, 0.6, 0.7, 0.1]),
  ...                mindspore.float32)
  >>> index, output = ops.max(input)
  >>> print(index, output)
  >>> 3 0.7
  </pre>
  </td>
  <td><pre>
  ops.max(input, axis=None, keepdims=False, *, initial=None, where=True, return_indices=False)
  >>> # Example:
  >>> input = Tensor(np.array([0.0, 0.4, 0.6, 0.7, 0.1]),
  ...                mindspore.float32)
  >>> output, index = ops.max(input, axis=0)
  >>> print(output, index)
  </pre>
  </td>
  </tr>
  </table>

- Interface: mindspore.ops.min

Change: Return value exchange sequence. The value is changed from "index, value" to "value, index".

<table>
  <tr>
  <td style="text-align:center"> Original Interface </td> <td style="text-align:center"> Interface v2.0.0-rc1 </td>
  </tr>
  <tr>
  <td><pre>
  ops.min(x, axis=0, keep_dims=False)
  >>> # Example:
  >>> input = Tensor(np.array([0.0, 0.4, 0.6, 0.7, 0.1]),
  ...                mindspore.float32)
  >>> index, output = ops.min(input)
  >>> 0 0.0
  </pre>
  </td>
  <td><pre>
  ops.min(input, axis=None, keepdims=False, *, initial=None, where=True, return_indices=False)
  >>> # Example:
  >>> input = Tensor(np.array([0.0, 0.4, 0.6, 0.7, 0.1]),
  ...                mindspore.float32)
  >>> output, index = ops.min(input, keepdims=True)
  >>> 0.0 0
  </pre>
  </td>
  </tr>
  </table>

- Interface: mindspore.ops.random_gamma

Change: The seed2 parameter is deleted and seed=0 is changed to None. The framework behavior is unified and complies with the actual application scenarios and habits of users.

<table>
  <tr>
  <td style="text-align:center"> Original Interface </td> <td style="text-align:center"> Interface v2.0.0-rc1 </td>
  </tr>
  <tr>
  <td><pre>
  ops.random_gamma(shape, alpha, seed=0, seed2=0)
  </pre>
  </td>
  <td><pre>
  ops.random_gamma(shape, alpha, seed=None)
  </pre>
  </td>
  </tr>
  </table>

- Interface: mindspore.ops.standard_laplace

Change: The seed2 parameter is deleted and seed=0 is changed to None. The framework behavior is unified and complies with the actual application scenarios and habits of users.

<table>
  <tr>
  <td style="text-align:center"> Original Interface </td> <td style="text-align:center"> Interface v2.0.0-rc1 </td>
  </tr>
  <tr>
  <td><pre>
  ops.standard_laplace(shape, seed=0, seed2=0)
  </pre>
  </td>
  <td><pre>
  ops.standard_laplace(shape, seed=None)
  </pre>
  </td>
  </tr>
  </table>

- Interface: mindspore.ops.standard_normal

Change: The seed2 parameter is deleted and seed=0 is changed to None. The framework behavior is unified and complies with the actual application scenarios and habits of users.

<table>
  <tr>
  <td style="text-align:center"> Original Interface </td> <td style="text-align:center"> Interface v2.0.0-rc1 </td>
  </tr>
  <tr>
  <td><pre>
  ops.standard_normal(shape, seed=0, seed2=0)
  </pre>
  </td>
  <td><pre>
  ops.standard_normal(shape, seed=None)
  </pre>
  </td>
  </tr>
  </table>

- Interface: mindspore.ops.bernoulli

Change: The default value of seed is changed from -1 to None. Meets the actual application scenario.

<table>
  <tr>
  <td style="text-align:center"> Original Interface </td> <td style="text-align:center"> Interface v2.0.0-rc1 </td>
  </tr>
  <tr>
  <td><pre>
  ops.bernoulli(x, p=0.5, seed=-1)
  </pre>
  </td>
  <td><pre>
  ops.bernoulli(input, p=0.5, seed=None)
  </pre>
  </td>
  </tr>
  </table>

- Interface: mindspore.data_sink

Change: Deleted the steps parameter. Parameter name jit is changed to jit_config, and new input_signature parameter is added. The usability is improved to meet the requirements of actual application scenarios.

<table>
  <tr>
  <td style="text-align:center"> Original Interface </td> <td style="text-align:center"> Interface v2.0.0-rc1 </td>
  </tr>
  <tr>
  <td><pre>
  mindspore.data_sink(fn, dataset, steps,
                      sink_size=1, jit=False)
  </pre>
  </td>
  <td><pre>
  mindspore.data_sink(fn, dataset, sink_size=1,
                      jit_config=None, input_signature=None)
  </pre>
  </td>
  </tr>
  </table>

- Interface: mindspore.ops.conv2d

Change: Extend Interface Function. Add the bias parameter and modify the parameter name and parameter sequence.

<table>
  <tr>
  <td style="text-align:center"> Original Interface </td> <td style="text-align:center"> Interface v2.0.0-rc1 </td>
  </tr>
  <tr>
  <td><pre>
  conv2d(inputs, weight, pad_mode="valid",
         padding=0, stride=1, dilation=1, group=1)
  </pre>
  </td>
  <td><pre>
  conv2d(input, weight, bias=None, stride=1,
         pad_mode="valid", padding=0, dilation=1, groups=1)
  </pre>
  </td>
  </tr>
  </table>

- Interface: mindspore.dataset.vision.Pad

Change: Adjust the input parameter padding of Pad, RandomCrop, and RandomCropWithBbox. When the input length of Padding is 2, the first value is used to fill the left/upper boundary, the second value is used to fill the right/lower boundary, and the first value is used to fill the left/right boundary. Fill the upper/lower boundary with the second value.

Description: The padding parameter whose size is 2 is not compatible with the effect of the earlier version. The padding parameter needs to be explicitly represented (left, right, top, and bottom).

<table>
  <tr>
  <td style="text-align:center"> Original Interface </td> <td style="text-align:center"> Interface v2.0.0-rc1 </td>
  </tr>
  <tr>
  <td><pre>
  mindspore.dataset.vision.Pad(padding=(1,2))
  Indicates that the left/upper part of the image is filled with 1 pixel,
  and the right/down part is filled with 2 pixels.
  </pre>
  </td>
  <td><pre>
  mindspore.dataset.vision.Pad(padding=(1,2,1,2))
  Indicates that the left/upper part of the image is filled with 1 pixel,
  and the right/down part is filled with 2 pixels.
  </pre>
  </td>
  </tr>
  </table>

- Interface: mindspore.dataset.Dataset.map

Change: Delete the column_order parameter. In most cases, output_columns and column_order have the same value. Therefore, column_order does not need to be transferred. To adjust the sequence of data columns, use mindspore.dataset.Dataset.project.

Description:

1. If the column sequence does not need to be changed, delete the column_order parameter.
  2. If you need to specify the data column sequence, delete the column_order parameter and add a project method to the end of the parameter for column transformation (as in the following example).

<table>
  <tr>
  <td style="text-align:center"> Original Interface </td> <td style="text-align:center"> Interface v2.0.0-rc1 </td>
  </tr>
  <tr>
  <td><pre>
  >>> dataset = dataset.map(operations=[transforms],
  ...                       input_columns=["column_a"],
  ...                       output_columns=["column_b", "column_c"],
  ...                       column_order=["column_c", "column_b"])
  </pre>
  </td>
  <td><pre>
  >>> dataset = dataset.map(operations=[transforms],
  ...                       input_columns=["column_a"],
  ...                       output_columns=["column_b", "column_c"])
  >>> dataset = dataset.project(["column_c", column_b"])")
  </pre>
  </td>
  </tr>
  </table>

- Interface: mindspore.dataset.Dataset.batch

Change: Delete the column_order parameter. In most cases, output_columns and column_order have the same value. Therefore, column_order does not need to be transferred. To adjust the sequence of data columns, use mindspore.dataset.Dataset.project.

Description:

1. If the column sequence does not need to be changed, delete the column_order parameter.
  2. If you need to specify the data column sequence, delete the column_order parameter and add a project method to the end of the parameter for column transformation (as in the following example).

<table>
  <tr>
  <td style="text-align:center"> Original Interface </td> <td style="text-align:center"> Interface v2.0.0-rc1 </td>
  </tr>
  <tr>
  <td><pre>
  >>> dataset = dataset.batch(batch_size=4,
  ...                         input_columns=["column_a"],
  ...                         output_columns=["column_b", "column_c"],
  ...                         column_order=["column_c", "column_b"])
  </pre>
  </td>
  <td><pre>
  >>> dataset = dataset.batch(batch_size=4, input_columns=["column_a"]
  ...                         output_columns=["column_b", "column_c"])
  >>> dataset = dataset.project(["column_c", column_b"])")
  </pre>
  </td>
  </tr>
  </table>

- Interface: mindspore.dataset.Dataset.batch

Change: Split the batch method into two methods: batch and padded_batch. The pad_info parameter is moved from the batch method to the padded_batch method.

Description: To use the pad_info parameter, use the padded_batch method instead.

<table>
  <tr>
  <td style="text-align:center"> Original Interface </td> <td style="text-align:center"> Interface v2.0.0-rc1 </td>
  </tr>
  <tr>
  <td><pre>
  >>> dataset = dataset.batch(batch_size=4,
  ...                         drop_remainder=True, pad_info=...)
  </pre>
  </td>
  <td><pre>
  >>> dataset = dataset.padded_batch(batch_size=4,
  ...                                drop_remainder=True, pad_info=...)
  </pre>
  </td>
  </tr>
  </table>

### Bug fixes

- [I62I3J] fix inference failure of BGCF network on Ascend 310
- [I7C2W3] fix error issuse of null pointer when enabling multiple loss in parallel pipeline scenarios

### Contributors

Thanks goes to these wonderful people:

alashkari,anzhengqi,archer2049,B.L.LAN,baihuawei,bichaoyang,BJ-WANG,Bokai Li,Brian-K,caifubi,caiyimeng,cathwong,changzherui,ChenDonYY,chenfei_mindspore,chengang,chengbin,chenhaozhe,chenjianping,chenkang,chenweifeng,chuht,chujinjin,davidanugraha,DavidFFFan,DeshiChen,douzhixing,emmmmtang,Erpim,Ethan,fangwenyi,fangzehua,fangzhou0329,fary86,fengyixing,gaoshuanglong,Gaoxiong,gaoyong10,gengdongjie,gongdaguo1,Greatpan,GuoZhibin,guozhijian,hangq,hanhuifeng,haozhang,hedongdong,Henry Shi,heterogeneous_to_backoff_2_0,huangbingjian,huanghui,huangxinjing,hujiahui8,hujingsong,huoxinyou,jachua,jiahongQian,jianghui58,jiangzhenguang,jiaorui,jiaoy1224,jijiarong,jjfeing,JoeyLin,json,JuiceZ,jxl,kairui_kou,KevinYi,kisnwang,KXiong,laiyongqiang,lanzhineng,liangchenghui,liangzelang,LiangZhibo,lianliguang,lichen,ligan,lijunbin,limingqi107,ling,linqingke,liubuyu,liuchao,liuchuting,liujunzhu,liuluobin,liutongtong9,liuyang811,lixiao,liyan2022,liyejun,liyuxia,looop5,luochao60,luojianing,luoyang,luoyuan,lyqlola,maning202007,maoyaomin,Margaret_wangrui,mayadong,MaZhiming,melody,mengyuanli,michaelzhu_70ab,Mohammad Motallebi,moran,NaCN,nomindcarry,OwenSec,panfengfeng,panshaowu,panzhihui,pkuliuliu,qinzheng,qiuzhongya,qujianwei,r1chardf1d0,Renyuan Zhang,RobinGrosman,shaojunsong,shenwei41,Soaringfish,tangdezhi_123,tanghuikang,tan-wei-cheng,TinaMengtingZhang,TronZhang,TuDouNi,VectorSL,wang_ziqi,wanghenchang,wangnan39,wangpingan,wangshaocong,wangshengnan123,wangtongyu6,weichaoran,wind-zyx,wqx,wtcheng,wujueying,wYann,XianglongZeng,xiaohanzhang,xiaotianci,xiaoyao,XinDu,xulei,xumengjuan1,xupan,xwkgch,yanghaoran,yangluhang,yangruoqi713,yangshuo,yangsijia,yangzhenzhang,yanzhenxiang2020,Yanzhi_YI,yao_yf,yefeng,yeyunpeng2020,Yi_zhang95,yide12,YijieChen,YingLai Lin,YingtongHu,youshu,yuchaojie,yuedongli,YuJianfeng,zangqx,ZengZitao,zhangbuxue,zhangdanyang,zhangdong,zhangfanghe,zhangqi,zhangqinghua,zhangyanhui,zhangyinxia,zhangyongxian,zhangzhaoju,zhanzhan,zhengzuohe,ZhidanLiu,zhixinaa,zhoufeng,zhouyaqiang0,zhuguodong,zhupuxu,zhuyuxiao,zichun_ye,zjun,zlq2020,zong_shuai,ZPaC,zuochuanyong,zyli2020,陈宇,范吉斌,冯一航,胡彬,宦晓玲,黄勇,雷元哲,李良灿,李林杰,刘崇鸣,刘力力,刘勇琪,吕浩宇,吕昱峰（Nate.River）,没有窗户的小巷,沈竞兴,十六夜,王程浩,王禹程,王振邦,徐安越,徐永飞,杨旭华,于振华,俞涵,张清华,张澍坤,张栩浩,张学同,赵英灼,周超,周洪叶,朱家兴

Contributions of any kind are welcome!

Last committed message: !55203 fix dockerfile mindinsight versions

v2.0.0-rc1

ea0b5f5

2023-07-29 17:32

MindSpore 2.0.0-rc1 Release Notes

## MindSpore 2.0.0-rc1 Release Notes

### Major Features and Improvements

#### FrontEnd

- [BETA] Statement with "return", "return None" and with no return of function are supported in `GRAPH_MODE`.
- [BETA] Object with `list` type are supported in `GRAPH_MODE`.
- [BETA] Statement with "raise" are supported in variable condition situation in `GRAPH_MODE`.
- [STABLE] Functional call supports data sinking mode.
- [BETA] The Transformer layer in nn module is added to provide easy-to-use Transformer APIs. Batch_size does not need to be defined. Dynamic seq_length is supported.

#### DataSet

- [STABLE] In the Ascend environment，the timeout waiting time in data sink mode is adjusted to 1900s by default. This solves the problem that the GetNext operator may time out due to environment resource competition and large computing workload in data sinking mode.
- [STABLE] MindRecord supports to query the schemas and number samples. MindRecord provides multi-process writing mode, allowing users to generate MindRecord data files in parallel.
- [STABLE] The Dataset pipeline can process any Python object. For details, see [Supporting Python Objects in Dataset Pipeline](https://www.mindspore.cn/tutorials/en/r2.0/advanced/dataset/python_objects.html).

#### AutoParallel

- [STABLE] The strategies of whole parameters can be saved when saving strategy.
- [STABLE] The Conv3D/MaxPool3D/AvgPool3D distributed operator is supported.
- [STABLE] Support operator-level parallelism and optimizer-level parallelism under the PyNative with shard: parallel training and the Model API are decoupled to provide basic parallel expression capabilities.
- [STABLE] Support operator-level parallelism, and optimizer-level parallelism under the Graph mode: parallel training and the Model API are decoupled to provide basic parallel expression capabilities.
- [BETA] Supports customized distributed graph segmentation, improving the flexibility of distributed training.

#### Runtime

- [STABLE] Control flow supports subgraph sink.
- [STABLE] Support CUDA 11.6.
- [STABLE] Support for operator selection and execution of List/Tuple/Scalar type kernel to match native Python expression.
- [STABLE] Kernel that is not supported by hardware can automatically select CPU kernel.
- [STABLE] Support heterogeneous execution within subgraph.

#### Ascend

- [STABLE] Support overflow detection scheme and HCCL runtime overflow check.
- [STABLE] Support dump of communication operators.

#### Profiler

- [STABLE] Rich Profiler collection item configuration, users can collect performance data in more detail.

#### Dump

- [BETA] Single card in PyNatvie mode supports operator overflow detection.
- [BETA] Graph mode supports hccl operator dump.

### API Change

- [STABLE] Add computing APIs, such as MaxUnpool, ReplicationPad, and GaussianNLLLoss.
  For details, visit <https://www.mindspore.cn/docs/en/r2.0/api_python/mindspore.html>.
- [STABLE] Extend inventory API functions, such as AvgPool, pad, norm, and interplate.

#### operator

- [BETA] Add operator primitive for `mindspore.ops.AdaptiveAvgPool3D`.
- [BETA] Add operator primitive for `mindspore.ops.AffineGrid`.
- [BETA] Add operator primitive for `mindspore.ops.Angle`.
- [BETA] Add operator primitive for `mindspore.ops.BartlettWindow`.
- [BETA] Add operator primitive for `mindspore.ops.Bernoulli`.
- [BETA] Add operator primitive for `mindspore.ops.BesselI0`.
- [BETA] Add operator primitive for `mindspore.ops.BesselI1`.
- [BETA] Add operator primitive for `mindspore.ops.BesselJ0`.
- [BETA] Add operator primitive for `mindspore.ops.BesselJ1`.
- [BETA] Add operator primitive for `mindspore.ops.BesselK0`.
- [BETA] Add operator primitive for `mindspore.ops.BesselK0e`.
- [BETA] Add operator primitive for `mindspore.ops.BesselK1`.
- [BETA] Add operator primitive for `mindspore.ops.BesselK1e`.
- [BETA] Add operator primitive for `mindspore.ops.BesselY0`.
- [BETA] Add operator primitive for `mindspore.ops.BesselY1`.
- [BETA] Add operator primitive for `mindspore.ops.Bincount`.
- [BETA] Add operator primitive for `mindspore.ops.BlackmanWindow`.
- [BETA] Add operator primitive for `mindspore.ops.ChannelShuffle`.
- [BETA] Add operator primitive for `mindspore.ops.Cholesky`.
- [BETA] Add operator primitive for `mindspore.ops.Col2Im`.
- [BETA] Add operator primitive for `mindspore.ops.Complex`.
- [BETA] Add operator primitive for `mindspore.ops.ComplexAbs`.
- [BETA] Add operator primitive for `mindspore.ops.Cross`.
- [BETA] Add operator primitive for `mindspore.ops.CTCLossV2`.
- [BETA] Add operator primitive for `mindspore.ops.Cummin`.
- [BETA] Add operator primitive for `mindspore.ops.Diag`.
- [BETA] Add operator primitive for `mindspore.ops.Digamma`.
- [BETA] Add operator primitive for `mindspore.ops.Expand`.
- [BETA] Add operator primitive for `mindspore.ops.Fmax`.
- [BETA] Add operator primitive for `mindspore.ops.Gcd`.
- [BETA] Add operator primitive for `mindspore.ops.Geqrf`.
- [BETA] Add operator primitive for `mindspore.ops.GLU`.
- [BETA] Add operator primitive for `mindspore.ops.GridSampler2D`.
- [BETA] Add operator primitive for `mindspore.ops.GridSampler3D`.
- [BETA] Add operator primitive for `mindspore.ops.HammingWindow`.
- [BETA] Add operator primitive for `mindspore.ops.Heaviside`.
- [BETA] Add operator primitive for `mindspore.ops.Hypot`.
- [BETA] Add operator primitive for `mindspore.ops.Igamma`.
- [BETA] Add operator primitive for `mindspore.ops.IndexFill`.
- [BETA] Add operator primitive for `mindspore.ops.InplaceIndexAdd`.
- [BETA] Add operator primitive for `mindspore.ops.InplaceUpdateV2`.
- [BETA] Add operator primitive for `mindspore.ops.Lcm`.
- [BETA] Add operator primitive for `mindspore.ops.LeftShift`.
- [BETA] Add operator primitive for `mindspore.ops.LogicalXor`.
- [BETA] Add operator primitive for `mindspore.ops.Logit`.
- [BETA] Add operator primitive for `mindspore.ops.LogSpace`.
- [BETA] Add operator primitive for `mindspore.ops.LuUnpack`.
- [BETA] Add operator primitive for `mindspore.ops.MatrixDiagPartV3`.
- [BETA] Add operator primitive for `mindspore.ops.MatrixDiagV3`.
- [BETA] Add operator primitive for `mindspore.ops.MatrixSetDiagV3`.
- [BETA] Add operator primitive for `mindspore.ops.MaxPool3DWithArgmax`.
- [BETA] Add operator primitive for `mindspore.ops.MaxUnpool2D`.
- [BETA] Add operator primitive for `mindspore.ops.MaxUnpool3D`.
- [BETA] Add operator primitive for `mindspore.ops.MultiMarginLoss`.
- [BETA] Add operator primitive for `mindspore.ops.MultinomialWithReplacement`.
- [BETA] Add operator primitive for `mindspore.ops.Mvlgamma`.
- [BETA] Add operator primitive for `mindspore.ops.NanToNum`.
- [BETA] Add operator primitive for `mindspore.ops.NextAfter`.
- [BETA] Add operator primitive for `mindspore.ops.Orgqr`.
- [BETA] Add operator primitive for `mindspore.ops.Polygamma`.
- [BETA] Add operator primitive for `mindspore.ops.ResizeBilinearV2`.
- [BETA] Add operator primitive for `mindspore.ops.RightShift`.
- [BETA] Add operator primitive for `mindspore.ops.ScatterNdDiv`.
- [BETA] Add operator primitive for `mindspore.ops.ScatterNdMul`.
- [BETA] Add operator primitive for `mindspore.ops.SearchSorted`.
- [BETA] Add operator primitive for `mindspore.ops.Sinc`.
- [BETA] Add operator primitive for `mindspore.ops.Trace`.
- [BETA] Add operator primitive for `mindspore.ops.Tril`.
- [BETA] Add operator primitive for `mindspore.ops.TrilIndices`.
- [BETA] Add operator primitive for `mindspore.ops.TriuIndices`.
- [BETA] Add operator primitive for `mindspore.ops.UniqueConsecutive`.
- [STABLE] Add operator primitive for `mindspore.ops.Cummax`.
- [STABLE] Add operator primitive for `mindspore.ops.FillV2`.
- [STABLE] Add operator primitive for `mindspore.ops.IsClose`.
- [STABLE] Add operator primitive for `mindspore.ops.MatrixSolve`.
- [STABLE] Add operator primitive for `mindspore.ops.Median`.
- [STABLE] Add operator primitive for `mindspore.ops.MultilabelMarginLoss`.
- [STABLE] Add operator primitive for `mindspore.ops.NonZero`.
- [STABLE] Add operator primitive for `mindspore.ops.Pdist`.
- [STABLE] Add operator primitive for `mindspore.ops.Polar`.
- [STABLE] Add operator primitive for `mindspore.ops.RandomGamma`.
- [STABLE] Add operator primitive for `mindspore.ops.RandomPoisson`.
- [STABLE] Add operator primitive for `mindspore.ops.RandomShuffle`.
- [STABLE] Add operator primitive for `mindspore.ops.Renorm`.
- [STABLE] Add operator primitive for `mindspore.ops.ScatterNdMax`.
- [STABLE] Add operator primitive for `mindspore.ops.ScatterNdMin`.
- [STABLE] Add operator primitive for `mindspore.ops.Svd`.
- [STABLE] Add operator primitive for `mindspore.ops.TripletMarginLoss`.

#### Deleted APIs

- The `mindspore.compression` feature was deprecated at MindSpore 1.8 and is removed in this version.
  The following `mindspore.nn.quant` interfaces are also removed simultaneously: `mindspore.nn.FakeQuantWithMinMaxObserver`, `mindspore.nn.Conv2dBnFoldQuantOneConv`, `mindspore.nn.Conv2dBnFoldQuant`, `mindspore.nn.Conv2dBnWithoutFoldQuant`, `mindspore.nn.Conv2dQuant`, `mindspore.nn.DenseQuant`, `mindspore.nn.ActQuant`, `mindspore.nn.TensorAddQuant`, `mindspore.nn.ActQuant`, `mindspore.nn.MulQuant`. Please use [MindSpore Golden Stick](https://gitee.com/mindspore/golden-stick) instead to implement QuantAwareTraining in MindSpore.
- The `mindspore.dataset.close_pool`, `mindspore.dataset.to_device`, and `mindspore.dataset.set_dynamic_columns` interfaces are discarded in earlier version and being removed in this version.

#### Backwards Incompatible Change

- Interface: mindspore.set_context(mode=PYNATIVE_MODE)

Change: The default value is changed from GRAPH_MODE to PYNATIVE_MODE.

Description: If the running mode is not set and the diagram mode needs to be set, use the following method:
  mindspore.set_context(mode=GRAPH_MODE).

<table>
  <tr>
  <td style="text-align:center"> Original Interface </td> <td style="text-align:center"> Interface v2.0.0-rc1 </td>
  </tr>
  <tr>
  <td><pre>
  mindspore.set_context(mode=GRAPH_MODE)
  </pre>
  </td>
  <td><pre>
  mindspore.set_context(mode=PYNATIVE_MODE)
  </pre>
  </td>
  </tr>
  </table>

- Interface: mindspore.train.Model.train

Change: The default value of dataset_sink_mode is changed from True to False.

Description: If dataset_sink_mode is not set and the data sinking mode needs to be set, use the following method:
  Model.train(dataset_sink_mode=True).

<table>
  <tr>
  <td style="text-align:center"> Original Interface </td> <td style="text-align:center"> Interface v2.0.0-rc1 </td>
  </tr>
  <tr>
  <td><pre>
  Model.train(dataset_sink_mode=True)
  </pre>
  </td>
  <td><pre>
  Model.train(dataset_sink_mode=False)
  </pre>
  </td>
  </tr>
  </table>

- Interface: mindspore.export

Change: The file_format parameter is changed from AIR to no default value.

Description: If file_format is not set in the original mode, you need to set file_format additionally. In this case, use the following method:
  mindspore.export(net, *inputs, file_name, file_format="AIR", **kwargs).

<table>
  <tr>
  <td style="text-align:center"> Original Interface </td> <td style="text-align:center"> Interface v2.0.0-rc1 </td>
  </tr>
  <tr>
  <td><pre>
  mindspore.export(net, *inputs, file_name,
                   file_format="AIR", **kwargs)
  </pre>
  </td>
  <td><pre>
  mindspore.export(net, *inputs, file_name,
                   file_format, **kwargs)
  </pre>
  </td>
  </tr>
  </table>

- Interface: mindspore.ops.norm

Change: The ord parameter function is extended to support multiple forms.

<table>
  <tr>
  <td style="text-align:center"> Original Interface </td> <td style="text-align:center"> Interface v2.0.0-rc1 </td>
  </tr>
  <tr>
  <td><pre>
  ops.norm(input_x, axis, p=2, keep_dims=False, epsilon=1e-12)
  >>> # Example:
  >>> input = Tensor(np.array([[[1.0, 2.0], [3.0, 4.0]],
  ...                          [[5.0, 6.0], [7.0, 8.0]]]).astype(np.float32))
  >>> output = ops.norm(input, [0, 1], p=2)
  </pre></td>
  <td><pre>
  ops.norm(A, ord=None, dim=None, keepdim=False, *, dtype=None)
  >>> # Example:
  >>> input = Tensor(np.array([[[1.0, 2.0], [3.0, 4.0]],
  ...                          [[5.0, 6.0], [7.0, 8.0]]]).astype(np.float32))
  >>> output = ops.norm(input, ord=2, dim=(0, 1))
  </pre>
  </td>
  </tr>
  </table>

- Interface: mindspore.Tensor.norm

Change: The ord parameter function is extended to support multiple forms.

Description: For details, see the example of ops.norm.

<table>
  <tr>
  <td style="text-align:center"> Original Interface </td> <td style="text-align:center"> Interface v2.0.0-rc1 </td>
  </tr>
  <tr>
  <td><pre>
  Tensor.norm(axis, p=2, keep_dims=False, epsilon=1e-12)
  </pre>
  </td>
  <td><pre>
  Tensor.norm(ord=None, dim=None, keepdim=False, *, dtype=None)
  </pre>
  </td>
  </tr>
  </table>

- Interface: mindspore.ops.dropout

Change: The seed0 and seed1 parameters are deleted and seed=None parameter is added. Instead of returning Tensors and masks, only Tensors are returned. The input parameter training=True is added.

<table>
  <tr>
  <td style="text-align:center"> Original Interface </td> <td style="text-align:center"> Interface v2.0.0-rc1 </td>
  </tr>
  <tr>
  <td><pre>
  ops.dropout(x, p=0.5, seed0=0, seed1=0)
  >>> # Example:
  >>> input = Tensor(((20, 16), (50, 50)),
  ...                mindspore.float32)
  >>> output, mask = dropout(x, p=0.5)
  </pre>
  </td>
  <td><pre>
  ops.dropout(input, p=0.5, training=True, seed=None)
  >>> # Example:
  >>> input = Tensor(((20, 16), (50, 50)),
  ...                mindspore.float32)
  >>> output = ops.dropout(input, p=0.5，training=True)
  </pre>
  </td>
  </tr>
  </table>

- Interface: mindspore.ops.dropout2d

Change: Return value is changed from Tensor and mask to Tensor only. The input parameter training=True is added.

<table>
  <tr>
  <td style="text-align:center"> Original Interface </td> <td style="text-align:center"> Interface v2.0.0-rc1 </td>
  </tr>
  <tr>
  <td><pre>
  ops.dropout2d(x, p=0.5)
  >>> # Example:
  >>> input = Tensor(np.ones([2, 1, 2, 3]),
  ...                mindspore.float32)
  >>> output, mask = dropout2d(input, 0.5)
  </pre>
  </td>
  <td><pre>
  ops.dropout2d(input, p=0.5, training=True)
  >>> # Example:
  >>> input = Tensor(np.ones([2, 1, 2, 3]),
  ...                mindspore.float32)
  >>> output = ops.dropout2d(input, 0.5, training=True)
  </pre>
  </td>
  </tr>
  </table>

- Interface: mindspore.ops.dropout3d

Change: Return value is changed from Tensor and mask to Tensor only. The input parameter training=True is added.

<table>
  <tr>
  <td style="text-align:center"> Original Interface </td> <td style="text-align:center"> Interface v2.0.0-rc1 </td>
  </tr>
  <tr>
  <td><pre>
  ops.dropout3d(x, p=0.5)
  >>> # Example:
  >>> input = Tensor(np.ones([2, 1, 2, 3]),
  ...                mindspore.float32)
  >>> output, mask = dropout3d(input, 0.5)
  </pre>
  </td>
  <td><pre>
  ops.dropout3d(input, p=0.5, training=True)
  >>> # Example:
  >>> input = Tensor(np.ones([2, 1, 2, 3]),
  ...                mindspore.float32)
  >>> output = ops.dropout3d(input, 0.5, training=True)
  </pre>
  </td>
  </tr>
  </table>

- Interface: mindspore.ops.std

Change: The interface is reconstructed, and the interface usage mode is more consistent with user habits.

Description: If parameter `unbiased` has been set, use the following alternative: `unbiased=False` -> `ddof=0`, `unbiased=True` -> `ddof=1`.

<table>
  <tr>
  <td style="text-align:center"> Original Interface </td> <td style="text-align:center"> Interface v2.0.0-rc1 </td>
  </tr>
  <tr>
  <td><pre>
  ops.std(input_x, axis=(), unbiased=True, keep_dims=False)
  </pre>
  </td>
  <td><pre>
  ops.std(input, axis=None, ddof=0, keepdims=False)
  </pre>
  </td>
  </tr>
  </table>

- Interface: mindspore.load_param_into_net

Change: Parameters that are not loaded in the ckpt are added as return values.

<table>
  <tr>
  <td style="text-align:center"> Original Interface </td> <td style="text-align:center"> Interface v2.0.0-rc1 </td>
  </tr>
  <tr>
  <td><pre>
  net_param = load_param_into_net()
  </pre>
  </td>
  <td><pre>
  net_param, ckpt_param = load_param_into_net()
  </pre>
  </td>
  </tr>
  </table>

- Interface: mindspore.nn.BCELoss

Change: The default value of `reduction` is changed from 'none' to 'mean'.

<table>
  <tr>
  <td style="text-align:center"> Original Interface </td> <td style="text-align:center"> Interface v2.0.0-rc1 </td>
  </tr>
  <tr>
  <td><pre>
  BCELoss(weight=None, reduction='none')
  >>> # Example:
  >>> weight = Tensor(np.array([[1.0, 2.0, 3.0],
  ...                           [4.0, 3.3, 2.2]]),
  ...                 mindspore.float32)
  >>> loss = nn.BCELoss(weight=weight, reduction='mean')
  >>> logits = Tensor(np.array([[0.1, 0.2, 0.3],
  ...                           [0.5, 0.7, 0.9]]),
  ...                 mindspore.float32)
  >>> labels = Tensor(np.array([[0, 1, 0], [0, 0, 1]]),
  ...                 mindspore.float32)
  >>> output = loss(logits, labels)
  >>> print(output)
  >>> 1.8952923
  </pre>
  </td>
  <td><pre>
  BCELoss(weight=None, reduction='mean')
  >>> # Example:
  >>> weight = Tensor(np.array([[1.0, 2.0, 3.0],
  ...                           [4.0, 3.3, 2.2]]),
  ...                 mindspore.float32)
  >>> loss = nn.BCELoss(weight=weight)
  >>> logits = Tensor(np.array([[0.1, 0.2, 0.3],
  ...                           [0.5, 0.7, 0.9]]),
  ...                 mindspore.float32)
  >>> labels = Tensor(np.array([[0, 1, 0], [0, 0, 1]]),
  ...                 mindspore.float32)
  >>> output = loss(logits, labels)
  >>> print(output)
  >>> 1.8952923
  </pre>
  </td>
  </tr>
  </table>

- Interface: mindspore.ops.split

Change: The interface is reconstructed. The interface usage mode is more suitable for users. The sequence of the second and third parameters is adjusted, and the split_size_or_sections function is modified and extended.

<table>
  <tr>
  <td style="text-align:center"> Original Interface </td> <td style="text-align:center"> Interface v2.0.0-rc1 </td>
  </tr>
  <tr>
  <td><pre>
  ops.split(input_x, axis=0, output_num=1)
  >>> # Example:
  >>> input = Tensor(np.array([[1, 1, 1, 1], [2, 2, 2, 2]]),
  ...                mindspore.int32)
  >>> output = ops.split(input, axis=1, output_num=4)
  </pre>
  </td>
  <td><pre>
  ops.split(tensor, split_size_or_sections, axis=0)
  >>> # Example:
  >>> input = Tensor(np.array([[1, 1, 1, 1], [2, 2, 2, 2]]),
  ...                mindspore.int32)
  >>> output = ops.split(input, split_size_or_sections=1, axis=1)
  </pre>
  </td>
  </tr>
  </table>

- Interface: mindspore.Tensor.split

Change: The interface is reconstructed. The interface usage mode is more suitable for users. The positions of the two parameters is adjusted, and the split_size_or_sections function is modified and extended.

Description: For details, see the example of ops.split.

<table>
  <tr>
  <td style="text-align:center"> Original Interface </td> <td style="text-align:center"> Interface v2.0.0-rc1 </td>
  </tr>
  <tr>
  <td><pre>
  Tensor.split(axis=0, output_num=1)
  </pre>
  </td>
  <td><pre>
  Tensor.split(split_size_or_sections, axis=0)
  </pre>
  </td>
  </tr>
  </table>

- Interface: mindspore.ops.pad

Change: Modify the parameter name paddings to padding, and the mode and value functions are added.

<table>
  <tr>
  <td style="text-align:center"> Original Interface </td> <td style="text-align:center"> Interface v2.0.0-rc1 </td>
  </tr>
  <tr>
  <td><pre>
  ops.pad(input_x, paddings)
  >>> # Example:
  >>> input_x = Tensor(np.array([[-0.1, 0.3, 3.6],
  ...                            [0.4, 0.5, -3.2]]),
  ...                  mindspore.float32)
  >>> paddings = ((1, 2), (2, 1))
  >>> output = ops.pad(input_x, paddings)
  </pre>
  </td>
  <td><pre>
  ops.pad(input_x, padding, mode='constant', value=None)
  >>> # Example:
  >>> input_x = Tensor(np.array([[-0.1, 0.3, 3.6],
  ...                            [0.4, 0.5, -3.2]]),
  ...                  mindspore.float32)
  >>> paddings = (2, 1, 1, 2)
  >>> output = ops.pad(input_x, paddings)
  </pre>
  </td>
  </tr>
  </table>

- Interface: mindspore.ops.meshgrid

Change: The input parameter is changed from `inputs` to `*input`.

<table>
  <tr>
  <td style="text-align:center"> Original Interface </td> <td style="text-align:center"> Interface v2.0.0-rc1 </td>
  </tr>
  <tr>
  <td><pre>
  ops.meshgrid(inputs, indexing='xy')
  >>> # Example:
  >>> x = Tensor(np.array([1, 2, 3, 4]).astype(np.int32))
  >>> y = Tensor(np.array([5, 6, 7]).astype(np.int32))
  >>> z = Tensor(np.array([8, 9, 0, 1, 2]).astype(np.int32))
  output = ops.meshgrid((x, y, z), indexing='xy')
  </pre>
  </td>
  <td><pre>
  ops.meshgrid(*inputs, indexing='xy')
  >>> # Example:
  >>> x = Tensor(np.array([1, 2, 3, 4]).astype(np.int32))
  >>> y = Tensor(np.array([5, 6, 7]).astype(np.int32))
  >>> z = Tensor(np.array([8, 9, 0, 1, 2]).astype(np.int32))
  output = ops.meshgrid(x, y, z, indexing='xy')
  </pre>
  </td>
  </tr>
  </table>

- Interface: mindspore.ops.max

Change: Return value exchange sequence. The value is changed from "index, value" to "value, index".

<table>
  <tr>
  <td style="text-align:center"> Original Interface </td> <td style="text-align:center"> Interface v2.0.0-rc1 </td>
  </tr>
  <tr>
  <td><pre>
  ops.max(x, axis=0, keep_dims=False)
  >>> # Example:
  >>> input = Tensor(np.array([0.0, 0.4, 0.6, 0.7, 0.1]),
  ...                mindspore.float32)
  >>> index, output = ops.max(input)
  >>> print(index, output)
  >>> 3 0.7
  </pre>
  </td>
  <td><pre>
  ops.max(input, axis=None, keepdims=False, *, initial=None, where=True, return_indices=False)
  >>> # Example:
  >>> input = Tensor(np.array([0.0, 0.4, 0.6, 0.7, 0.1]),
  ...                mindspore.float32)
  >>> output, index = ops.max(input, axis=0)
  >>> print(output, index)
  </pre>
  </td>
  </tr>
  </table>

- Interface: mindspore.ops.min

Change: Return value exchange sequence. The value is changed from "index, value" to "value, index".

<table>
  <tr>
  <td style="text-align:center"> Original Interface </td> <td style="text-align:center"> Interface v2.0.0-rc1 </td>
  </tr>
  <tr>
  <td><pre>
  ops.min(x, axis=0, keep_dims=False)
  >>> # Example:
  >>> input = Tensor(np.array([0.0, 0.4, 0.6, 0.7, 0.1]),
  ...                mindspore.float32)
  >>> index, output = ops.min(input)
  >>> 0 0.0
  </pre>
  </td>
  <td><pre>
  ops.min(input, axis=None, keepdims=False, *, initial=None, where=True, return_indices=False)
  >>> # Example:
  >>> input = Tensor(np.array([0.0, 0.4, 0.6, 0.7, 0.1]),
  ...                mindspore.float32)
  >>> output, index = ops.min(input, keepdims=True)
  >>> 0.0 0
  </pre>
  </td>
  </tr>
  </table>

- Interface: mindspore.ops.random_gamma

Change: The seed2 parameter is deleted and seed=0 is changed to None. The framework behavior is unified and complies with the actual application scenarios and habits of users.

<table>
  <tr>
  <td style="text-align:center"> Original Interface </td> <td style="text-align:center"> Interface v2.0.0-rc1 </td>
  </tr>
  <tr>
  <td><pre>
  ops.random_gamma(shape, alpha, seed=0, seed2=0)
  </pre>
  </td>
  <td><pre>
  ops.random_gamma(shape, alpha, seed=None)
  </pre>
  </td>
  </tr>
  </table>

- Interface: mindspore.ops.standard_laplace

Change: The seed2 parameter is deleted and seed=0 is changed to None. The framework behavior is unified and complies with the actual application scenarios and habits of users.

<table>
  <tr>
  <td style="text-align:center"> Original Interface </td> <td style="text-align:center"> Interface v2.0.0-rc1 </td>
  </tr>
  <tr>
  <td><pre>
  ops.standard_laplace(shape, seed=0, seed2=0)
  </pre>
  </td>
  <td><pre>
  ops.standard_laplace(shape, seed=None)
  </pre>
  </td>
  </tr>
  </table>

- Interface: mindspore.ops.standard_normal

Change: The seed2 parameter is deleted and seed=0 is changed to None. The framework behavior is unified and complies with the actual application scenarios and habits of users.

<table>
  <tr>
  <td style="text-align:center"> Original Interface </td> <td style="text-align:center"> Interface v2.0.0-rc1 </td>
  </tr>
  <tr>
  <td><pre>
  ops.standard_normal(shape, seed=0, seed2=0)
  </pre>
  </td>
  <td><pre>
  ops.standard_normal(shape, seed=None)
  </pre>
  </td>
  </tr>
  </table>

- Interface: mindspore.ops.bernoulli

Change: The default value of seed is changed from -1 to None. Meets the actual application scenario.

<table>
  <tr>
  <td style="text-align:center"> Original Interface </td> <td style="text-align:center"> Interface v2.0.0-rc1 </td>
  </tr>
  <tr>
  <td><pre>
  ops.bernoulli(x, p=0.5, seed=-1)
  </pre>
  </td>
  <td><pre>
  ops.bernoulli(input, p=0.5, seed=None)
  </pre>
  </td>
  </tr>
  </table>

- Interface: mindspore.data_sink

Change: Deleted the steps parameter. Parameter name jit is changed to jit_config, and new input_signature parameter is added. The usability is improved to meet the requirements of actual application scenarios.

<table>
  <tr>
  <td style="text-align:center"> Original Interface </td> <td style="text-align:center"> Interface v2.0.0-rc1 </td>
  </tr>
  <tr>
  <td><pre>
  mindspore.data_sink(fn, dataset, steps,
                      sink_size=1, jit=False)
  </pre>
  </td>
  <td><pre>
  mindspore.data_sink(fn, dataset, sink_size=1,
                      jit_config=None, input_signature=None)
  </pre>
  </td>
  </tr>
  </table>

- Interface: mindspore.ops.conv2d

Change: Extend Interface Function. Add the bias parameter and modify the parameter name and parameter sequence.

<table>
  <tr>
  <td style="text-align:center"> Original Interface </td> <td style="text-align:center"> Interface v2.0.0-rc1 </td>
  </tr>
  <tr>
  <td><pre>
  conv2d(inputs, weight, pad_mode="valid",
         padding=0, stride=1, dilation=1, group=1)
  </pre>
  </td>
  <td><pre>
  conv2d(input, weight, bias=None, stride=1,
         pad_mode="valid", padding=0, dilation=1, groups=1)
  </pre>
  </td>
  </tr>
  </table>

- Interface: mindspore.dataset.vision.Pad

Change: Adjust the input parameter padding of Pad, RandomCrop, and RandomCropWithBbox. When the input length of Padding is 2, the first value is used to fill the left/upper boundary, the second value is used to fill the right/lower boundary, and the first value is used to fill the left/right boundary. Fill the upper/lower boundary with the second value.

Description: The padding parameter whose size is 2 is not compatible with the effect of the earlier version. The padding parameter needs to be explicitly represented (left, right, top, and bottom).

<table>
  <tr>
  <td style="text-align:center"> Original Interface </td> <td style="text-align:center"> Interface v2.0.0-rc1 </td>
  </tr>
  <tr>
  <td><pre>
  mindspore.dataset.vision.Pad(padding=(1,2))
  Indicates that the left/upper part of the image is filled with 1 pixel,
  and the right/down part is filled with 2 pixels.
  </pre>
  </td>
  <td><pre>
  mindspore.dataset.vision.Pad(padding=(1,2,1,2))
  Indicates that the left/upper part of the image is filled with 1 pixel,
  and the right/down part is filled with 2 pixels.
  </pre>
  </td>
  </tr>
  </table>

- Interface: mindspore.dataset.Dataset.map

Change: Delete the column_order parameter. In most cases, output_columns and column_order have the same value. Therefore, column_order does not need to be transferred. To adjust the sequence of data columns, use mindspore.dataset.Dataset.project.

Description:

1. If the column sequence does not need to be changed, delete the column_order parameter.
  2. If you need to specify the data column sequence, delete the column_order parameter and add a project method to the end of the parameter for column transformation (as in the following example).

<table>
  <tr>
  <td style="text-align:center"> Original Interface </td> <td style="text-align:center"> Interface v2.0.0-rc1 </td>
  </tr>
  <tr>
  <td><pre>
  >>> dataset = dataset.map(operations=[transforms],
  ...                       input_columns=["column_a"],
  ...                       output_columns=["column_b", "column_c"],
  ...                       column_order=["column_c", "column_b"])
  </pre>
  </td>
  <td><pre>
  >>> dataset = dataset.map(operations=[transforms],
  ...                       input_columns=["column_a"],
  ...                       output_columns=["column_b", "column_c"])
  >>> dataset = dataset.project(["column_c", column_b"])")
  </pre>
  </td>
  </tr>
  </table>

- Interface: mindspore.dataset.Dataset.batch

Change: Delete the column_order parameter. In most cases, output_columns and column_order have the same value. Therefore, column_order does not need to be transferred. To adjust the sequence of data columns, use mindspore.dataset.Dataset.project.

Description:

1. If the column sequence does not need to be changed, delete the column_order parameter.
  2. If you need to specify the data column sequence, delete the column_order parameter and add a project method to the end of the parameter for column transformation (as in the following example).

<table>
  <tr>
  <td style="text-align:center"> Original Interface </td> <td style="text-align:center"> Interface v2.0.0-rc1 </td>
  </tr>
  <tr>
  <td><pre>
  >>> dataset = dataset.batch(batch_size=4,
  ...                         input_columns=["column_a"],
  ...                         output_columns=["column_b", "column_c"],
  ...                         column_order=["column_c", "column_b"])
  </pre>
  </td>
  <td><pre>
  >>> dataset = dataset.batch(batch_size=4, input_columns=["column_a"]
  ...                         output_columns=["column_b", "column_c"])
  >>> dataset = dataset.project(["column_c", column_b"])")
  </pre>
  </td>
  </tr>
  </table>

- Interface: mindspore.dataset.Dataset.batch

Change: Split the batch method into two methods: batch and padded_batch. The pad_info parameter is moved from the batch method to the padded_batch method.

Description: To use the pad_info parameter, use the padded_batch method instead.

<table>
  <tr>
  <td style="text-align:center"> Original Interface </td> <td style="text-align:center"> Interface v2.0.0-rc1 </td>
  </tr>
  <tr>
  <td><pre>
  >>> dataset = dataset.batch(batch_size=4,
  ...                         drop_remainder=True, pad_info=...)
  </pre>
  </td>
  <td><pre>
  >>> dataset = dataset.padded_batch(batch_size=4,
  ...                                drop_remainder=True, pad_info=...)
  </pre>
  </td>
  </tr>
  </table>

### Bug fixes

- [I66PE6] fix AssignSub primitive abnormal input leads to coredump.

- [I6F5E6] fix data_sink function timeout on Ascend.

### Others

- Windows support is still being optimized,this version does not support now.It will be available for download in version 2.0.

### Contributors

Thanks goes to these wonderful people:

alashkari,anzhengqi,archer2049,B.L.LAN,baihuawei,bichaoyang,BJ-WANG,Bokai Li,Brian-K,caifubi,caiyimeng,cathwong,changzherui,ChenDonYY,chenfei_mindspore,chengang,chengbin,chenhaozhe,chenjianping,chenkang,chenweifeng,chuht,chujinjin,davidanugraha,DavidFFFan,DeshiChen,douzhixing,emmmmtang,Erpim,Ethan,fangwenyi,fangzehua,fangzhou0329,fary86,fengyixing,gaoshuanglong,Gaoxiong,gaoyong10,gengdongjie,gongdaguo1,Greatpan,GuoZhibin,guozhijian,hangq,hanhuifeng,haozhang,hedongdong,Henry Shi,heterogeneous_to_backoff_2_0,huangbingjian,huanghui,huangxinjing,hujiahui8,hujingsong,huoxinyou,jachua,jiahongQian,jianghui58,jiangzhenguang,jiaorui,jiaoy1224,jijiarong,jjfeing,JoeyLin,json,JuiceZ,jxl,kairui_kou,KevinYi,kisnwang,KXiong,laiyongqiang,lanzhineng,liangchenghui,liangzelang,LiangZhibo,lianliguang,lichen,ligan,lijunbin,limingqi107,ling,linqingke,liubuyu,liuchao,liuchuting,liujunzhu,liuluobin,liutongtong9,liuyang811,lixiao,liyan2022,liyejun,liyuxia,looop5,luochao60,luojianing,luoyang,luoyuan,lyqlola,maning202007,maoyaomin,Margaret_wangrui,mayadong,MaZhiming,melody,mengyuanli,michaelzhu_70ab,Mohammad Motallebi,moran,NaCN,nomindcarry,OwenSec,panfengfeng,panshaowu,panzhihui,pkuliuliu,qinzheng,qiuzhongya,qujianwei,r1chardf1d0,Renyuan Zhang,RobinGrosman,shaojunsong,shenwei41,Soaringfish,tangdezhi_123,tanghuikang,tan-wei-cheng,TinaMengtingZhang,TronZhang,TuDouNi,VectorSL,wang_ziqi,wanghenchang,wangnan39,wangpingan,wangshaocong,wangshengnan123,wangtongyu6,weichaoran,wind-zyx,wqx,wtcheng,wujueying,wYann,XianglongZeng,xiaohanzhang,xiaotianci,xiaoyao,XinDu,xulei,xumengjuan1,xupan,xwkgch,yanghaoran,yangluhang,yangruoqi713,yangshuo,yangsijia,yangzhenzhang,yanzhenxiang2020,Yanzhi_YI,yao_yf,yefeng,yeyunpeng2020,Yi_zhang95,yide12,YijieChen,YingLai Lin,YingtongHu,youshu,yuchaojie,yuedongli,YuJianfeng,zangqx,ZengZitao,zhangbuxue,zhangdanyang,zhangdong,zhangfanghe,zhangqi,zhangqinghua,zhangyanhui,zhangyinxia,zhangyongxian,zhangzhaoju,zhanzhan,zhengzuohe,ZhidanLiu,zhixinaa,zhoufeng,zhouyaqiang0,zhuguodong,zhupuxu,zhuyuxiao,zichun_ye,zjun,zlq2020,zong_shuai,ZPaC,zuochuanyong,zyli2020,陈宇,范吉斌,冯一航,胡彬,宦晓玲,黄勇,雷元哲,李良灿,李林杰,刘崇鸣,刘力力,刘勇琪,吕浩宇,吕昱峰（Nate.River）,没有窗户的小巷,沈竞兴,十六夜,王程浩,王禹程,王振邦,徐安越,徐永飞,杨旭华,于振华,俞涵,张清华,张澍坤,张栩浩,张学同,赵英灼,周超,周洪叶,朱家兴

Contributions of any kind are welcome!

Last committed message: !52829 fix conv3d group

v2.0.0-alpha

1ffbc36

2023-07-29 17:30

MindSpore 2.0.0-alpha Release Notes

## MindSpore 2.0.0-alpha Release Notes

### Major Features and Improvements

#### PyNative

- The default mode of MindSpore is switched to PyNative. If you want to manually set the mode, please refer to [Computational Graph](https://www.mindspore.cn/tutorials/en/r2.0.0-alpha/advanced/compute_graph.html).
- Support dynamic shape without padding, three networks are supported as demos: Transformer-GPU, YOLOV5-GPU, ASR-Ascend. Transformer-GPU and YOLOV5-GPU can be downloaded from [models](https://gitee.com/mindspore/models/tree/dynamic_shape). Only the following operators are available on Ascend backend: Add、Assign、BatchMatMul、BiasAdd、BiasAddGrad、Cast、Conv2D、Conv2DBackpropFilter、Conv2DBackpropInput、CTCLoss、Div、Dropout、DropoutDoMask、Equal、ExpandDims、Gather、GetNext、LayerNorm、LayerNormGrad、LessEqual、Load、Log、LogicalAnd、LogicalNot、LogicalOr、LogSoftmax、LogSoftmaxGrad、MatMul、Maximum、Mul、Neg、NotEqual、NPUAllocFloatStatus、NPUClearFloatStatus、OneHot、RealDiv、Reciprocal、ReduceMean、ReduceSum、ReLU、ReluGrad、Reshape、Select、Softmax、StridedSlice、Sub、Tile、Transpose、UnsortedSegmentSum、ZerosLike。The remaining operators have not been fully verified, please use them as appropriate.

#### DataSet

- The TFRecordDataset API can directly read TFRecord files compressed by GZIP or ZLIB.
- The NumpySlicesDataset API can process data of different dimensions at the same time.
- Optimize the structure of error log to display more clear call stack information for debugging.
- Fixed `mindspore.dataset.config.set_seed` does not take effect for random seeds in distributed training scenarios.

#### AutoParallel

- Supports more operators with distributed implements.

Element Wise Operators:AddN, BitwiseAnd, BitwiseOr, BitwiseXor, CumProd, HShrink, HSigmoid, IsFinite, Mish, MulNoNan, Rint, SeLU, SoftShrink, TruncateDiv, TruncateMod, Xdivy Xlogy, InplaceAdd, InplacSub, InplaceUpdate, Cdist, L2Loss, Lerp.

Math Operators:SquaredDifference, Erfinv, MaskedFill, SplitV, Gamma, KLDivLoss, LinSpace.

Scatter Operators:ScatterAdd,ScatterDiv,ScatterMax,ScatterMul,ScatterNdAdd,ScatterNdSub,ScatterNdUpdate,ScatterSub,TensorScatterAdd,TensorScatterDiv,TensorScatterMax,TensorScatterMax,TensorScatterMul,TensorScatterAdd,TensorScatterUpdate.

- Add new apis `transform_checkpoints` and `transform_checkpoint_by_rank` to transfer the distributed checkpoint files by strategy files. Please refer to [Distributed Resilience Training and Inference](https://www.mindspore.cn/tutorials/experts/en/r2.0.0-alpha/parallel/resilience_train_and_predict.html)。

### API Change

#### operator

- [STABLE] Add operator primitive for `mindspore.ops.AdaptiveMaxPool3D`.
- [STABLE] Add operator primitive for `mindspore.ops.AdjustHue`.
- [STABLE] Add operator primitive for `mindspore.ops.BartlettWindow`.
- [STABLE] Add operator primitive for `mindspore.ops.BesselJ0`.
- [STABLE] Add operator primitive for `mindspore.ops.BesselJ1`.
- [STABLE] Add operator primitive for `mindspore.ops.BesselK0`.
- [STABLE] Add operator primitive for `mindspore.ops.BesselK0e`.
- [STABLE] Add operator primitive for `mindspore.ops.BesselK1`.
- [STABLE] Add operator primitive for `mindspore.ops.BesselK1e`.
- [STABLE] Add operator primitive for `mindspore.ops.BesselY0`.
- [STABLE] Add operator primitive for `mindspore.ops.BesselY1`.
- [STABLE] Add operator primitive for `mindspore.ops.Betainc`.
- [STABLE] Add operator primitive for `mindspore.ops.Bincount`.
- [STABLE] Add operator primitive for `mindspore.ops.BlackmanWindow`.
- [STABLE] Add operator primitive for `mindspore.ops.Bucketize`.
- [STABLE] Add operator primitive for `mindspore.ops.CombinedNonMaxSuppression`.
- [STABLE] Add operator primitive for `mindspore.ops.CompareAndBitpack`.
- [STABLE] Add operator primitive for `mindspore.ops.Complex`.
- [STABLE] Add operator primitive for `mindspore.ops.DataFormatVecPermute`.
- [STABLE] Add operator primitive for `mindspore.ops.EuclideanNorm`.
- [STABLE] Add operator primitive for `mindspore.ops.Expand`.
- [STABLE] Add operator primitive for `mindspore.ops.ExtractGlimpse`.
- [STABLE] Add operator primitive for `mindspore.ops.FillDiagonal`.
- [STABLE] Add operator primitive for `mindspore.ops.FractionalAvgPool`.
- [STABLE] Add operator primitive for `mindspore.ops.FractionalMaxPool`.
- [STABLE] Add operator primitive for `mindspore.ops.Gcd`.
- [STABLE] Add operator primitive for `mindspore.ops.HammingWindow`.
- [STABLE] Add operator primitive for `mindspore.ops.Histogram`.
- [STABLE] Add operator primitive for `mindspore.ops.HSVToRGB`.
- [STABLE] Add operator primitive for `mindspore.ops.Lcm`.
- [STABLE] Add operator primitive for `mindspore.ops.LeftShift`.
- [STABLE] Add operator primitive for `mindspore.ops.ListDiff`.
- [STABLE] Add operator primitive for `mindspore.ops.LogSpace`.
- [STABLE] Add operator primitive for `mindspore.ops.Lstsq`.
- [STABLE] Add operator primitive for `mindspore.ops.MatrixDiagPartV3`.
- [STABLE] Add operator primitive for `mindspore.ops.MatrixDiagV3`.
- [STABLE] Add operator primitive for `mindspore.ops.MatrixExp`.
- [STABLE] Add operator primitive for `mindspore.ops.MatrixPower`.
- [STABLE] Add operator primitive for `mindspore.ops.MaxPool3DWithArgmax`.
- [STABLE] Add operator primitive for `mindspore.ops.MaxUnpool2D`.
- [STABLE] Add operator primitive for `mindspore.ops.MultilabelMarginLoss`.
- [STABLE] Add operator primitive for `mindspore.ops.NextAfter`.
- [STABLE] Add operator primitive for `mindspore.ops.Orgqr`.
- [STABLE] Add operator primitive for `mindspore.ops.ReduceStd`.
- [STABLE] Add operator primitive for `mindspore.ops.RGBToHSV`.
- [STABLE] Add operator primitive for `mindspore.ops.RightShift`.
- [STABLE] Add operator primitive for `mindspore.ops.SampleDistortedBoundingBoxV2`.
- [STABLE] Add operator primitive for `mindspore.ops.ScaleAndTranslate`.
- [STABLE] Add operator primitive for `mindspore.ops.ScatterAddWithAxis`.
- [STABLE] Add operator primitive for `mindspore.ops.ScatterNdDiv`.
- [STABLE] Add operator primitive for `mindspore.ops.ScatterNdMax`.
- [STABLE] Add operator primitive for `mindspore.ops.ScatterNdMul`.
- [STABLE] Add operator primitive for `mindspore.ops.STFT`.
- [STABLE] Add operator primitive for `mindspore.ops.Trace`.
- [STABLE] Add operator primitive for `mindspore.ops.UpsampleNearest3D`.
- [STABLE] Add operator primitive for `mindspore.ops.UpsampleTrilinear3D`.
- [STABLE] Add distributed weight conversion interface `mindspore.parallel.transform_checkpoints`.
- [STABLE] Add distributed weight conversion interface `mindspore.parallel.transform_checkpoint_by_rank`.

#### Backwards Incompatible Change

##### Python API

- The `mindspore.ms_function` interface is renamed to `mindspore.jit`, and `mindspore.ms_function` will be deprecated and removed in a future version.
- The `mindspore.ms_class` interface is renamed to `mindspore.jit_class`, and `mindspore.ms_class` will be deprecated and removed in a future version.
- The `mindspore.ops.ms_kernel` interface is renamed to `mindspore.ops.kernel`, and `mindspore.ops.ms_kernel` will be deprecated and removed in a future version.
- The `mindspore.dataset.map` interface parameter `column_order` does not take effect, use`mindspore.dataset.project`.
- The `mindspore.dataset.close_pool` and `mindspore.dataset.to_device` and `mindspore.dataset.set_dynamic_columns` are deprecated and removed in this version.

### Bug fixes

- Fixed an issue where the mixed precision functional interface could not modify the backend driver in graph mode
- Fixed the problem that users can automatically transfer device_id in the single-P scenario for the following networks:（mobilenetv1/fasterrcnn/yolov3/yolov4/yolov5/unet/openpose/simplepose/crnn/gnmtv2/faceattribute/facequality/facedetection）

### Contributors

Thanks goes to these wonderful people:

AGroupofProbiotocs, anzhengqi, askmiao, baihuawei, baiyangfan, bai-yangfan, bingyaweng, BowenK, buxue, caifubi, CaoJian, caojian05, caozhou, Cathy, changzherui, chenbo116, chenfei, chengxianbin, chenhaozhe, chenjianping, chenzomi, chenzupeng, chujinjin, cj, cjh9368, Corleone, damon0626, danish, Danish, davidmc, dayschan, doitH, dong-li001, fary86, fuzhiye, Gaoxiong, GAO_HYP_XYJ, gengdongjie, Gogery, gongdaguo, gray0v0, gukecai, guoqi, gzhcv, hangq, hanhuifeng2020, Harshvardhan, He, heleiwang, hesham, hexia, Hoai, HuangBingjian, huangdongrun, huanghui, huangxinjing, huqi, huzhifeng, hwjiaorui, Jiabin Liu, jianghui58, Jiaqi, jin-xiulang, jinyaohui, jjfeing, John, jonyguo, JulyAi, jzg, kai00, kingfo, kingxian, kpy, kswang, liuyongqi, laiyongqiang, leonwanghui, liangchenghui, liangzelang, lichen_101010, lichenever, lihongkang, lilei, limingqi107, ling, linqingke, Lin Xh, liubuyu, liuwenhao4, liuxiao78, liuxiao93, liuyang_655, liuzhongkai, Lixia, lixian, liyanliu, liyong, lizhenyu, luopengting, lvchangquan, lvliang, lz, maning202007, Margaret_wangrui, mengyuanli, Ming_blue, ms_yan, ougongchang, panfengfeng, panyifeng, Payne, Peilin, peixu_ren, Pengyongrong, qianlong, qianjiahong, r1chardf1d0, riemann_penn, rmdyh, Sheng, shenwei41, simson, Simson, Su, sunsuodong, tao_yunhao, tinazhang, VectorSL, , Wan, wandongdong, wangdongxu, wangmin,  wangyue01, wangzhe, wanyiming, Wei, wenchunjiang, wilfChen, WilliamLian, wsc, wudenggang, wukesong, wuweikang, wuxuejian, Xiao Tianci, Xiaoda, xiefangqi, xinyunfan, xuanyue, xuyongfei, yanghaitao, yanghaitao1, yanghaoran, YangLuo, yangruoqi713, yankai, yanzhenxiang2020, yao_yf, yepei6, yeyunpeng, Yi, yoni, yoonlee666, yuchaojie, yujianfeng, yuximiao, zengzitao, Zhang,  zhanghuiyao, zhanghui_china, zhangxinfeng3, zhangyihui, zhangz0911gm, zhanke, zhanyuan, zhaodezan, zhaojichen, zhaoting, zhaozhenlong, zhengjun10, zhiqwang, zhoufeng, zhousiyi, zhouyaqiang, zhouyifengCode, Zichun, Ziyan, zjun, ZPaC, wangfengwfwf, zymaa, gerayking, shu-kun-zhang.

Contributions of any kind are welcome!

Last committed message: !48176 修改argmin接口入参默认值