In graph mode, Python code is not executed by the Python interpreter. Instead, the code is compiled into a static computation graph, and then the static computation graph is executed.
Currently, only the function, Cell, and subclass instances modified by the @ms_function
decorator can be built.
For a function, build the function definition. For the network, build the construct
method and other methods or functions called by the construct
method.
For details about how to use ms_function
, click https://www.mindspore.cn/docs/en/r1.8/api_python/mindspore/mindspore.ms_function.html#mindspore.ms_function.
For details about the definition of Cell
, click https://www.mindspore.cn/docs/en/r1.8/api_python/nn/mindspore.nn.Cell.html.
Due to syntax parsing restrictions, the supported data types, syntax, and related operations during graph building are not completely consistent with the Python syntax. As a result, some usage is restricted.
The following describes the data types, syntax, and related operations supported during static graph building. These rules apply only to graph mode.
All the following examples run on the network in graph mode. For brevity, the network definition is not described.
Currently, the following built-in Python
data types are supported: Number
, String
, List
, Tuple
, and Dictionary
.
Supports int
, float
, and bool
, but does not support complex numbers.
Number
can be defined on the network. That is, the syntax y = 1
, y = 1.2
, and y = True
are supported.
When the data is constant, the value of the data can be achieved at compile time, the forcible conversion to Number
is supported in the network. That is, the syntax y = int(x)
, y = float(x)
, and y = bool(x)
are supported.
String
can be constructed on the network. That is, the syntax y = "abcd"
is supported.
Use str() to change the constant value to string, str.format() can use to format the string, but not supported to input a kwarts type arguments and the argument of format function cannot be a variable.
For example:
from mindspore import ms_function
@ms_function()
def test_str_format():
x = "{} is zero".format(0)
return x
x = test_str_format()
print(x)
The result is as follows:
0 is zero
List
can be constructed on the network, that is, the syntax y = [1, 2, 3]
is supported.
List
to be output in the computation graph will be converted into Tuple
.
When using List index to get the element,the reference type between MindSpore and Python interpreter may be different. Due to MindSpore using ListGetItem to implement getting value of the list, and the operator ListGetItem will return a copy of the variable, that make the reference type may not same with Python interpreter.
For example:
Python:
>>>a = [[1,2,3],4,5]
>>>b = a[0]
>>>b[0] = 123123
>>>a
[123123, 2, 3], 4, 5]
MindSpore:
from mindspore import ms_function
@ms_function
def test_list():
x = [[1,2,3],4,5]
b = x[0]
b[0] = 123123
return x
x = test_list()
print('x:{}'.format(x))
The result is as follows:
x: ((1, 2, 3), 4, 5)
Supported APIs
append
: adds an element to list
.
For example:
from mindspore import ms_function
@ms_function()
def test_list():
x = [1, 2, 3]
x.append(4)
return x
x = test_list()
print('x:{}'.format(x))
The result is as follows:
x: (1, 2, 3, 4)
Supported index values and value assignment
Single-level and multi-level index values and value assignment are supported.
The index value supports only int
and slice
.
The element of slice
data should be constant that can be deduced in the state of compiling graph.
The assigned value can be Number
, String
, Tuple
, List
, or Tensor
.
When the value of the current slice is Tensor, the Tensor needs to be converted to a List, which is currently implemented through JIT Fallback. Therefore, variable scenarios cannot be supported temporarily.
For example:
import numpy as np
from mindspore import ms_function
t = ms.Tensor(np.array([1, 2, 3]))
@ms_function()
def test_index():
x = [[1, 2], 2, 3, 4]
m = x[0][1]
z = x[1::2]
x[1] = t
x[2] = "ok"
x[3] = (1, 2, 3)
x[0][1] = 88
n = x[-3]
return m, z, x, n
m, z, x, n = test_index()
print('m:{}'.format(m))
print('z:{}'.format(z))
print('x:{}'.format(x))
print('n:{}'.format(n))
The result is as follows:
m:2
z:[2, 4]
x:[[1, 88], Tensor(shape=[3], dtype=Int64, value= [1, 2, 3]), 'ok', (1, 2, 3)]
n:[1 2 3]
Tuple
can be constructed on the network, that is, the syntax y = (1, 2, 3)
is supported.
Forcible conversion to Tuple
is not supported on the network. That is, the syntax y = tuple(x)
is not supported.
The reference type of tuple is same as List, please refer to List.
Supported index values
The index value can be int
, slice
, Tensor
, and multi-level index value. That is, the syntax data = tuple_x[index0][index1]...
is supported.
Restrictions on the index value Tensor
are as follows:
Tuple
stores Cell
. Each Cell
must be defined before a tuple is defined. The number of input parameters, input parameter type, and input parameter shape
of each Cell
must be the same. The number of outputs of each Cell
must be the same. The output type must be the same as the output shape.
The index Tensor
is a scalar Tensor
whose dtype
is int32
. The value range is [-tuple_len, tuple_len)
, negative index is not supported in Ascend
backend.
This syntax does not support the running branches whose control flow conditions if
, while
, and for
are variables. The control flow conditions can be constants only.
GPU
and Ascend
backend is supported.
An example of the int
and slice
indexes is as follows:
import numpy as np
from mindspore import ms_function
t = ms.Tensor(np.array([1, 2, 3]))
@ms_function()
def test_index():
x = (1, (2, 3, 4), 3, 4, t)
y = x[1][1]
z = x[4]
m = x[1:4]
n = x[-4]
return y, z, m, n
y, z, m, n = test_index()
print('y:{}'.format(y))
print('z:{}'.format(z))
print('m:{}'.format(m))
print('n:{}'.format(n))
The result is as follows:
y:3
z:[1 2 3]
m:((2, 3, 4), 3, 4)
n:(2, 3, 4)
An example of the Tensor
index is as follows:
import mindspore as ms
from mindspore import nn
class Net(nn.Cell):
def __init__(self):
super(Net, self).__init__()
self.relu = nn.ReLU()
self.softmax = nn.Softmax()
self.layers = (self.relu, self.softmax)
def construct(self, x, index):
ret = self.layers[index](x)
return ret
x = ms.Tensor([-1.0], ms.float32)
net = Net()
ret = net(x, 0)
print('ret:{}'.format(ret))
The result is as follows:
ret:[0.]
Dictionary
can be constructed on the network. That is, the syntax y = {"a": 1, "b": 2}
is supported. Currently, only String
can be used as the key
value.
Dictionary
to be output in the computational graph will extract all value
values to form the Tuple
output.
Supported APIs
keys
: extracts all key
values from dict
to form Tuple
and return it.
values
: extracts all value
values from dict
to form Tuple
and return it.
items
: extracts Tuple
composed of each pair of value
values and key
values in dict
to form Tuple
and return it.
For example:
import mindspore as ms
import numpy as np
from mindspore import ms_function
x = {"a": ms.Tensor(np.array([1, 2, 3])), "b": ms.Tensor(np.array([4, 5, 6])), "c": ms.Tensor(np.array([7, 8, 9]))}
@ms_function()
def test_dict():
y = x.keys()
z = x.values()
q = x.items()
return y, z, q
y, z, q = test_dict()
print('y:{}'.format(y))
print('z:{}'.format(z))
The result is as follows:
y:('a', 'b', 'c')
z:(Tensor(shape=[3], dtype=Int64, value= [1, 2, 3]), Tensor(shape=[3], dtype=Int64, value= [4, 5, 6]), Tensor(shape=[3], dtype=Int64, value= [7, 8, 9]))
q:[('a', Tensor(shape=[3], dtype=Int64, value= [1, 2, 3])), ('b', Tensor(shape=[3], dtype=Int64, value= [4, 5, 6])), ('c', Tensor(shape=[3], dtype=Int64, value= [7, 8, 9]))]
Supported index values and value assignment
The index value supports only String
. The assigned value can be Number
, Tuple
, or Tensor
.
For example:
import mindspore as ms
import numpy as np
from mindspore import ms_function
x = {"a": ms.Tensor(np.array([1, 2, 3])), "b": ms.Tensor(np.array([4, 5, 6])), "c": ms.Tensor(np.array([7, 8, 9]))}
@ms_function()
def test_dict():
y = x["b"]
x["a"] = (2, 3, 4)
return x, y
x, y = test_dict()
print('x:{}'.format(x))
print('y:{}'.format(y))
The result is as follows:
x:{'a': (2, 3, 4), 'b': Tensor(shape=[3], dtype=Int64, value= [4, 5, 6]), 'c': Tensor(shape=[3], dtype=Int64, value= [7, 8, 9])}
y:[4 5 6]
Currently, MindSpore supports the following user-defined data types: Tensor
, Primitive
, and Cell
.
Currently, tensors cannot be constructed on the network. That is, the syntax x = Tensor(args...)
is not supported.
You can use the @constexpr
decorator to modify the function and generate the Tensor
in the function.
For details about how to use @constexpr
, click https://www.mindspore.cn/docs/en/r1.8/api_python/ops/mindspore.ops.constexpr.html.
The constant Tensor
used on the network can be used as a network attribute and defined in init
, that is, self.x = Tensor(args...)
. Then the constant can be used in construct
.
In the following example, Tensor
of shape = (3, 4), dtype = int64
is generated by @constexpr
.
import mindspore as ms
from mindspore.ops import constexpr
@constexpr
def generate_tensor():
return ms.Tensor(np.ones((3, 4)))
x = generate_tensor()
print('x:{}'.format(x))
The result is as follows:
x:[[1. 1. 1. 1.]
[1. 1. 1. 1.]
[1. 1. 1. 1.]]
The following describes the attributes, APIs supported by the Tensor
.
Supported attributes
shape
: obtains the shape of Tensor
and returns a Tuple
.
dtype
: obtains the data type of Tensor
and returns a data type defined by MindSpore
.
Supported APIs
all
: reduces Tensor
through the all
operation. Only Tensor
of the Bool
type is supported.
any
: reduces Tensor
through the any
operation. Only Tensor
of the Bool
type is supported.
view
: reshapes Tensor
into input shape
.
expand_as
: expands Tensor
to the same shape
as another Tensor
based on the broadcast rule.
For example:
import mindspore as ms
import numpy as np
x = ms.Tensor(np.array([[True, False, True], [False, True, False]]))
y = ms.Tensor(np.ones((2, 3), np.float32))
z = ms.Tensor(np.ones((2, 2, 3)))
x_shape = x.shape
x_dtype = x.dtype
x_all = x.all()
x_any = x.any()
x_view = x.view((1, 6))
y_as_z = y.expand_as(z)
print('x_shape:{}'.format(x_shape))
print('x_dtype:{}'.format(x_dtype))
print('x_all:{}'.format(x_all))
print('x_any:{}'.format(x_any))
print('x_view:{}'.format(x_view))
print('y_as_z:{}'.format(y_as_z))
The result is as follows:
x_shape:(2, 3)
x_dtype:Bool
x_all:False
x_any:True
x_view:[[ True False True False True False]]
y_as_z:[[[1. 1. 1.]
[1. 1. 1.]]
[[1. 1. 1.]
[1. 1. 1.]]]
Currently, Primitive
and its subclass instances can be constructed on the network. That is, the reduce_sum = ReduceSum(True)
syntax is supported.
However, during construction, the parameter can be specified only in position parameter mode, and cannot be specified in the key-value pair mode. That is, the syntax reduce_sum = ReduceSum(keep_dims=True)
is not supported.
Currently, the attributes and APIs related to Primitive
and its subclasses cannot be called on the network.
For details about the defined Primitive
, click https://www.mindspore.cn/docs/en/r1.8/api_python/ops/mindspore.ops.Primitive.html#mindspore.ops.Primitive.
Currently, Cell
and its subclass instances can be constructed on the network. That is, the syntax cell = Cell(args...)
is supported.
However, during construction, the parameter can be specified only in position parameter mode, and cannot be specified in the key-value pair mode. That is, the syntax cell = Cell(arg_name=value)
is not supported.
Currently, the attributes and APIs related to Cell
and its subclasses cannot be called on the network unless they are called through self
in construct
of Cell
.
For details about the definition of Cell
, click https://www.mindspore.cn/docs/en/r1.8/api_python/nn/mindspore.nn.Cell.html.
For details about the defined Cell
, click https://www.mindspore.cn/docs/en/r1.8/api_python/nn/mindspore.nn.Cell.html#mindspore.nn.Cell.
Parameter
is a variable tensor, indicating the parameters that need to be updated during network training.
For details about the definition of Parameter
:https://www.mindspore.cn/docs/en/r1.8/api_python/mindspore/mindspore.Parameter.html#mindspore.Parameter
Primaries represent the most tightly bound operations of the language Which contains Attribute references
, Subscriptions
, Calls
.
An attribute reference is a primary followed by a period and a name.
In Cell
instance of MindSpore, using attribute reference as left operands must meet the restrictions below:
The attribute must belong to self, such as self.xxx. It is not supported to change attribute of other instance.
The attribute type must be Parameter
and be initialized in __init__
function.
For example:
import mindspore as ms
from mindspore import nn
import numpy as np
from mindspore.ops import constexpr
class Net(nn.Cell):
def __init__(self):
super().__init__()
self.weight = ms.Parameter(ms.Tensor(3, ms.float32), name="w")
self.m = 2
def construct(self, x, y):
self.weight = x # restictions matched, success
# self.m = 3 # self.m not Parameter type, failure
# y.weight = x # not attribute of self, failure
return x
net = Net()
ret = net(1, 2)
print('ret:{}'.format(ret))
The result is as follows:
ret:1
Index value of a sequence Tuple
, List
, Dictionary
, Tensor
which called subscription in Python.
Index value of Tuple
refers to chapter Tuple of this page.
Index value of List
refers to chapter List of this page.
Index value of Dictionary
refers to chapter Dictionary of this page.
Index value of Tensor
refers to https://www.mindspore.cn/docs/en/r1.8/note/index_support.html#index-values
A call calls a callable object (e.g., Cell
or Primitive
) with a possibly empty series of arguments.
For example:
import mindspore as ms
from mindspore import nn, ops
import numpy as np
class Net(nn.Cell):
def __init__(self):
super().__init__()
self.matmul = ops.MatMul()
def construct(self, x, y):
out = self.matmul(x, y) # A call of Primitive
return out
x = ms.Tensor(np.ones(shape=[1, 3]), ms.float32)
y = ms.Tensor(np.ones(shape=[3, 4]), ms.float32)
net = Net()
ret = net(x, y)
print('ret:{}'.format(ret))
The result is as follows:
ret:[[3. 3. 3. 3.]]
Arithmetic operators and assignment operators support the Number
and Tensor
operations, as well as the Tensor
operations of different dtype
.
This is because these operators are converted to operators with the same name for computation, and they support implicit type conversion.
For details about the rules, click https://www.mindspore.cn/docs/en/r1.8/note/operator_list_implicit.html#conversion-rules.
Unary Arithmetic Operator | Supported Type |
---|---|
+ |
Number , Tensor
|
- |
Number , Tensor
|
~ |
Tensor with Bool data type |
notes:
~
operator get the bitwise inversion of its integer argument; in Mindspore the ~
redefined to get logic not for Tensor(Bool)
.Binary Arithmetic Operator | Supported Type |
---|---|
+ |
Number + Number , String + String , Number + Tensor , Tensor + Number , Tuple + Tensor , Tensor + Tuple , List + Tensor , Tensor +List , List +List , Tensor + Tensor , Tuple + Tuple . |
- |
Number - Number , Tensor - Tensor , Number -Tensor , Tensor - Number , Tuple -Tensor , Tensor -Tuple , List -Tensor , Tensor -List . |
* |
Number * Number , Tensor * Tensor , Number * Tensor , Tensor * Number , List * Number , Number * List , Tuple * Number , Number * Tuple , Tuple * Tensor , Tensor * Tuple , List *Tensor , Tensor * List . |
/ |
Number / Number , Tensor / Tensor , Number / Tensor , Tensor / Number , Tuple / Tensor , Tensor / Tuple , List / Tensor , Tensor / List . |
% |
Number % Number , Tensor % Tensor , Number % Tensor , Tensor % Number , Tuple % Tensor , Tensor % Tuple , List % Tensor , Tensor % List . |
** |
Number ** Number , Tensor ** Tensor , Number ** Tensor , Tensor ** Number , Tuple ** Tensor , Tensor ** Tuple , List ** Tensor , Tensor ** List . |
// |
Number // Number , Tensor // Tensor , Number // Tensor , Tensor // Number , Tuple // Tensor , Tensor // Tuple , List // Tensor , Tensor // List . |
Restrictions:
Number
type, value of Number can't be Bool
.Number
type, operations between Float64
and Int32
are not supported.Tensor
type, left and right operands can't both be Bool
value.List * Number
is concatenate duplicate List Number times, data type of the List
must be Number
, String
, None
or List
/Tuple
that contains these types. This rule applies to Number * List
, Tuple * Number
, Number * Tuple
too.Assignment Operator | Supported Type、 |
---|---|
= |
All Built-in Python Types that MindSpore supported and MindSpore User-defined Data Types. |
+= |
Number += Number , String += String , Number += Tensor , Tensor += Number , Tuple += Tensor , Tensor += Tuple , List += Tensor , Tensor += List , List += List , Tensor += Tensor , Tuple += Tuple . |
-= |
Number -= Number , Tensor -= Tensor , Number -= Tensor , Tensor -= Number , Tuple -= Tensor , Tensor -= Tuple , List -= Tensor , Tensor -= List . |
*= |
Number *= Number , Tensor *= Tensor , Number *= Tensor , Tensor *= Number , List *= Number , Number *= List , Tuple *= Number , Number *= Tuple , Tuple *= Tensor , Tensor *= Tuple , List *= Tensor , Tensor *= List . |
/= |
Number /= Number , Tensor /= Tensor , Number /= Tensor , Tensor /= Number , Tuple /= Tensor , Tensor /= Tuple , List /= Tensor , Tensor /= List . |
%= |
Number %= Number , Tensor %= Tensor , Number %= Tensor , Tensor %= Number , Tuple %= Tensor , Tensor %= Tuple , List %= Tensor 、Tensor %= List . |
**= |
Number **= Number , Tensor **= Tensor , Number **= Tensor , Tensor **= Number , Tuple **= Tensor , Tensor **= Tuple , List **= Tensor , Tensor **= List . |
//= |
Number //= Number , Tensor //= Tensor , Number //= Tensor , Tensor //= Number , Tuple //= Tensor , Tensor //= Tuple , List //= Tensor , Tensor //= List . |
Notes:
For =
the scenarios below are not allowed:
Only instance of Cell
and Primitve
can be created in function construct, the statement like xx = Tensor(...)
is forbidden.
Only Parameter
attribute of self can be assigned, for more detail refer to Attribute Reference.
If all operands of AugAssign
are Number
type, value of Number can't be Bool
.
If all operands of AugAssign
are Number
type, operations between Float64
and Int32
are not supported.
If either operand of AugAssign
is Tensor
type, left and right operands can't both be Bool
value.
The result of List *= Number
is concatenate duplicate List Number times, data type of the List
must be Number
, String
, None
or List
/Tuple
that contains these types. This rule applies to Number * List
, Tuple * Number
, Number * Tuple
too.
Logical Operator | Supported Type |
---|---|
and |
String , Number , Tuple , List , Dict , None , Scalar , Tensor . |
or |
String , Number , Tuple , List , Dict , None , Scalar , Tensor . |
not |
Number , tuple , List and Tensor with only one element. |
Restrictions:
and
, or
, if left operand is a Tensor
, right operand should be Tensor
which has same data type with left operand, and bothTensor
must have only one element.and
, or
, if left operand not Tensor
, right operand can be any supported type.Compare Operator | Supported Type |
---|---|
in |
Number in tuple , String in tuple , Tensor in Tuple , Number in List , String in List , Tensor in List , and String in Dictionary . |
not in |
Same as in . |
is |
The value can only be None , True , or False . |
is not |
The value can only be None , True , or False . |
< |
Number < Number , Number < Tensor , Tensor < Tensor , Tensor < Number . |
<= |
Number <= Number , Number <= Tensor , Tensor <= Tensor , Tensor <= Number . |
> |
Number > Number , Number > Tensor , Tensor > Tensor , Tensor > Number . |
>= |
Number >= Number , Number >= Tensor , Tensor >= Tensor , Tensor >= Number . |
!= |
Number != Number , Number != Tensor , Tensor != Tensor , Tensor != Number , mstype != mstype , String != String , Tuple ! = Tuple , List != List . |
== |
Number == Number , Number == Tensor , Tensor == Tensor , Tensor == Number , mstype == mstype , String == String , Tuple == Tuple , List == List . |
Restrictions:
<
, <=
, >
, >=
, !=
, if all operators are Number
type, value of Number can't be Bool
.<
, <=
, >
, >=
, !=
, ==
, if all operands are Number
type, operations between Float64
and Int32
are not supported.<
, <=
, >
, >=
, !=
, ==
, if either operand is Tensor
type, left and right operands can't both be Bool
value.==
, if all operands are Number
type, support both Number
have Bool
value, not support only one Number
has Bool
value.!=
, ==
, all supported types but mstype
can compare with None
.a>b>c
is not supported.Usage:
if (cond): statements...
x = y if (cond) else z
Parameter: cond
-- Variables of Bool
type and constants of Bool
, List
, Tuple
, Dict
and String
types are supported.
Restrictions:
If cond
is not a constant, the variable or constant assigned to a same sign in different branches should have same data type.If the data type of assigned variables or constants is Tensor
, the variables and constants should have same shape and element type.
The number of if
cannot exceed 100.
Example 1:
import mindspore as ms
from mindspore import ms_function
x = ms.Tensor([1, 2], ms.int32)
y = ms.Tensor([0, 3], ms.int32)
m = 'xx'
n = 'yy'
@ms_function()
def test_cond(x, y):
if (x > y).any():
return m
else:
return n
ret = test_cond(x, y)
print('ret:{}'.format(ret))if (x > y).any():
return m
else:
return n
The data type of m
returned by the if
branch and n
returned by the else
branch must be same.
The result is as follows:
ret:xx
Example 2:
import mindspore as ms
from mindspore import ms_function
x = ms.Tensor([1, 2], ms.int32)
y = ms.Tensor([0, 3], ms.int32)
m = 'xx'
n = 'yy'
@ms_function()
def test_cond(x, y):
out = 'init'
if (x > y).any():
out = m
else:
out = n
return out
ret = test_cond(x, y)
print('ret:{}'.format(ret))
The variable or constant m
assigned to out
in if
branch and the variable or constant n
assigned to out in false
branch must have same data type.
The result is as follows:
ret:xx
Example 3:
import mindspore as ms
from mindspore import ms_function
x = ms.Tensor([1, 2], ms.int32)
y = ms.Tensor([0, 3], ms.int32)
m = 'xx'
@ms_function()
def test_cond(x, y):
out = 'init'
if (x > y).any():
out = m
return out
ret = test_cond(x, y)
print('ret:{}'.format(ret))
The variable or constant m
assigned to out
in if
branch and the variable or constant init
initially assigned to out
must have same data type.
The result is as follows:
ret:xx
Usage:
for i in sequence statements...
for i in sequence statements... if (cond) break
for i in sequence statements... if (cond) continue
Parameter: sequence
-- Iterative sequences (Tuple
, List
, range
and so on).
Restrictions:
The total number of graph operations is a multiple of number of iterations of the for
loop. Excessive number of iterations of the for
loop may cause the graph to occupy more memory than usage limit.
The for...else...
statement is not supported.
Example:
import numpy as np
from mindspore import ms_function
z = ms.Tensor(np.ones((2, 3)))
@ms_function()
def test_cond():
x = (1, 2, 3)
for i in x:
z += i
return z
ret = test_cond()
print('ret:{}'.format(ret))
The result is as follows:
ret:[[7. 7. 7.]
[7. 7. 7.]]
Usage:
while (cond) statements...
while (cond) statements... if (cond1) break
while (cond) statements... if (cond1) continue
Parameter: cond
-- Variables of Bool
type and constants of Bool
, List
, Tuple
, Dict
and String
types are supported.
Restrictions:
If cond
is not a constant, the variable or constant assigned to a same sign inside body of while
and outside body of while
should have same data type.If the data type of assigned variables or constants is Tensor
, the variables and constants should have same shape and element type.
The while...else...
statement is not supported.
If cond
is not a constant, in while body, the data with type of Number
, List
, Tuple
are not allowed to update and the shape of Tensor
data are not allowed to change.
The number of while
cannot exceed 100.
Example 1:
from mindspore import ms_function
m = 1
n = 2
@ms_function()
def test_cond(x, y):
while x < y:
x += 1
return m
return n
ret = test_cond(1, 5)
print('ret:{}'.format(ret))
The data type of m
returned inside while
and data type of n
returned outside while
must have same data type.
The result is as follows:
ret:1
Example 2:
from mindspore import ms_function
m = 1
n = 2
def ops1(a, b):
return a + b
@ms_function()
def test_cond(x, y):
out = m
while x < y:
x += 1
out = ops1(out, x)
return out
ret = test_cond(1, 5)
print('ret:{}'.format(ret))
The variable op1
assigned to out
inside while
and the variable or constant init
initially assigned to out
must have same data type.
The result is as follows:
ret:15
Defines functions.
Usage:
def function_name(args): statements...
For example:
from mindspore import ms_function
def number_add(x, y):
return x + y
@ms_function()
def test(x, y):
return number_add(x, y)
ret = test(1, 5)
print('ret:{}'.format(ret))
The result is as follows:
ret: 6
Restrictions:
return
statement.Construct
function of the outermost network is not support kwargs, like:def construct(**kwargs):
.def function(x, y, *args)
and def function(x = 1, y = 1, **kwargs)
.Generates functions.
Usage: lambda x, y: x + y
For example:
from mindspore import ms_function
@ms_function()
def test(x, y):
number_add = lambda x, y: x + y
return number_add(x, y)
ret = test(1, 5)
print('ret:{}'.format(ret))
The result is as follows:
ret: 6
Support List Comprehension and Generator Expression.
Generates a list. Own to the implicit converting during compiling, the result of expression is a tuple.
Usage: refer to Python official syntax description.
For example:
from mindspore import ms_function
@ms_function()
def test(x, y):
l = [x * x for x in range(1, 11) if x % 2 == 0]
return l
ret = test(1, 5)
print('ret:{}'.format(ret))
The result is as follows:
ret:(4, 16, 36, 64, 100)
Restrictions:
Use multiple nested iterations comprehension in the generator.
For example (Use two nested iterations):
l = [y for x in ((1, 2), (3, 4), (5, 6)) for y in x]
The result would be:
TypeError: The `generators` supports one `comprehension` in ListComp/GeneratorExp, but got 2 comprehensions.
Generates a list. The same as List Comprehension. The expression would generate a list immediately, not like the behavior running in Python.
Usage: Referencing List Comprehension.
For example:
from mindspore import ms_function
@ms_function()
def test(x, y):
l = (x * x for x in range(1, 11) if x % 2 == 0)
return l
ret = test(1, 5)
print('ret:{}'.format(ret))
The result is as follows:
ret:(4, 16, 36, 64, 100)
Restrictions: The same as List Comprehension.
Currently, the following built-in Python functions are supported: len
, isinstance
, partial
, map
, range
, enumerate
, super
, pow
, and filter
.
Returns the length of a sequence.
Calling: len(sequence)
Input parameter: sequence
-- Tuple
, List
, Dictionary
, or Tensor
.
Return value: length of the sequence, which is of the int
type. If the input parameter is Tensor
, the length of dimension 0 is returned.
For example:
import mindspore as ms
import numpy as np
from mindspore import ms_function
z = ms.Tensor(np.ones((6, 4, 5)))
@ms_function()
def test():
x = (2, 3, 4)
y = [2, 3, 4]
d = {"a": 2, "b": 3}
x_len = len(x)
y_len = len(y)
d_len = len(d)
z_len = len(z)
return x_len, y_len, d_len, z_len
x_len, y_len, d_len, z_len = test()
print('x_len:{}'.format(x_len))
print('y_len:{}'.format(y_len))
print('d_len:{}'.format(d_len))
print('z_len:{}'.format(z_len))
The result is as follows:
x_len:3
y_len:3
d_len:2
z_len:6
Determines whether an object is an instance of a class. Different from operator Isinstance
, the second input parameter of Isinstance
is the type defined in the dtype
module of MindSpore.
Calling: isinstance(obj, type)
Input parameters:
obj
-- Any instance of any supported type.
type
-- A type in the MindSpore dtype
module.
Return value: If obj
is an instance of type
, return True
. Otherwise, return False
.
For example:
import mindspore as ms
import numpy as np
from mindspore import ms_function
z = ms.Tensor(np.ones((6, 4, 5)))
@ms_function()
def test():
x = (2, 3, 4)
y = [2, 3, 4]
x_is_tuple = isinstance(x, tuple)
y_is_list = isinstance(y, list)
z_is_tensor = isinstance(z, ms.Tensor)
return x_is_tuple, y_is_list, z_is_tensor
x_is_tuple, y_is_list, z_is_tensor = test()
print('x_is_tuple:{}'.format(x_is_tuple))
print('y_is_list:{}'.format(y_is_list))
print('z_is_tensor:{}'.format(z_is_tensor))
The result is as follows:
x_is_tuple:True
y_is_list:True
z_is_tensor:True
A partial function used to fix the input parameter of the function.
Calling: partial(func, arg, ...)
Input parameters:
func
--Function.
arg
-- One or more parameters to be fixed. Position parameters and key-value pairs can be specified.
Return value: functions with certain input parameter values fixed
For example:
from mindspore import ops
from mindspore import ms_function
def add(x, y):
return x + y
@ms_function()
def test():
add_ = ops.partial(add, x=2)
m = add_(y=3)
n = add_(y=5)
return m, n
m, n = test()
print('m:{}'.format(m))
print('n:{}'.format(n))
The result is as follows:
m:5
n:7
Maps one or more sequences based on the provided functions and generates a new sequence based on the mapping result. If the number of elements in multiple sequences is inconsistent, the length of the new sequence is the same as that of the shortest sequence.
Calling: map(func, sequence, ...)
Input parameters:
func
-- Function.
sequence
-- One or more sequences (Tuple
or List
).
Return value: A Tuple
For example:
from mindspore import ms_function
def add(x, y):
return x + y
@ms_function()
def test():
elements_a = (1, 2, 3)
elements_b = (4, 5, 6)
ret = map(add, elements_a, elements_b)
return ret
ret = test()
print('ret:{}'.format(ret))
The result is as follows:
ret: (5, 7, 9)
Packs elements in the corresponding positions in multiple sequences into tuples, and then uses these tuples to form a new sequence. If the number of elements in each sequence is inconsistent, the length of the new sequence is the same as that of the shortest sequence.
Calling: zip(sequence, ...)
Input parameter: sequence
-- One or more sequences (Tuple
or List
)`.
Return value: A Tuple
For example:
from mindspore import ms_function
@ms_function()
def test():
elements_a = (1, 2, 3)
elements_b = (4, 5, 6)
ret = zip(elements_a, elements_b)
return ret
ret = test()
print('ret:{}'.format(ret))
The result is as follows:
ret:((1, 4), (2, 5), (3, 6))
Creates a Tuple
based on the start value, end value, and step.
Calling:
range(start, stop, step)
range(start, stop)
range(stop)
Input parameters:
start
-- start value of the count. The type is int
. The default value is 0.
stop
-- end value of the count (exclusive). The type is int
.
step
-- Step. The type is int
. The default value is 1.
Return value: A Tuple
For example:
from mindspore import ms_function
@ms_function()
def test():
x = range(0, 6, 2)
y = range(0, 5)
z = range(3)
return x, y, z
x, y, z = test()
print('x:{}'.format(x))
print('y:{}'.format(y))
print('z:{}'.format(z))
The result is as follows:
x:(0, 2, 4)
y:(0, 1, 2, 3, 4)
z:(0, 1, 2)
Generates an index sequence of a sequence. The index sequence contains data and the corresponding subscript.
Calling:
enumerate(sequence, start)
enumerate(sequence)
Input parameters:
sequence
-- A sequence (Tuple
, List
, or Tensor
).
start
-- Start position of the subscript. The type is int
. The default value is 0.
Return value: A Tuple
For example:
import mindspore as ms
import numpy as np
from mindspore import ms_function
y = ms.Tensor(np.array([[1, 2], [3, 4], [5, 6]]))
@ms_function()
def test():
x = (100, 200, 300, 400)
m = enumerate(x, 3)
n = enumerate(y)
return m, n
m, n = test()
print('m:{}'.format(m))
print('n:{}'.format(n))
The result is as follows:
m:((3, 100), (4, 200), (5, 300), (6, 400))
n:((0, Tensor(shape=[2], dtype=Int64, value= [1, 2])), (1, Tensor(shape=[2], dtype=Int64, value= [3, 4])), (2, Tensor(shape=[2], dtype=Int64, value= [5, 6])))
Calls a method of the parent class (super class). Generally, the method of the parent class is called after super
.
Calling:
super().xxx()
super(type, self).xxx()
Input parameters:
type
-- Class.
self
-- Object.
Return value: method of the parent class.
For example:
from mindspore import nn
class FatherNet(nn.Cell):
def __init__(self, x):
super(FatherNet, self).__init__(x)
self.x = x
def construct(self, x, y):
return self.x * x
def test_father(self, x):
return self.x + x
class SingleSubNet(FatherNet):
def __init__(self, x, z):
super(SingleSubNet, self).__init__(x)
self.z = z
def construct(self, x, y):
ret_father_construct = super().construct(x, y)
ret_father_test = super(SingleSubNet, self).test_father(x)
return ret_father_construct, ret_father_test
Returns the power.
Calling: pow(x, y)
Input parameters:
x
-- Base number, Number
, or Tensor
.
y
-- Power exponent, Number
, or Tensor
.
Return value: y
power of x
, Number
, or Tensor
For example:
import mindspore as ms
import numpy as np
from mindspore import ms_function
x = ms.Tensor(np.array([1, 2, 3]))
y = ms.Tensor(np.array([1, 2, 3]))
@ms_function()
def test(x, y):
return pow(x, y)
ret = test(x, y)
print('ret:{}'.format(ret))
The result is as follows:
ret:[ 1 4 27]
Prints logs.
Calling: print(arg, ...)
Input parameter: arg
-- Information to be printed (int
, float
, bool
, String
or Tensor
).
When the arg
is int
, float
, or bool
, it will be printed out as a 0-D
tensor.
Return value: none
For example:
import mindspore as ms
import numpy as np
from mindspore import ms_function
x = ms.Tensor(np.array([1, 2, 3]), ms.int32)
y = ms.Tensor(3, ms.int32)
@ms_function()
def test(x, y):
print(x)
print(y)
return x, y
ret = test(x, y)
The result is as follows:
Tensor(shape=[3], dtype=Int32, value= [1 2 3])
3
According to the provided function to judge the elements of a sequence. Each element is passed into the function as a parameter in turn, and the elements whose return result is not 0 or False form a new sequence.
Calling: filter(func, sequence)
Input parameters:
func
-- Function.sequence
-- A sequence (Tuple
or List
).Return value: A Tuple
.
For example:
from mindspore import ms_function
def is_odd(x):
if x % 2:
return True
return False
@ms_function()
def test():
elements = (1, 2, 3, 4, 5)
ret = filter(is_odd, elements)
return ret
ret = test()
print('ret:{}'.format(ret))
The result is as follows:
ret:(1, 3, 5)
Default parameter value: The data types int
, float
, bool
, None
, str
, tuple
, list
, and dict
are supported, whereas Tensor
is not supported.
Variable parameters: Inference and training of networks with variable parameters are supported.
Key-value pair parameter: Functions with key-value pair parameters cannot be used for backward propagation on computational graphs.
Variable key-value pair parameter: Functions with variable key-value pairs cannot be used for backward propagation on computational graphs.
The input parameters of the outermost network only can be lool
, int
, float
, Tensor
, None
, mstype.number(mstype.bool, mstype.int, mstype.float, mstype.uint)
, List
or Tuple
that contains these types, and Dictionary
whose values are these types.
While calculating gradient for outermost network, only Tensor
input could be calculated, input of other type will be ignored. For example, input parameter (x, y, z)
of outermost network, x
and z
are Tensor
type, y
is other type. While calculating gradient for the network, only gradients of x
and z
are calculated, and (grad_x, grad_y)
is returned.
If you want to use other types of input for the network, please transfer them to the network while initializing network in the __init__
function, and save them as network attributes, then use in the construct
.
The input parameters of inner network do not have this restriction.
For example:
import mindspore as ms
from mindspore import nn, ops
import numpy as np
class Net(nn.Cell):
def __init__(self, flag):
super(Net, self).__init__()
self.flag = flag
def construct(self, x, y, z):
if self.flag == "ok":
return x + y + z
return x - y - z
class GradNet(nn.Cell):
def __init__(self, net):
super(GradNet, self).__init__()
self.grad_all = ops.GradOperation(get_all=True)
self.forward_net = net
def construct(self, x, y, z):
return self.grad_all(self.forward_net)(x, y, z)
flag = "ok"
input_x = ms.Tensor(np.ones((2, 3)).astype(np.float32))
input_y = 2
input_z = ms.Tensor(np.ones((2, 3)).astype(np.float32) * 2)
net = Net(flag)
grad_net = GradNet(net)
ret = grad_net(input_x, input_y, input_z)
print('ret:{}'.format(ret))
The result is as follows:
ret:(Tensor(shape=[2, 3], dtype=Float32, value=
[[ 1.00000000e+00, 1.00000000e+00, 1.00000000e+00],
[ 1.00000000e+00, 1.00000000e+00, 1.00000000e+00]]), Tensor(shape=[2, 3], dtype=Float32, value=
[[ 1.00000000e+00, 1.00000000e+00, 1.00000000e+00],
[ 1.00000000e+00, 1.00000000e+00, 1.00000000e+00]]))
In the Net
defined above, string
flag is transferred during initialization and saved as attribute self.flag
, then used in the construct
.
The input parameter x
and z
are Tensor
, y
is int
. While grad_net
calculates gradient of the input parameters (x, y, z)
for the outermost network, gradient of y
is automatically ignored, only the gradient of x
and z
is calculated, ret = (grad_x, grad_z)
.
Common Python function with the @ms_function decorator.
Cell subclass inherited from nn.Cell.
Category | Content |
---|---|
Cell instance |
mindspore/nn/* and user-defined Cell. |
Member function of a Cell instance |
Member functions of other classes in the construct function of Cell can be called. |
Primitive operator |
Class decorated with @ms_class. |
Composite operator |
mindspore/ops/operations/* |
constexpr generation operator |
mindspore/ops/composite/* |
constexpr 生成算子 |
Value computation operator generated by @constexpr. |
Function | User-defined Python functions and system functions listed in the preceding content. |
You are not allowed to modify non-Parameter
data members of the network.
For example:
import mindspore as ms
from mindspore import nn
import numpy as np
class Net(nn.Cell):
def __init__(self):
super(Net, self).__init__()
self.x = 2
self.par = ms.Parameter(ms.Tensor(np.ones((2, 3, 4))), name="par")
def construct(self, x, y):
self.par[0] = y
self.x = x
return x + y
net = Net()
net(1, 2)
In the preceding defined network, self.x
is not a Parameter
and cannot be modified. self.par
is a Parameter
and can be modified.
The result would be:
TypeError: 'self.x' should be initialized as a 'Parameter' type in the '__init__' function
When an undefined class member is used in the construct
function, AttributeError
is not thrown like the Python interpreter. Instead, it is processed as None
.
For example:
from mindspore import nn
class Net(nn.Cell):
def __init__(self):
super(Net, self).__init__()
def construct(self, x):
return x + self.y
net = Net()
net(1)
In the preceding defined network, construct
uses the undefined class member self.y
. In this case, self.y
is processed as None
.
The result would be:
RuntimeError: mindspore/ccsrc/frontend/operator/composite/multitype_funcgraph.cc:161 GenerateFromTypes] The 'add' operation does not support the type [Int64, kMetaTypeNone]
此处可能存在不合适展示的内容,页面不予展示。您可通过相关编辑功能自查并修改。
如您确认内容无涉及 不当用语 / 纯广告导流 / 暴力 / 低俗色情 / 侵权 / 盗版 / 虚假 / 无价值内容或违法国家有关法律法规的内容,可点击提交进行申诉,我们将尽快为您处理。