335 Star 1.5K Fork 862

MindSpore / docs

Create your Gitee Account
Explore and code with more than 12 million developers,Free private repositories !:)
Sign up
Clone or Download
BatchNorm2d.md 5.70 KB
Copy Edit Raw Blame History
luojianing authored 2023-07-21 15:16 . replace target=blank

Function Differences with tf.keras.layers.BatchNormalization

View Source On Gitee

tf.keras.layers.BatchNormalization

tf.keras.layers.BatchNormalization(
    axis=-1,
    momentum=0.99,
    epsilon=0.001,
    center=True,
    scale=True,
    beta_initializer='zeros',
    gamma_initializer='ones',
    moving_mean_initializer='zeros',
    moving_variance_initializer='ones',
    beta_regularizer=None,
    gamma_regularizer=None,
    beta_constraint=None,
    gamma_constraint=None,
    **kwargs
)(inputs, training) -> Tensor

For more information, see tf.keras.layers.BatchNormalization.

mindspore.nn.BatchNorm2d

class mindspore.nn.BatchNorm2d(
    num_features,
    eps=1e-5,
    momentum=0.9,
    affine=True,
    gamma_init='ones',
    beta_init='zeros',
    moving_mean_init='zeros',
    moving_var_init='ones',
    use_batch_statistics=None,
    data_format='NCHW'
)(x) -> Tensor

For more information, see mindspore.nn.BatchNorm2d.

Differences

TensorFlow: Perform batch normalization on the input data.

MindSpore: Batch Normalization Layer (BNL) is applied to the input four-dimensional data. Batch normalization is applied on four-dimensional inputs (small batches of two-dimensional inputs with additional channel dimensionality) to avoid internal covariate shifts.

Categories Subcategories TensorFlow MindSpore Differences
Parameters Parameter 1 axis - The axis that should be normalized (usually the feature axis). MindSpore does not have this parameter
Parameter 2 momentum momentum -
Parameter 3 epsilon eps Same function, different parameter names
Parameter 4 center - If True, the offset beta is added to the normalization tensor. If False, beta is ignored. MindSpore does not have this parameter
Parameter 5 scale - If True, multiply by gamma. if False, do not use gamma. MindSpore does not have this parameter
Parameter 6 beta_initializer beta_init Same function, different parameter names
Parameter 7 gamma_initializer gamma_init Same function, different parameter names
Parameter 8 moving_mean_initializer moving_mean_init Same function, different parameter names
Parameter 9 moving_variance_initializer moving_var_init Same function, different parameter names
Parameter 10 beta_regularizer - Optional regularizer for beta weights. MindSpore does not have this parameter
Parameter 11 gamma_regularizer - Optional regularizer for gamma weights. MindSpore does not have this parameter
Parameter 12 beta_constraint - Optional constraint on the beta weight. MindSpore does not have this parameter
Parameter 13 gamma_constraint - Optional constraint on gamma weights. MindSpore does not have this parameter
Parameter 14 **kwargs - Not involved
Parameter 15 - num_features Number of channels, input C in Tensor shape (N,C,H,W)(N,C,H,W). TensorFlow does not have this parameter
Parameter 16 - affine bool type. When set to True, the γ and β values can be learned. Default value: True. TensorFlow does not have this parameter
Parameter 17 - use_batch_statistics If True, the mean and variance values of the current batch data are used, and the running mean and running variance are tracked.
If False, the mean and variance values of the specified values are used and no statistical values are tracked.
If None, use_batch_statistics is automatically set to True or False depending on the training and validation modes. use_batch_statistics is set to True for training. use_batch_statistics is automatically set to False for validation. Default value: None. TensorFlow does not have this parameter
Parameter 18 - data_format MindSpore can specify the input data format as "NHWC" or "NCHW". Default value: "NCHW". TensorFlow does not have this parameter
Inputs Input 1 inputs x Same function, different parameter names
Input 2 training - Not involved

Code Example

Both APIs have the same function and are used in the same way.

# TensorFlow
import tensorflow as tf

inputx = [[[[1, 2],
           [2, 1]],
          [[3, 4],
           [4, 3]]],
         [[[5, 6],
           [6, 5]],
          [[7, 8],
           [8, 7]]]]
input_tf = tf.constant(inputx, dtype=tf.float32)
output_tf = tf.keras.layers.BatchNormalization(axis=3, momentum=0.1, epsilon=1e-5)
output = output_tf(input_tf, training=False)
print(output.numpy())
# [[[[0.999995  1.99999  ]
#    [1.99999   0.999995 ]]
#
#   [[2.999985  3.99998  ]
#    [3.99998   2.999985 ]]]
#
#
#  [[[4.999975  5.99997  ]
#    [5.99997   4.999975 ]]
#
#   [[6.9999647 7.99996  ]
#    [7.99996   6.9999647]]]]

# MindSpore
from mindspore import Tensor, nn
import numpy as np

m = nn.BatchNorm2d(num_features=2, momentum=0.9)
input_ms = Tensor(np.array([[[[1, 2], [2, 1]],
                          [[3, 4], [4, 3]]],
                          [[[5, 6], [6, 5]],
                          [[7, 8], [8, 7]]]]).astype(np.float32))
output = m(input_ms)
print(output)
# [[[[0.99999493 1.9999899 ]
#    [1.9999899  0.99999493]]
#
#   [[2.9999847  3.9999797 ]
#    [3.9999797  2.9999847 ]]]
#
#
#  [[[4.9999747  5.9999695 ]
#    [5.9999695  4.9999747 ]]
#
#   [[6.9999647  7.9999595 ]
#    [7.9999595  6.9999647 ]]]]
1
https://gitee.com/mindspore/docs.git
git@gitee.com:mindspore/docs.git
mindspore
docs
docs
r2.0

Search

53164aa7 5694891 3bd8fe86 5694891