Tensor operation includes structural operation and mathematical operation.
The structural operation includes tensor creation, index slicing, dimension transform, combining & splitting, etc.
The mathematical operation includes scalar operation, vector operation, and matrix operation. We will also introduce the broadcasting mechanism of tensor operation.
This section is about the mathematical operation of tensor.
The mathematical operation includes scalar operation, vector operation, and matrix operation.
The scalar operation includes add, subtract, multiply, divide, power, and trigonometric functions, exponential functions, log functions, and logical comparison, etc.
The scalar operation is an element-by-element operation.
Some of the scalar operators are overloaded from the normal mathematical operators and support broadcasting similar as numpy.
Most scalar operators are under the module tf.math
.
import tensorflow as tf
import numpy as np
a = tf.constant([[1.0,2],[-3,4.0]])
b = tf.constant([[5.0,6],[7.0,8.0]])
a+b # Operator overloading
<tf.Tensor: shape=(2, 2), dtype=float32, numpy=
array([[ 6., 8.],
[ 4., 12.]], dtype=float32)>
a-b
<tf.Tensor: shape=(2, 2), dtype=float32, numpy=
array([[ -4., -4.],
[-10., -4.]], dtype=float32)>
a*b
<tf.Tensor: shape=(2, 2), dtype=float32, numpy=
array([[ 5., 12.],
[-21., 32.]], dtype=float32)>
a/b
<tf.Tensor: shape=(2, 2), dtype=float32, numpy=
array([[ 0.2 , 0.33333334],
[-0.42857143, 0.5 ]], dtype=float32)>
a**2
<tf.Tensor: shape=(2, 2), dtype=float32, numpy=
array([[ 1., 4.],
[ 9., 16.]], dtype=float32)>
a**(0.5)
<tf.Tensor: shape=(2, 2), dtype=float32, numpy=
array([[1. , 1.4142135],
[ nan, 2. ]], dtype=float32)>
a%3 # Reloading of mod operator, identical to: m = tf.math.mod(a,3)
<tf.Tensor: shape=(3,), dtype=int32, numpy=array([1, 2, 0], dtype=int32)>
a//3 # Divid and round towards negative infinity
<tf.Tensor: shape=(2, 2), dtype=float32, numpy=
array([[ 0., 0.],
[-1., 1.]], dtype=float32)>
(a>=2)
<tf.Tensor: shape=(2, 2), dtype=bool, numpy=
array([[False, True],
[False, True]])>
(a>=2)&(a<=3)
<tf.Tensor: shape=(2, 2), dtype=bool, numpy=
array([[False, True],
[False, False]])>
(a>=2)|(a<=3)
<tf.Tensor: shape=(2, 2), dtype=bool, numpy=
array([[ True, True],
[ True, True]])>
a==5 #tf.equal(a,5)
<tf.Tensor: shape=(3,), dtype=bool, numpy=array([False, False, False])>
tf.sqrt(a)
<tf.Tensor: shape=(2, 2), dtype=float32, numpy=
array([[1. , 1.4142135],
[ nan, 2. ]], dtype=float32)>
a = tf.constant([1.0,8.0])
b = tf.constant([5.0,6.0])
c = tf.constant([6.0,7.0])
tf.add_n([a,b,c])
<tf.Tensor: shape=(2,), dtype=float32, numpy=array([12., 21.], dtype=float32)>
tf.print(tf.maximum(a,b))
[5 8]
tf.print(tf.minimum(a,b))
[1 6]
# clip value
x = tf.constant([0.9,-0.8,100.0,-20.0,0.7])
y = tf.clip_by_value(x,clip_value_min=-1,clip_value_max=1)
z = tf.clip_by_norm(x,clip_norm = 3)
tf.print(y)
tf.print(z)
[0.9 -0.8 1 -1 0.7]
[0.0264732055 -0.0235317405 2.94146752 -0.588293493 0.0205902718]
Vector operation manipulate along one specific axis. It projects one vector to a scalar or another vector. Many names of vector operator starts with "reduce".
# Vector "reduce"
a = tf.range(1,10)
tf.print(tf.reduce_sum(a))
tf.print(tf.reduce_mean(a))
tf.print(tf.reduce_max(a))
tf.print(tf.reduce_min(a))
tf.print(tf.reduce_prod(a))
45
5
9
1
362880
# "reduce" along the specific dimension
b = tf.reshape(a,(3,3))
tf.print(tf.reduce_sum(b, axis=1, keepdims=True))
tf.print(tf.reduce_sum(b, axis=0, keepdims=True))
[[6]
[15]
[24]]
[[12 15 18]]
# "reduce" for bool type
p = tf.constant([True,False,False])
q = tf.constant([False,False,True])
tf.print(tf.reduce_all(p))
tf.print(tf.reduce_any(q))
0
1
# Implement tf.reduce_sum using tf.foldr
s = tf.foldr(lambda a,b:a+b,tf.range(10))
tf.print(s)
45
# Cumulative sum
a = tf.range(1,10)
tf.print(tf.math.cumsum(a))
tf.print(tf.math.cumprod(a))
[1 3 6 ... 28 36 45]
[1 2 6 ... 5040 40320 362880]
# Index of max and min values in the arguments
a = tf.range(1,10)
tf.print(tf.argmax(a))
tf.print(tf.argmin(a))
8
0
# Sort the elements in the tensor using tf.math.top_k
a = tf.constant([1,3,7,5,4,8])
values,indices = tf.math.top_k(a,3,sorted=True)
tf.print(values)
tf.print(indices)
# tf.math.top_k is able to implement KNN algorithm in TensorFlow
[8 7 5]
[5 2 3]
Matrix must be two-dimensional. Something such as tf.constant([1,2,3])
is not a matrix.
Matrix operation includes matrix multiply, transpose, inverse, trace, norm, determinant, eigenvalue, decomposition, etc.
Most of the matrix operations are in the tf.linalg
except for some popular operations.
# Matrix multiplication
a = tf.constant([[1,2],[3,4]])
b = tf.constant([[2,0],[0,2]])
a@b # Identical to tf.matmul(a,b)
<tf.Tensor: shape=(2, 2), dtype=int32, numpy=
array([[2, 4],
[6, 8]], dtype=int32)>
# Matrix transpose
a = tf.constant([[1.0,2],[3,4]])
tf.transpose(a)
<tf.Tensor: shape=(2, 2), dtype=float32, numpy=
array([[1., 3.],
[2., 4.]], dtype=float32)>
# Matrix inverse, must be in type of tf.float32 or tf.double
a = tf.constant([[1.0,2],[3.0,4]],dtype = tf.float32)
tf.linalg.inv(a)
<tf.Tensor: shape=(2, 2), dtype=float32, numpy=
array([[-2.0000002 , 1.0000001 ],
[ 1.5000001 , -0.50000006]], dtype=float32)>
# Matrix trace
a = tf.constant([[1.0,2],[3,4]])
tf.linalg.trace(a)
<tf.Tensor: shape=(), dtype=float32, numpy=5.0>
# Matrix norm
a = tf.constant([[1.0,2],[3,4]])
tf.linalg.norm(a)
<tf.Tensor: shape=(), dtype=float32, numpy=5.477226>
# Determinant
a = tf.constant([[1.0,2],[3,4]])
tf.linalg.det(a)
<tf.Tensor: shape=(), dtype=float32, numpy=-2.0>
# Eigenvalues
a = tf.constant([[1.0,2],[5,4]])
tf.linalg.eigvals(a)
<tf.Tensor: shape=(2,), dtype=complex64, numpy=array([-0.99999994+0.j, 5.9999995 +0.j], dtype=complex64)>
# QR decomposition
a = tf.constant([[1.0,2.0],[3.0,4.0]],dtype = tf.float32)
q,r = tf.linalg.qr(a)
tf.print(q)
tf.print(r)
tf.print(q@r)
[[-0.316227794 -0.948683321]
[-0.948683321 0.316227734]]
[[-3.1622777 -4.4271884]
[0 -0.632455349]]
[[1.00000012 1.99999976]
[3 4]]
# SVD decomposition
a = tf.constant([[1.0,2.0],[3.0,4.0],[5.0,6.0]], dtype = tf.float32)
s,u,v = tf.linalg.svd(a)
tf.print(u,"\n")
tf.print(s,"\n")
tf.print(v,"\n")
tf.print(u@tf.linalg.diag(s)@tf.transpose(v))
# SVD decomposition is used for dimension reduction in PCA
[[0.229847744 -0.88346082]
[0.524744868 -0.240782902]
[0.819642067 0.401896209]]
[9.52551842 0.51429987]
[[0.619629562 0.784894466]
[0.784894466 -0.619629562]]
[[1.00000119 2]
[3.00000095 4.00000048]
[5.00000143 6.00000095]]
The rules of broadcasting in TensorFlow is the same as numpy:
tf.broadcast_to
expand the dimension of tensor explicitly.
a = tf.constant([1,2,3])
b = tf.constant([[0,0,0],[1,1,1],[2,2,2]])
b + a # Identical to b + tf.broadcast_to(a,b.shape)
<tf.Tensor: shape=(3, 3), dtype=int32, numpy=
array([[1, 2, 3],
[2, 3, 4],
[3, 4, 5]], dtype=int32)>
tf.broadcast_to(a,b.shape)
<tf.Tensor: shape=(3, 3), dtype=int32, numpy=
array([[1, 2, 3],
[1, 2, 3],
[1, 2, 3]], dtype=int32)>
# Shape after broadcasting using static shape, requires arguments in TensorShape type
tf.broadcast_static_shape(a.shape,b.shape)
TensorShape([3, 3])
# Shape after broadcasting using dynamic shape, requires arguments in Tensor type
c = tf.constant([1,2,3])
d = tf.constant([[1],[2],[3]])
tf.broadcast_dynamic_shape(tf.shape(c),tf.shape(d))
<tf.Tensor: shape=(2,), dtype=int32, numpy=array([3, 3], dtype=int32)>
# Results of broadcasting
c+d # Identical to tf.broadcast_to(c,[3,3]) + tf.broadcast_to(d,[3,3])
<tf.Tensor: shape=(3, 3), dtype=int32, numpy=
array([[2, 3, 4],
[3, 4, 5],
[4, 5, 6]], dtype=int32)>
Please leave comments in the WeChat official account "Python与算法之美" (Elegance of Python and Algorithms) if you want to communicate with the author about the content. The author will try best to reply given the limited time available.
You are also welcomed to join the group chat with the other readers through replying 加群 (join group) in the WeChat official account.
此处可能存在不合适展示的内容,页面不予展示。您可通过相关编辑功能自查并修改。
如您确认内容无涉及 不当用语 / 纯广告导流 / 暴力 / 低俗色情 / 侵权 / 盗版 / 虚假 / 无价值内容或违法国家有关法律法规的内容,可点击提交进行申诉,我们将尽快为您处理。