# week7 **Repository Path**: sika0819/week7 ## Basic Information - **Project Name**: week7 - **Description**: 第7周作业 - **Primary Language**: Python - **License**: Not specified - **Default Branch**: master - **Homepage**: None - **GVP Project**: No ## Statistics - **Stars**: 0 - **Forks**: 0 - **Created**: 2019-05-29 - **Last Updated**: 2020-12-19 ## Categories & Tags **Categories**: Uncategorized **Tags**: None ## README 导入包 ```python import numpy as np import tensorflow as tf from tensorflow.examples.tutorials.mnist import input_data from matplotlib import pyplot as plt %matplotlib inline tf.logging.set_verbosity(tf.logging.INFO) ``` 看一下数据大小，训练集55000条，校验集5000条，测试集10000条 ```python mnist = input_data.read_data_sets("./") print(mnist.train.images.shape) print(mnist.train.labels.shape) print(mnist.validation.images.shape) print(mnist.validation.labels.shape) print(mnist.test.images.shape) print(mnist.test.labels.shape) ``` WARNING:tensorflow:From :1: read_data_sets (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version. Instructions for updating: Please use alternatives such as official/mnist/dataset.py from tensorflow/models. WARNING:tensorflow:From C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\contrib\learn\python\learn\datasets\mnist.py:260: maybe_download (from tensorflow.contrib.learn.python.learn.datasets.base) is deprecated and will be removed in a future version. Instructions for updating: Please write your own downloading logic. WARNING:tensorflow:From C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\contrib\learn\python\learn\datasets\mnist.py:262: extract_images (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version. Instructions for updating: Please use tf.data to implement this functionality. Extracting ./train-images-idx3-ubyte.gz WARNING:tensorflow:From C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\contrib\learn\python\learn\datasets\mnist.py:267: extract_labels (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version. Instructions for updating: Please use tf.data to implement this functionality. Extracting ./train-labels-idx1-ubyte.gz Extracting ./t10k-images-idx3-ubyte.gz Extracting ./t10k-labels-idx1-ubyte.gz WARNING:tensorflow:From C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\contrib\learn\python\learn\datasets\mnist.py:290: DataSet.__init__ (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version. Instructions for updating: Please use alternatives such as official/mnist/dataset.py from tensorflow/models. (55000, 784) (55000,) (5000, 784) (5000,) (10000, 784) (10000,) 打印16张训练集的图片，显示图片带有的标签，，每张图片是28x28长度的一个一维向量（28*28=784），图片大小为28*28 ```python plt.figure(figsize=(8,8)) for idx in range(16): plt.subplot(4,4, idx+1) plt.axis('off') plt.title('[{}]'.format(mnist.train.labels[idx])) plt.imshow(mnist.train.images[idx].reshape((28,28))) ``` ![png](output_5_0.png) 这里我们直接使用上面的数据作为输入，所以定义两个placeholder分别用于图像和lable数据，另外，定义一个float类型的变量用于设置学习率。 为了让网络更高效的运行，多个数据会被组织成一个batch送入网络，两个placeholder的第一个维度就是batchsize，因为我们这里还没有确定batchsize，所以第一个维度留空。 ```python x = tf.placeholder("float", [None, 784])#定义输入，输入数据是图片的784列一维数据，定义一个占位符，后面把数据输入进来 y = tf.placeholder("int64", [None])#输出数据自然是整数型的标签 learning_rate = tf.placeholder("float")#定义学习率 def initialize(shape, stddev=0.1):#定义初始化函数。 return tf.truncated_normal(shape, stddev=0.1)#用truncated_normal函数进行初始化，truncated_normal是截断正态分布，会删除大于2个stddev的x值 L1_units_count = 100 #第一层神经元的个数 W_1 = tf.Variable(initialize([784, L1_units_count])) #variable主要用于数据存储,在计算图的运算过程中，其值会一直保存到程序运行结束 b_1 = tf.Variable(initialize([L1_units_count]))#比如神经网络中的权重和bias等，在训练过后，总是希望这些参数能够保存下来，而不是直接就消失了，所以这个时候要用到Variable logits_1 = tf.matmul(x, W_1) + b_1 #logits函数，即y=wx+b在tensorflow中计算图的定义方法 output_1 = tf.nn.relu(logits_1) #激活函数为relu ``` WARNING:tensorflow:From C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\framework\op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version. Instructions for updating: Colocations handled automatically by placer. ```python L2_units_count = 10 #最终输出的结果总过有10个 W_2 = tf.Variable(initialize([L1_units_count, L2_units_count])) b_2 = tf.Variable(initialize([L2_units_count])) logits_2 = tf.matmul(output_1, W_2) + b_2 logits = logits_2 ``` 接下来定义损失函数和优化器 ```python cross_entropy_loss = tf.reduce_mean( tf.nn.sparse_softmax_cross_entropy_with_logits(logits=logits, labels=y))#损失函数是交叉熵损失 optimizer = tf.train.GradientDescentOptimizer( #定义优化方法为梯度下降 learning_rate=learning_rate).minimize(cross_entropy_loss) ``` 需要注意的是，上面的网络，最后输出的是未经softmax的原始logits，而不是概率分布， 要想看到概率分布，还需要做一下softmax。 将输出的结果与正确结果进行对比，即可得到我们的网络输出结果的准确率。 ```python pred = tf.nn.softmax(logits)# 原始logit用softmax转换一下 correct_pred = tf.equal(tf.argmax(pred, 1), y)# 预测的准确率 accuracy = tf.reduce_mean(tf.cast(correct_pred, tf.float32))#取平均值得到最终的准确率 ``` saver用于保存或恢复训练的模型。 ```python batch_size = 32 #一轮喂入的数据为32个 trainig_step = 1000 #一轮训练训练1000个step saver = tf.train.Saver()#模型保存 ``` 所有的基本定义到这里就完成了，接下来就可以正式开始定义计算图了 ```python with tf.Session() as sess: sess.run(tf.global_variables_initializer()) #定义验证集与测试集 validate_data = { x: mnist.validation.images, y: mnist.validation.labels, } test_data = {x: mnist.test.images, y: mnist.test.labels} for i in range(trainig_step): xs, ys = mnist.train.next_batch(batch_size) _, loss = sess.run( [optimizer, cross_entropy_loss], feed_dict={ x: xs, y: ys, learning_rate: 0.3 }) #每100次训练打印一次损失值与验证准确率 if i > 0 and i % 100 == 0: validate_accuracy = sess.run(accuracy, feed_dict=validate_data) print( "after %d training steps, the loss is %g, the validation accuracy is %g" % (i, loss, validate_accuracy)) saver.save(sess, './model.ckpt', global_step=i) print("the training is finish!") #最终的测试准确率 acc = sess.run(accuracy, feed_dict=test_data) print("the test accuarcy is:", acc) ``` after 100 training steps, the loss is 0.351709, the validation accuracy is 0.882 after 200 training steps, the loss is 0.0963676, the validation accuracy is 0.9088 after 300 training steps, the loss is 0.284616, the validation accuracy is 0.9212 after 400 training steps, the loss is 0.128543, the validation accuracy is 0.9304 after 500 training steps, the loss is 0.285667, the validation accuracy is 0.9254 after 600 training steps, the loss is 0.111476, the validation accuracy is 0.937 WARNING:tensorflow:From C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\training\saver.py:966: remove_checkpoint (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version. Instructions for updating: Use standard file APIs to delete files with this prefix. after 700 training steps, the loss is 0.0604077, the validation accuracy is 0.9458 after 800 training steps, the loss is 0.262678, the validation accuracy is 0.9498 after 900 training steps, the loss is 0.273349, the validation accuracy is 0.947 the training is finish! the test accuarcy is: 0.9456 下面用训练好的模型做一个测试 ```python with tf.Session() as sess: ckpt = tf.train.get_checkpoint_state('./') if ckpt and ckpt.model_checkpoint_path: saver.restore(sess, ckpt.model_checkpoint_path) final_pred, acc = sess.run( [pred, accuracy], feed_dict={ x: mnist.test.images[:16], y: mnist.test.labels[:16] }) orders = np.argsort(final_pred) plt.figure(figsize=(8, 8)) print(acc) for idx in range(16): order = orders[idx, :][-1] prob = final_pred[idx, :][order] plt.subplot(4, 4, idx + 1) plt.axis('off') plt.title('{}: [{}]-[{:.1f}%]'.format(mnist.test.labels[idx], order, prob * 100)) plt.imshow(mnist.test.images[idx].reshape((28, 28))) else: pass ``` WARNING:tensorflow:From C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\training\saver.py:1266: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version. Instructions for updating: Use standard file APIs to check for files with this prefix. INFO:tensorflow:Restoring parameters from ./model.ckpt-900 0.9375 ![png](output_18_1.png) ``` 可以看到结果比较差。推测原因：可能是由于神经层数，神经元数目也比较少，学习轮数比较少。增加神经网络的深度，增加训练轮数可能会提高效果。 ```