# CarND_behavioral_cloning **Repository Path**: software_test/CarND_behavioral_cloning ## Basic Information - **Project Name**: CarND_behavioral_cloning - **Description**: 行为克隆网络，基于Nvida论文，使用udacity仿真器 - **Primary Language**: Python - **License**: Not specified - **Default Branch**: master - **Homepage**: None - **GVP Project**: No ## Statistics - **Stars**: 3 - **Forks**: 0 - **Created**: 2019-11-18 - **Last Updated**: 2022-08-01 ## Categories & Tags **Categories**: Uncategorized **Tags**: None ## README **Behavioral Cloning Project** The goals / steps of this project are the following: * Use the simulator to collect data of good driving behavior * Build, a convolution neural network in Keras that predicts steering angles from images * Train and validate the model with a training and validation set * Test that the model successfully drives around track one without leaving the road * Summarize the results with a written report ## [Rubric Points](https://review.udacity.com/#!/rubrics/432/view) --- ## Files Submitted & Code Quality ### 1. Submission includes all required files My project includes the following files: * [model.ipynb](./model.ipynb) containing the script to create and train the model * [drive.py](./drive.py) for driving the car in autonomous mode. I added the image preprocessing in this file at line 66. * [model.h5](./model.h5) containing a trained convolution neural network. * README.md summarizing the results * [video.mp4](./video.mp4) recording the performance of the model in the simulator. * [helper.py](./helper.py) containing all the image processing, augmentation and batch generator functions. * [project_env.yml](./project_env.yml) discribing the environment I trained the model. You can import it with `conda env create -f project_env.yml` ### 2. Submission includes functional code Using the Udacity provided simulator and my drive.py file, the car can be driven autonomously around the track by executing ```sh python drive.py model.h5 ``` You can check the performance of this model by watching [video.mp4](./video.mp4) ### 3. Submission code is usable and readable The [model.ipynb](./model.ipynb) file contains the code for training and saving the convolution neural network. The file shows the pipeline I used for training and validating the model, and it contains comments to explain how the code works. ## Model Architecture and Training Strategy ### 1. An appropriate model architecture has been employed My model consists of 5 convolutional layers and 4 fully-connected layers (4th code cell in [model.ipynb](./model.ipynb)).The structure of my network is based on the [NVIDIA model](https://devblogs.nvidia.com/deep-learning-self-driving-cars/), which was designed for end-to-end deep learning of self driving car. But there are some differences between NVIDIA model and mine. * 1.I added max pooling layers after every convolutional layer to reduce the quantity of parameters. * 2.I added a dropout layer after all the convolutional layers to avoid overfitting and accelerate training. The detailed parameters of the model are shown in the following table. |Layer |Kernel Size |Number of Kernels|Output Shape |Param| |:-------------:|:-------------:|:-------------:|:-------------:|:-------------:| |Lambda |/|/|(batch_size, 66, 200, 3) |0| |Conv2D_1 |(5, 5)|24|(batch_size, 33, 100, 24) |1824| |ReLu |/|/|(batch_size, 33, 100, 24) |0| |MaxPooling2D |(2, 2)|/|(batch_size, 32, 99, 24) |0| |Conv2D_2 |(5, 5)|36|(batch_size, 16, 50, 36) |21636| |ReLu |/|/|(batch_size, 16, 50, 36) |0| |MaxPooling2D |(2, 2)|/|(batch_size, 15, 49, 36) |0| |Conv2D_3 |(5, 5)|48|(batch_size, 8, 25, 48) |43248| |ReLu |/|/|(batch_size, 8, 25, 48) |0| |MaxPooling2D |(2, 2)|/|(batch_size, 7, 24, 48) |0| |Conv2D_4 |(3, 3)|64|(batch_size, 7, 24, 64) |27712| |Relu |/|/|(batch_size, 7, 24, 64) |0| |MaxPooling2D |(2, 2)|/|(batch_size, 6, 23, 64) |0| |Conv2D_5 |(3, 3)|64|(batch_size, 6, 23, 64) |36928| |ReLu |/|/|(batch_size, 6, 23, 64) |0| |MaxPooling2D |(2, 2)|/|(batch_size, 5, 22, 64) |0| |Dropout |/|/|(batch_size, 5, 22, 64) |0| |Flatten |/|/|(batch_size, 7040) |0| |Dense_1 |/|/|(batch_size, 100) |704100| |ReLu |/|/|(batch_size, 100) |0| |Dense_2 |/|/|(batch_size, 50) |5050| |ReLu |/|/|(batch_size, 50) |0| |Dense_3 |/|/|(batch_size, 10) |510| |ReLu |/|/|(batch_size, 10) |0| |Dense_4 |/|/|(batch_size, 1) |11| |Total params: |841,019| |Trainable params: |841,019| There is a ReLu layer after every convolutional layer and fully-connected layer (except the last one) to introduce nonlinearity, and the data is normalized in the model using a Keras lambda layer (3rd line of 4th code cell in [model.ipynb](./model.ipynb)). ### 2. Attempts to reduce overfitting in the model The model contains dropout layers in order to reduce overfitting (24th line of 4th code cell in [model.ipynb](./model.ipynb)). The model was trained on a random augmented data set to ensure that the model was not overfitting (code line 21-125 of [helper.py](./helper.py)). The model was tested by running it through the simulator and ensuring that the vehicle could stay on the track. ### 3. Model parameter tuning The model used an adam optimizer, so the learning rate was not tuned manually (2nd line of 5th code cell in [model.ipynb](./model.ipynb)). ### 4. Appropriate training data I use the [training data provided by Udacity](https://d17h27t6h515a5.cloudfront.net/topher/2016/December/584f6edd_data/data.zip), which consists of photos taken by center, left and right camera with 8036 defferent timestamps. Then I design the pre-processing and augmentation functions. The pre-processing process is as follows. * 1.Crop 70 rows of pixels upper and 25 rows lower, only leave road area on the image (line 14 in [helper.py](./helper.py)). * 2.Resize the image to (200,66) (line 16 in [helper.py](./helper.py)). * 3.Convert the color space from RGB to HSV (line 18 in [helper.py](./helper.py)). Then I test the preprocessing() function in the 4th code cell of [test.ipynb](./test.ipynb). The result is shown below. [image1]: ./imgs_readme/2.croped.png "Pre-processed" ![alt text][image1] All the training images, validation images and photos taken by center camera in autonomous mode would be pre-processed. I added the pre-processing in [drive.py](./drive.py) (line 66). In addition, to generalize the model I applied augmentation on traing set. The augmentation process is as follows. * 1.Randomly load the center, left or right image of the same timestamp (line 24-33 in [helper.py](./helper.py)). Adjust the steering angle by +0.2/-0.2 for left/right image. * 2.Randomly flip the image with a probability of one-half (line 36-41 in [helper.py](./helper.py)). [image2]: ./imgs_readme/3.fliped.png "Fliped" ![alt text][image2] * 3.Randomly add shadow on the image by adjust the V channel in HSV color space (line 45-61 in [helper.py](./helper.py)). [image3]: ./imgs_readme/4.shadowed.png "Shadowed" ![alt text][image3] * 4.Randomly adjust the gamma value of the image with cv2.LUT() (line 65-70 in [helper.py](./helper.py)). [image4]: ./imgs_readme/5.gammaed.png "Gammaed" ![alt text][image4] * 5.Randomly shift the image horizontally or vertically (line 73-88 in [helper.py](./helper.py)). [image5]: ./imgs_readme/6.shifted.png "Shifted" ![alt text][image5] And then I designed the generator (line 104-125 in [helper.py](./helper.py)), which will receive a list of image paths, a list of steering angles and the batch size (64 as default). All data will be shuffled, pre-processed and half of them will be augmented randomly. It will yield a batch of augmented data when next() is applied. ### 5. Training. I chose Adam as the optimizer and mean-squared-error as the loss function. The model was trained for 10 epoches (125 batches for each epoch). ### 6. Result. I recorded the performance of the model in the simulator and saved as [video.mp4](./video.mp4).