# AdvancedEAST **Repository Path**: ztty8888/advanced-east ## Basic Information - **Project Name**: AdvancedEAST - **Description**: AdvancedEAST - **Primary Language**: Python - **License**: MIT - **Default Branch**: master - **Homepage**: None - **GVP Project**: No ## Statistics - **Stars**: 0 - **Forks**: 2 - **Created**: 2021-11-19 - **Last Updated**: 2021-11-19 ## Categories & Tags **Categories**: Uncategorized **Tags**: None ## README # AdvancedEAST ## introduction AdvancedEAST is inspired by EAST [EAST:An Efficient and Accurate Scene Text Detector](https://arxiv.org/abs/1704.03155v2). The architecture of AdvancedEAST is showed below![AdvancedEast network arch](AdvancedEast.network.png). This project is inherited by [huoyijie/AdvancedEAST](https://github.com/huoyijie/AdvancedEAST)(preprocess, network architecture, predict) and [BaoWentz/AdvancedEAST-PyTorch](https://github.com/BaoWentz/AdvancedEAST-PyTorch)(performance). This model contains features extractor, features merging branch, and output layers. The features extractor is VGG16, which is a sequence of convolution layers and pooling layers. Features merging branch composes nearest upsampling. ## Environment * Ubuntu 18.04 Server * python 3.7.5 ## Dependences * mindspore==1.1.1 * shapely==1.7.1 * numpy==1.19.4 * tqdm==4.36.1 * opencv==4.4.0.46 ## Project files ```bash ├── vgg16 ├── README.md // introduction of the project ├── scripts │ ├── run_standalone_train_gpu.sh // train on single GPU │ ├── run_distribute_train_gpu.sh // train on multi GPUs │ ├── run_eval.sh // evaluation ├── src │ ├── advanced_east.py // execute training │ ├── config.py // configuration of file, control parameters │ ├── dataset.py // dataset │ ├── label.py // label data │ ├── logging.py // logging │ ├── losses.py // define loss │ ├── model.py // define network │ ├── nms.py // nms │ ├── predict.py // predict verteices of image │ ├── preprocess.py // pre-process data, resize image │ ├── score.py // scoring ├── data_preprocess.py // preprocess data ├── train.py // training ├── eval.py // evaluation ``` ## dataset ICPR MTWI 2018 challenge 2：Text detection of network image，[Link](https://tianchi.aliyun.com/competition/entrance/231651/introduction). It is not available to download dataset on the origin webpage, the dataset is now provided by the author of the original project，[Baiduyun link](https://pan.baidu.com/s/1NSyc-cHKV3IwDo6qojIrKA), password: ye9y. There are 10000 images and corresponding label information in total in the dataset, which is divided into 2 directories with 9000 and 1000 samples respectively. In the origin training setting, training set and validation set are partitioned at the ratio of 9:1. If you want to use your own dataset, please modify the configuration of dataset in /src/config.py. The organization of dataset file is listed as below： > ```bash > . > └─data_dir > ├─images # dataset > └─txt # vertex of text boxes > ``` Some parameters in config.py： ```bash 'validation_split_ratio': 0.1, # ratio of validation dataset 'total_img': 9000, # total number of samples in dataset 'data_dir': './icpr/', # dir of dataset 'train_fname': 'train.txt', # the file which stores the images file name in training dataset 'val_fname': 'val.txt', # the file which stores the images file name in validation dataset 'mindsrecord_train_file': 'advanced-east.mindrecord', # mindsrecord of training dataset 'mindsrecord_test_file': 'advanced-east-val.mindrecord', # mindsrecord of validation dataset 'origin_image_dir_name': 'images_9000/', # dir which stores the original images. 'train_image_dir_name': 'images_train/', # dir which stores the preprocessed images. 'origin_txt_dir_name': 'txt_9000/', # dir which stores the original text verteices. 'train_label_dir_name': 'labels_train/', # dir which stores the preprocessed text verteices. ``` ## Quick Start After installing MindSpore via the official website and modify configuration of the project, you can start training and evaluation as follows: Running on GPU ```bash run training example python train.py --device_target="GPU" --device_id=[DEVICE_ID] > output.train.log 2>&1 & run distributed training example sh run_distribute_train_gpu.sh run evaluation example python eval.py --device_target="GPU" --device_id=[DEVICE_ID] --ckpt ckpt_filename --method DATASET_TYPE --path img_filename > output.eval.log 2>&1 & 2>&1 & ``` ## Run the project ### Data preprocess Resize all the images to fixed size, and convert the label information(the vertex of text box) into the format used in training and evaluation, then the Mindsrecord files are generated. To achieve this, you need to execute data_preprocess.py. ### Training #### Training on GPU(s) single GPU（1p） ```bash usage: train.py [--device_target TARGET][--is_distributed VALUE] [--device_id DEVICE_ID][--pre_trained PRE_TRAINED] [--ckpt_path CHECKPOINT_PATH][--ckpt_interval INTERVAL_STEP] parameters/options: --device_target the training backend type, Ascend or GPU, default is Ascend. --is_distributed the way of traing, whether do distribute traing, value can be 0 or 1. --device_id the device which used to train model. --pre_trained use pre_trained model or not. --ckpt_path the path to save output logging. --ckpt_interval the epoch interval for saving checkpoint. ``` multi GPUs ```bash # running on distributed environment（8p） bash scripts/run_distribute_train_gpu.sh ``` The detailed training parameters are in /src/config.py. config.py： ```bash 'initial_epoch': 0, # epoch to init 'epoch_num': 24, # number of epoch 'learning_rate': 1e-3, # learning rate when initialization 'decay': 5e-4, # weightdecay parameter 'epsilon': 1e-4, # the value of epsilon in loss computation 'batch_size': 8, # batch size 'ckpt_interval': 2, # interval to save ckpt files 'lambda_inside_score_loss': 4.0, # coef of inside_score_loss 'lambda_side_vertex_code_loss': 1.0, # coef of vertex_code_loss "lambda_side_vertex_coord_loss": 1.0, # coef of vertex_coord_loss 'max_train_img_size': 256, # max size of training images 'max_predict_img_size': 256, # max size of the images to predict 'ckpt_save_max': 10, # maximum of ckpt in dir 'saved_model_file_path': './saved_model/', # dir of saved model 'vgg_weights': './src/0-150_5004.ckpt', # path to load pretained weights ``` ## Evaluation ### Evaluate ```bash usage：eval.py [--device_target TARGET][--ckpt ckpt_filename] [--method DATASET_TYPE][--path img_filename] options： --device_target the evaluation backend type, Ascend or GPU, default is Ascend. --ckpt checkpoint filename --method method of evaluation --path if method is 'pred', this parameter is the path of image, which is required to feed in model to get verteices of image. ``` The above python command will run in the background, you can view the results through the file output.eval.log. ## performance ### Training performance The performance listed below are acquired with the default configurations in /src/config.py | Parameters | single GPU | 8 GPUs | | -------------------------- | ---------------------------------------------- |------------------------------------| | Model Version | AdvancedEAST | AdvancedEAST | | Resources | Tesla V100-32G |Tesla V100-32G x 8 | | MindSpore Version | 1.1.1 |1.1.1 | | Dataset | MTWI-2018 |MTWI-2018 | | Training Parameters | epoch=24, steps=1012, batch_size = 8, lr=1e-3 |epoch=144, steps=1012, batch_size = 8, lr=1e-3 | | Optimizer | AdamWeightDecay | AdamWeightDecay | | Loss Function | QuadLoss | QuadLoss | | Outputs | matrix with size of 3x64x64 | matrix with size of 3x64x64 | | Loss | 0.2 |0.2 | | Speed | 60 ms/step，60 s/epoch | 60 ms/step，8 s/epoch | | Total Time | 25.3 mins | 18.75mins | | Checkpoints | 172.3 MB（.ckpt file） | 172.3 MB（.ckpt file） | ### Evaluation Performance On the default setting. | Parameters | single GPU | 8 GPUs| | ------------------- | --------------------------- |---------------------| | Model Version | AdvancedEAST | AdvancedEAST | | Resources | Tesla V100-32G | Tesla V100-32G x 8 | | MindSpore Version | 1.1.1 |1.1.1 | | Dataset | 900 images | 900 images | | Batch_size | 8 | 8 | | Outputs | precision, recall, F score | precision, recall, F score | | Performance | 84.84% 35.35% 45.96% |84.74% 35.15% 45.83% |