43 Star 209 Fork 45

AI柠檬 / ASRT_SpeechRecognition

Create your Gitee Account
Explore and code with more than 8 million developers,Free private repositories !:)
Sign up
Clone or Download
Notice: Creating folder will generate an empty file .keep, because not support in Git

ASRT: A Deep-Learning-Based Chinese Speech Recognition System

GPL-3.0 Licensed TensorFlow Version Python Version

ReadMe Language | 中文版 | English |

ASRT Project Home Page | Released Download | View this project's wiki document (Chinese) | Experience Demo | Donate

If you have any questions in your works with this project, welcome to put up issues in this repo and I will response as soon as possible.

You can check the FAQ Page (Chinese) first before asking questions to avoid repeating questions.

A post about ASRT's introduction

About how to use ASRT to train and deploy:

For questions about the principles of the statistical language model that are often asked, see:

For questions about CTC, see:

For more infomation please refer to author's blog website: AILemon Blog (Chinese)


This project uses tensorFlow.keras based on deep convolutional neural network and long-short memory neural network, attention mechanism and CTC to implement.

  • Steps

First, clone the project to your computer through Git, and then download the data sets needed for the training of this project. For the download links, please refer to End of Document

$ git clone https://github.com/nl8590687/ASRT_SpeechRecognition.git

Or you can use the "Fork" button to copy a copy of the project and then clone it locally with your own SSH key.

After cloning the repository via git, go to the project root directory; create a subdirectory dataset/ (you can use a soft link instead) for datasets, and then extract the downloaded datasets directly into it.

$ cd ASRT_SpeechRecognition

$ mkdir dataset

$ tar zxf <dataset zip files name> -C dataset/ 

Then, you need to copy all the files in the 'datalist' directory to the dataset directory, that is, put them together with the data set.

Note that in the current version, in the configuration file, two data sets, Thchs30 and ST-CMDS, are added by default, please delete them if you don’t need them. If you want to use other data sets, you need to add data configuration yourself, and use the standard format supported by ASRT to organize the data in advance.

$ cp -rf datalist/* dataset/

Currently available models are 24, 25 and 251

Before running this project, please install the necessary Python3 version dependent library

To start training this project, please execute:

$ python3 train_speech_model.py

To start the test of this project, please execute:

$ python3 evaluate_speech_model.py

Before testing, make sure the model file path filled in the code files exists.

ASRT API Server startup please execute:

$ python3 asrserver.py

Please note that after opening the API server, you need to use the client software corresponding to this ASRT project for voice recognition. For details, see the Wiki documentation to download ASRT Client Demo.

If you want to train and use other model(not Model 251), make changes in the corresponding position of the import speech_model_zoo in the code files.

If there is any problem during the execution of the program or during use, it can be promptly put forward in the issue, and I will reply as soon as possible.

Deploy ASRT by docker:

$ docker pull ailemondocker/asrt_service:1.1.0
$ docker run --rm -it -p 20000:20000 --name asrt-server -d ailemondocker/asrt_service:1.1.0

It will start a api server for recognition rather than training.


Speech Model


The maximum length of the input audio is 16 seconds, and the output is the corresponding Chinese pinyin sequence.

  • Questions about downloading trained models

The released finished software that includes trained model weights can be downloaded from ASRT download page.

Github Releases page includes the archives of the various versions of the software released and it's introduction. Under each version module, there is a zip file that includes trained model weights files.

Language Model

Maximum Entropy Hidden Markov Model Based on Probability Graph.

The input is a Chinese pinyin sequence, and the output is the corresponding Chinese character text.

About Accuracy

At present, the best model can basically reach 80% of Pinyin correct rate on the test set.

However, as the current international and domestic teams can achieve 98%, the accuracy rate still needs to be further improved.

Python Dependency Library

  • tensorFlow (1.15 - 2.x)
  • numpy
  • wave
  • matplotlib
  • math
  • scipy
  • requests

If you have trouble when install those packages, please run the following script to do it as long as you have a GPU and CUDA 11.2 and cudnn 8.1 have been installed:

$ pip install -r requirements.txt

Dependent Environment Details and Hardware Requirement

Data Sets

Some free Chinese speech datasets (Chinese)

  • Tsinghua University THCHS30 Chinese voice data set

    data_thchs30.tgz Download

    test-noise.tgz Download

    resource.tgz Download

  • Free ST Chinese Mandarin Corpus

    ST-CMDS-20170001_1-OS.tar.gz Download

  • AIShell-1 Open Source Dataset

    data_aishell.tgz Download

    Note:unzip this dataset

    $ tar xzf data_aishell.tgz
    $ cd data_aishell/wav
    $ for tar in *.tar.gz;  do tar xvf $tar; done
  • Primewords Chinese Corpus Set 1

    primewords_md_2018_set1.tar.gz Download

  • aidatatang_200zh

    aidatatang_200zh.tgz Download

  • MagicData

    train_set.tar.gz Download

    dev_set.tar.gz Download

    test_set.tar.gz Download

    metadata.tar.gz Download

Special thanks! Thanks to the predecessors' public voice data set.

If the provided dataset link cannot be opened and downloaded, click this link OpenSLR


GPL v3.0 © nl8590687 Author: ailemon

Cite this project

DOI: 10.5281/zenodo.5808435


Contributors Page

@nl8590687 (repo owner)

Repository Comments ( 18 )

Sign in to post a comment


A Deep-Learning-Based Chinese Speech Recognition System 基于深度学习的中文语音识别系统 expand collapse
Python and 2 more languages


No release





Load More
can not load any more