In this paper, deep learning is briefly introduced. Then, a review of the research progress of deep learning based speech recognition is presented from the following five points: Training criterions for deep learning based acoustic models, different model architectures for deep learning based speech recognition acoustic modeling, scalable and distributed optimization methods for deep learning based acoustic model training, speaker adaptation for deep learning based acoustic model, and deep leaning based end-to-end speech recognition. At the end of this paper, the future possible research points of deep learning based speech recognition are also proposed.