In this paper we introduce the latest and future progress in speech emotion recognition, especially on the practical speech emotion research considering the real world applications. We mainly discuss: the history and development of affective computing research, the practical applications in affective computing, and review the general speech emotion recognition methods, including emotion model, emotion database, feature extraction and emotion recognition algorithm. Consider the needs in real world applications, we focus on the key methods in practical speech emotion research. We analysis the current challenges in practical speech emotion recognition, especially for the fidgetiness emotion, we review the methods in database establishment, feature analysis and modeling techniques. Finally, we give the outlook of future speech emotion research, and discuss the future challenges and possible solutions.