An improved algorithm of speech emotion recognition is proposed based on modified multiple kernel learning. The algorithm based on Gaussian radial basis function improves classification performance by gathering different samples,utilizing different evaluation criteria and acquiring different parameters .In addition, with the help of multiple kernel technology, the trained Gauss kernel functions are used to construct the basis of multiple kernel learning and the efficiency of learning are greatly improved by utilizing the relaxation factor to construct objective function of soft margin .Experimental results show the presented algorithm can effectively improve the performance of speech emotion recognition.