Short time Fourier transform and wavelet transform are not effective in extracting features of electrocardiogram (ECG) signal for arrhythmia detection. Therefore,a novel algorithm based on the feature selection of S-transform is proposed for arrhythmia classification. First, ECG signals are processed by S-transform, and the time-frequency features are extracted from both the amplitude and the phase of ST results. Then, time-frequency features, morphological features, and RR interval are combined as the original feature vector. Second, the genetic algorithm (GA) and support vector machine (SVM) are combined as a Wrapper approach to search an optimal feature subset. The feature weights are computed by ReliefF algorithm, and the initialization of genetic population depends on the feature weights. Moreover,GA searches an optimal feature subset using classification performance as the fitness function. Finally, a multi-SVM model with one against all (OAA) strategy is built for the classification of eight types of ECG beats from the MIT-BIH arrhythmia database. Experimental results indicate that the proposed approach has the best performance among other state-of-the-art approaches, and the sensitivity, specificity, and accuracy reach 96.14%, 99.75%, and 99.81%, respectively.