Fuzzy support vector machine (FSVM) is used on road roughness recognition. The general SVM is particularly sensitive to the noise points and outliers in the samples, so a method is proposed, in which the distance from sample to the center of class is taken as the fuzzy membership of the sample and the parameters of FSVM are optimized by improved particle swarm optimization (PSO) algorithm. After training and testing the experimental data, the highest average recognition rate increases to 77.5%, which is higher than 72.5% that of the method with the general support vector machine. Data processing indicates that FSVM strengthens the influence of effective samples on classification and weaken influence of noise points and outliers. Furthermore, the recognition rate of road roughness has been improved.