Recognition of atherosclerotic plaque is premise of adopting treatment measures in clinic; Automatic recognition can overcome the effect of subjective factors. Characters of plaque are extracted by gray level co-occurrence matrix, and then SVM is applied to construct classifier combining PSO algorithm, which optimizes parameters of SVM based on Gaussian radius basis kernel function. The result shows that rate of recognition reaches 92%.The method improves accuracy, scientific and objectivity in diagnosing cardio-cerebrovascular disease in clinic.