Spectral clustering transforms data clustering problem into a graph partitioning problem and classifies data points by finding the optimal sub-graphs. The key to spectral clustering is constructing a suitable similarity matrix, which can truly describe the intrinsic structure of the dataset. However, traditional spectral clustering algorithms adopt Gaussian kernel function to construct the similarity matrix, which results in their sensitivity of selection for scale parameter. In addition, the initial cluster centers need randomly determing at the clustering stage and the clustering performance is not stable. The paper presents an algorithm based on message passing. The algorithm uses a density adaptive similarity measure, which can well describe the relations between data points, and it can obtain high-quality cluster centers through message passing mechanism in affinity propagation (AP) clustering. Moreover, the performance of clustering is optimized by the method. Experiments show that the proposed algorithm can effectively deal with the clustering problem of multi-scale datasets. Its clustering performance is very stable, and the clustering quality is better than traditional spectral clustering algorithm and k-means algorithm.