For the problem of underdetermined blind source separation (UBSS), a method to enhance signal sparsity is proposed, and the density based spatial clustering of applications with noise (DBSCAN) combined with the clustering by fast search and find of density peaks (CFSFDP) is used to estimate the mixing matrix. Firstly, the time domain observed signals are transformed into sparse signals in the time-frequency domain, the single-source-point (SSP) detection is used to highlight the linear clustering characteristics, and the mirroring mapping is used to transform the linear clustering into compact clustering for density-based clustering analysis. Then, in the dense data heaps, the DBSCAN is used to search for high-density points and their corresponding neighborhoods to automatically find the number of clusters and the initial cluster centers. Finally, the number of clusters is used as the input parameter of CFSFDP, and the corresponding density peaks are searched by CFSFDP in the range of data clusters to achieve further correction of the cluster centers position. The above method not only improves the estimation accuracy of the underdetermined mixing matrix, but also provides a highly consistent estimator.