In order to realize the automatic removing of impurities in silicon particles, an impurity identification algorithm is presented and an automated impurity removal device is designed. Firstly, an image of silicon particles is collected by the camera in real time, and it is preprocessed, morphologically operated, and edge detected by Visual C++ combined with OpenCV to identify and locate the impurity. Then, based on the RS-232 protocol, the communication with the single-chip microcomputer is realized, the position information of the impurity is transmitted, and the action of the corresponding device is controlled. Finally, the separation of the silicon and the impurity is accomplished. It is verified that the system can effectively identify and remove impurities, and the removal rate reaches 95%, which meets the design requirements. The device adopts the “subregional removal algorithm” combined with the “high-pressure gas” method for impurity removal, and has the advantages of fast action response and high rejection precision, improving production efficiency and reducing production cost.