Interference factors such as seal cover, invoice crease and so on, cause noise adhesion in number area of some invoice, which would seriously lead to the invoice number segmentation error. Aiming at this problem, a noise adhesion area repairing algorithm is proposed. At the same time, according to the font structure and characteristics of ordinary invoice number, invoice number recognition algorithm based on characteristics of digital structure is proposed. Firstly, define number structure features, including four kinds of fill area, two kinds of number of passing through the character, and four kinds of hollow area, which constitute a 10-dimensional feature vector of the number to be identified. Then, match the feature vector with the template features in the standard template library, by obtaining the Euclidean distance, and regard the corresponding number with the minimum Euclidean distances as the last recognition result. The proposed method and printed number recognition method based on the improved left and right contour features are compared. Experimental results indicate that the proposed identification algorithm has higher accuracy, faster recognition speed and stronger robustness to noise.