As an important part of the household appliances, the refrigerator becomes more intelligent. Object recognition of the food in a refrigerator is a key technology of a smart refrigerator. However, the foods in the refrigerator are diverse and disordered, which brings a lot of challenges to identify the varieties of foods. A method using an integrated convolutional neural network is proposed to solve this problem. The basic idea is that two convolutional neural networks are firstly trained separately. One is used to recognize the kinds of fruits and vegetables, the other is to recognize the color of them. Then, a multilayer perceptron is used to integrate the two independent networks to carry out classification. The two separate convolutional neural networks can complement and improve each other in the integrated network. In the method, color information, an important feature in the recognition, can be enhanced. The proposed structure also improves the recognition rate which is influenced by object occlusion and view variations. Finally, the effectiveness of the proposed method is validated on a dataset which contains a large amount of images obtained from a real situation of a refrigerator.