How to improve the accuracy of face attributes recognition in natural environment or unrestricted environment is an important question in applying face attributes. In daily life, the uncontrollable factors, such as face postures and light, have a great influence on the recognition of human face attributes. How to improve the accuracy under the influence of the above factors is a key problem in the study of face attribute recognition. Given the success of convolutional neural network (CNN) in image classification, a new network structure is built by using multi-level sub-network and ranked Dropout mechanism algorithm. The structure has strong robustness to deal with face changes, thus achieving better results in the CelebA dataset and LFWA dataset, and reducing the network size significantly as well.