The effectiveness of deep learning algorithms depends on a large amount of labeled data. The purpose of transfer learning is to use a dataset with known labels （source domain） to classify a dataset with unknown labels （target domain）， so the research of deep transfer learning has become a hotspot. For the problem of insufficient training data labels， a model of multi-scale domain adversarial network（MSDAN） based on multi-scale feature fusion is proposed. This method uses the idea of generating adversarial networks and multi-scale feature fusion to obtain the feature representation of the domain data and the target domain data in a high-dimensional feature space. The feature representation extracts common geometric features and common semantic features of the source domain data and the target domain data. The feature representation of the source domain data and the source domain label are input into the classifier for classification， and finally more advanced effect is obtained in the test of the target domain dataset.