In view of the fact that the existing 3D databases have fewer behavioral categories, few interactions with scenes, and single and fixed perspectives, this paper provides a large-scale human body complex behavior database DMV action3D based on RGB-D cameras, from two fixed perspectives and a mobile robot records human behavior from a dynamic perspective. There are 31 different behavioral classes in the database, including daily behaviors, interaction behaviors, and abnormal behaviors angles. Validated database collected more than 620 behavioral videos, about 600 000 frames of color images and depth images, to provide robots with optimal viewing. In order to verify the reliability and practicability of the data set, this paper adopts four methods for human behavior recognition, which are HOG3D features extracted by CRFasRNN method based on the information features of customs nodes, CNN and conditional random field (CRF), and then adopts SVM method for human behavior recognition. Spatial and temporal characteristics are extracted based on the three-dimensional convolutional network (C3D) and the 3D dense connection residual network, and the motion tags are predicted by softmax layer. The results show that DMV action3D human behavior database is characterized by a variety of scenes and complicated movements, and the difficulty of recognition is greatly increased. The DMV action 3D database has great advantages for studying human behavior in real environments, and provides a better resource for serving robots to recognize human behavior in real environments.