动态多视角复杂3D人体行为数据库及行为识别
作者:
作者单位:

上海理工大学光电信息与计算机工程学院,上海,200093

作者简介:

通讯作者:

基金项目:

国家自然科学基金 61673276,61603255;上海青年科技英才扬帆计划 17YF1427000国家自然科学基金 (61673276,61603255) 资助项目;上海青年科技英才扬帆计划(17YF1427000)资助项目。


Dynamic and Multi-view Complicated 3D Database of Human Activity and Activity Recognition
Author:
Affiliation:

School of Optional-Electrical and Computer Engineering, University of Shanghai for Science and Technology, Shanghai, 200093, China

Fund Project:

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
    摘要:

    提供了一个较大规模的基于RGB-D摄像机的人体复杂行为数据库DMV (Dynamic and multi-view) action3D,从2个固定视角和一台移动机器人动态视角录制人体行为。数据库现有31个不同的行为类,包括日常行为、交互行为和异常行为类等三大类动作,收集了超过620个行为视频约60万帧彩色图像和深度图像,为机器人寻找最佳视角提供了可供验证的数据库。为验证数据集的可靠性和实用性,本文采取4种方法进行人体行为识别,分别是基于关节点信息特征、基于卷积神经网络(Convolutional neural networks,CNN)和条件随机场(Conditional random field,CRF)结合的CRFasRNN方法提取的彩色图像HOG3D特征,然后采用支持向量机(Support vector machine,SVM)方法进行了人体行为识别;基于3维卷积网络(C3D)和3D密集连接残差网络提取时空特征,通过softmax层以预测动作标签。实验结果表明:DMV action3D人体行为数据库由于场景多变、动作复杂等特点,识别的难度也大幅增大。DMV action3D数据集对于研究真实环境下的人体行为具有较大的优势,为服务机器人识别真实环境下的人体行为提供了一个较佳的资源。

    Abstract:

    In view of the fact that the existing 3D databases have fewer behavioral categories, few interactions with scenes, and single and fixed perspectives, this paper provides a large-scale human body complex behavior database DMV action3D based on RGB-D cameras, from two fixed perspectives and a mobile robot records human behavior from a dynamic perspective. There are 31 different behavioral classes in the database, including daily behaviors, interaction behaviors, and abnormal behaviors angles. Validated database collected more than 620 behavioral videos, about 600 000 frames of color images and depth images, to provide robots with optimal viewing. In order to verify the reliability and practicability of the data set, this paper adopts four methods for human behavior recognition, which are HOG3D features extracted by CRFasRNN method based on the information features of customs nodes, CNN and conditional random field (CRF), and then adopts SVM method for human behavior recognition. Spatial and temporal characteristics are extracted based on the three-dimensional convolutional network (C3D) and the 3D dense connection residual network, and the motion tags are predicted by softmax layer. The results show that DMV action3D human behavior database is characterized by a variety of scenes and complicated movements, and the difficulty of recognition is greatly increased. The DMV action 3D database has great advantages for studying human behavior in real environments, and provides a better resource for serving robots to recognize human behavior in real environments.

    参考文献
    相似文献
    引证文献
引用本文

王永雄,李璇,李梁华.动态多视角复杂3D人体行为数据库及行为识别[J].数据采集与处理,2019,34(1):68-79

复制
分享
文章指标
  • 点击次数:
  • 下载次数:
历史
  • 收稿日期:2018-06-23
  • 最后修改日期:2018-12-05
  • 录用日期:
  • 在线发布日期: 2019-04-12