基于区域划分与标准时间的手部异常行为检测

梁宇宁; 王绍华; 金向明; 周栋

doi:10.13700/j.bh.1001-5965.2020.0369

基于区域划分与标准时间的手部异常行为检测

doi: 10.13700/j.bh.1001-5965.2020.0369

1.
北京航空航天大学可靠性与系统工程学院, 北京 100083
2.
中国航发湖南动力机械研究所, 株洲 412002

详细信息

通讯作者:
周栋, E-mail: zhoudong@buaa.edu.cn

中图分类号: TP391
计量
- 文章访问数: 538
- HTML全文浏览量: 179
- PDF下载量: 36
- 被引次数: 0
出版历程
- 收稿日期: 2020-07-29
- 录用日期: 2020-10-09
- 网络出版日期: 2021-10-20

Abnormal hand behavior detection based on area division and standard time

1.
School of Reliability and Systems Engineering, Beihang University, Beijing 100083, China
2.
AECC Hunan Aviation Powerplant Research Institute, Zhuzhou 412002, China

More Information

Corresponding author: ZHOU Dong, E-mail: zhoudong@buaa.edu.cn

摘要

摘要:
通过监控视频自动检测操作任务中手部异常行为，能够预防人因差错，提高人因可靠性。针对手部操作任务运动特征不明显、常用异常检测和手势识别方法不适用的问题，提出基于区域划分与标准时间的手部异常行为检测技术。使用基于椭圆模型的肤色检测方法，检测视频中手部形心位置；提出工作区域划分方法，根据手部形心在操作过程中所处区域的不同，将连续操作分割为单元任务，获得各段单元任务的起止时间和持续时长；以正常工作时间为标准，对超出标准时间范围的单元任务提出异常警告。实验表明：所提方法的单元任务分割正确率高于93%，异常行为检测率高于86%，能够有效检测手部异常行为，为人为差错的监测与预警提供技术支持。
- 异常行为检测 /
- 肤色检测 /
- 人为差错 /
- 人因可靠性 /
- 单元任务分割
Abstract:
Abnormal hand behavior detection during operation based on intelligent video surveillance systems can prevent human errors and improve human reliability. In order to solve the problems that the motion characteristics of hand operation are not obvious, and common abnormal detection and gesture recognition methods are not applicable, a detection technology of abnormal hand behavior based on area division and standard time is proposed. The skin color detection method based on ellipse model was used to detect the hand centroids in the video. The work area division method is proposed to define the unit task. The continuous operation was divided into unit tasks according to the work area of the hand centroid in each frame, and the start and end time and the duration of each unit task were obtained. Standard time was defined by normal working hours. And warnings were given to the unit tasks which exceed the standard time range. Experimental results show that the accuracy rate of unit task segmentation is higher than 93%, and the detection rate of abnormal behavior is higher than 86%. The proposed method can effectively detect abnormal hand behavior and provide technical support for human error monitoring and early warning.
- abnormal behavior detection /
- skin color detection /
- human error /
- human reliability /
- unit task segmentation

HTML全文

图 1 基于椭圆模型的肤色检测方法

Figure 1. Skin color detection method based on ellipse model

下载: 全尺寸图片幻灯片

图 2 开运算处理手部图像的过程

Figure 2. Steps for processing hand images by open operation

下载: 全尺寸图片幻灯片

图 3 利用腐蚀处理手部交叠区域的过程

Figure 3. Steps for eliminating overlapping parts of hands by erosion

下载: 全尺寸图片幻灯片

图 4 拼图任务中右手所处的不同区域

Figure 4. Different areas of right hand in puzzle task

下载: 全尺寸图片幻灯片

图 5 单元任务示意图

Figure 5. Schematic diagram of unit tasks

下载: 全尺寸图片幻灯片

图 6 形心分组示意图

Figure 6. Schematic diagram of centroids grouping

下载: 全尺寸图片幻灯片

图 7 拼图任务中的犹豫过程

Figure 7. Hesitation process in puzzle task

下载: 全尺寸图片幻灯片

图 8 犹豫点剔除算法流程

Figure 8. Flowchart of hesitant points elimination algorithm

下载: 全尺寸图片幻灯片

图 9 单元任务分割算法流程

Figure 9. Flowchart of unit task segmentation algorithm

下载: 全尺寸图片幻灯片

图 10 实验所用的3个案例

Figure 10. Three cases used in experiment

下载: 全尺寸图片幻灯片

图 11 基于区域划分与标准时间的手部异常行为检测流程

Figure 11. Flowchart of abnormal hand behavior detection based on area division and standard time

下载: 全尺寸图片幻灯片

表 1 实验结果

Table 1. Results of experiment

参数	案例1	案例2	案例3
视频时长	00:06:04	00:11:13	01:01:50
实际异常行为数量	11	5	15
算法判断异常行为总数	14	6	13
算法正确判断异常数量	10	5	13
算法分割正确率/%	95.83	93.75	95.65
异常行为检测率/%	90.91	100	86.67

下载: 导出CSV

表 2 不同检测方法的检测率对比

Table 2. Comparison of detection rates of different detection methods

方法	检测率/%
本文方法	90.91
基于手部轨迹识别^[24]	80.00
基于姿态-动作模型^[25]	70~90
基于手部图像特征分析^[26]	80~85

下载: 导出CSV

参考文献(26)

[1]	TAO J, QIU D, YANG F, et al. A bibliometric analysis of human reliability research[J]. Journal of Cleaner Production, 2020, 260: 121041. doi: 10.1016/j.jclepro.2020.121041
[2]	朱旭东, 刘志镜. 基于主题隐马尔科夫模型的人体异常行为识别[J]. 计算机科学, 2012, 39(3): 251-255. doi: 10.3969/j.issn.1002-137X.2012.03.057 ZHU X D, LIU Z J. Human abnormal behavior recognition based on topic hidden Markov model[J]. Computer Science, 2012, 39(3): 251-255(in Chinese). doi: 10.3969/j.issn.1002-137X.2012.03.057
[3]	YI X, DONG H, DONG X, et al. Human reliability analysis method on armored vehicle system considering error correction[J]. Journal of Shanghai Jiaotong University (Science), 2016, 21(4): 472-477. doi: 10.1007/s12204-016-1749-5
[4]	李鹏程, 陈国华, 张力, 等. 人因可靠性分析技术的研究进展与发展趋势[J]. 原子能科学技术, 2011, 45(3): 329-340. https://www.cnki.com.cn/Article/CJFDTOTAL-YZJS201103015.htm LI P C, CHEN G H, ZHANG L, et al. Research review and development trends of human reliability analysis technique[J]. Atomic Energy Science and Technology, 2011, 45(3): 329-340(in Chinese). https://www.cnki.com.cn/Article/CJFDTOTAL-YZJS201103015.htm
[5]	LIU P, QIU Y, HU J, et al. Expert judgments for performance shaping factors' multiplier design in human reliability analysis[J]. Reliability Engineering and System Safety, 2020, 194: 106343. doi: 10.1016/j.ress.2018.12.022
[6]	BEN MABROUK A, ZAGROUBA E. Abnormal behavior recognition for intelligent video surveillance systems: A review[J]. Expert Systems with Applications, 2018, 91: 480-491. doi: 10.1016/j.eswa.2017.09.029
[7]	ZHU B, XIE Y, LUO G, et al. An abnormal behavior detection method using optical flow model and openpose[J]. International Journal of Advanced Computer Science and Applications, 2020, 11(5): 28-34.
[8]	LAZARIDIS L, DIMOU A, DARAS P. Abnormal behavior detection in crowded scenes using density heatmaps and optical flow[C]//Proceedings of the European Signal Processing Conference, 2018: 2074-2078.
[9]	MABROUK A B, ZAGROUBA E. Spatio-temporal feature using optical flow based distribution for violence detection[M]. Amsterdam: Elsevier Science, 2017.
[10]	MIN W, ZOU S, LI J. Human fall detection using normalized shape aspect ratio[J]. Multimedia Tools and Applications, 2019, 78(11): 14331-14353. doi: 10.1007/s11042-018-6794-7
[11]	WANG J, XU Z. Crowd anomaly detection for automated video surveillance[C]//Proceedings of the IET Seminar Digest, 2015: 15382040.
[12]	LI K, HUANG H, ZHENG Z, et al. Research of crowed abnormal behavior detection technology based on trajectory gradient[C]//International Symposium on Intelligence Computation and Applications, 2018: 486-500.
[13]	WANG T, LI Y, HU J, et al. A survey on vision-based hand gesture recognition[C]//1st International Conference on Smart Multimedia, 2018: 219-231.
[14]	XIA Z, LEI Q, YANG Y, et al. Vision-based hand gesture recognition for human-robot collaboration: A survey[C]//Proceedings of the 20195th International Conference on Control, Automation and Robotics, 2019: 198-205.
[15]	郭全利. 动态手势识别关键算法研究[D]. 锦州: 辽宁工业大学, 2018. GUO Q L. Research on key algorithms of dynamic gesture recognition[D]. Jinzhou: Liaoning University of Technology, 2018(in Chinese).
[16]	张兆杨, 杨高波, 刘志. 视频对象分割提取的原理与应用[M]. 北京: 科学出版社, 2009. ZHANG Z Y, YANG G B, LIU Z. Principle and application of video object segmentation and extraction[M]. Beijing: Science Press, 2009(in Chinese).
[17]	ZIVKOVIC Z, HEIJDEN F V D. Efficient adaptive density estimation per image pixel for the task of background subtraction[J]. Pattern Recognition Letters, 2006, 27(7): 773-780. doi: 10.1016/j.patrec.2005.11.005
[18]	ZIVKOVIC Z. Improved adaptive Gaussian mixture model for background subtraction[C]//Proceedings of the 17th International Conference on Pattern Recognition, 2004: 28-31.
[19]	SUTTON R S. Learning to predict by the methods of temporal differences[J]. Machine Learning, 1988, 3(1): 9-44.
[20]	HSU R L, ABDEL-MOTTALEB M, JAIN A K. Face detection in color images[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, 24(5): 696-706. doi: 10.1109/34.1000242
[21]	孙燮华. 数字图像处理: 原理与算法[M]. 北京: 机械工业出版社, 2010. SUN X H. Digital image processing: Principle and algorithm[M]. Beijing: China Machine Press, 2010(in Chinese).
[22]	易树平, 郭伏. 基础工业工程[M]. 北京: 机械工业出版社, 2014. YI S P, GUO F. Basic industrial engineering[M]. Beijing: China Machine Press, 2014(in Chinese).
[23]	邱慧慧. 面向离散制造业的标准时间制定系统研究[D]. 济南: 山东大学, 2009. QIU H H. Research on discrete manufacturing industries-oriented standard time formulation system[D]. Jinan: Shandong University, 2009(in Chinese).
[24]	陈琼, 鱼滨. 基于手部轨迹识别的ATM智能视频监控系统[J]. 计算机工程, 2012, 38(11): 143-146. https://www.cnki.com.cn/Article/CJFDTOTAL-JSJC201211044.htm CHEN Q, YU B. Intelligent video surveillance system of automatic teller machine based on hand trajectory recognition[J]. Computer Engineering, 2012, 38(11): 143-146(in Chinese). https://www.cnki.com.cn/Article/CJFDTOTAL-JSJC201211044.htm
[25]	陈若愚. 超市中人体异常行为识别方法的研究[D]. 长沙: 国防科学技术大学, 2013. CHEN R Y. A research on abnormal behavior detection methods in supermarket[D]. Changsha: National University of Defense Technology, 2013(in Chinese).
[26]	刘刚. 基于手部图像特征分析的超市中异常行为的检测[D]. 成都: 西华大学, 2017. LIU G. Detection of abnormal behavior in supermarket based on hand image feature analysis[D]. Chengdu: Xihua University, 2017(in Chinese).