多因素飞行冲突探测中的认知负荷评估方法

杨越; 马博凯; 程龙

doi:10.13700/j.bh.1001-5965.2023.0816

多因素飞行冲突探测中的认知负荷评估方法

doi: 10.13700/j.bh.1001-5965.2023.0816

中国民航大学空中交通管理学院，天津 300300

基金项目:

天津市教委科研计划(2023KJ234)

详细信息

通讯作者:
E-mail：yueyang0124@126.com

中图分类号: V355
计量
- 文章访问数: 217
- HTML全文浏览量: 55
- PDF下载量: 9
- 被引次数: 0
出版历程
- 收稿日期: 2023-12-15
- 录用日期: 2024-05-10
- 网络出版日期: 2024-05-22
- 整期出版日期: 2026-03-31

Cognitive load assessment method for multifactorial flight conflict detection

College of Air Traffic Management，Civil Aviation University of China，Tianjin 300300，China

Funds:

Education Commission Research Program of Tianjin (2023KJ234)

More Information

Corresponding author: E-mail：yueyang0124@126.com

摘要

摘要:
管制员认知负荷的实时监测与评估对空管系统的安全运行具有重要意义，以生理参数反映管制指挥中认知负荷的变化情况，便于及时发现影响管制效能的不良工作状态，从而实现风险管控关口前移。基于雷达管制仿真实验平台搭建不同空域复杂性条件下的冲突探测场景，设计最小间距（4 km、12 km、16 km）×汇聚角度（45°、90°、135°）×速度特性（速度相同、快速优先、慢速优先）的被试内三因素重复测量实验方案；采集眼动和心电数据，基于多因素方差分析研究不同复杂性因素的影响效应及生理指标的变化规律，筛选能够有效表征空域复杂性的特征生理指标；在此基础上，采用随机森林（RF）、支持向量机（SVM）和长短期记忆网络（LSTM）3种机器学习算法对被试的认知负荷进行预测。研究表明，在同类因素的不同水平中，被试的认知负荷分别在最小间距（MD）接近告警间隔和汇聚角度（CA）为锐角的水平下达到最高，在速度特性（SC）为速度相同时达到最低。注视点总数（NrF）、扫视点总数（NrS）、平均扫视持续时间（SD）、平均扫视幅度（SA）、平均扫视峰值速度（SPV）、眨眼频率（BF）、瞳孔直径（PD）、平均呼吸频率（RR）及心率变异性中低频与高频功率的比值（LF/HF）9项生理指标可以作为有效表征不同MD水平下认知负荷的特征指标。基于单模态眼动信号的SVM模型预测准确率为94.69%，高于单模态心电信号预测模型和眼动+心电双模态信号预测模型，删除强相关的特征指标会在一定程度影响模型性能。
- 冲突探测 /
- 认知负荷 /
- 眼动指标 /
- 心电指标 /
- 机器学习 /
- 模型性能
Abstract:
Real-time monitoring and evaluation of air traffic controllers’ cognitive load is of great significance to the operational safety of the air traffic control system. Cognitive load variation in control command, which is reflected by the physiological parameters, facilitates timely discovery of the undesirable working condition that affecting control effectiveness, in order to realize forward movement of the risk control gate. The radar control simulation experimental platform is used to create conflict detection scenarios under various airspace complexity conditions. A repeated within-subjects measurement experimental scheme is designed with three factors: minimum distance (4 km, 12 km, 16 km), convergence angle (45°, 90°, 135°), and speed characteristics (fast speed priority, same speed, slow speed priority). The effect of different complexity factors on the cognitive load and the changing laws of physiological indexes are studied based on multifactorial analysis of variance through collecting the subjects' eye movement and electrocardiogram data, so as to select feature physiological indexes that can effectively express airspace complexity factors. Based on this, the individuals' cognitive load is assessed using three machine learning algorithms: random forest (RF), support vector machine (SVM), and long short-term memory network (LSTM). The results show that among different levels of the same type factor, the cognitive load is highest when the minimum distance (MD) is closed to the warning interval and the convergence angle (CA) is an acute angle, as well as lowest when the speed characteristics (SC) is the same speed respectively. Nine physiological indexes can be used as feature indexes to effectively express cognitive load at different levels of MD, including number of fixations (NrF), number of saccades (NrS), mean saccade duration (SD), average saccade amplitude (SA), average saccade peak velocity (SPV), blinking frequency (BF), pupil diameter (PD), mean respiratory rate (RR) and power ratio of low-frequency to high-frequency in heart rate variability (LF/HF). The assessment accuracy of the SVM model based on a single-modal eye movement signal is 94.69%, which is higher than the single-modal assessment model with electrocardiographic signal and the dual-modal assessment model with eye movement and electrocardiographic signals. Removing strongly correlated feature indexes will affect model performance to some extent.
- conflict detection /
- cognitive load /
- eye movement index /
- electrocardiographic index /
- machine learning /
- model performance

HTML全文

图 1 冲突探测实验场景

Figure 1. Conflict detection experimental scenarios

下载: 全尺寸图片幻灯片

图 2 CA×MD×SC交互作用显著的生理指标分析结果

Figure 2. Analysis results of physiological indexes with significant effect of CA × MD × SC interactions

下载: 全尺寸图片幻灯片

图 3 认知负荷评估方法流程图

Figure 3. Flowchart of cognitive load assessment method

下载: 全尺寸图片幻灯片

表 1 各项生理指标的显著性检验p值

Table 1. Analysis of variance results of physiological indexes

因素	NrF	FD	NrS	SD	SA	SPV	BF	PD	RR	HR	SDNN	PNN50	LF/HF
MD	0.027*	0.114	0.006**	0.000**	0.000**	0.000**	0.000**	0.010*	0.028*	0.102	0.33	0.229	0.032*
CA	0.531	0.213	0.332	0.000**	0.000**	0.000**	0.012*	0.102	0.399	0.032*	0.396	0.292	0.721
SC	0.158	0.445	0.086	0.002**	0.08	0.598	0.002**	0.32	0.45	0.001**	0.397	0.789	0.619
注：** 表示在置信度p=0.01 级别（双尾）相关性显著；* 表示在置信度p=0.05 级别（双尾）相关性显著。

下载: 导出CSV

表 2 不同模态生理指标的模型评估结果

Table 2. Model assessment results by different modal physiological indexes

指标模态	评估模型	准确率/%	R²	R
眼动特征指标	RF	91.13	0.78	0.92
	SVM	94.69	0.91	0.95
	LSTM	93.37	0.88	0.93
心电特征指标	RF	74.25	0.029	0.23
	SVM	83.59	0.029	0.38
	LSTM	74.68	0.020	0.17
眼动+心电指标	RF	88.46	0.76	0.91
	SVM	93.28	0.89	0.94
	LSTM	91.58	0.82	0.90

下载: 导出CSV

表 3 去除强相关特征指标的模型评估结果

Table 3. Model assessment results after removing strongly correlated feature indexes

指标模态	评估模型	准确率/%	R²	R
眼动特征指标	RF	90.05	0.74	0.88
	SVM	92.90	0.90	0.93
	LSTM	92.35	0.86	0.93
心电特征指标	RF	74.25	0.029	0.23
	SVM	83.59	0.029	0.38
	LSTM	74.68	0.020	0.17
眼动+心电指标	RF	88.00	0.71	0.87
	SVM	92.39	0.89	0.93
	LSTM	91.15	0.73	0.92

下载: 导出CSV

参考文献(32)

[1]	HAO S Q, CHENG S W, ZHANG Y P. A multi-aircraft conflict detection and resolution method for 4-dimensional trajectory-based operation[J]. Chinese Journal of Aeronautics, 2018, 31(7): 1579-1593.
[2]	ARICO P, BORGHINI G, DI FLUMERI G, et al. Human factors and neurophysiological metrics in air traffic control: a critical review[J]. IEEE Reviews in Biomedical Engineering, 2017, 10: 250-263.
[3]	STANKOVIC S, RAUFASTE É, AVERTY P. Determinants of conflict detection: a model of risk judgments in air traffic control[J]. Human Factors, 2008, 50(1): 121-134.
[4]	MARCHITTO M, BENEDETTO S, BACCINO T, et al. Air traffic control: ocular metrics reflect cognitive complexity[J]. International Journal of Industrial Ergonomics, 2016, 54: 120-130.
[5]	RANTANEN E M, NUNES A. Hierarchical conflict detection in air traffic control[J]. The International Journal of Aviation Psychology, 2005, 15(4): 339-362.
[6]	CHARLES R L, NIXON J. Measuring mental workload using physiological measures: a systematic review[J]. Applied Ergonomics, 2019, 74: 221-232.
[7]	FRANK N, LE P, MILLS E, et al. Micromovements and discomfort associated with flight mission with helmet operation tasks with different levels of cognitive workload[J]. International Journal of Industrial Ergonomics, 2023, 95: 103441.
[8]	GOH S Y M, LYE S W, SOU K L, et al. Effects of increased cognitive load on field of view in multi-task operations involving surveillance[J]. Human-Intelligent Systems Integration, 2021, 3(4): 277-285.
[9]	JOHN A R, SINGH A K, DO T N, et al. Unraveling the physiological correlates of mental workload variations in tracking and collision prediction tasks[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2022, 30: 770-781.
[10]	BERNHARDT K A, POLTAVSKI D, PETROS T, et al. The effects of dynamic workload and experience on commercially available EEG cognitive state metrics in a high-fidelity air traffic control environment[J]. Applied Ergonomics, 2019, 77: 83-91.
[11]	HE X L, WANG L J, GAO X H, et al. The eye activity measurement of mental workload based on basic flight task[C]//Proceedings of the IEEE 10th International Conference on Industrial Informatics. Piscataway: IEEE Press, 2012: 502-507.
[12]	BODALA I P, KE Y, MIR H, et al. Cognitive workload estimation due to vague visual stimuli using saccadic eye movements[C]//Proceedings of the 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. Piscataway: IEEE Press, 2014: 2993-2996.
[13]	MALLICK R, SLAYBACK D, TOURYAN J, et al. The use of eye metrics to index cognitive workload in video games[C]//Proceedings of the IEEE Second Workshop on Eye Tracking and Visualization. Piscataway: IEEE Press, 2016: 60-64.
[14]	PRABHAKAR G, MUKHOPADHYAY A, MURTHY L, et al. Cognitive load estimation using ocular parameters in automotive[J]. Transportation Engineering, 2020, 2: 100008.
[15]	DE RIVECOURT M, N KUPERUS M, J POST W, et al. Cardiovascular and eye activity measures as indices for momentary changes in mental effort during simulated flight[J]. Ergonomics, 2008, 51(9): 1295-1319.
[16]	李鹏杰, 姚志, 王萌, 等. 心率变异性在手控交会对接操作脑力负荷评价中的应用[J]. 人类工效学, 2013, 19(3): 1-5. LI P J, YAO Z, WANG M, et al. The application of heart rate variability in the evaluation of mental workload during manual controlled rendezvous and docking[J]. Chinese Journal of Ergonomics, 2013, 19(3): 1-5(in Chinese).
[17]	GRASSMANN M, VLEMINCX E, VON LEUPOLDT A, et al. Individual differences in cardiorespiratory measures of mental workload: an investigation of negative affectivity and cognitive avoidant coping in pilot candidates[J]. Applied Ergonomics, 2017, 59: 274-282.
[18]	AYGUN A, LYU B Y, NGUYEN T, et al. Cognitive workload assessment via eye gaze and EEG in an interactive multi-modal driving task[C]//Proceedings of the International Conference on Multimodal Interaction. New York: ACM, 2022: 337-348.
[19]	OPPELT M P, FOLTYN A, DEUSCHEL J, et al. ADABase: a multimodal dataset for cognitive load estimation[J]. Sensors, 2022, 23(1): 340.
[20]	CHEN J, XUE L, LIU Z C. A pilot workload evaluation method based on EEG data and physiological data[C]//Proceedings of the IEEE International Conference on Signal Processing, Communications and Computing. Piscataway: IEEE Press, 2020: 1-6.
[21]	王煜文, 王盛, 韩明秀, 等. 基于机器学习的飞行员脑力负荷评估研究进展[J]. 载人航天, 2021, 27(6): 789-796. WANG Y W, WANG S, HAN M X, et al. Research progress of mental workload assessment based on machine learning in pilots[J]. Manned Spaceflight, 2021, 27(6): 789-796(in Chinese).
[22]	MEDEIROS J, COUCEIRO R, DUARTE G, et al. Can EEG be adopted as a neuroscience reference for assessing software programmers' cognitive load?[J]. Sensors, 2021, 21(7): 2338.
[23]	LOFT S, BOLLAND S, HUMPHREYS M S, et al. A theory and model of conflict detection in air traffic control: incorporating environmental constraints[J]. Journal of Experimental Psychology Applied, 2009, 15(2): 106-124.
[24]	MARCHITTO M, DI STASI L L, CAÑAS J J. Ocular movements under taskload manipulations: influence of geometry on saccades in air traffic control simulated tasks[J]. Human Factors and Ergonomics in Manufacturing & Service Industries, 2012, 22(5): 407-419.
[25]	NEAL A, KWANTES P J. An evidence accumulation model for conflict detection performance in a simulated air traffic control task[J]. Human Factors, 2009, 51(2): 164-180.
[26]	SALVUCCI D D, GOLDBERG J H. Identifying fixations and saccades in eye-tracking protocols[C]//Proceedings of the Symposium on Eye Tracking Research & Applications - ETRA '00. New York: ACM, 2000: 71-78.
[27]	ANDERSSON R, LARSSON L, HOLMQVIST K, et al. One algorithm to rule them all? An evaluation and discussion of ten eye movement event-detection algorithms[J]. Behavior Research Methods, 2017, 49(2): 616-637.
[28]	PAN T, WANG H B, SI H Q, et al. Identification of pilots' fatigue status based on electrocardiogram signals[J]. Sensors, 2021, 21(9): 3003.
[29]	STATSENKO Y, HABUZA T, GORKOM K N, et al. Applying the inverse efficiency score to visual-motor task for studying speed-accuracy performance while aging[J]. Frontiers in Aging Neuroscience, 2020, 12: 574401.
[30]	DING Y, CAO Y Q, DUFFY V G, et al. Measurement and identification of mental workload during simulated computer tasks with multimodal methods and machine learning[J]. Ergonomics, 2020, 63(7): 896-908.
[31]	张奇良, 杨坤华, 曲行达, 等. 基于多模态生理信号的驾驶人脑力负荷评估[J]. 深圳大学学报(理工版), 2022, 39(3): 278-286. ZHANG Q L, YANG K H, QU X D, et al. Evaluation of drivers' mental workload based on multi-modal physiological signals[J]. Journal of Shenzhen University (Science and Engineering), 2022, 39(3): 278-286(in Chinese).
[32]	郑玲, 乔旭强, 倪涛, 等. 基于多维信息特征分析的驾驶人认知负荷研究[J]. 中国公路学报, 2021, 34(4): 240-250. ZHENG L, QIAO X Q, NI T, et al. Driver cognitive loads based on multi-dimensional information feature analysis[J]. China Journal of Highway and Transport, 2021, 34(4): 240-250(in Chinese).