Software robot-based application behavior simulation for cyber security range in industrial control field
-
摘要:
工控靶场为开展工业控制系统(ICS)安全研究提供重要支撑。面向工控靶场关键任务之一的应用软件行为模拟，提出一种软件机器人方法以实现工控靶场应用软件行为的逼真模拟。考虑软件图形界面及软件内在显隐式规则，提出基于尺度不变特征变换(SIFT)图像相似度的软件菜单采集算法及混合状态机模型对应用软件行为进行建模。针对软件机器人的智能化问题，使用深度Q网络(DQN)算法驱动软件机器人对应用软件行为进行自主学习，同时结合多重经验回访和多重目标网络对DQN算法进行优化。实验结果表明：基于DQN的软件机器人能够对工控软件进行有效学习，且优化后的DQN算法自主学习效果更佳。
Abstract:The cyber security range in the industrial control field provides important support for studies on industrial control system (ICS) security. The application behavior simulation is a crucial task for the cyber security range in the industrial control field. Therefore, a software robot method was proposed to realistically simulate the application behavior for the cyber security range in the industrial control field. By considering software graphical interfaces and explicit and implicit software rules, a software menu acquisition algorithm based on scale invariant feature transform (SIFT) for image similarity, as well as a hybrid hierarchical state machine model was developed to model application behaviors. In view of the intelligent problem of the software robot, a deep Q network (DQN) algorithm was utilized to drive the software robot to autonomously learn the application behavior. At the same time, the DQN algorithm was optimized by combining multiple experience replays and multiple target networks. The experiment results show that the software robot based on DQN can effectively learn the industrial control software, and the optimized DQN algorithm has a better autonomous learning effect.
-
图 1 用户和机器人使用软件过程的行为模型示意图
Figure 1. Model of behavior users and robots in process of using software
图 2 西门子典型工控软件的SM-HHSM
Figure 2. SM-HHSM of Siemens typical industrial control software
图 6 FSM和SM-HHSM遍历时间和内存占用对比
Figure 6. Comparison of traversal time and memory usage between FSM and SM-HHSM
图 8 $ \gamma = 0.9,\alpha = 0.03 $时软件机器人的训练损失曲线
Figure 8. Training loss curve of software robot when $ \gamma $ = 0.9 and $ \alpha $ = 0.03
图 9 $ \varepsilon $不同取值下平均奖励、累积奖励情况和成功到达率对比
Figure 9. Comparison of average and accumulative rewards and successful arrival rates with different $ \varepsilon $ values
图 10 DQN与其优化方案的平均奖励、累积奖励和成功到达率对比
Figure 10. Comparison of average and accumulative rewards and successful arrival rates between DQN and its optimization schemes
表 1 实验参数设置
Table 1. Parameter setting in experiment
软件行为模拟机器人位置 目标按钮位置 学习率$ \alpha $ 迭代训练次数$ I $ 折扣因子$ \gamma $ 经验池大小 批大小 最大移动次数 (0,3) (1,0) 0.03 5000 0.9 1000 200 50 
下载: 导出CSV
-
[1] 方滨兴, 贾焰, 李爱平, 等. 网络空间靶场技术研究[J]. 信息安全学报, 2016, 1(3): 1-9.FANG B X, JIA Y, LI A P, et al. Cyber ranges: State-of-the-art and research challenges[J]. Journal of Cyber Security, 2016, 1(3): 1-9(in Chinese). [2] 王佰玲, 刘红日, 张耀方, 等. 工业控制系统安全仿真关键技术研究综述[J]. 系统仿真学报, 2021, 33(6): 1466-1488.WANG B L, LIU H R, ZHANG Y F, et al. Review on key technologies of industrial control system security simulation[J]. Journal of System Simulation, 2021, 33(6): 1466-1488(in Chinese). [3] OZKAYA M. Analysing UML-based software modelling languages[J]. Journal of Aeronautics and Space Technologies, 2018, 11(2): 119-134. [4] KOSE M A, OZKAYA M. Towards extending UML’s activity diagram for the architectural modeling, analysis, and implementation[C]//Proceedings of the Federated Conference on Computer Science and Information Systems. Piscataway: IEEE Press, 2020: 639-648. [5] BOUTEKKOUK F. Architecture description languages taxonomies review: A special focus on self-adaptive distributed embedded systems[J]. International Journal of Technology Diffusion, 2021, 12(1): 53-74. doi: 10.4018/IJTD.2021010103 [6] SMOLYAKOV I Y, BELYAEV S A. Design of the software architecture for starcraft video game on the basis of finite state machines[C]//Proceedings of the IEEE Conference of Russian Young Researchers in Electrical and Electronic Engineering. Piscataway: IEEE Press, 2019: 356-359. [7] LEE E, KIM Y G, SEO Y D, et al. RINGA: Design and verification of finite state machine for self-adaptive software at runtime[J]. Information and Software Technology, 2018, 93: 200-222. doi: 10.1016/j.infsof.2017.09.008 [8] MARTÍN A G, FERNÁNDEZ-ISABEL A, DE DIEGO I M, et al. A survey for user behavior analysis based on machine learning techniques: Current models and applications[J]. Applied Intelligence, 2021, 51(8): 6029-6055. doi: 10.1007/s10489-020-02160-x [9] LIAN J. Implementation of computer network user behavior forensic analysis system based on speech data system log[J]. International Journal of Speech Technology, 2020, 23(3): 559-567. doi: 10.1007/s10772-020-09747-2 [10] ZHAO X, BHUIYAN M Z A, QI L Y, et al. TrCMP: A dependable app usage inference design for user behavior analysis through cyber-physical parameters[J]. Journal of Systems Architecture, 2020, 102: 101665. doi: 10.1016/j.sysarc.2019.101665 [11] REN F J, BAO Y W. A review on human-computer interaction and intelligent robots[J]. International Journal of Information Technology & Decision Making, 2020, 19(1): 5-47. [12] MU F R, LIU C Y. Design and research of intelligent logistics robot based on STM32[J]. Recent Advances in Electrical & Electronic Engineering, 2021, 14(1): 44-51. [13] BAI X J, DING X Y, WU M X. Intelligent classification robot based on image recognition[C]//Proceedings of the International Conference on Optoelectronic Materials and Devices. Bellinham: SPIE, 2021: 117671M. [14] WANG T M, TAO Y, LIU H. Current researches and future development trend of intelligent robot: A review[J]. International Journal of Automation and Computing, 2018, 15(5): 525-546. doi: 10.1007/s11633-018-1115-1 [15] HE W, LI Z J, CHEN C L P. A survey of human-centered intelligent robots: Issues and challenges[J]. IEEE/CAA Journal of Automatica Sinica, 2017, 4(4): 602-609. doi: 10.1109/JAS.2017.7510604 [16] LOWE D G. Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision, 2004, 60(2): 91-110. doi: 10.1023/B:VISI.0000029664.99615.94 -
下载:
点击查看大图
计量
- 文章访问数: 211
- HTML全文浏览量: 103
- PDF下载量: 1
- 被引次数: 0
