基于LSTM循环神经网络的故障时间序列预测

doi:10.13700/j.bh.1001-5965.2017.0285

北京航空航天大学学报 ›› 2018, Vol. 44 ›› Issue (4): 772-784.doi: 10.13700/j.bh.1001-5965.2017.0285

基于LSTM循环神经网络的故障时间序列预测

王鑫¹, 吴际¹, 刘超¹, 杨海燕¹, 杜艳丽², 牛文生^1,3

1. 北京航空航天大学计算机学院, 北京 100083;
2. 丰台职业教育中心学校, 北京 100076;
3. 中航工业西安航空计算技术研究所, 西安 710068

收稿日期:2017-05-08 出版日期:2018-04-20 发布日期:2018-04-27
通讯作者: 吴际 E-mail:wuji@buaa.edu.cn
作者简介:王鑫,男,博士研究生。主要研究方向:数据驱动技术;吴际,男,博士,副教授,硕士生导师。主要研究方向:模型驱动、软件可靠性分析;刘超,男,博士,教授,博士生导师。主要研究方向:软件工程、软件测试。
基金资助:
中国民用航空专项研究项目（MJ-S-2013-10）；国防科工局技术基础项目（JSZL2014601B008）；国家自然科学基金（61602237）

Exploring LSTM based recurrent neural network for failure time series prediction

WANG Xin¹, WU Ji¹, LIU Chao¹, YANG Haiyan¹, DU Yanli², NIU Wensheng^1,3

1. School of Computer Science and Engineering, Beijing University of Aeronautics and Astronautics, Beijing 100083, China;
2. Fengtai Vocational Education Central School, Beijing 100076, China;
3. Aeronautical Computing Technique Research Institute, Aviation Industry Corporation of China, Xi'an 710068, China

Received:2017-05-08 Online:2018-04-20 Published:2018-04-27
Supported by:
China Civil Aviation Special Research Project (MJ-S-2013-10); Technology Foundation Program of the National Defense Technology Industry Ministry (JSZL2014601B008); National Natural Science Foundation of China (61602237)

摘要/Abstract

摘要： 有效地预测使用阶段的故障数据对于合理制定可靠性计划以及开展可靠性维护活动等具有重要的指导意义。从复杂系统的历史故障数据出发，提出了一种基于长短期记忆（LSTM）循环神经网络的故障时间序列预测方法，包括网络结构设计、网络训练和预测过程实现算法等，进一步以预测误差最小为目标，提出了一种基于多层网格搜索的LSTM预测模型参数优选算法，通过与多种典型时间序列预测模型的实验对比，验证了所提出的LSTM预测模型及其参数优选算法在故障时间序列分析中具有很强的适用性和更高的准确性。

关键词: 长短期记忆(LSTM)模型, 循环神经网络, 故障时间序列预测, 多层网格搜索, 深度学习

Abstract: Effectively forecasting the failure data in the usage stage is essential to reasonably make reliability plans and carry out reliability maintaining activities. Beginning with the historical failure data of complex system, a long short-term memory (LSTM) based recurrent neural network for failure time series prediction is presented, in which the design of network structure, the procedures and algorithms of network training and forecasting are involved. Furthermore, a multilayer grid search algorithm is proposed to optimize the parameters of LSTM prediction model. The experimental results are compared with various typical time series prediction models, and validate that the proposed LSTM prediction model and the corresponding parameter optimization algorithm have strong adaptiveness and higher accuracy in failure time series prediction.

Key words: long short-term memory (LSTM) model, recurrent neural network, failure time series prediction, multilayer grid search, deep learning

中图分类号:

王鑫, 吴际, 刘超, 杨海燕, 杜艳丽, 牛文生. 基于LSTM循环神经网络的故障时间序列预测[J]. 北京航空航天大学学报, 2018, 44(4): 772-784.

WANG Xin, WU Ji, LIU Chao, YANG Haiyan, DU Yanli, NIU Wensheng. Exploring LSTM based recurrent neural network for failure time series prediction[J]. JOURNAL OF BEIJING UNIVERSITY OF AERONAUTICS AND A, 2018, 44(4): 772-784.

参考文献

[1] VICHARE N M,PECHT M G.Prognostics and health management of electronics[J].IEEE Transactions on Components & Packaging Technologies,2006,29(1):222-229.
[2] SAPANKEVYCH N I,SANKAR R.Time series prediction using support vector machines:A survey[J].IEEE Computational Intelligence Magazine,2009,4(2):24-38.
[3] 王鑫,吴际,刘超,等.奇异谱分析在故障时间序列分析中的应用[J].北京航空航天大学学报,2016,42(11):2321-2331.WANG X,WU J,LIU C,et al.Application of singular spectrum analysis for failure time series[J].Journal of Beijing University of Aeronautics and Astronautics,2016,42(11):2321-2331(in Chinese).
[4] 李瑞莹,康锐.基于ARMA模型的故障率预测方法研究[J].系统工程与电子技术,2008,30(8):1588-1591.LI R Y,KANG R.Research on failure rate forecasting method based on ARMA model[J].Systems Engineering and Electronics,2008,30(8):1588-1591(in Chinese).
[5] ROCCO S C M.Singular spectrum analysis and forecasting of failure time series[J].Reliability Engineering & System Safety,2013,114(6):126-136.
[6] MOURA M D C,ZIO E,LINS I D,et al.Failure and reliability prediction by support vector machines regression of time series data[J].Reliability Engineering & System Safety,2011,96(11):1527-1534.
[7] XU K,XIE M,TANG L C,et al.Application of neural networks in forecasting engine systems reliability[J].Applied Soft Computing,2003,2(4):255-268.
[8] WANG X,WU J,LIU C,et al.A hybrid model based on singular spectrum analysis and support vector machines regression for failure time series prediction[J].Quality & Reliability Engineering International,2016,32(8):2717-2738.
[9] LECUN Y,BENGIO Y,HINTON G.Deep learning[J].Nature,2015,521(7553):436-444.
[10] GRAVES A.Long short-term memory[M].Berlin:Springer,2012:1735-1780.
[11] SRIVASTAVA N,MANSIMOV E,SALAKHUTDINOV R.Unsupervised learning of video representations using LSTMs[C]//Proceedings of the 32nd International Conference on Machine Learning.Lille:JMLR W&CP,2015:843-852.
[12] DONAHUE J,HENDRICKS L A,ROHRBACH M,et al.Long-term recurrent convolutional networks for visual recognition and description[J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2015,39(4):677-691.
[13] VINYALS O,TOSHEV A,BENGIO S,et al.Show and tell:A neural image caption generator[C]//IEEE Conference on Computer Vision and Pattern Recognition.Piscataway,NJ:IEEE Press,2015:3156-3164.
[14] MA X,TAO Z,WANG Y,et al.Long short-term memory neural network for traffic speed prediction using remote microwave sensor data[J].Transportation Research Part C Emerging Technologies,2015,54:187-197.
[15] HANSON J,YANG Y,PALIWAL K,et al.Improving protein disorder prediction by deep bidirectional long short-term memory recurrent neural networks[J].Bioinformatics,2017,33(5):685.
[16] GREFF K,SRIVASTAVA R K,KOUTNIK J,et al.LSTM:A search space odyssey[J].IEEE Transactions on Neural Networks & Learning Systems,2016,PP(99):1-11.
[17] GRAVES A,SCHMIDHUBER J.Framewise phoneme classification with bidirectional LSTM and other neural network architectures[J].Neural Networks,2005,18(5-6):602.
[18] AMARI S I.Backpropagation and stochastic gradient descent method[J].Neurocomputing,1993,5(4-5):185-196.
[19] DUCHI J,HAZAN E,SINGER Y.Adaptive subgradient methods for online learning and stochastic optimization[J].Journal of Machine Learning Research,2011,12(7):257-269.
[20] YEUNG S,RUSSAKOVSKY O,NING J,et al.Every moment counts:Dense detailed labeling of actions in complex videos[J].International Journal of Computer Vision,2017(8):1-15.
[21] KINGMA D P,BA J.Adam:A method for stochastic optimization[C]//ICLR 2015,2015:1-15.
[22] CHUNG J,GULCEHRE C,CHO K H,et al.Empirical evaluation of gated recurrent neural networks on sequence modeling[C]//NIPS 2014 Deep Learning and Representation Learning Workshop,2014:1-9.
[23] CHEN P W,WANG J Y,LEE H M.Model selection of SVMs using GA approach[C]//IEEE International Joint Conference on Neural Networks.Piscataway,NJ:IEEE Press,2004:2035-2040.
[24] BRATTON D,KENNEDY J.Defining a standard for particle swarm optimization[C]//IEEE Swarm Intelligence Symposium.Piscataway,NJ:IEEE Press,2007:120-127.
[25] SNOEK J,LAROCHELLE H,ADAMS R P.Practical Bayesian optimization of machine learning algorithms[C]//International Conference on Neural Information Processing Systems.Lake Tahoe:Curran Associates Inc.,2012:2951-2959.
[26] HSU C W,CHANG C C,LIN C J.A practical guide to support vector classification[EB/OL].(2016-05-19)[2017-03-20]. https://www.csie.ntu.edu.tw/~cjlin/papers/guide/guide.pdf.
[27] SU C,JIN Q,FU Y.Correlation analysis for wind speed and failure rate of wind turbines using time series approach[J].Journal of Renewable & Sustainable Energy,2012,4(3):687-700.
[28] CHATFIELD C.The Holt-Winters forecasting procedure[J].Journal of the Royal Statistical Society,1978,27(3):264-279.
[29] BARTHOLOMEW D J.Time series analysis forecasting and control[J].Journal of the Operational Research Society,1971,22(2):199-201.
[30] HYNDMAN R J,KHANDAKAR Y.Automatic time series forecasting:The forecast package for R[J].Journal of Statistical Software,2008,27(3):1-22.
[31] VAUTARD R,YIOU P,GHIL M.Singular-spectrum analysis:A toolkit for short,noisy chaotic signals[J].Physica D-Nonlinear Phenomena,1992,58(1-4):95-126.
[32] GOLYANDINA N,KOROBEYNIKOV A.Basic singular spectrum analysis and forecasting with R[J].Computational Statistics & Data Analysis,2014,71(1):934-954.
[33] NIKOLOPOULOS K,GOODWIN P,PATELIS A,et al.Forecasting with cue information:A comparison of multiple regression with alternative forecasting approaches[J].European Journal of Operational Research,2007,180(1):354-368.
[34] BIANCO V,MANCA O,NARDINI S.Electricity consumption forecasting in Italy using linear regression models[J].Energy,2009,34(9):1413-1421.
[35] BRERETON R G,LLOYD G R.Support vector machines for classification and regression[J].Analyst,2010,135(2):230-267.
[36] CHERKASSKY V,MA Y.Practical selection of SVM parameters and noise estimation for SVM regression[J].Neural Networks,2004,17(1):113-126.
[37] Google.tensorflow rminer 1.4.2[EB/OL].[2017-03-20].https://www.tensorflow.org/versions/r0.12.
[38] R Core Team.The R project for statistical computing[EB/OL].[2017-03-20].https://www.r-project.org.
[39] HYNDMAN R.robjhyndman/forecast[EB/OL].[2017-03-20].https://github.com/robjhyndman/forecast.
[40] KOROBEYNIKOV A.asl/rssa[EB/OL].[2017-03-20].https://github.com/asl/rssa.
[41] PAULO C.rminer:Data mining classification and regression methods[EB/OL].[2017-03-20].https://cran.r-project.org/package=rminer.

基于LSTM循环神经网络的故障时间序列预测

Exploring LSTM based recurrent neural network for failure time series prediction

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 10

编辑推荐

Metrics

本文评价

[1]	张克明, 蔡远文, 任元. 基于生成对抗网络的航天异常事件检测方法[J]. 北京航空航天大学学报, 2019, 45(7): 1329-1336.
[2]	魏宏喜, 张越. 基于生成对抗网络的零样本图像分类[J]. 北京航空航天大学学报, 2019, 45(12): 2345-2350.
[3]	张新峰, 郭宇桐, 蔡轶珩, 孙萌. 基于DCNN和全连接CRF的舌图像分割算法[J]. 北京航空航天大学学报, 2019, 45(12): 2364-2374.
[4]	张梦琴, 孟权令, 张维刚. 基于深度视觉语义嵌入的视频缩略图推荐[J]. 北京航空航天大学学报, 2019, 45(12): 2479-2486.
[5]	赵国, 秦世引. PMMWI与VI优势互补的人体隐蔽违禁物检测与定位[J]. 北京航空航天大学学报, 2019, 45(10): 2011-2025.
[6]	郭琳, 秦世引. 遥感图像飞机目标高效搜检深度学习优化算法[J]. 北京航空航天大学学报, 2019, 45(1): 159-173.
[7]	韩京冶, 许福, 陈志泊, 刘辉. 一种基于深度学习的交互式电话号码识别方法[J]. 北京航空航天大学学报, 2018, 44(5): 1074-1080.
[8]	车畅畅, 王华伟, 倪晓梅, 洪骥宇. 基于深度学习的航空发动机故障融合诊断[J]. 北京航空航天大学学报, 2018, 44(3): 621-628.
[9]	许轲, 吴凤鸽, 赵军锁. 基于深度强化学习的软件定义卫星姿态控制算法[J]. 北京航空航天大学学报, 2018, 44(12): 2651-2659.
[10]	王鑫, 侯志强, 余旺盛, 戴铂, 金泽芬芬. 基于深度稀疏学习的鲁棒视觉跟踪[J]. 北京航空航天大学学报, 2017, 43(12): 2554-2563.