| Citation: | JI X，WU T X，YU T，et al. Power text information extraction based on multi-task learning[J]. Journal of Beijing University of Aeronautics and Astronautics，2024，50（8）：2461-2469 （in Chinese） doi: 10.13700/j.bh.1001-5965.2022.0683
|
In order to improve the analysis and processing speed of power system fault text in actual business scenarios, a power fault text information extraction model based on pre-training and multi-task learning was proposed. The pre-training model was used to learn the context information of power text words. The first-order and second-order fusion features of words were mined, which enhanced the representation ability of features. The multi-task learning framework was used to combine named entity recognition and relation extraction, which realized the mutual supplement and mutual promotion of entity recognition and relationship extraction, so as to improve the performance of power fault text information extraction. The model was verified by the daily business data of a power data center. Compared with other models, the proposed model’s accuracy and recall of power fault text entity recognition and relation extraction were improved.
| [1] |
周念成, 廖建权, 王强钢, 等. 深度学习在智能电网中的应用现状分析与展望[J]. 电力系统自动化, 2019, 43(4): 180-191. doi: 10.7500/AEPS20180323002
ZHOU N C, LIAO J Q, WANG Q G, et al. Analysis and prospect of deep learning application in smart grid[J]. Automation of Electric Power Systems, 2019, 43(4): 180-191 (in Chinese). doi: 10.7500/AEPS20180323002
|
| [2] |
李刚, 张博, 赵文清, 等. 电力设备状态评估中的数据科学问题: 挑战与展望[J]. 电力系统自动化, 2018, 42(21): 10-20. doi: 10.7500/AEPS20180331006
LI G, ZHANG B, ZHAO W Q, et al. Data science issues in state evaluation of power equipment: Challenges and prospects[J]. Automation of Electric Power Systems, 2018, 42(21): 10-20 (in Chinese). doi: 10.7500/AEPS20180331006
|
| [3] |
薛禹胜, 赖业宁. 大能源思维与大数据思维的融合(一)大数据与电力大数据[J]. 电力系统自动化, 2016, 40(1): 1-8. doi: 10.7500/AEPS20151208005
XUE Y S, LAI Y N. Integration of macro energy thinking and big data thinking part one big data and power big data[J]. Automation of Electric Power Systems, 2016, 40(1): 1-8 (in Chinese). doi: 10.7500/AEPS20151208005
|
| [4] |
邱剑, 王慧芳, 应高亮, 等. 文本信息挖掘技术及其在断路器全寿命状态评价中的应用[J]. 电力系统自动化, 2016, 40(6): 107-112. doi: 10.7500/AEPS20150812003
QIU J, WANG H F, YING G L, et al. Text mining technique and application of lifecycle condition assessment for circuit breaker[J]. Automation of Electric Power Systems, 2016, 40(6): 107-112(in Chinese). doi: 10.7500/AEPS20150812003
|
| [5] |
邵冠宇, 王慧芳, 吴向宏, 等. 基于依存句法分析的电力设备缺陷文本信息精确辨识方法[J]. 电力系统自动化, 2020, 44(12): 178-185. doi: 10.7500/AEPS20190401001
SHAO G Y, WANG H F, WU X H, et al. Precise information identification method of power equipment defect text based on dependency parsing[J]. Automation of Electric Power Systems, 2020, 44(12): 178-185 (in Chinese). doi: 10.7500/AEPS20190401001
|
| [6] |
郑伟彦, 杨勇, 卢家驹, 等. 面向配电网知识图谱的配电调度文本实体链接方法[J]. 电力系统保护与控制, 2021, 49(4): 111-117.
ZHENG W Y, YANG Y, LU J J, et al. Entity linking method of distribution dispatching texts for a distribution network knowledge graph[J]. Power System Protection and Control, 2021, 49(4): 111-117 (in Chinese).
|
| [7] |
余建明, 王小海, 张越, 等. 面向智能调控领域的知识图谱构建与应用[J]. 电力系统保护与控制, 2020, 48(3): 29-35.
YU J M, WANG X H, ZHANG Y, et al. Construction and application of knowledge graph for intelligent dispatching and control[J]. Power System Protection and Control, 2020, 48(3): 29-35 (in Chinese).
|
| [8] |
何炎祥, 罗楚威, 胡彬尧. 基于CRF和规则相结合的地理命名实体识别方法[J]. 计算机应用与软件, 2015, 32(1): 179-185. doi: 10.3969/j.issn.1000-386x.2015.01.046
HE Y X, LUO C W, HU B Y. Geographic entity recognition method based on CRF model and rules combination[J]. Computer Applications and Software, 2015, 32(1): 179-185 (in Chinese). doi: 10.3969/j.issn.1000-386x.2015.01.046
|
| [9] |
JACOBS P S, RAU L F. SCISOR: Extracting information from on-line news[J]. Communications of the ACM, 1990, 33(11): 88-97. doi: 10.1145/92755.92769
|
| [10] |
WILLIAMS R J, ZIPSER D. A learning algorithm for continually running fully recurrent neural networks[J]. Neural Computation, 1989, 1(2): 270-280. doi: 10.1162/neco.1989.1.2.270
|
| [11] |
BEKOULIS G, DELEU J, DEMEESTER T, et al. Joint entity recognition and relation extraction as a multi-head selection problem[J]. Expert Systems with Applications, 2018, 114: 34-45. doi: 10.1016/j.eswa.2018.07.032
|
| [12] |
BEKOULIS G, DELEU J, DEMEESTER T, et al. Adversarial training for multi-context joint entity and relation extraction [EB/OL]. (2018-08-21) [2019-01-14].https://arxiv.org/abs/ 1808.06876.
|
| [13] |
HOCHREITER S, SCHMIDHUBER J. Long short-term memory[J]. Neural Computation, 1997, 9(8): 1735-1780. doi: 10.1162/neco.1997.9.8.1735
|
| [14] |
HUANG Z H, XU W, YU K. Bidirectional LSTM-CRF models for sequence tagging[EB/OL]. (2015-08-09) [2019-01-20]. http://arxiv.org/abs/1508.01991.
|
| [15] |
KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6): 84-90. doi: 10.1145/3065386
|
| [16] |
NGUYEN T H, GRISHMAN R. Relation extraction: Perspective from convolutional neural networks[C]//Proceedings of the Workshop on Vector Space Modeling for Natural Language Processing. Stroudsburg: Association for Computational Linguistics, 2015: 39-48.
|
| [17] |
VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Proceedings of the International Conference on Neural Information Processing Systems. New York: ACM, 2017: 6000-6010.
|
| [18] |
LI C, TIAN Y. Downstream model design of pre-trained language model for relation extraction task[EB/OL]. (2020-04-08) [2021-01-05]. http://arxiv.org/abs/2004.03786.
|
| [19] |
佟佳弘, 武志刚, 管霖, 等. 电力调度文本的自然语言理解与解析技术及应用[J]. 电网技术, 2020, 44(11): 4148-4156.
TONG J H, WU Z G, GUAN L, et al. Power dispatching text analysis and application based on natural language understanding[J]. Power System Technology, 2020, 44(11): 4148-4156 (in Chinese).
|
| [20] |
蒋晨, 王渊, 胡俊华, 等. 基于深度学习的电力实体信息识别方法[J]. 电网技术, 2021, 45(6): 2141-2149.
JIANG C, WANG Y, HU J H, et al. Power entity information recognition based on deep learning[J]. Power System Technology, 2021, 45(6): 2141-2149 (in Chinese).
|
| [21] |
LIU Y H, OTT M, GOYAL N, et al. RoBERTa: A robustly optimized BERT pretraining approach[EB/OL]. (2019-07-26) [2021-01-06]. http://arxiv.org/abs/1907.11692.
|
| [22] |
LAFFERTY J D, MCCALLUM A, PEREIRA F C N. Conditional random fields: Probabilistic models for segmenting and labeling sequence data[C]//Proceedings of the Eighteenth International Conference on Machine Learning. New York: ACM, 2001: 282-289.
|
| [23] |
WEI M, XU Z P, HU J W. Entity relationship extraction based on Bi-LSTM and attention mechanism[C]//Proceedings of the International Conference on Artificial Intelligence and Information Systems. New York: ACM, 2021: 1-5.
|
| [24] |
杨秋勇 , 彭泽武, 苏华权, 等. 基于Bi-LSTM-CRF的中文电力实体识别[J]. 信息技术, 2021, 9: 45-50.
YANG Q Y, PENG Z W, SU H Q, et al. Chinese electric power entity recognition based on Bi-LSTM-CRF[J]. Information Technology, 2021, 9: 45-50 (in Chinese).
|
| [25] |
DOZAT T, MANNING C D. Deep biaffine attention for neural dependency parsing [EB/OL]. (2016-11-06) [2021-02-10].https://arxiv.org/abs/1611.01734.
|
| [26] |
ZHOU W X, HUANG K, MA T Y, et al. Document-level relation extraction with adaptive thresholding and localized context pooling[C]//Proceedings of the AAAI Conference on Artificial Intelligence. Palo Alto: AAAI Press, 2021, 35(16): 14612-14620.
|
| [27] |
RAM R V S, AKILANDESWARI A, DEVI S L. Linguistic features for named entity recognition using CRFs[C]//Proceedings of the International Conference on Asian Language Processing. Piscataway: IEEE Press, 2010: 158-161.
|
| [28] |
MA X Z, HOVY E. End-to-end sequence labeling via Bi-directional LSTM-CNNs-CRF[C]//Proceedings of the Annual Meeting of the Association for Computational Linguistics. Stroudsburg: Association for Computational Linguistics, 2016: 781-792.
|
Figures(3) / Tables(9)
Copyright © Journal of Beijing University of Aeronautics and Astronautics
Address: Editorial Department of Journal of Beijing University of Aeronautics and Astronautics, 37 Xueyuan Road, Haidian District, Beijing Post Code: 100191 Email: jbuaa@buaa.edu.cn
Supported by:
Beijing Renhe Information Technology Co., Ltd.
DownLoad:
