基于图结构和序列特征融合的关系抽取

武同心; 纪鑫; 王宏刚; 杨智伟; 陈屹婷; 赵加奎

doi:10.13700/j.bh.1001-5965.2022.0706

基于图结构和序列特征融合的关系抽取

doi: 10.13700/j.bh.1001-5965.2022.0706

武同心^1, ,,
纪鑫^{1, 2},
王宏刚¹,
杨智伟¹,
陈屹婷¹,
赵加奎¹

1.
国家电网有限公司大数据中心，北京 100052
2.
北京航空航天大学计算机学院，北京 100191

基金项目: 国家电网有限公司大数据中心科技项目(52999021N005)

详细信息

通讯作者:
E-mail：tongxin-wu@sgcc.com.cn

中图分类号: TP391.1
计量
- 文章访问数: 241
- HTML全文浏览量: 111
- PDF下载量: 21
- 被引次数: 0
出版历程
- 收稿日期: 2022-08-11
- 录用日期: 2022-10-04
- 网络出版日期: 2022-11-07
- 整期出版日期: 2024-09-27

Relation extraction based on fusion of graph structure and sequence features

1.
Big Data Center of State Grid Corporation of China，Beijing 100052，China
2.
School of Computer Science and Engineering，Beihang University，Beijing 100191，China

Funds: Big Data Center of State Grid Corporation of China Technology Project (52999021N005)

More Information

Corresponding author: E-mail：tongxin-wu@sgcc.com.cn

摘要

摘要:
关系抽取是自然语言处理应用中的一项重要任务。现有的关系抽取方法主要基于语言序列特征或句子结构信息来预测关系，并不能有效地反映实体间关系的内在结构和特征。为此，提出一种融合句子中图结构和序列特征信息的关系抽取模型。该模型利用基于注意力的图卷积网络（GCN）学习语句中的结构信息，利用双向长短期记忆（BiLSTM）网络学习语句的序列语义特征，通过注意力机制结合句子结构特征和序列语义对关系进行分类。在公共数据集和手工构建的数据集上进行了大量实验，验证了所提模型的优越性。
- 信息抽取 /
- 关系抽取 /
- 图神经网络 /
- 序列模型 /
- 特征融合
Abstract:
Relation extraction is an important task for natural language processing applications. Most of the existing relation extraction methods mainly predict the relation based on language sequence features or structure information of sentences, which fails to effectively reflect the internal structure and features of the relation between entities. In this paper, a relation extraction model fusing graph structure and sequence feature information in sentences was proposed. The model used an attention-based graph convolutional neural network (GCN) to learn the structure information of sentences and utilized bi-directional long short-term memory (BiLSTM) to learn the sequence semantics. The relation was classified by considering the two features through the attention mechanism. Extensive experiments were conducted on a public dataset and a manually constructed dataset, which demonstrated the priority of the proposed model.
- information extraction /
- relation extraction /
- graph neural network /
- sequence model /
- feature fusion

HTML全文

图 1 CaStr模型结构

Figure 1. Structure of CaStr model

下载: 全尺寸图片幻灯片

图 2 基于密集连接的级联连接

Figure 2. Cascaded connections based on dense connections

下载: 全尺寸图片幻灯片

图 3 基于注意力机制的融合

Figure 3. Attention mechanism-based fusion

下载: 全尺寸图片幻灯片

表 1 CSGC-3数据子集中不同关系的数量

Table 1. Number of relations in CSGC-3 subdatasets

子集	从属/个	缺陷/个	原因/个
训练集	1796	860	930
验证集	143	64	83
测试集	151	62	105

下载: 导出CSV

表 2 在Semeval-10^[38]数据集上的句级关系抽取性能表现

Table 2. Sentence-level relation extraction performance on Semeval-10^[38] dataset %

模型	精确率	召回率	F₁
CNN^[7]	63.63	57.76	60.55
BiLSTM^[18]	66.01	59.21	62.40
Tree-LSTM^[25]	67.73	60.03	63.65
AGGCN^[10]	69.92	60.91	65.11
CaStr	71.03	61.76	66.07

下载: 导出CSV

表 3 在 CSGC-3数据集上的句级关系抽取性能表现

Table 3. Sentence-level relation extraction performance on CSGC-3 dataset %

模型	精确率	召回率	F₁
CNN^[7]	94.13	87.62	90.75
BiLSTM^[18]	94.48	88.17	91.21
Tree-LSTM^[25]	96.74	89.64	93.05
AGGCN^[10]	97.14	89.84	93.35
CaStr	98.22	90.27	94.08

下载: 导出CSV

表 4 在 PubMed^[39]数据集上的跨句关系抽取

Table 4. Cross-sentence relation extraction on PubMed^[39] dataset %

模型	精确率	召回率	F₁
CNN^[7]	81.85	76.79	79.24
BiLSTM^[18]	84.78	78.74	81.65
Tree-LSTM^[25]	86.87	80.34	83.48
AGGCN^[10]	88.56	81.67	84.98
CaStr	90.23	83.17	86.56

下载: 导出CSV

表 5 在 CSGC-3数据集上的跨句关系抽取性能

Table 5. Cross-sentence relation extraction performance on CSGC-3 dataset %

模型	精确率	召回率	F₁
CNN^[7]	68.78	63.56	66.06
BLSTM^[18]	70.08	65.27	67.59
Tree-LSTM^[25]	73.75	68.34	70.94
AGGCN^[10]	77.91	72.11	74.90
CaStr	79.83	73.22	76.38

下载: 导出CSV

表 6 在 Semeval-10^[38]数据集上的消融实验

Table 6. Ablation experiments on Semeval-10^[38] dataset %

模型	精确率	召回率	F₁
CaStr-NO-BiLSTM	65.35	56.78	60.76
CaStr-NO-ATT	68.41	59.01	63.36
CaStr	71.03	61.76	66.07

下载: 导出CSV

表 7 在 CSGC-3 数据集上的消融实验

Table 7. Ablation experiments on CSGC-3 dataset %

模型	精确率	召回率	F₁
CaStr-NO-BiLSTM	93.12	84.91	88.82
CaStr-NO-ATT	96.03	86.85	91.21
CaStr	98.22	90.27	94.08

下载: 导出CSV

表 8 不同模型从CSGC-3数据集中抽取出的关系实例

Table 8. Examples of relation extracted by different models from CSGC-3 dataset

文本	由CaStr模型输出的三元组	由BiLSTM^[18]网络模型输出的三元组	由AGGCN^[10]模型输出的三元组
1号主变压器控制箱内液晶显示器与变压器的温差较大，怀疑显示不准确。	{主变压器，控制箱，从属}， {控制箱，温差，缺陷}， {温差，显示不准确，原因}。	{主变压器，控制箱，从属}， {主变压器，温差，缺陷}， {温差，显示不准确，原因}。	{主变压器，机械限制，从属}， {机械限制，松动，缺陷}， {松动，开关的多个齿轮滑动，原因}。
变压器的机械限制松动，导致开关的多个齿轮滑动。	{主变压器，机械限制，从属}， {机械限制，开关的多个齿轮，缺陷}， {开关的多个齿轮，松动，原因}。

下载: 导出CSV

参考文献(41)

[1]	BOLLACKER K, EVANS C, PARITOSH P, et al. Freebase: A collaboratively created graph database for structuring human knowledge[C]//Proceedings of the ACM SIGMOD International Conference on Management of Data. New York: ACM, 2008: 1247-1250.
[2]	YU M, YIN W P, HASAN K S, et al. Improved neural relation detection for knowledge base question answering[EB/OL]. (2017-04-20)[2022-04-10]. http://arxiv.org/abs/1704.06194.
[3]	宁康, 陈挺. 生物医学大数据的现状与展望[J]. 科学通报, 2015, 60(5): 534-546. NING K, CHEN T. Big Data for biomedical research: Current status and prospects[J]. Chin Sci Bull, 2015, 60(5): 534-546(in Chinese).
[4]	王群弼. 电力领域实体关系抽取及知识图谱构建研究[D]. 北京: 中国地质大学(北京), 2020: 23-70. WANG Q B. Research on relation extraction and knowledge graph construction in the filed of electric power[D]. Beijing: China University of Geosciences (Beijing), 2020: 23-70(in Chinese).
[5]	BEKOULIS G, DELEU J, DEMEESTER T, et al. Adversarial training for multi-context joint entity and relation extraction[EB/OL]. (2018-08-21)[2022-04-12]. http://arxiv.org/abs/1808.06876.
[6]	LECUN Y, BOTTOU L, BENGIO Y, et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998, 86(11): 2278-2324. doi: 10.1109/5.726791
[7]	LIU C Y, SUN W B, CHAO W H, et al. Convolution neural network for relation extraction[C]//Proceedings of the International Conference on Advanced Data Mining and Applications. Berlin: Springer, 2013: 231-242.
[8]	CHENG J P, DONG L, LAPATA M. Long short-term memory-networks for machine reading[EB/OL]. (2016-01-25)[2022-04-12]. http://arxiv.org/abs/1601.06733.
[9]	TAI K S, SOCHER R, MANNING C D. Improved semantic representations from tree-structured long short-term memory networks[EB/OL]. (2015-02-28)[2022-04-12]. http://arxiv.org/abs/1503.00075.
[10]	GUO Z J, ZHANG Y, LU W. Attention guided graph convolutional networks for relation extraction[EB/OL]. (2019-06-18)[2022-04-13]. http://arxiv.org/abs/1906.07510.
[11]	KIPF T N, WELLING M. Semi-supervised classification with graph convolutional networks[EB/OL]. (2016-09-09)[2022-04-13]. http://arxiv.org/abs/1609.02907.
[12]	SHI M Y, HUANG J Y, LI C F. Entity relationship extraction based on BLSTM model[C]//Proceedings of the IEEE/ACIS International Conference on Computer and Information Science. Piscataway: IEEE Press, 2019: 266-269.
[13]	LI J, WANG X, TU Z P, et al. On the diversity of multi-head attention[J]. Neurocomputing, 2021, 454: 14-24. doi: 10.1016/j.neucom.2021.04.038
[14]	ZENG D J, LIU K, LAI S W, et al. Relation classification via convolutional deep neural network[C]//Proceedings of the COLING International Conference on Computational Linguistics: Technical Papers. Stroudsburg: Association for Computational Linguistics, 2014: 2335-2344.
[15]	JIANG X T, WANG Q, LI P, et al. Relation extraction with multi-instance multi-label convolutional neural networks[C]//Proceedings of the COLING International Conference on Computational Linguistics: Technical Papers. Stroudsburg: Association for Computational Linguistics, 2016: 1471-1480.
[16]	LIN Y K, SHEN S Q, LIU Z Y, et al. Neural relation extraction with selective attention over instances[C]//Proceedings of the Annual Meeting of the Association for Computational Linguistics. Stroudsburg: Association for Computational Linguistics, 2016: 2124-2133.
[17]	NGUYEN T H, GRISHMAN R. Relation extraction: Perspective from convolutional neural networks[C]//Proceedings of the Workshop on Vector Space Modeling for Natural Language Processing. Stroudsburg: Association for Computational Linguistics, 2015: 39-48.
[18]	ZHOU P, SHI W, TIAN J, et al. Attention-based bidirectional long short-term memory networks for relation classification[C]//Proceedings of the Annual Meeting of the Association for Computational Linguistics. Stroudsburg: Association for Computational Linguistics, 2016: 207-212.
[19]	ZHANG Y H, ZHONG V, CHEN D Q, et al. Position-aware attention and supervised data improve slot filling[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing. Stroudsburg: Association for Computational Linguistics, 2017: 35-45.
[20]	HUANG Z H, XU W, YU K. Bidirectional LSTM-CRF models for sequence tagging[EB/OL]. (2015-08-09)[2022-04-15] http://arxiv.org/abs/1508.01991.
[21]	SCARSELLI F, GORI M, TSOI A C, et al. The graph neural network model[J]. IEEE Transactions on Neural Networks, 2009, 20(1): 61-80. doi: 10.1109/TNN.2008.2005605
[22]	FU T J, LI P H, MA W Y. GraphRel: Modeling text as relational graphs for joint entity and relation extraction[C]//Proceedings of the Annual Meeting of the Association for Computational Linguistics. Stroudsburg: Association for Computational Linguistics, 2019: 1409-1418.
[23]	ZHANG Y H, QI P, MANNING C D. Graph convolution over pruned dependency trees improves relation extraction[EB/OL]. (2018-09-26)[2022-04-16]. http://arxiv.org/abs/1809.10185.
[24]	GENG Z Q, CHEN G F, HAN Y M, et al. Semantic relation extraction using sequential and tree-structured LSTM with attention[J]. Information Sciences, 2020, 509: 183-192. doi: 10.1016/j.ins.2019.09.006
[25]	MIWA M, BANSAL M. End-to-end relation extraction using LSTMs on sequences and tree structures[EB/OL]. (2016-01-05)[2022-04-16]. http://arxiv.org/abs/1601.00770.
[26]	VERGA P, STRUBELL E, MCCALLUM A. Simultaneously self-attending to all mentions for full-abstract biological relation extraction[EB/OL]. (2018-02-28)[2022-04-17]. http://arxiv.org/abs/1802.10569.
[27]	YAO Y, YE D M, LI P, et al. DocRED: A large-scale document-level relation extraction dataset[EB/OL]. (2019-06-14)[2022-04-17]. http://arxiv.org/abs/1906.06127.
[28]	QUIRK C, POON H. Distant supervision for relation extraction beyond the sentence boundary[EB/OL]. (2019-09-15)[2022-04-17]. http://arxiv.org/abs/1609.04873.
[29]	NAN G S, GUO Z J, SEKULIĆ I, et al. Reasoning with latent structure refinement for document-level relation extraction[EB/OL]. (2020-05-13)[2022-04-18]. http://arxiv.org/abs/2005.06312.
[30]	CHRISTOPOULOU F, MIWA M, ANANIADOU S. Connecting the dots: Document-level neural relation extraction with edge-oriented graphs[EB/OL]. (2019-08-31)[2022-04-18]. http://arxiv.org/abs/1909.00228.
[31]	JIA R, WONG C, POON H. Document-level $N$-ary relation extraction with multiscale representation learning[EB/OL]. (2019-04-04)[2022-04-18]. http://arxiv.org/abs/1904.02347.
[32]	GUPTA P, RAJARAM S, SCHÜTZE H, et al. Neural relation extraction within and across sentence boundaries[C]//Proceedings of the AAAI Conference on Artificial Intelligence. Washton, D. C. : AAAI, 2019, 33(1): 6513-6520.
[33]	CHRISTOPOULOU F, MIWA M, ANANIADOU S. A walk-based model on entity graphs for relation extraction[C]//Proceedings of the Annual Meeting of the Association for Computational Linguistics. Stroudsburg: Association for Computional Linguistics, 2018: 81-88.
[34]	LIU Y, LAPATA M. Learning structured text representations[J]. Transactions of the Association for Computational Linguistics, 2018, 6: 63-75. doi: 10.1162/tacl_a_00005
[35]	PENNINGTON J, SOCHER R, MANNING C. Glove: Global vectors for word representation[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing. Stroudsburg: Association for Computational Linguistics, 2014: 1532-1543.
[36]	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Proceedings of the International Conference on Neural Information Processing Systems. New York: ACM, 2017: 6000–6010.
[37]	HUANG G, LIU Z, VAN DER MAATEN L, et al. Densely connected convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2017: 2261-2269.
[38]	HENDRICKX I, KIM S N, KOZAREVA Z, et al. SemEval-2010 task 8: Multi-way classification of semantic relations between pairs of nominals[EB/OL]. (2019-11-23)[2022-04-20]. http://arxiv.org/abs/1911.10422.
[39]	PENG N Y, POON H, QUIRK C, et al. Cross-sentence N-ary relation extraction with graph LSTMs[J]. Transactions of the Association for Computational Linguistics, 2017, 5: 101-115. doi: 10.1162/tacl_a_00049
[40]	MANNING C, SURDEANU M, BAUER J, et al. The stanford CoreNLP natural language processing toolkit[C]//Proceedings of the Annual Meeting of the Association for Computational Linguistics: System Demonstrations. Stroudsburg: Association for Computational Linguistics, 2014: 55-60.
[41]	KINGMA D P, BA J. Adam: A method for stochastic optimization[EB/OL]. (2014-12-22)[2022-04-20]. https://arxiv.org/abs/1412.6980.