无对齐实体场景的多语言知识图谱补全

唐榕氚; 徐秋程; 汤闻易; 翟飞飞; 周玉

doi:10.13700/j.bh.1001-5965.2023.0709

无对齐实体场景的多语言知识图谱补全

doi: 10.13700/j.bh.1001-5965.2023.0709

唐榕氚^{1, 2},
徐秋程²,
汤闻易²,
翟飞飞^{1, 3},
周玉^{1, 3, ,}

1.
中国科学院自动化研究所，北京 100190
2.
南京航空航天大学民航学院空中交通管理系统全国重点实验室，南京 210007
3.
北京中科凡语科技有限公司，北京 100190

基金项目:

空中交通管理系统全国重点实验室开放课题(SKLATM202203)

详细信息

通讯作者:
E-mail：yzhou@nlpr.ia.ac.cn

中图分类号: TP391
计量
- 文章访问数: 973
- HTML全文浏览量: 223
- PDF下载量: 65
- 被引次数: 0
出版历程
- 收稿日期: 2023-10-31
- 录用日期: 2023-12-29
- 网络出版日期: 2024-03-12
- 整期出版日期: 2026-01-31

Multilingual knowledge graph completion without aligned entity pairs

TANG Rongchuan^{1, 2},
XU Qiucheng²,
TANG Wenyi²,
ZHAI Feifei^{1, 3},
ZHOU Yu^{1, 3
, ,}

1.
Institute of Automation，Chinese Academy of Science，Beijing 100190，China
2.
State Key Laboratory of Air Traffic Management System，College of Civil Aviation，Nanjing University of Aeronautics and Astronautics，Nanjing 210007，China
3.
Beijing Fanyu Technology Co.，Ltd.，Beijing 100190，China

Funds:

Open Research Projects of State Key Laboratory of Air Traffic Management System (SKLATM202203)

More Information

Corresponding author: E-mail：yzhou@nlpr.ia.ac.cn

摘要

摘要:
多语言知识图谱补全(MKGC)旨在利用其他语言知识图谱的信息增强目标语言知识图谱上的链接预测性能。现有方法通常利用不同语言知识图谱之间预先对齐的实体对作为知识迁移的媒介，然而在实际场景中，不同语言知识图谱之间通常没有预先对齐的实体，导致难以实现知识迁移。针对上述无对齐实体场景，提出一种融合预训练语言模型信息的伪对齐实体生成模块，不断迭代生成新的对齐实体进行知识迁移。为区分不同语言知识图谱中信息对目标语言知识图谱的贡献度，提出一种基于多图注意力的图神经网络(MGA-GNN)用于对三元组进行编码，通过该网络输出的嵌入表征计算得到三元组的合理性得分，完成链接预测任务。为验证所提方法的有效性，在2个公开数据集DBP-5L和E-PKG上进行了实验验证，结果表明：所提方法在多个语言知识图谱上链接预测的性能超过了有对齐实体的MKGC方法，证明了该方法在更加实际场景下的优越性能。
- 多语言知识图谱补全 /
- 实体对齐 /
- 多图注意力 /
- 图神经网络 /
- 链接预测
Abstract:
The goal of multilingual knowledge graph completion (MKGC) is to improve link prediction performance on the target knowledge graph by leveraging data from other language-specific knowledge graphs. Existing methods usually use pre-aligned entities between different knowledge graphs to accomplish knowledge transfer. However, there are usually no pre-aligned entities between different knowledge graphs in practical scenarios, making knowledge transfer difficult to achieve. Considering the MKGC without aligned entity pairs, a pseudo-aligned entity generation module that integrates a pre-trained language model is proposed to iteratively generate new aligned entities for knowledge transfer. It is suggested to use a graph neural network based on multi-graph attention (MGA-GNN) to encode the triples in order to differentiate the information in various language-specific wisdom graphs. Finally, the plausibility of the triples is calculated via the embeddings output by the network to conduct the link prediction task. Experimental results on the DBP-5L and E-PKG datasets show the effectiveness of the proposed method and its superior performance in more practical scenarios.
- multilingual knowledge graph completion /
- entity alignment /
- multi-graph attention /
- graph neural network /
- link prediction

HTML全文

图 1 无对齐实体的多语言知识图谱补全框架

Figure 1. The framework of multilingual knowledge graph completion without alignment pairs

下载: 全尺寸图片幻灯片

表 1 数据集统计数据数量

Table 1. The statistic numbers of datasets

数据集	语言	实体数	关系数	训练数据量	验证数据量	测试数据量
DBP-5L^[6]	EL	5231	111	8670	4152	1017
	EN	13996	831	48652	24051	7464
	ES	12382	144	33036	16220	4810
	FR	13176	178	30139	14705	4171
	JA	11805	128	17979	8633	2162
E-PKG^[8]	DE	17223	21	45515	22753	7602
	EN	16544	21	60310	39150	10071
	ES	9595	21	18090	9039	3034
	FR	17068	21	47999	23994	8022
	IT	15670	21	42767	21377	7148
	JA	2642	21	10013	5002	1688

下载: 导出CSV

表 2 DBP-5L数据集实验结果

Table 2. Experimental results on the DBP-5L dataset

方法分类	方法	语言	H@1	H@10	MRR
单语言 KGC方法	TransE^[9]	EL	13.1	43.7	24.3
		EN	7.3	29.3	16.9
		ES	13.5	45.0	24.4
		FR	17.5	48.8	27.6
		JA	21.1	48.5	25.3
	RotatE^[11]	EL	14.5	36.2	26.2
		EN	12.3	30.4	20.7
		ES	21.2	53.9	33.8
		FR	23.2	55.5	35.1
		JA	26.4	60.2	39.8
	KG-BERT^[23]	EL	17.3	40.1	27.3
		EN	12.9	31.9	21.0
		ES	21.9	54.1	34.0
		FR	23.5	55.9	35.4
		JA	26.9	59.8	38.7
有对齐实体的 MKGC方法	KEnS^[6]	EL	28.1	56.9
		EN	15.1	39.8
		ES	23.6	60.1
		FR	25.5	62.9
		JA	32.1	65.3
	AlignKGC^[7]	EL	27.6	56.2	33.8
		EN	15.5	39.2	22.3
		ES	24.2	60.9	35.1
		FR	24.1	62.3	37.4
		JA	31.6	64.3	41.6
	SS-AGA^[8]	EL	30.8	58.6	35.3
		EN	16.3	41.3	23.1
		ES	25.5	61.9	36.6
		FR	27.1	65.5	38.3
		JA	34.6	66.9	42.9
无对齐实体的 MKGC方法	MGA-GNN	EL	33.6	77.1	49.3
		EN	19.5	59.8	33.7
		ES	26.1	64.7	39.3
		FR	31.2	70.5	44.6
		JA	28.5	60.6	39.8
注：加粗数值表示性能最优。

下载: 导出CSV

表 3 E-PKG数据集实验结果

Table 3. Experimental results on the E-PKG dataset

方法分类	方法	语言	H@1	H@10	MRR
单语言 KGC方法	TransE^[9]	DE	21.2	65.5	37.4
		EN	23.2	67.5	39.4
		ES	17.2	58.4	33.0
		FR	20.8	66.9	37.5
		IT	22.0	63.8	37.8
		JA	25.1	72.7	43.6
	RotatE^[11]	DE	22.3	64.3	38.2
		EN	24.2	66.8	40.0
		ES	18.3	58.9	33.7
		FR	22.1	64.3	38.2
		IT	22.5	64.0	38.1
		JA	26.3	71.9	41.8
	KG-BERT^[23]	DE	21.8	64.7	38.4
		EN	24.3	66.4	39.6
		ES	18.7	58.8	33.2
		FR	22.3	67.2	38.3
		IT	22.9	63.7	37.2
		JA	26.9	72.4	44.1
有对齐实体的 MKGC方法	KEnS^[6]	DE	24.3	65.8
		EN	26.2	69.5
		ES	21.3	59.5
		FR	25.4	68.2
		IT	25.1	64.6
		JA	33.5	73.6
	AlignKGC^[7]	DE	22.1	65.1	38.5
		EN	25.6	68.3	40.5
		ES	19.4	59.1	34.2
		FR	22.8	67.2	38.8
		IT	24.2	63.4	37.3
		JA	31.2	72.3	46.2
	SS-AGA^[8]	DE	24.6	66.3	39.4
		EN	26.7	69.8	41.5
		ES	21.0	60.1	36.3
		FR	25.9	68.7	40.2
		IT	24.9	63.8	38.4
		JA	33.9	74.1	48.3
无对齐实体的 MKGC方法	MGA-GNN	DE	24.9	68.4	40.3
		EN	29.3	70.4	44.4
		ES	22.2	60.3	35.6
		FR	24.1	68.9	39.6
		IT	26.1	65.9	40.9
		JA	43.0	73.2	54.5
注：加粗数值表示性能最优。

下载: 导出CSV

表 4 DBP-5L数据集中生成实体精确率和召回率

Table 4. The precision and recall of the generated aligned entities on the DBP-5L dataset %

语言对	精确率	召回率
EL-JA	97.1	25.4
EL-ES	98.6	68.4
EL-EN	99.2	66.8
EL-FR	99.0	64.6
JA-ES	97.6	22.1
JA-EN	98.0	25.3
JA-FR	98.6	21.2
ES-EN	99.7	74.5
ES-FR	99.6	70.7
EN-FR	99.7	71.1

下载: 导出CSV

表 5 E-PKG数据集中生成实体精确率和召回率

Table 5. The precision and recall of the generated aligned entities on the E-PKG dataset %

语言对	精确率	召回率
DE-EN	58.5	24.5
DE-ES	51.9	21.0
DE-FR	56.1	21.4
DE-IT	55.6	22.2
DE-JA	60.9	16.3
ES-EN	51.7	23.5
ES-FR	50.1	22.8
ES-IT	55.0	25.2
ES-JA	50.6	16.0
FR-EN	56.1	23.9
FR-IT	58.7	25.2
FR-JA	58.6	16.6
IT-EN	60.3	24.3
IT-JA	61.6	15.8
JA-EN	42.8	17.7

下载: 导出CSV

参考文献(23)

[1]	REN H Y, DAI H J, DAI B, et al. LEGO: latent execution-guided reasoning for multi-hop question answering on knowledge graphs[C]//Proceedings of the 38th International Conference on Machine Learning. San Diego: PMLR, 2021, 139: 8959-8970.
[2]	SAXENA A, KOCHSIEK A, GEMULLA R. Sequence-to-sequence knowledge graph completion and question answering[C]//Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: ACL, 2022: 2814-2828.
[3]	WANG X, HUANG T L, WANG D X, et al. Learning intents behind interactions with knowledge graph for recommendation[C]//Proceedings of the Web Conference 2021. New York: ACM, 2021: 878-887.
[4]	WANG X T, LIU K P, WANG D J, et al. Multi-level recommendation reasoning over knowledge graphs with reinforcement learning[C]//Proceedings of the ACM Web Conference 2022. New York: ACM, 2022: 2098-2108.
[5]	LIN Q K, LIU J, XU F Z, et al. Incorporating context graph with logical reasoning for inductive relation prediction[C]//Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM, 2022: 893-903.
[6]	CHEN X L, CHEN M H, FAN C J, et al. Multilingual knowledge graph completion via ensemble knowledge transfer[C]//Findings of the Association for Computational Linguistics: EMNLP 2020. Stroudsburg: ACL, 2020: 3227-3238.
[7]	SINGH H, JAIN P, MAUSAM, et al. Multilingual knowledge graph completion with joint relation and entity alignment[EB/OL]. (2021-04-18)[2023-10-29]. https://arxiv.org/abs/2104.08804.
[8]	HUANG Z J, LI Z, JIANG H M, et al. Multilingual knowledge graph completion with self-supervised adaptive graph alignment[C]//Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: ACL, 2022: 474-485.
[9]	BORDES A, USUNIER N, GARCIA-DURÁN A, et al. Translating embeddings for modeling multi-relational data[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems. New York: ACM, 2013: 2787-2795.
[10]	YANG B S, YIH W T, HE X D, et al. Embedding entities and relations for learning and inference in knowledge bases[C]//Proceedings of the International Conference on Learning Representations. Phoenix: ICLR, 2014: 2768038.
[11]	SUN Z Q, DENG Z H, NIE J Y, et al. RotatE: knowledge graph embedding by relational rotation in complex space[EB/OL]. (2019-02-26)[2023-10-29]. https://arxiv.org/abs/1902.10197.
[12]	SCHLICHTKRULL M, KIPF T N, BLOEM P, et al. Modeling relational data with graph convolutional networks[C]//Proceedings of the Semantic Web. Berlin: Springer, 2018: 593-607.
[13]	ZHANG Y Q, YAO Q M. Knowledge graph reasoning with relational digraph[C]//Proceedings of the ACM Web Conference. New York: ACM, 2022: 912-924.
[14]	CHEN M H, TIAN Y T, YANG M H, et al. Multilingual knowledge graph embeddings for cross-lingual knowledge alignment[C]//Proceedings of the 26th International Joint Conference on Artificial Intelligence. New York: ACM, 2017: 1511-1517.
[15]	GUO L B, ZHANG Q, SUN Z Q, et al. Understanding and improving knowledge graph embedding for entity alignment[C]//Proceedings of the 39th International Conference on Machine Learning. San Diego: PMLR, 2022, 162: 8145-8156.
[16]	ZHU Y, LIU H Z, WU Z H, et al. Relation-aware neighborhood matching model for entity alignment[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2021, 35(5): 4749-4756.
[17]	SUN Z Q, WANG C M, HU W, et al. Knowledge graph alignment network with gated multi-hop neighborhood aggregation[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2020, 34(1): 222-229.
[18]	YAN Y C, LIU L H, BAN Y K, et al. Dynamic knowledge graph alignment[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2021, 35(5): 4564-4572.
[19]	DEVLIN J, CHANG M W, LEE K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[EB/OL]. (2019-05-24)[2023-10-29]. https://arxiv.org/abs/1810.04805.
[20]	CONNEAU A, LAMPLE G, RANZATO M, et al. Word translation without parallel data[EB/OL]. (2018-01-30)[2023-10-29]. https://arxiv.org/abs/1710.04087.
[21]	HENDRYCKS D, GIMPEL K. Gaussian error linear units (GELUs)[EB/OL]. (2023-06-06)[2023-10-29]. https://arxiv.org/abs/1606.08415.
[22]	LEHMANN J, ISELE R, JAKOB M, et al. DBpedia—a large-scale, multilingual knowledge base extracted from Wikipedia[J]. Semantic Web, 2015, 6(2): 167-195.
[23]	YAO L, MAO C S, LUO Y. KG-BERT: BERT for knowledge graph completion[EB/OL]. (2019-09-11)[2023-10-29]. https://arxiv.org/abs/1909.03193.