基于图卷积网络的表格隶属关系抽取

张宇童; 李启元; 刘树衎

doi:10.13700/j.bh.1001-5965.2022.0382

基于图卷积网络的表格隶属关系抽取

doi: 10.13700/j.bh.1001-5965.2022.0382

张宇童¹,
李启元¹,
刘树衎^{1, 2, ,}

1.
中国人民解放军海军工程大学电子工程学院，武汉 430033
2.
东南大学计算机科学与工程学院，南京 211102

基金项目: 湖北省自然科学基金（2018CFC800）

详细信息

通讯作者:
E-mail：liusk@seu.edu.cn

中图分类号: TP183
计量
- 文章访问数: 152
- HTML全文浏览量: 104
- PDF下载量: 2
- 被引次数: 0
出版历程
- 收稿日期: 2022-05-18
- 录用日期: 2022-08-12
- 网络出版日期: 2022-09-14
- 整期出版日期: 2024-04-29

Tabular subordination relation extraction based on graph convolutional networks

ZHANG Yutong¹,
LI Qiyuan¹,
LIU Shukan^{1, 2
, ,}

1.
School of Electronic Engineering，Naval University of Engineering，PLA，Wuhan 430033，China
2.
School of Computer Science and Engineering，Southeast University，Nanjing 211102，China

Funds: Natural Science Foundation of Hubei Province (2018CFC800)

More Information

Corresponding author: E-mail：liusk@seu.edu.cn

摘要

摘要:
针对表格识别与分析领域中表内单元格间隶属关系抽取问题，定义表格隶属关系抽取任务，结合表格与图结构的相似性，给出表内单元格的图表示方法，并提出一种基于图卷积网络（GCN）的表格隶属关系抽取模型。所提模型通过GCN对表内单元格及其邻近格进行特征的聚合，预测单元格间是否存在隶属关系，实现关系抽取。为验证所提模型的有效性，标注中文表单Rel-forms及英文表格Rel-SciTSR 这2个数据集。通过实验，在上述2类数据集及联合数据集上F₁分数分别达到98.61%、96.55%、97.05%，验证所提模型在此2个数据集上的有效性，并分别分析文本内容、坐标信息、单元格属性及格间相对方向等不同因素对隶属关系抽取实验结果的影响。
- 表格分析 /
- 隶属关系 /
- 图表示 /
- 关系抽取 /
- 图卷积网络
Abstract:
This study addresses the lack of research on extracting subordination relations between the cells in a table in the field of table recognition and analysis. It defines the table subordination relation extraction task. Based on the similarity between table and graph structures and the graph representation method for table cells, a subordination relation extraction model is proposed using graph convolutional networks(GCN). The proposed model predicts the subordination relation between cells by aggregating the features of table cells and their adjacent cells through the GCN, and then realizes relation extraction. To verify the effectiveness of the proposed model, two datasets, Chinese Rel-forms and English Rel-SciTSR, were annotated. The experiments achieve 98.61%, 96.55% and 97.05% F₁ scores on the above two datasets and their joint datasets respectively, thus verifying the effectiveness of the proposed model. The effects of different factors such as text content, coordinate information, cell attributes and relative orientation between cells on the experimental results of subordination relation extraction are also analyzed.
- tabular analysis /
- subordination relation /
- graph representation /
- relation extraction /
- graph convolutional network

HTML全文

图 1 2类表格数据的隶属关系定义及抽取过程

Figure 1. Definition and extraction of subordination relation in two types of tabular data

下载: 全尺寸图片幻灯片

图 2 GCN的传播方式

Figure 2. Dissemination mode of GCN

下载: 全尺寸图片幻灯片

图 3 GCN工作流程

Figure 3. GCN flowchart

下载: 全尺寸图片幻灯片

图 4 本文模型

Figure 4. The proposed model

下载: 全尺寸图片幻灯片

图 5 Rel-SciTSR数据集样例

Figure 5. Sample images from dataset of Rel-SciTSR

下载: 全尺寸图片幻灯片

图 6 Rel-forms数据集样例

Figure 6. Sample images from dataset of Rel-forms

下载: 全尺寸图片幻灯片

图 7 正确率变化趋势

Figure 7. Change trend of accuracy

下载: 全尺寸图片幻灯片

图 8 正确率及损失变化趋势

Figure 8. Change trend of accuracy and loss

下载: 全尺寸图片幻灯片

表 1 文本特征关系抽取结果

Table 1. Result of relation extraction in text feature

数据集	正确率/％
Rel-SciTSR	86.46
Rel-forms	91.76
Rel-SciTSR+ Rel-forms	88.93

下载: 导出CSV

表 2 隶属关系抽取结果

Table 2. Result of subordination relation extraction

特征	P			R			F₁
特征	数据集①	数据集②	数据集③	数据集①	数据集②	数据集③	数据集①	数据集②	数据集③
Po	97.51	93.59	95.05	95.19	93.96	95.56	96.34	93.77	95.30
Po+Cl	98.16	95.88	96.93	97.90	95.65	96.67	98.03	95.76	96.80
Po+Cl+Rd	98.82	96.49	97.18	98.40	96.61	96.93	98.61	96.55	97.05
注：①论文表格数据集Rel-SciTSR；②表单类数据集Rel-forms；③联合数据集Rel-SciTSR+ Rel-forms。

下载: 导出CSV

参考文献(18)

[1]	WANG H Y, CHENG Y H, PHILTP CHEN C L , et al. Semisupervised classification of hyperspectral image based on graph convolutional broad network[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2021, 14: 2995-3005. doi: 10.1109/JSTARS.2021.3062642
[2]	ZOU Y J, MA J W. A deep semantic segmentation model for image-based table structure recognition[C]//Proceeclings of the 2020 15th IEEE International Conference on Signal Processing. Piscataway: IEEE Press, 2020: 274-280.
[3]	SCHREIBER S, AGNE S, WOLF I, et al. DeepDeSRT: Deep learning for detection and structure recognition of tables in document images[C]//Proceeclings of the 2017 14th IAPR International Conference on Document Analysis and Recognition. Piscataway: IEEE Press, 2017: 1162-1167.
[4]	SIDDIQUI S A, KHAN P I, DENGEL A, et al. Rethinking semantic segmentation for table structure recognition in documents[C]//Proceeclings of the 2019 International Conference on Document Analysis and Recognition. Piscataway: IEEE Press, 2019: 1397-1402.
[5]	RAJA S, MONDAL A, JAWAHAR C V . Table structure recognition using top-down and bottom-up cues[C]//Proceeclings of the European Conference on Computer Vision.Berlin: Springer, 2020: 70-86.
[6]	LIN T Y, DOLLÁR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2017: 2117-2125.
[7]	GUO X L, ZHU S J, YANG Z W, et al. Consecutive missing data recovery method based on long-short term memory network[C]//Proceeclings of the 2021 3rd Asia Energy and Electrical Engineering Symposium. Piscataway: IEEE Press, 2021: 988-992.
[8]	KONG L J, BAO Y C, WANG Q W, et al. A gradient heatmap based table structure recognition[C]//Proceeclings of the 2021 13th International Conference on Machine Learning and Computing. New York: ACM, 2021: 456-463.
[9]	QIAO LS, LI ZS, CHENG Z, et al. LGPMA: Complicated table structure recognition with local and global pyramid mask alignment[C]//Proceeclings of the International Conference on Document Analysis and Recognition. Berlin: Springer, 2021: 99-114.
[10]	LONG R J, WANG W, XUE N, et al. Parsing table structures in the wild[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE Press, 2021: 944-952.
[11]	ZHOU X Y, WANG D Q, KRHENBÜHL P. Objects as points[EB/OL]. (2019-04-25) [2022-03-24]. https://arxiv.org/abs/1904.07850.
[12]	CHI Z W, HUANG H Y, XU H D, et al. Complicated table structure recognition[EB/OL]. (2019-08-28)[2022-03-10]. https://arxiv.org/abs/1908.04729.
[13]	XUE W Y, YU B S, WANG W, et al. TGRNet: A table graph reconstruction network for table structure recognition[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE Press, 2021: 1295-1304.
[14]	RIBA P, DUTTA A, GOLDMANN L, et al. Table detection in invoice documents by graph neural networks[C]//Proceeclings of the 2019 International Conference on Document Analysis and Recognition . Piscataway: IEEE Press, 2019: 122-127.
[15]	SCARSELLI F, GORI M, TSOI A C, et al. Computational capabilities of graph neural networks[J]. IEEE Transactions on Neural Networks, 2009, 20(1): 81-102. doi: 10.1109/TNN.2008.2005141
[16]	QASIM S R, MAHMOOD H, SHAFAIT F. Rethinking table recognition using graph neural networks[C]//Proceeclings of the 2019 International Conference on Document Analysis and Recognition. Piscataway: IEEE Press, 2019: 142-147.
[17]	LI Y R, HUANG Z, YAN J C, et al. GFTE: Graph-based financial table extraction[C]//Proceeclings of the International Conference on Pattern Recognition. Berlin: Springer, 2021: 644-658.
[18]	郑海潇, 文斌. 基于图卷积网络的比特币非法交易识别方法[J]. 信息网络安全, 2021, 21(9): 74-79. doi: 10.3969/j.issn.1671-1122.2021.09.011 ZHENG H X, WEN B. Bitcoin illegal transaction identification method based on graph convolutional network[J]. Netinfo Security, 2021, 21(9): 74-79(in Chinese). doi: 10.3969/j.issn.1671-1122.2021.09.011