-
摘要:
针对表情符号与文本数据结合时可能改变原语义，且其与文本数据相互作用机制尚未被充分探讨的问题，提出融合双重表情符号注意力机制的文本情感分类。使用BERT预训练模型获得文本的动态词向量表示；构建CNN-BiGRU双通道模型分别进行局部特征和全局特征提取；使用Emoji2vec模型获取表情符号向量表示，并构建双重表情符号注意力机制，分别从局部级和全局级表情符号注意力机制层面加强表情符号与文本数据结合后的关键信息；将输出的特征向量融合，进行情感分类。为验证所提模型的有效性，设置对比实验及消融实验，分别在Emoji-phone数据集和EmojifyData数据集上进行情感分类训练，结果表明：所提模型在准确率上相比于RoBERTa-3xBiGRU模型分别提升了
0.0176 和0.0166 。Abstract:Addressing the issue that the combination of emojis and text data may alter the original semantics, and the mechanism of their interaction with text data has not been fully explored. For this reason, textual sentiment classification incorporating dual emoji attention mechanisms is proposed in the paper. First, a BERT pre-training model is used to obtain the dynamic word vector representation of text; then a CNN-BiGRU dual-channel model is constructed to extract local and global features respectively; after that, the Emoji2vec model is used to obtain the emoji vector representation and construct a dual emoji attention mechanism, which strengthens the key information of the combination of emoji and text from the level of local and global emoji attention mechanisms respectively; then the output feature vectors are fused to classify emotions. In order to verify the effectiveness of the proposed model, contrast and ablation experiments were set up. The Emoji-phone and EmojifyData datasets were used for sentiment classification training, and the findings indicate that the model in this article outperforms the more recent RoBERTa-3xBiGRU model by
0.0176 and0.0166 , respectively.-
Key words:
- sentiment classification /
- emoji /
- dual-channel model /
- BERT /
- attention mechanism
-
图 1 融合双重表情符号注意力机制的文本情感分类流程
Figure 1. Flowchart of text sentiment classification incorporating dual emoji attention mechanisms
图 5 Emoji-phone数据集上模型的准确率和损失函数值变化
Figure 5. Variation of accuracy and loss values of model on Emoji-phone dataset
图 6 EmojifyData数据集上模型的准确率和损失函数值变化
Figure 6. Variation of accuracy and loss values of model on EmojifyData dataset
图 8 消融实验3在Emoji-phone数据集上的测试结果
Figure 8. Results of ablation experiment 3 on Emoji-phone dataset
图 9 消融实验3在EmojifyData数据集上的测试结果
Figure 9. Results of ablation experiment 3 on EmojifyData dataset
表 1 数据集示例
Table 1. Example dataset
数据集 文本 情感标签 Emoji-phone 好喜欢这个手机，拍照超级好看$\begin{gathered}{\includegraphics[width=0.025\paperwidth]{2.jpg}}\end{gathered} $ 1 屏幕卡顿，系统卡$\begin{gathered}{\includegraphics[width=0.025\paperwidth]{3.jpg}}\end{gathered} $，闪退游戏 0 EmojifyData That’s awesome! Plus you’re a Meredith
Angel!! $\begin{gathered}{\includegraphics[width=0.07\paperwidth]{4.jpg}}\end{gathered} $1 I feel the frustration aagh$\begin{gathered}{\includegraphics[width=0.025\paperwidth]{5.jpg}}\end{gathered} $ 0 
下载: 导出CSV
表 2 模型参数设置
Table 2. Model parameters settings
参数 数值 Batch_size 64 epoch 10 词向量维度 768 学习率 10−5 Dropout 0.3 优化器 Adam 卷积核尺寸 (2,3,4) 卷积核通道数 (128,128,128) BiGRU层数 1 Hidden_size 256
下载: 导出CSV
表 3 对比实验在2个数据集上的测试结果
Table 3. Test results of comparison experiment on two datasets
模型 准确率 F1值 Emoji-phone EmojifyData Emoji-phone EmojifyData 模型1 0.8675 0.8408 0.8701 0.8398 模型2 0.8827 0.8438 0.8828 0.8422 模型3 0.9035 0.8588 0.9038 0.8595 模型4 0.9137 0.8805 0.9134 0.8797 模型5 0.9195 0.8870 0.9196 0.8858 模型6 0.9139 0.8872 0.9129 0.8848 本文模型 0.9315 0.9038 0.9317 0.9035
下载: 导出CSV
表 4 消融实验1在2个数据集上的测试结果
Table 4. Test results of ablation experiment 1 on two datasets
模型 准确率 F1值 Emoji-phone EmojifyData Emoji-phone EmojifyData 模型7 0.9107 0.8772 0.9090 0.8743 模型8 0.9231 0.8922 0.9226 0.8923 本文模型 0.9315 0.9038 0.9317 0.9035
下载: 导出CSV
表 5 消融实验2在2个数据集上的测试结果
Table 5. Test results of ablation experiment 2 on two datasets
模型 准确率 F1值 Emoji-phone EmojifyData Emoji-phone EmojifyData 模型9 0.9203 0.8855 0.9209 0.8868 模型10 0.9215 0.8888 0.9217 0.8871 本文模型 0.9315 0.9038 0.9317 0.9035
下载: 导出CSV
表 6 文本情感分类结果示例
Table 6. Example of text emotion classification results
文本 模型12 本文模型 True ① 这手机续航简直离谱！昨天早上充满电，到现在还有30%$\begin{gathered}{\includegraphics[width=0.07\paperwidth]{7.jpg}}\end{gathered} $，害得我充电器都白带了！ 负向 正向 正向 ② 略重但质感无敌，游戏不适配，老公当备用机真香$\begin{gathered}{\includegraphics[width=0.025\paperwidth]{6.jpg}}\end{gathered} $ 负向 正向 正向 ③ 手机用一天没问题，昨天899买的，今天799。呵呵，真牛$\begin{gathered}{\includegraphics[width=0.025\paperwidth]{8.jpg}}\end{gathered} $ 正向 负向 负向 ④ 商品目前看来，没有什么大的损坏，但这包装，$\begin{gathered}{\includegraphics[width=0.025\paperwidth]{9.jpg}}\end{gathered} $呵呵的无话可说！ 正向 负向 负向 ⑤ Lord knows ion wanna get up $\begin{gathered}{\includegraphics[width=0.025\paperwidth]{10.jpg}}\end{gathered} $ 正向 负向 负向 ⑥ I wish I had someone to talk to about music man $\begin{gathered}{\includegraphics[width=0.025\paperwidth]{3.jpg}}\end{gathered} $ 正向 负向 负向 ⑦ I suppose I’ll be an adult and just dye my hair black or brown for this job$\begin{gathered}{\includegraphics[width=0.025\paperwidth]{9.jpg}}\end{gathered} $ 正向 负向 负向
下载: 导出CSV
-
[1] Walther J B, D’addario K P. The impacts of emoticons on message interpretation in computer-mediated communication[J]. Social Science Computer Review, 2001, 19(3): 324-347. [2] Schmidhuber J. Deep learning in neural networks: an overview[J]. Neural Networks, 2015, 61: 85-117. [3] Liu B. Text sentiment analysis based on CBOW model and deep learning in big data environment[J]. Journal of Ambient Intelligence and Humanized Computing, 2020, 11(2): 451-458. [4] Xiao Z, Liang P J. Chinese sentiment analysis using bidirectional LSTM with word embedding[C]//Proceedings of the 2nd International Conference on Cloud Computing and Security. Berlin: Springer, 2016: 601-610. [5] Zhang C Y, Wang D S, Wang L L, et al. Temporal data-driven failure prognostics using BiGRU for optical networks[J]. Journal of Optical Communications and Networking, 2020, 12(8): 277-287. [6] Zhang X S, Wu Z Q, Liu K, et al. Text sentiment classification based on BERT embedding and sliced multi-head self-attention Bi-GRU[J]. Sensors, 2023, 23(3): 1481. [7] 孙红, 陈强越. 融合BERT词嵌入和注意力机制的中文文本分类[J]. 小型微型计算机系统, 2022, 43(1): 22-26.Sun H, Chen Q Y. Chinese text classification based on BERT and attention[J]. Journal of Chinese Computer Systems, 2022, 43(1): 22-26(in Chinese). [8] Zhang S Y, Gong H F, She L N. An aspect sentiment classification model for graph attention networks incorporating syntactic, semantic, and knowledge[J]. Knowledge-Based Systems, 2023, 275: 110662. [9] Jia K L, Meng F X, Liang J, et al. Text sentiment analysis based on BERT-CBLBGA[J]. Computers and Electrical Engineering, 2023, 112: 109019. [10] 程艳, 尧磊波, 张光河, 等. 基于注意力机制的多通道CNN和BiGRU的文本情感倾向性分析[J]. 计算机研究与发展, 2020, 57(12): 2583-2595.Cheng Y, Yao L B, Zhang G H, et al. Text sentiment orientation analysis of multi channels CNN and BiGRU based on attention mechanism[J]. Journal of Computer Research and Development, 2020, 57(12): 2583-2595(in Chinese). [11] Qiu H, Fan C D, Yao J, et al. Chinese microblog sentiment detection based on CNN-BiGRU and multihead attention mechanism[J]. Scientific Programming, 2020, 2020: 8865983. [12] Bao T, Ren N, Luo R, et al. A BERT-based hybrid short text classification model incorporating CNN and attention-based BiGRU[J]. Journal of Organizational and End User Computing (JOEUC), 2021, 33(6): 1-21. [13] Yan C, Liu J H, Liu W, et al. Research on public opinion sentiment classification based on attention parallel dual-channel deep learning hybrid model[J]. Engineering Applications of Artificial Intelligence, 2022, 116: 105448. [14] 陈黎, 刘雨欣, 周耘立, 等. 融合表情符号图像特征学习的微博情感分类[J]. 四川大学学报(自然科学版), 2021, 58(1): 74-80.Chen L, Liu Y X, Zhou Y L, et al. Incorporating image features of emoticons into microblog sentiment classification[J]. Journal of Sichuan University (Natural Science Edition), 2021, 58(1): 74-80(in Chinese). [15] Jiang F, Liu Y Q, Luan H B, et al. Microblog sentiment analysis with emoticon space model[J]. Journal of Computer Science and Technology, 2015, 30(5): 1120-1129. [16] 赵晓芳, 金志刚. 融合表情符号与短文本的微博多维情感分类[J]. 哈尔滨工业大学学报, 2020, 52(5): 113-120.Zhao X F, Jin Z G. Multi-dimensional sentiment classification of microblog based on emoticons and short texts[J]. Journal of Harbin Institute of Technology, 2020, 52(5): 113-120(in Chinese). [17] Su Y J, Chen C H, Chen T Y, et al. Chinese microblog sentiment analysis by adding emoticons to attention-based CNN[J]. Journal of Internet Technology, 2020, 21(3): 821-829. [18] 张仰森, 郑佳, 黄改娟, 等. 基于双重注意力模型的微博情感分析方法[J]. 清华大学学报(自然科学版), 2018, 58(2): 122-130.Zhang Y S, Zheng J, Huang G J, et al. Microblog sentiment analysis method based on a double attention model[J]. Journal of Tsinghua University(Science and Technology), 2018, 58(2): 122-130(in Chinese). [19] 何炎祥, 孙松涛, 牛菲菲, 等. 用于微博情感分析的一种情感语义增强的深度学习模型[J]. 计算机学报, 2017, 40(4): 773-790.He Y X, Sun S T, Niu F F, et al. A deep learning model enhanced with emotion semantics for microblog sentiment analysis[J]. Chinese Journal of Computers, 2017, 40(4): 773-790(in Chinese). [20] Prottasha N J, Sami A A, Kowsher M, et al. Transfer learning for sentiment analysis using BERT based supervised fine-tuning[J]. Sensors, 2022, 22(11): 4157. [21] 陈亚茹, 陈世平. 融合自注意力机制和BiGRU网络的微博情感分析模型[J]. 小型微型计算机系统, 2020, 41(8): 1590-1595.Chen Y R, Chen S P. Microblog sentiment analysis model with the combination of self-attention and BiGRU[J]. Journal of Chinese Computer Systems, 2020, 41(8): 1590-1595(in Chinese). [22] Abdelgwad M M, Soliman T H A, Taloba A I. Arabic aspect sentiment polarity classification using BERT[J]. Journal of Big Data, 2022, 9(1): 1-15. [23] Eisner B, Rocktäschel T, Augenstein I, et al. Emoji2vec: learning emoji representations from their description[C]//Proceedings of the 4th International Workshop on Natural Language Processing for Social Media. Austin: Association for Computational Linguistics, 2016: 48-54. [24] Devlin J, Chang M W, Lee K, et al. BERT: bidirectional encoder representations from transformers for language understanding[J]. Computation and Language, 2018, 23(2): 3-19. [25] 韩永鹏, 陈彩, 苏航, 等. 融合通道特征的混合神经网络文本分类模型[J]. 中文信息学报, 2021, 35(2): 78-88.Han Y P, Chen C, Su H, et al. Hybrid neural network text classification model with channel features[J]. Journal of Chinese Information Processing, 2021, 35(2): 78-88(in Chinese). [26] 王根生, 黄学坚, 闵潞. 多元特征融合的GRU神经网络文本情感分类模型[J]. 小型微型计算机系统, 2019, 40(10): 2130-2138.Wang G S, Huang X J, Min L. GRU neural network text emotion classification model based on multi-feature fusion[J]. Journal of Chinese Computer Systems, 2019, 40(10): 2130-2138(in Chinese). [27] Talaat A S. Sentiment analysis classification system using hybrid BERT models[J]. Journal of Big Data, 2023, 10(1): 1-18. -
下载:
点击查看大图
计量
- 文章访问数: 375
- HTML全文浏览量: 130
- PDF下载量: 26
- 被引次数: 0
