多输入傅里叶神经网络及其麻雀搜索优化

黎亮亮; 张著洪; 张永丹

doi:10.13700/j.bh.1001-5965.2022.0404

多输入傅里叶神经网络及其麻雀搜索优化

doi: 10.13700/j.bh.1001-5965.2022.0404

贵州大学大数据与信息工程学院，贵州省系统优化与科学计算特色重点实验室，贵阳 550025

基金项目: 国家自然科学基金(62063002)

详细信息

通讯作者:
E-mail：zhzhang@gzu.edu.cn

中图分类号: TP301.6
计量
- 文章访问数: 119
- HTML全文浏览量: 58
- PDF下载量: 4
- 被引次数: 0
出版历程
- 收稿日期: 2022-05-21
- 录用日期: 2022-11-25
- 网络出版日期: 2022-12-14
- 整期出版日期: 2024-02-27

Multi-input Fourier neural network and its sparrow search optimization

Guizhou Provincial Characteristic Key Laboratory of System Optimization and Scientific Computation，College of Big Data and Information Engineering，Guizhou University，Guiyang 550025，China

Funds: National Natural Science Foundation of China (62063002)

More Information

Corresponding author: E-mail：zhzhang@gzu.edu.cn

摘要

摘要:
鉴于反向传播（BP）神经网络存在灵敏度高但收敛速度慢，以及已有傅里叶神经网络不具备多输入数据特征提取能力，借助多个傅里叶神经网络构建能接收多维数据的堆叠神经网络，进而将其与多层感知器融合，获得基于梯度下降的多输入傅里叶神经网络。结合此神经网络获取全局最优参数值难的因素，通过在麻雀搜索算法中引入Cat混沌映射、动态种群规模调节机制及参数自适应调节方案，提出改进型麻雀搜索算法，并将其应用于多输入傅里叶神经网络的参数优化及高维函数优化问题的求解。理论分析可得，所提算法的计算复杂度主要由种群规模和优化问题的维度决定。比较性的数值实验表明，所获神经网络提取多源数据特征的能力和泛化能力强，同时所提算法处理高维优化问题具有明显优势且收敛速度快。
- 傅里叶神经网络 /
- 多层感知器 /
- 麻雀搜索 /
- 高维函数优化 /
- 多属性分类
Abstract:
In engineering applications, the back-propagation (BP) neural network often encounters many limitations due to its slow convergence and high noise sensitivity, and the reported Fourier neural networks cannot extract the features of multi-attribute input data. Hereby, this work proposes a gradient descent-based multi-input Fourier neural network after integrating the multi-layer perceptron with an overlapping Fourier neural network. Then to address the difficulty in deciding the global optimal parameter settings, the Cat chaotic map and the mechanisms of population-size adjustment and parameter adaptiveness are designed to promote the sparrow search algorithm’s ability to balance global exploration and local exploitation. An improved sparrow search algorithm is thus developed, optimizing the parameter settings and solving high dimensional function optimization problems. The theoretical analysis shows that the improved algorithm’s computational complexity is decided by its population size and the optimization problem dimension. Numerically comparative experiments have validated that the acquired Fourier neural network can effectively extract the features of multi-attribute data with strong generalization ability, and that the improved algorithm has significant advantages in coping with high dimensional function optimization problems.
- Fourier neural network /
- multi-layer perceptron /
- sparrow search /
- high dimensional function optimization /
- multi-attribute classification

HTML全文

图 1 MIFNN的结构示意图

Figure 1. Schematic diagram of MIFNN structure

下载: 全尺寸图片幻灯片

图 2 各算法的平均搜索曲线

Figure 2. Average search curves of algorithms

下载: 全尺寸图片幻灯片

图 3 各神经网络的损失函数变化曲线

Figure 3. Loss function variation curves of neural networks

下载: 全尺寸图片幻灯片

表 1 各算法独立运行25次后获得的统计结果（F₁~F₄）

Table 1. Statistical results acquired by each algorithm after 25 runs per example（F₁-F₄）

算法	μ				σ
算法	F₁	F₂	F₃	F₄	F₁	F₂	F₃	F₄
SSA	2.47×10	3.20×10⁻¹	5.14×10⁻⁴	1.80×10⁻³	1.91×10⁻¹	5.51 ×10⁻¹	1.10×10⁻³	1.60×10⁻³
HHO	6.45×10⁻¹	7.44×10⁰	1.26 ×10⁻¹	5.79×10⁻⁴	2.58 ×10⁻¹	9.25×10⁰	1.22 ×10⁻¹	5.52×10⁻⁴
AO	7.07×10⁻¹	7.05×10⁰	2.96×10⁻²	3.04×10⁻⁴	1.24 ×10⁻¹	1.68×10	4.57×10⁻²	3.42×10⁻⁴
DOA	7.50×10⁻¹	2.00×10³	4.97×10²	1.90×10⁻³	1.46×10⁻⁴	5.80×10⁻²	1.30 ×10⁰	2.50×10⁻³
WOA	7.50×10⁻¹	1.99×10³	1.56×10²	1.49×10⁻²	1.49×10⁻⁵	1.74×10⁰	4.02×10	1.67×10⁻²
ISSA	2.17×10⁻⁴	6.90×10⁻⁴	9.43×10⁻⁶	6.35×10⁻⁴	8.19×10⁻⁴	2.80×10⁻³	3.66×10⁻⁵	5.99×10⁻⁴

下载: 导出CSV

表 2 各算法独立运行25次后获得的统计结果（F₅~F₈）

Table 2. Statistical results acquired by each algorithm after 25 runs per example（F₅-F₈）

算法	μ				σ
算法	F₅	F₆	F₇	F₈	F₅	F₆	F₇	F₈
SSA	−2.10×10⁵	6.72×10⁻⁷	1.91×10⁻⁴	1.55×10⁻²	2.30×10⁴	1.16×10⁻⁶	1.98×10⁻⁴	5.27×10⁻²
HHO	−8.31×10⁵	1.32×10⁻⁵	1.42×10⁻²	5.40×10⁻³	3.46×10⁴	2.12×10⁻⁵	1.34×10⁻²	9.10×10⁻³
AO	−1.28×10⁵	5.33×10⁻⁶	1.77×10⁻²	2.15×10	3.76×10⁴	9.32×10⁻⁶	4.21×10⁻²	1.23×10
DOA	−4.68×10⁴	1.17×10⁰	2.00×10²	4.53×10	7.49×10³	6.90×10⁻³	5.30×10⁻³	2.05×10⁰
WOA	−6.75×10⁵	1.17×10⁻¹	7.76×10	1.05×10	1.18×10⁵	4.34×10⁻²	2.28×10	9.39×10⁰
ISSA	−8.38×10⁵	1.03×10⁻⁸	6.26×10⁻⁶	7.22×10⁻⁵	2.59×10⁻²	2.58×10⁻⁸	1.87×10⁻⁵	1.25×10⁻⁴

下载: 导出CSV

表 3 各神经网络的训练分类准确率比较

Table 3. Comparison of training classification accuracy of neutral networks %

神经网络	TrAcc
神经网络	Iris	Wine	Haberman	Hayes	Immunotherapy	Cryotherapy	Heart	BreastCancer	Transfusion	Audit	Banknote
MGD-MIFNN	100	100	81.30	100	100	100	85.17	100	67.50	100	74.27
BPNN	95.24	95.16	73.36	70.65	80.65	93.55	75.42	88.89	72.08	100	100
ELM	98.10	100	74.77	64.13	85.48	96.77	84.69	90.12	74.95	100	100
RBF	95.24	95.97	73.36	64.13	82.26	80.65	68.42	82.72	77.44	46.03	91.35
WNN	95.24	99.19	78.50	77.17	96.77	98.39	83.73	88.89	67.50	100	99.48
SSA-MIFNN	100	89.52	78.50	86.96	95.16	98.39	74.64	97.53	82.22	100	70.83
HHO-MIFNN	93.33	79.30	74.30	90.22	96.77	95.16	78.95	88.89	77.44	100	56.15
AO-MIFNN	96.19	87.10	82.71	90.22	95.16	93.55	77.51	91.36	74.28	100	66.15
DOA-MIFNN	100	100	83.18	98.91	100	100	86.6	100	86.23	100	70.52
WOA-MIFNN	100	91.13	80.84	94.57	98.39	100	83.73	86.42	84.32	100	67.71
ISSA-MIFNN	99.05	98.39	86.45	97.83	100	98.39	87.08	95.06	87.95	100	73.13

下载: 导出CSV

表 4 各神经网络的测试分类准确率比较

Table 4. Comparison of test classification accuracy of neutral networks %

神经网络	TeAcc
神经网络	Iris	Wine	Haberman	Hayes	Immunotherapy	Cryotherapy	Heart	BreastCancer	Transfusion	Audit	Banknote
MGD-MIFNN	100	87.04	82.60	82.50	71.43	89.29	85.56	82.86	48.00	100	56.55
BPNN	100	92.59	79.35	42.50	75.00	85.71	83.33	71.43	76.00	100	100
ELM	100	96.30	77.17	65.00	82.14	89.29	81.11	82.86	78.22	99.14	100
RBF	100	87.04	77.17	67.50	75.00	82.14	81.11	68.57	78.67	71.12	92.96
WNN	100	88.89	71.74	52.50	42.86	82.14	76.67	62.86	70.22	100	98.54
SSA-MIFNN	100	77.78	80.43	80.00	75.00	92.86	82.22	78.57	76.00	99.57	57.04
HHO-MIFNN	100	68.52	84.87	85.00	78.57	92.86	87.78	68.57	76.89	99.14	56.80
AO-MIFNN	97.78	83.33	80.43	80.00	78.57	85.71	75.56	64.19	80.44	99.14	55.34
DOA-MIFNN	97.78	88.89	83.70	77.50	82.14	96.43	85.56	82.86	73.78	100	56.07
WOA-MIFNN	97.78	68.52	81.52	75.00	75.00	96.43	78.89	65.71	76.00	100	57.04
ISSA-MIFNN	100	96.30	85.78	90.00	89.29	100	91.11	85.71	80.89	100	58.50

下载: 导出CSV

表 5 各神经网络在不同网络结构下的分类准确率比较

Table 5. Comparison of neural networks’ classification accuracy rates based on different network structures %

神经网络	TrAcc				TeAcc
	Transfusion		Banknote		Transfusion		Banknote
	[10,5]	[20,15]	[10,5]	[20,15]	[10,5]	[20,15]	[10,5]	[20,15]
MGD-MIFNN	55.26	86.81	72.25	77.50	37.33	61.33	56.31	54.13
SSA-MIFNN	81.26	80.88	72.19	76.56	75.56	78.22	56.31	52.67
HHO-MIFNN	81.07	74.19	58.13	70.63	76.44	79.11	56.80	55.83
AO-MIFNN	79.35	73.80	69.17	57.40	75.56	76.89	56.55	57.28
DOA-MIFNN	80.88	87.38	73.33	73.85	76.00	74.22	56.80	57.52
WOA-MIFNN	80.31	83.37	73.75	71.04	75.56	77.33	53.16	59.47
ISSA-MIFNN	82.79	87.57	72.25	77.50	76.44	79.11	57.28	58.50

下载: 导出CSV

参考文献(24)

[1]	杨旭华. 神经网络及其在控制中的应用研究[D]. 杭州: 浙江大学, 2004: 24-32. YANG X H. Study on neural networks machine and its application in control[D]. Hangzhou: Zhejiang University, 2004: 24-32(in Chinese).
[2]	杨晓帆, 陈廷槐. 人工神经网络固有的优点和缺点[J]. 计算机科学, 1994, 21(2): 23-26. YANG X F, CHEN T H. Advantages and disadvantages inherent in artificial neural networks[J]. Computer Science, 1994, 21(2): 23-26(in Chinese).
[3]	邹阿金, 沈建中. 傅立叶神经网络建模研究[J]. 湘潭大学自然科学学报, 2001, 23(2): 23-26. doi: 10.13715/j.cnki.nsjxu.2001.02.005 ZOU A J, SHEN J Z. A neural network model structure study based on Fourier[J]. Natural Science Journal of Xiangtan University, 2001, 23(2): 23-26(in Chinese). doi: 10.13715/j.cnki.nsjxu.2001.02.005
[4]	GASHLER M S, ASHMORE S C. Modeling time series data with deep Fourier neural networks[J]. Neurocomputing, 2016, 188(5): 3-11.
[5]	党选举, 潘登. 基于电流注入法的开关磁阻电机转矩脉动抑制[J]. 组合机床与自动化加工技术, 2021(5): 115-119. doi: 10.13462/j.cnki.mmtamt.2021.05.027 DANG X J, PAN D. Torque ripple reduction of switched reluctance motor based on current injection method[J]. Modular Machine Tool & Automatic Manufacturing Technique, 2021(5): 115-119(in Chinese). doi: 10.13462/j.cnki.mmtamt.2021.05.027
[6]	BENRABAH M, KARA K, AITSAHED O, et al. Adaptive Fourier series neural network PID controller[J]. International Journal of Control, Automation and Systems, 2021, 19(10): 3388-3399.
[7]	PENG P, XIE L, WEI H. A deep Fourier neural network for seizure prediction using convolutional neural network and ratios of spectral power[J]. International Journal of Neural Systems, 2021, 31(8): 2150022.
[8]	张雨浓, 李钧, 张智军, 等. SIMO傅里叶三角基神经网络的权值直接确定法和结构自确定算法[J]. 信息与控制, 2011, 40(4): 507-513. ZHANG Y N, LI J, ZHANG Z J, et al. A weights-direct-determination method and structure-automatic-determination algorithm for SIMO trigonometrically-activated Fourier neural networks[J]. Information and Control, 2011, 40(4): 507-513(in Chinese).
[9]	LIN L, WU X H, QI J, et al. Power quality disturbance classification based on a novel Fourier neural network and hyperbolic S-transform[J]. International Journal of Signal Processing, Image Processing and Pattern Recognition, 2016, 9(1): 111-124. doi: 10.14257/ijsip.2016.9.1.11
[10]	龚鹍, 邓方, 陈杰. 基于自适应差分进化算法和傅里叶神经网络的电子罗盘的标定[C]//中国自动化学会控制理论专业委员会B卷. 北京: 中国自动化学会控制理论专业委员会, 2011: 741-746. GONG P, DENG F, CHEN J. Calibration of electronic compass based on adaptive differential evolution algorithm and Fourier neural network[C]// Technical Committee on Control Theory, Chinese Association of Automation, Vol. B. Beijing: Technical Committee on Control Theory, Chinese Association of Automation, 2011: 741-746(in Chinese).
[11]	陈诗雨, 李小勇, 杜杨杨, 等. Fourier神经网络非线性拟合性能优化研究[J]. 武汉大学学报(工学版), 2020, 53(3): 277-282. doi: 10.14188/j.1671-8844.2020-03-013 CHEN S Y, LI X Y, DU Y Y, et al. Optimization study of Fourier neural network nonlinear fitting performance[J]. Engineering Journal of Wuhan University, 2020, 53(3): 277-282(in Chinese). doi: 10.14188/j.1671-8844.2020-03-013
[12]	薛建凯. 一种新型的群智能优化技术的研究与应用[D]. 上海: 东华大学, 2020. XUE J K. Research and application of a novel swarm intelligence optimization technique[D]. Shanghai: Donghua University, 2020(in Chinese).
[13]	HEIDARI A A, MIRJALILI S, FARIS H, et al. Harris hawks optimization: Algorithm and applications[J]. Future Generation Computer Systems, 2019, 97(8): 849-872.
[14]	ABUALIGAH L, YOUSRI D, ABD ELAZIZ M, et al. Aquila optimizer: A novel meta-heuristic optimization algorithm[J]. Computers & Industrial Engineering, 2021, 157: 107250.
[15]	BAIRWA A K, JOSHI S, SINGH D. Dingo optimizer: A nature-inspired metaheuristic approach for engineering problems[J]. Mathematical Problems in Engineering, 2021, 2021(26): 2571863.
[16]	MIRJALILI S, LEWIS A. The whale optimization algorithm[J]. Advances in Engineering Software, 2016, 95: 51-67. doi: 10.1016/j.advengsoft.2016.01.008
[17]	李爱莲, 全凌翔, 崔桂梅, 等. 融合正余弦和柯西变异的麻雀搜索算法[J]. 计算机工程与应用, 2022, 58(3): 91-99. doi: 10.3778/j.issn.1002-8331.2106-0148 LI A L, QUAN L X, CUI G M, et al. Sparrow search algorithm combining sine-cosine and cauchy mutation[J]. Computer Engineering and Applications, 2022, 58(3): 91-99(in Chinese). doi: 10.3778/j.issn.1002-8331.2106-0148
[18]	付华, 刘昊. 多策略融合的改进麻雀搜索算法及其应用[J]. 控制与决策, 2022, 37(1): 87-96. doi: 10.13195/j.kzyjc.2021.0582 FU H, LIU H. Improved sparrow search algorithm with multi-strategy integration and its application[J]. Control and Decision, 2022, 37(1): 87-96(in Chinese). doi: 10.13195/j.kzyjc.2021.0582
[19]	YANG Q, CHEN W N, GU T, et al. A distributed swarm optimizer with adaptive communication for large-scale optimization[J]. IEEE Transactions on Cybernetics, 2020, 50(7): 3393-3408. doi: 10.1109/TCYB.2019.2904543
[20]	LI D, GUO W, LERCH A, et al. An adaptive particle swarm optimizer with decoupled exploration and exploitation for large scale optimization[J]. Swarm and Evolutionary Computation, 2021, 60(7): 100789.
[21]	ZHANG Z H, LI L, LU J X. Gradient-based fly immune visual recurrent neural network solving large-scale global optimization[J]. Neurocomputing, 2021, 454: 238-253. doi: 10.1016/j.neucom.2021.05.002
[22]	ZHANG Z H, XIAO T Y, QIN X C. Fly visual evolutionary neural network solving large-scale global optimization[J]. International Journal of Intelligent Systems, 2021, 36(11): 6680-6712. doi: 10.1002/int.22564
[23]	徐辰华, 李成县, 喻昕, 等. 基于Cat混沌与高斯变异的改进灰狼优化算法[J]. 计算机工程与应用, 2017, 53(4): 1-9. XU C H, LI C X, YU X, et al. Improved grey wolf optimization algorithm based on chaotic Cat mapping and Gaussian mutation[J]. Computer Engineering and Applications, 2017, 53(4): 1-9(in Chinese).
[24]	杨彤. 改进鲸鱼算法在函数优化中的应用[D]. 西安: 西安电子科技大学, 2020: 15-17. YANG T. Applications of improved whale optimization algorithm in function optimization[D]. Xi’an: Xidian University, 2020: 15-17(in Chinese).