Dirichlet混合样本的EM算法与动态聚类算法比较

doi:10.13700/j.bh.1001-5965.2018.0752

北京航空航天大学学报 ›› 2019, Vol. 45 ›› Issue (9): 1805-1811.doi: 10.13700/j.bh.1001-5965.2018.0752

Dirichlet混合样本的EM算法与动态聚类算法比较

夏棒¹, EMILION Richard², 王惠文³

1. 中国工商银行博士后科研工作站, 北京 100032;
2. 奥尔良大学 MAPMO研究所, 奥尔良 45000;
3. 北京航空航天大学经济管理学院, 北京 100083

收稿日期:2018-12-25 出版日期:2019-09-20 发布日期:2019-09-29
通讯作者: 夏棒 E-mail:15011176214@163.com
作者简介:夏棒,男,博士。主要研究方向:成分数据路径分析;人机交互、结构方程模型等。EMILION Richard,男,博士,教授,博士生导师。主要研究方向:Dirichlet分布数据及其数学理论。王惠文,女,博士,教授,博士生导师。主要研究方向:复杂数据分析、投入产出表预测等。
基金资助:
国家自然科学基金（71420107025）

Comparison between EM algorithm and dynamical clustering algorithm for Dirichlet mixture samples

XIA Bang¹, EMILION Richard², WANG Huiwen³

1. Postdoctoral Workstation, Industrial and Commercial Bank of China, Beijing 100032, China;
2. MAPMO, University of Orleans, Orleans 45000, France;
3. School of Economics and Management, Beihang University, Beijing 100083, China

Received:2018-12-25 Online:2019-09-20 Published:2019-09-29
Supported by:
National Natural Science Foundation of China (71420107025)

摘要/Abstract

摘要： Dirichlet分布是一类包含正参数向量的连续多元概率分布，在比例结构问题中具有广泛的应用。针对Dirichlet混合样本的聚类问题，进行了最大期望（EM）算法和动态聚类算法研究。首先，推导其数学过程，并给出算法迭代步骤。然后，利用数字仿真实验，比较了EM算法与动态聚类算法两种机器学习算法在Dirichlet混合样本中的聚类效果。最后，计算对数似然函数值、程序运行时间、收敛迭代次数、聚类正确率、真正率（TPR）和假正率（FPR）6个评价指标。仿真实验结果表明，EM算法聚类正确率更高但是运算效率相对较低，而动态聚类算法运算效率较高但是损失了部分聚类正确率。因此，实际应用中建议综合权衡聚类正确率与运算效率的相对需求后，再选取合适算法进行Dirichlet混合样本聚类。

关键词: Dirichlet分布, 混合样本, 最大期望(EM)算法, 动态聚类, 机器学习

Abstract: Dirichlet distribution is a kind of continuous multivariate probability distribution with positive parameter vectors, which is widely used in proportional structure problems. Expectation maximization (EM) algorithm and dynamical clustering algorithm of Dirichlet mixture samples are presented, their mathematical process is deduced, and the iteration steps of the algorithms are given. Then, using digital simulation experiments, the clustering effects of the two machine learning algorithms with Dirichlet samples are compared. By calculating six evaluation factors which are log-likelihood function value, program running time, convergence iteration times, clustering accuracy, true positive rate (TPR) and false positive rate (FPR), the simulation results show that EM algorithm has higher clustering accuracy but lower operational efficiency, while dynamical clustering algorithm has higher operational efficiency but loses some clustering accuracy. Therefore, in practical application, it is suggested to weigh the relative requirements of accuracy and operational efficiency before selecting a suitable algorithm to cluster Dirichlet samples.

Key words: Dirichlet distribution, mixture sample, expectation marimization (EM) algorithm, dynamical clustering, machine learning

中图分类号:

O212

夏棒, EMILION Richard, 王惠文. Dirichlet混合样本的EM算法与动态聚类算法比较[J]. 北京航空航天大学学报, 2019, 45(9): 1805-1811.

XIA Bang, EMILION Richard, WANG Huiwen. Comparison between EM algorithm and dynamical clustering algorithm for Dirichlet mixture samples[J]. JOURNAL OF BEIJING UNIVERSITY OF AERONAUTICS AND A, 2019, 45(9): 1805-1811.

参考文献

[1] FRALEY C,RAFTERY A E.Model-based clustering,discriminant analysis,and density estimation[J]. Publications of the American Statistical Association,2002,97(458):611-631.
[2] DEMPSTER A P.Maximum likelihood from incomplete data via the EM algorithm[J].Journal of Royal Statistical Society B,1977,39(1):1-38.
[3] CELEUX G,GOVAERT G.Stochastic algorithms for clustering[M].Heidelberg:Physica-Verlag,1990.
[4] CELEUX G,GOVAERT G.A classification EM algorithm for clustering and two stochastic versions[J].Computational Statistics & Data Analysis,1992,14(3):315-332.
[5] TANNER M A,WONG W H.The calculation of posterior distributions by data augmentation[J].Publications of the American Statistical Association,1987,82(398):528-540.
[6] RENDER E.Mixture densities,maximum likelihood and the EM algorithm[J].SIAM Review,1984,26(2):195-239.
[7] PEEL D,MCLACHLAN G J.Robust mixture modelling using the t distribution[J].Statistics and Computing,2000,10(4):339-348.
[8] DIDAY E,SCHROEDER A,OK Y.The dynamic clusters method in pattern recognition[C]//Proceedings of International Federation for Information Processing Congress,1974:691-697.
[9] SCOTT A J,SYMONS M J.Clustering methods based on likelihood ratio criteria[J].Biometrics,1971,27(1):387-397.
[10] SYMONS M J.Clustering criteria and multivariate normal mixtures[J].Biometrics,1981,37(1):35-43.
[11] WONG M A,LANE T.A kTH nearest neighbour clustering procedure[J].Journal of the Royal Statistical Society,1981,45(3):362-368.
[12] BROSSIER G.Piecewise hierarchical clustering[J].Journal of Classification,1990,7(2):197-216.
[13] BISHOP C M.Neural networks for pattern recognition[M].Oxford:Oxford University Press,1995.
[14] BOCK H H.Clustering and neural networks[M]//RIZZI A,VICHI M,BOCK H H.Advances in data science and classification.Berlin:Springer,1998:265-277.
[15] ARABIE P, CARROLL J D.Mapclus:A mathematical programming approach to fitting the adclus model[J].Psychometrika,1980,45(2):211-235.
[16] DIDAY E.An intuitive form of expression:Pyramids[D].Paris:INRIA,1984.
[17] JAMES G M, SUGAR C A.Clustering for sparsely sampled functional data[J].Publications of the American Statistical Association,2003,98(462):397-408.
[18] WINSBERG S,SOETE G D.Latent class models for time series analysis[J].Applied Stochastic Models in Business & Industry,1999,15(15):183-194.
[19] VRAC M,BILLARD L,DIDAY E.Copula analysis of mixture models[J].Computational Statistics,2012,27(3):427-457.
[20] KOSMIDIS I,KARLIS D.Model-based clustering using copulas with applications[J].Statistics and Computing,2016,26(5):1079-1099.
[21] MARBAC M,BIERNACKI C,VANDEWALLE V.Model-based clustering of Gaussian copulas for mixed data[J].Communications in Statistics-Theory and Methods,2017,46(23):11635-11656.

Dirichlet混合样本的EM算法与动态聚类算法比较

Comparison between EM algorithm and dynamical clustering algorithm for Dirichlet mixture samples

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 2

编辑推荐

Metrics

本文评价

[1]	聂春雨, 祝明, 郑泽伟, 武哲. 基于Q-Learning算法和神经网络的飞艇控制[J]. 北京航空航天大学学报, 2017, 43(12): 2431-2438.
[2]	李可, 杨奕臻, 颜波. 基于学习的图像智能适配显示技术[J]. 北京航空航天大学学报, 2015, 41(6): 1147-1154.