基于Parameter Server框架的大数据挖掘优化算法

doi:10.6040/j.issn.1672-3961.0.2016.339

山东大学学报(工学版) ›› 2017, Vol. 47 ›› Issue (4): 1-6.doi: 10.6040/j.issn.1672-3961.0.2016.339

• • 下一篇

基于Parameter Server框架的大数据挖掘优化算法

刘洋¹,刘博²,王峰¹

1. 河南财经政法大学云计算与大数据研究所, 河南郑州 450046;2. 华中科技大学计算机学院, 湖北武汉 430074

收稿日期:2016-09-03 出版日期:2017-08-20 发布日期:2016-09-03
作者简介:刘洋(1980— ),男,河南方城人,讲师,博士,主要研究方向为计算机系统结构,机器学习,大数据等.E-mail: liuyang@huel.edu.cn
基金资助:
河南省重点科技攻关资助项目(162102210096,152102210088,142102210090);河南省高等学校重点科研资助项目(18A520014)

Optimization algorithm for big data mining based on parameter server framework

LIU Yang¹, LIU Bo², WANG Feng¹

1. Institute of Cloud Computing and Big Data, Henan University of Economics and Law, Zhengzhou 450046, Henan, China;
2. School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan 430074, Hubei, China

Received:2016-09-03 Online:2017-08-20 Published:2016-09-03

摘要/Abstract

摘要： 基于大数据挖掘的实时性要求和数据样本的多样性特征,提出一种面向大数据挖掘的机器学习模型训练优化算法。分析当前算法的迭代计算过程,根据模型向量的改变量将迭代过程分为粗调和微调两个阶段,并发现在微调阶段绝大部分样本对计算结果的影响极小,因此可以在微调阶段不计算此类样本的梯度而直接采用上次迭代的计算结果,从而减小计算量,提升计算效率。试验结果表明,算法在分布式集群环境下可以减小模型训练约35%的计算量,且训练得到的模型准确度在正常范围内,可有效提高大数据挖掘的实时性。

关键词: 优化算法, 分布式系统, 大数据, 样本差异性, 机器学习

Abstract: Traditional machine learning algorithms for small data were not applicable for mining of big data. An optimization algorithm for machine learning and big data mining was proposed. The iterative computation of machine learning algorithms was divided into two phases according to the change of model vector. According to the observation that most samples contributed little to the model update during the iteration, the computation load of machine learning algorithms could be reduced by reusing the iterative computing results of this kind of samples. The experimental results showed that the proposed method could reduce the computation load by 35%, with little effect on prediction accuracy of the training model.

Key words: big data, sample diversity, machine learning, distributed system, optimization

中图分类号:

TU457

刘洋,刘博,王峰. 基于Parameter Server框架的大数据挖掘优化算法[J]. 山东大学学报(工学版), 2017, 47(4): 1-6.

LIU Yang, LIU Bo, WANG Feng. Optimization algorithm for big data mining based on parameter server framework[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2017, 47(4): 1-6.

参考文献

[1] 张引,陈敏,廖小飞. 大数据应用的现状与展望[J]. 计算机研究和发展,2013, 50(S2):216-233 ZHANG Yin, CHEN Min, LIAO Xiaofei. Big data applications: a survey[J]. Journal of Computer Research and Development, 2013, 50(S2):216-233.
[2] 王元卓,靳小龙,程学旗. 网络大数据:现状与展望[J]. 计算机学报,2013,36(6):1125-1138. WANG Yuanzhuo, JIN Xiaolong, CHENG Xueqi. Network big data: present and future[J]. Chinese Journal of Computers, 2013, 36(6):1125-1138.
[3] 张蕾,章毅. 大数据分析的无限深度神经网络方法[J]. 计算机研究与发展,2016,53(1):68-79. ZHANG Lei, ZHANG Yi. Big data analysis by infinite deep neural networks[J].Journal of Computer Research and Development, 2016, 53(1):68-79.
[4] 耿丽娟,李星毅. 用于大数据分类的KNN算法研究[J]. 计算机应用研究,2014, 31(5):1342-1344. GENG Lijuan, LI Xingyi. Improvements of KNN algorithm for big data classification[J]. Application Research of Computers, 2014, 31(5):1342-1344.
[5] 刘红岩,陈剑,陈国青. 数据挖掘中的数据分类算法综述[J].清华大学学报(自然科学版),2002,42(6):727-730. LIU Hongyan, CHEN Jian, CHEN Guoqing. Review of classification algorithms for data mining[J]. Journal of Tsinghua University(Science & Technology), 2002, 42(6):727-730.
[6] 何清,李宁,罗文娟,等. 大数据下的机器学习算法综述[J]. 模式识别与人工智能,2014,27(4):327-336. HE Qing, LI Ning, LUO Wenjuan, et al. A survey of machine learning algorithms for big data[J]. Pattern Recognition and Artificial Intelligence, 2014, 27(4):327-336.
[7] 吴启晖,邱俊飞,丁国如. 面向频谱大数据处理的机器学习方法[J].数据采集与处理,2015,30(4):703-713. WU Qihui, QIU Junfei, DING Guoru. Machine learning methods for big spectrum data processing[J]. Journal of Data Acquisition and Processing, 2015, 30(4):703-713.
[8] 程学旗,靳小龙,王元卓. 大数据系统和分析技术综述[J]. 软件学报,2014,25(9):1889-1908. CHENG Xueqi, JIN Xiaolong, WANG Yuanzhuo. Survey on big data system and analytic technology[J]. Journal of Software, 2014, 25(9):1889-1908.
[9] 郭迟,刘经南,方媛,等. 位置大数据的价值提取与协同挖掘方法[J]. 软件学报,2014, 25(4):713-730. GUO Chi, LIU Jingnan, FANG Yuan, et al. Value extraction and collaborative mining methods for location big data[J]. Journal of Software, 2014, 25(4):713-730.
[10] 陈国良,毛睿,陆克中. 大数据并行计算框架[J]. 科学通报,2015,60:566-569. CHEN Guoliang, MAO Rui, LU Kezhong. Parallel computing framework for big data[J]. Chinese Science Bulletin, 2015, 60:566-569.
[11] YUAN Jinhui, GAO Fei, HO Qirong, et al. Light LDA: big topic models on modest computer clusters[C] //Proceedings of the 24th International Conference on World Wide Web. Florence, Italy: Springer, 2015:1351-1361
[12] KUMAR Abhimanu, BEUTEL Alex, HO Qirong, et al. Fugue: slow-worker-agnostic distributed learning for big models on big data[C] //Proceedings of the Seventeenth International Conference on Artificial Intelligence and Statistics. Reykjavik, Iceland: JMLR, 2014:531-539.
[13] LIU Ji, WRIGHT S J, RE Christopher, et al. An asynchronous parallel stochastic coordinate descent algorithm[J]. Journal of Machine Learning Research, 2015, 16(1):285-322.
[14] HSIEH C J, YU H F, DHILLON I S. PASSCoDe: parallel asynchronous stochastic dual coordinate descent[C] //Proceedings of the 32nd International Conference on Machine Learning. Lille, France: ACM, 2015: 2370-2379.
[15] CHU Chengtao, KIM Sangkyun, LIN Yian, et al. Map-reduce for machine learning on multicore[C] //20th Annual Conference on Neural Information Processing Systems Vancouver. British Columbia, Canada: MIT Press, 2006:281-288.
[16] POWER Russell, LI Jinyang. Piccolo: building fast, distributed programs with partitioned tables[C] //9th USENIX Symposium on Operating Systems Design and Implementation. Vancouver, Canada: USENIX, 2010: 293-306.
[17] CHILIMBI Trishul, SUZUE Yutaka, APACIBLE Johnson, et al. Project adam: building an efficient and scalable deep learning training system[C] //11th USENIX Symposium on Operating Systems Design and Implementation. Broomfield, USA: USENIX, 2014: 571-582.
[18] XING Eric P, HO Qirong, DAI Wei, et al. Petuum: a new platform for distributed machine learning on big data[C] //Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Sydney, NSW, Australia: ACM, 2015: 1335-1344.
[19] LI Mu, ANDERSEN David G, PARK Jun Woo, et al. Scaling distributed machine learning with the parameter server[C] //11th USENIX Symposium on Operating Systems Design and Implementation. Broomfield, USA: USENIX, 2014:583-598.
[20] LI Mu, ANDERSEN David G, SMOLA Alexander J, et al. Communication efficient distributed machine learning with the parameter server[C] //28th Annual Conference on Neural Information Processing Systems. Montreal, Canada: MIT Press, 2014: 19-27.
[21] HO Qirong, CIPAR James, CUI Henggang, et al. More effective distributed ML via a stale synchronous parallel parameter server[C] //27th Annual Conference on Neural Information Processing Systems. Lake Tahoe, United States: MIT Press, 2013: 1223-1231.
[22] LANGFORD John, SMOLA Alexander J, ZINKEVICH Martin. Slow learners are fast[C] //23rd Annual Conference on Neural Information Processing Systems. Vancouver, Canada: MIT Press, 2009: 2331-2339.
[23] ZINKEVICH Martin A, WEIMER Markus, SMOLA Alex, et al. Parallelized stochastic gradient descent[C] //24th Annual Conference on Neural Information Processing Systems. Vancouver, Canada: MIT Press, 2009: 2331-233.
[24] LEWIS David D, YANG Yiming, ROSE Tony G, et al. RCV1: a new benchmark collection for text categorization research[J]. Journal of Machine Learning Research, 2004, 5:361-397.

多维度评价

Viewed

Full text

Abstract

Cited

Shared

Discussed

基于Parameter Server框架的大数据挖掘优化算法

Optimization algorithm for big data mining based on parameter server framework

PDF (PC)

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

多维度评价

本文评价

推荐阅读 0

[1]	张冕,黄颖,梅海艺,郭毓. 基于Kinect的配电作业机器人智能人机交互方法[J]. 山东大学学报(工学版), 2018, 48(5): 103-108.
[2]	王婷婷,翟俊海,张明阳,郝璞. 基于HBase和SimHash的大数据K-近邻算法[J]. 山东大学学报(工学版), 2018, 48(3): 54-59.
[3]	于曰伟,周长城,赵雷雷,邢玉清,石沛林. 基于交替迭代的车辆主动悬架LQG控制器设计[J]. 山东大学学报(工学版), 2017, 47(4): 50-58.
[4]	魏波,张文生,李元香,夏学文,吕敬钦. 一种选择特征的稀疏在线学习算法[J]. 山东大学学报(工学版), 2017, 47(1): 22-27.
[5]	周旺,张晨麟,吴建鑫. 一种基于Hartigan-Wong和Lloyd的定性平衡聚类算法[J]. 山东大学学报(工学版), 2016, 46(5): 37-44.
[6]	孟令恒,丁世飞. 基于单静态图像的深度感知模型[J]. 山东大学学报(工学版), 2016, 46(3): 37-43.
[7]	刘杰, 杨鹏, 吕文生, 刘阿古达木, 刘俊秀. 基于气象因素的PM_2.5质量浓度预测模型[J]. 山东大学学报(工学版), 2015, 45(6): 76-83.
[8]	董红斌, 张广江, 逄锦伟, 韩启龙. 一种基于协同进化方法的聚类集成算法[J]. 山东大学学报(工学版), 2015, 45(2): 1-9.
[9]	郑毅, 朱成璋. 基于深度信念网络的PM_2.5预测[J]. 山东大学学报(工学版), 2014, 44(6): 19-25.
[10]	王惠芳, 赵志诚, 张井岗. 一种高阶系统的分数阶IMC-ID^μ控制器设计[J]. 山东大学学报(工学版), 2014, 44(6): 77-82.
[11]	谢琳1,殷熙尧2,李凡长3,吴佳3. 一种逆归结学习表示[J]. 山东大学学报(工学版), 2013, 43(4): 46-50.
[12]	徐龙琴1,刘双印1,2,3,4*. 基于APSO-WLSSVR的水质预测模型[J]. 山东大学学报(工学版), 2012, 42(5): 80-86.
[13]	蔡荣英,王李进,吴超,钟一文*. 一种求解旅行商问题的迭代改进蚁群优化算法[J]. 山东大学学报(工学版), 2012, 42(1): 6-11.
[14]	何雪英1,2, 秦伟1, 尹义龙1*, 赵联征1,乔昊3. 基于机器学习的视频指纹识别[J]. 山东大学学报(工学版), 2011, 41(4): 29-33.
[15]	梁春林1,彭凌西2*. 基于免疫网络的无监督式分类算法[J]. 山东大学学报(工学版), 2010, 40(5): 82-86.