JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE) ›› 2017, Vol. 47 ›› Issue (4): 1-6.doi: 10.6040/j.issn.1672-3961.0.2016.339

    Next Articles

Optimization algorithm for big data mining based on parameter server framework

LIU Yang1, LIU Bo2, WANG Feng1   

  1. 1. Institute of Cloud Computing and Big Data, Henan University of Economics and Law, Zhengzhou 450046, Henan, China;
    2. School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan 430074, Hubei, China
  • Received:2016-09-03 Online:2017-08-20 Published:2016-09-03

PDF (PC)

629

Abstract: Traditional machine learning algorithms for small data were not applicable for mining of big data. An optimization algorithm for machine learning and big data mining was proposed. The iterative computation of machine learning algorithms was divided into two phases according to the change of model vector. According to the observation that most samples contributed little to the model update during the iteration, the computation load of machine learning algorithms could be reduced by reusing the iterative computing results of this kind of samples. The experimental results showed that the proposed method could reduce the computation load by 35%, with little effect on prediction accuracy of the training model.

Key words: big data, sample diversity, machine learning, distributed system, optimization

CLC Number: 

  • TU457
[1] 张引,陈敏,廖小飞. 大数据应用的现状与展望[J]. 计算机研究和发展,2013, 50(S2):216-233 ZHANG Yin, CHEN Min, LIAO Xiaofei. Big data applications: a survey[J]. Journal of Computer Research and Development, 2013, 50(S2):216-233.
[2] 王元卓,靳小龙,程学旗. 网络大数据:现状与展望[J]. 计算机学报,2013,36(6):1125-1138. WANG Yuanzhuo, JIN Xiaolong, CHENG Xueqi. Network big data: present and future[J]. Chinese Journal of Computers, 2013, 36(6):1125-1138.
[3] 张蕾,章毅. 大数据分析的无限深度神经网络方法[J]. 计算机研究与发展,2016,53(1):68-79. ZHANG Lei, ZHANG Yi. Big data analysis by infinite deep neural networks[J].Journal of Computer Research and Development, 2016, 53(1):68-79.
[4] 耿丽娟,李星毅. 用于大数据分类的KNN算法研究[J]. 计算机应用研究,2014, 31(5):1342-1344. GENG Lijuan, LI Xingyi. Improvements of KNN algorithm for big data classification[J]. Application Research of Computers, 2014, 31(5):1342-1344.
[5] 刘红岩,陈剑,陈国青. 数据挖掘中的数据分类算法综述[J].清华大学学报(自然科学版),2002,42(6):727-730. LIU Hongyan, CHEN Jian, CHEN Guoqing. Review of classification algorithms for data mining[J]. Journal of Tsinghua University(Science & Technology), 2002, 42(6):727-730.
[6] 何清,李宁,罗文娟,等. 大数据下的机器学习算法综述[J]. 模式识别与人工智能,2014,27(4):327-336. HE Qing, LI Ning, LUO Wenjuan, et al. A survey of machine learning algorithms for big data[J]. Pattern Recognition and Artificial Intelligence, 2014, 27(4):327-336.
[7] 吴启晖,邱俊飞,丁国如. 面向频谱大数据处理的机器学习方法[J].数据采集与处理,2015,30(4):703-713. WU Qihui, QIU Junfei, DING Guoru. Machine learning methods for big spectrum data processing[J]. Journal of Data Acquisition and Processing, 2015, 30(4):703-713.
[8] 程学旗,靳小龙,王元卓. 大数据系统和分析技术综述[J]. 软件学报,2014,25(9):1889-1908. CHENG Xueqi, JIN Xiaolong, WANG Yuanzhuo. Survey on big data system and analytic technology[J]. Journal of Software, 2014, 25(9):1889-1908.
[9] 郭迟,刘经南,方媛,等. 位置大数据的价值提取与协同挖掘方法[J]. 软件学报,2014, 25(4):713-730. GUO Chi, LIU Jingnan, FANG Yuan, et al. Value extraction and collaborative mining methods for location big data[J]. Journal of Software, 2014, 25(4):713-730.
[10] 陈国良,毛睿,陆克中. 大数据并行计算框架[J]. 科学通报,2015,60:566-569. CHEN Guoliang, MAO Rui, LU Kezhong. Parallel computing framework for big data[J]. Chinese Science Bulletin, 2015, 60:566-569.
[11] YUAN Jinhui, GAO Fei, HO Qirong, et al. Light LDA: big topic models on modest computer clusters[C] //Proceedings of the 24th International Conference on World Wide Web. Florence, Italy: Springer, 2015:1351-1361
[12] KUMAR Abhimanu, BEUTEL Alex, HO Qirong, et al. Fugue: slow-worker-agnostic distributed learning for big models on big data[C] //Proceedings of the Seventeenth International Conference on Artificial Intelligence and Statistics. Reykjavik, Iceland: JMLR, 2014:531-539.
[13] LIU Ji, WRIGHT S J, RE Christopher, et al. An asynchronous parallel stochastic coordinate descent algorithm[J]. Journal of Machine Learning Research, 2015, 16(1):285-322.
[14] HSIEH C J, YU H F, DHILLON I S. PASSCoDe: parallel asynchronous stochastic dual coordinate descent[C] //Proceedings of the 32nd International Conference on Machine Learning. Lille, France: ACM, 2015: 2370-2379.
[15] CHU Chengtao, KIM Sangkyun, LIN Yian, et al. Map-reduce for machine learning on multicore[C] //20th Annual Conference on Neural Information Processing Systems Vancouver. British Columbia, Canada: MIT Press, 2006:281-288.
[16] POWER Russell, LI Jinyang. Piccolo: building fast, distributed programs with partitioned tables[C] //9th USENIX Symposium on Operating Systems Design and Implementation. Vancouver, Canada: USENIX, 2010: 293-306.
[17] CHILIMBI Trishul, SUZUE Yutaka, APACIBLE Johnson, et al. Project adam: building an efficient and scalable deep learning training system[C] //11th USENIX Symposium on Operating Systems Design and Implementation. Broomfield, USA: USENIX, 2014: 571-582.
[18] XING Eric P, HO Qirong, DAI Wei, et al. Petuum: a new platform for distributed machine learning on big data[C] //Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Sydney, NSW, Australia: ACM, 2015: 1335-1344.
[19] LI Mu, ANDERSEN David G, PARK Jun Woo, et al. Scaling distributed machine learning with the parameter server[C] //11th USENIX Symposium on Operating Systems Design and Implementation. Broomfield, USA: USENIX, 2014:583-598.
[20] LI Mu, ANDERSEN David G, SMOLA Alexander J, et al. Communication efficient distributed machine learning with the parameter server[C] //28th Annual Conference on Neural Information Processing Systems. Montreal, Canada: MIT Press, 2014: 19-27.
[21] HO Qirong, CIPAR James, CUI Henggang, et al. More effective distributed ML via a stale synchronous parallel parameter server[C] //27th Annual Conference on Neural Information Processing Systems. Lake Tahoe, United States: MIT Press, 2013: 1223-1231.
[22] LANGFORD John, SMOLA Alexander J, ZINKEVICH Martin. Slow learners are fast[C] //23rd Annual Conference on Neural Information Processing Systems. Vancouver, Canada: MIT Press, 2009: 2331-2339.
[23] ZINKEVICH Martin A, WEIMER Markus, SMOLA Alex, et al. Parallelized stochastic gradient descent[C] //24th Annual Conference on Neural Information Processing Systems. Vancouver, Canada: MIT Press, 2009: 2331-233.
[24] LEWIS David D, YANG Yiming, ROSE Tony G, et al. RCV1: a new benchmark collection for text categorization research[J]. Journal of Machine Learning Research, 2004, 5:361-397.
[1] HU Jianping, LI Xin, XIE Qi, LI Ling, ZHANG Daochang. An unconstrained optimization EMD approach in 2D based on Delaunay triangulation [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2018, 48(5): 9-15.
[2] ZHANG Mian, HUANG Ying, MEI Haiyi, GUO Yu. Intelligent interaction method for power distribution robot based on Kinect [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2018, 48(5): 103-108.
[3] QIAN Shuqu, WU Huihong, XU Guofeng, JIN Jingliang. Immune clonal evolutionary algorithm of dynamic economic dispatch considering gas pollution emission [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2018, 48(4): 1-9.
[4] WANG Tingting, ZHAI Junhai, ZHANG Mingyang, HAO Pu. K-NN algorithm for big data based on HBase and SimHash [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2018, 48(3): 54-59.
[5] XIAO Miaomiao, WEI Benzheng, YIN Yilong. A hybrid intrusion detection system based on BFOA and K-means algorithm [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2018, 48(3): 115-119.
[6] YE Mingquan, GAO Lingyun, WAN Chunyuan. Gene expression data classification based on artificial bee colony and SVM [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2018, 48(3): 10-16.
[7] TIAN Jing, LI Hongwei, CHEN Rui, YU Wentao. Structural optimization and finite element analysis of a type of axial AMBs [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2018, 48(2): 107-113.
[8] LIU Zhiqing, GAO Haohan, AN Molin, ZHANG Xuekai. Project schedule optimization based on critical chain method with modified completion probability [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2018, 48(1): 104-111.
[9] CHU Xiaodong, TANG Maosen, GAO Xu, LIU Weisheng, JIA Shanjie, LI Sun. Robust optimal dispatch of active distribution networks based on centralized information system [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2017, 47(6): 20-25.
[10] LIU Xiaoming, XU Naiyuan, YANG Bin, WEI Xin, ZHANG Lina, CAO Yongji. Bi-stage optimization method for receiving-end ultra-high voltage network planning under global energy interconnection [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2017, 47(6): 1-6.
[11] SUN Donglei, LI Shan, LI Xueliang, HAN Xueshan, LI Wenbo, YANG Si. Interval optimization method for reference network adaptable to the uncertainties of power sources and electric loads [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2017, 47(6): 26-31.
[12] ZHANG Qisong, FU Jun, SHI Zhaohui, YANG Xiao, JIA Chunjuan, HUANG Yulong. SOC optimization on power supply reliability of wind-storage system [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2017, 47(6): 95-99.
[13] MA Hanjie, LIN Xia, XU Xiaohui, ZHANG Jian, ZHANG Zhisheng. Load optimization model of smart home management system based on adaptive particle swarm optimization [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2017, 47(6): 57-62.
[14] YU Yuewei, ZHOU Changcheng, ZHAO Leilei, XING Yuqing, SHI Peilin. Design of LQG controller for vehicle active suspension system based on alternate iteration [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2017, 47(4): 50-58.
[15] BAI Fengyue, WEN Bangyao, BAI Shuzhan. Analysis and optimization of urea deposits formation in urea-SCR systems [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2017, 47(4): 70-76.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
Copyright 2018 © Editorial office of Journal of Shandong University (Engineering Science)
Supported by Beijing Magtech Co.Ltd