一种面向矩阵分解模型的推荐系统训练加速方法

摘要/Abstract

参考文献

相关文章 15

多维度评价

doi:10.6040/j.issn.1672-3961.0.2023.285

摘要： 为降低矩阵分解(matrix factorization, MF)模型面向推荐系统应用的训练时间,特别针对细粒度稀疏的特征矩阵在训练过程中存在大量无效乘法运算的问题,提出一种基于特征矩阵联合稀疏性进行近似计算的训练加速方法。基于隐因子向量稀疏性强弱基本不变的特点,提出在模型训练初期,根据隐因子向量的稀疏性,对特征矩阵重新排列;在训练过程中,采用早停法,避免无效乘法运算。试验结果表明,模型训练过程中乘法运算次数可最多降低28.41%,加速前后评分预测值相关系数约0.95。所提出方法可以保证预测准确性小幅降低的同时,显著减少训练中的乘法运算次数,针对更大规模的矩阵分解模型训练,能实现更好的加速效果。

关键词: 推荐系统, 矩阵分解, 稀疏性, 算法加速, 近似计算

中图分类号:

TP391

段圣宇,吴伊宁,赛高乐. 一种面向矩阵分解模型的推荐系统训练加速方法[J]. 山东大学学报 (工学版), 2025, 55(1): 24-29.

DUAN Shengyu, WU Yining, SAI Gaole. Algorithmic acceleration of matrix factorization based recommendation system[J]. Journal of Shandong University(Engineering Science), 2025, 55(1): 24-29.

[1] Amazon Web Services Inc. Amazon personalize[EB/OL].(2023-12-16)[2023-02-16]. https://aws.amazon.com/personalize/.
[2] COVINGTON P, ADAMS J, SARGI E. Deep neural networks for YouTube recommendations[C] //Proceedings of the 10^th Conference on Recommender Systems. New York, USA: ACM, 2016: 191-198.
[3] 王磊,熊于宁,李云鹏,等. 一种基于增强图卷积神经网络的协同推荐模型[J]. 计算机研究与发展, 2021, 58(9): 1987-1996. WANG Lei, XIONG Yuning, LI Yunpeng, et al. A collaborative recommendation model based on enhanced graph convolutional neural network[J]. Journal of Computer Research and Development, 2021, 58(9): 1987-1996.
[4] HE Xiangnan, LIAO Lizi, ZHANG Hanwang, et al. Neural collaborative filtering[C] //Proceedings of the 26th International Conference on World Wide Web. New York, USA: ACM, 2017: 173-182.
[5] RESNICK P, VARIAN H R. Recommender systems[J]. Communications of the ACM, 1997, 40(3): 56-58.
[6] SALAKHUTDINOV R, MNIH A. Probabilistic matrix factorization[C] //Proceedings of the 20th International Conference on Neural Information Processing Systems. New York, USA: ACM, 2007: 1257-1264.
[7] KOREN Y. Factor in the neighbors: Scalable and accurate collaborative filtering[J]. ACM Transactions on Knowledge Discovery from Data, 2010, 4(1): 1-24.
[8] KOREN Y, BELL R, VOLINSKY C. Matrix factorization techniques for recommender systems[J]. Computer, 2009, 42(8): 30-37.
[9] YUE Xiaochen, LIU Qicheng. Parallel algorithms of improved FunkSVD based on GPU[J]. IEEE Access, 2022, 10: 26002-26010.
[10] BOTTOU L. Large-scale machine learning with stochastic gradient descent[C] //Proceedings of the 19th International Conference on Computational Statistics. Berlin, Germany: Springer, 2010: 177-186.
[11] YU H F, HSIEH C J, SI S, et al. Scalable coordinate descent approaches to parallel matrix factorization for recommender systems[C] //12th International Con-ference on Data Mining. New Jersey, USA: IEEE, 2012: 765-774.
[12] CHIN W S, ZHUANG Y, JUAN Y C, et al. A fast parallel stochastic gradient method for matrix factorization in shared memory systems[J]. ACM Transactions on Intelligent Systems and Technology, 2015, 6(1): 1-24.
[13] TAN Wei, CAO Liangliang, FONG L. Faster and cheaper: parallelizing large-scale matrix factorization on GPUs[C] //Proceedings of the 25th International Symposium on High-Performance Parallel and Distributed Computing. New York, USA: ACM, 2016: 219-230.
[14] WEI Feng, GUO Hao, CHENG Shaoyin, et al. AALRSMF: An adaptive learning rate schedule for matrix factorization[C] //Asia-Pacific Web Conference. Berlin, Germany: Springer, 2016: 410-413.
[15] XIE X L, TAN W, FONG L, et al. Cumf_sgd: parallelized stochastic gradient descent for matrix factorization on GPUs[C] //Proceedings of the 26th International Symposium on High-Performance Parallel and Distributed Computing. New York, USA: ACM, 2017: 79-92.
[16] LIAN Xiangru, YUAN Binhang, ZHU Xuefeng, et al. Persia: an open, hybrid system scaling deep learning-based recommenders up to 100 trillion parameters[C] // Proceedings of the 28^th SIGKDD Conference on Knowledge Discovery and Data Mining. New York, USA: ACM, 2022: 3288-3298.
[17] DONÀ J, GALLINARI P. Differentiable feature selection, a reparameterization approach[C] //Machine Learning and Knowledge Discovery in Databases. Research Track: European Conference. Berlin, Germany: Springer, 2021: 13-17.
[18] WU Yining, SAI Gaole, DUAN Shengyu. Work-in-Progress: Accelerated matrix factorization by approximate computing for recommendation system[C] //International Conference on Embedded Software. Shanghai, China: IEEE, 2022: 1-2.
[19] HARPER F M, KONSTAN J A. The MovieLens datasets: history and context[J]. ACM Transactions on Interactive Intelligent Systems, 2016, 5(4): 1-19.
[20] CHIN W S, YUAN B W, YANG M Y, et al. LIBMF: a library for parallel matrix factorization in shared-memory systems[J]. Journal of Machine Learing Research, 2016, 17(1): 2971-2977.

[1]	李璐,张志军,范钰敏,王星,袁卫华. 面向冷启动用户的元学习与图转移学习序列推荐[J]. 山东大学学报 (工学版), 2024, 54(2): 69-79.
[2]	郑顺,王绍卿,刘玉芳,李可可,孙福振. 基于动态掩码和多对对比学习的序列推荐模型[J]. 山东大学学报 (工学版), 2023, 53(6): 47-55.
[3]	王冰,马文明,武聪,郝昱猛. 融合信任相似度的偏置概率矩阵分解算法[J]. 山东大学学报 (工学版), 2022, 52(4): 110-117.
[4]	林晓炜,陈黎飞. 结构扩展的非负矩阵分解社区发现算法[J]. 山东大学学报 (工学版), 2021, 51(2): 57-64.
[5]	张文凯,禹可,吴晓非. 基于元图归一化相似性度量的实体推荐[J]. 山东大学学报 (工学版), 2020, 50(2): 66-75.
[6]	何文杰 ,何伟超,孙权森. 压缩感知重构算法的并行化及GPU加速[J]. 山东大学学报(工学版), 2018, 48(3): 110-114.
[7]	魏波,张文生,李元香,夏学文,吕敬钦. 一种选择特征的稀疏在线学习算法[J]. 山东大学学报(工学版), 2017, 47(1): 22-27.
[8]	黄丹,王志海,刘海洋. 一种局部协同过滤的排名推荐算法[J]. 山东大学学报(工学版), 2016, 46(5): 29-36.
[9]	林耀进,张佳,林梦雷,王娟. 一种基于模糊信息熵的协同过滤推荐方法[J]. 山东大学学报(工学版), 2016, 46(5): 13-20.
[10]	庞俊涛, 张晖, 杨春明, 李波, 赵旭剑. 基于概率矩阵分解的多指标协同过滤算法[J]. 山东大学学报(工学版), 2016, 46(3): 65-73.
[11]	张佳,林耀进,林梦雷,刘景华,李慧宗. 基于信息熵的协同过滤算法[J]. 山东大学学报(工学版), 2016, 46(2): 43-50.
[12]	李新玉, 徐桂云,任世锦,杨茂云. 基于鉴别流形的不相关稀疏投影非负矩阵分解[J]. 山东大学学报 (工学版), 2015, 45(5): 1-12.
[13]	陈大伟,闫昭*,刘昊岩. SVD系列算法在评分预测中的过拟合现象[J]. 山东大学学报(工学版), 2014, 44(3): 15-21.
[14]	孙远帅,陈垚,刘向荣,陈珂,林琛. 基于项目层次相似性的推荐算法[J]. 山东大学学报(工学版), 2014, 44(3): 8-14.
[15]	李改1,2,3, 李磊2,3. 一种解决协同过滤系统冷启动问题的新算法[J]. 山东大学学报(工学版), 2012, 42(2): 11-17.

Just accepted

Online first

Just accepted

Online first

Viewed

Full text

	From	local

	Times	39
	Rate	100%

Abstract

Just accepted	Online first	Issue

0	0	83

	From	Others

	Times	83
	Rate	100%

Cited

Web of Science	Crossref	ScienceDirect	Search for Citations in Google Scholar >>


This page requires you have already subscribed to WoS.

Shared

Discussed