Journal of Shandong University(Engineering Science) ›› 2023, Vol. 53 ›› Issue (6): 47-55.doi: 10.6040/j.issn.1672-3961.0.2022.381

• Machine Learning & Data Mining • Previous Articles    

Sequential recommendation model based on dynamic mask and multi-pair contrastive learning

ZHENG Shun, WANG Shaoqing*, LIU Yufang, LI Keke, SUN Fuzhen   

  1. School of Computer Science and Technology, Shandong University of Technology, Zibo 255000, Shandong, China
  • Published:2023-12-19

CLC Number: 

  • TP399
[1] DEVLIN J, CHANG M W, LEE K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[C] //Proceedings of NAACL-HLT. Minneapolis, USA: ACL, 2019: 4171-4186.
[2] SUN F, LIU J, WU J, et al. BERT4Rec: sequential recommendation with bidirectional encoder representations from transformer[C] //Proceedings of the 28th ACM International Conference on Information and Knowledge Management. Beijing, China: ACM, 2019: 1441-1450.
[3] DU H, SHI H, ZHAO P, et al. Contrastive learning with bidirectional transformers for sequential recommendation[C] //Proceedings of the 31st ACM International Conference on Information & Knowledge Management. Atlanta, USA: ACM, 2022: 396-405.
[4] WETTIG A, GAO T, ZHONG Z, et al. Should you mask 15% in masked language modeling?[C] // Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics. Dubrovnik, Croatia: ACL, 2023: 2977-2992.
[5] SHANI G, HECKERMAN D, BRAFMAN R I, et al. An MDP-based recommender system[J]. Journal of Machine Learning Research, 2005, 6(9):1532-4435.
[6] RENDLE S, FREUDENTHALER C, SCHMIDT-THIEME L. Factorizing personalized Markov chains for next-basket recommendation[C] //Proceedings of the 19th International Conference on World Wide Web. Raleigh, USA: ACM, 2010: 811-820.
[7] HE R, MCAULEY J. Fusing similarity models with Markov chains for sparse sequential recommendation[C] //2016 16th International Conference on Data Mining(ICDM). Barcelona, Spain: IEEE, 2016: 191-200.
[8] 冯兴杰, 曾云泽. 基于评分矩阵与评论文本的深度推荐模型[J]. 计算机学报, 2020, 43(5):884-900. FENG Xingjie, ZENG Yunze. Joint deep modeling of rating matrix and reviews for recommendation[J]. Chinese Journal of Computers, 2020, 43(5):884-900.
[9] 沈学利, 杜志伟. 融合自注意力机制与长短期偏好的序列推荐模型[J]. 计算机应用研究, 2021, 38(5):1371-1375. SHEN Xueli, DU Zhiwei. Sequential recommendation model that combines self-attention mechanism with long-term and short-term preferences [J]. Application Research of Computers, 2021, 38(5):1371-1375.
[10] KANG W C, MCAULEY J. Self-attentive sequential recommendation[C] //2018 IEEE International Conference on Data Mining(ICDM). Singapore: IEEE, 2018: 197-206.
[11] XU C, FENG J, ZHAO P, et al. Long- and short- term self-attention network for sequential recommendation[J]. Neurocomputing, 2021, 423: 580-589.
[12] WU S, TANG Y, ZHU Y, et al. Session-based recommendation with graph neural networks[C] //Proceedings of the AAAI Conference on Artificial Intelligence. Honolulu, USA: AAAI, 2019, 33(1): 346-353.
[13] XU C, ZHAO P, LIU Y, et al. Graph contextualized self-attention network for session-based recommendation[C] //Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence. Macao, China:Morgan Kaufmann, 2019, 19: 3940-3946.
[14] ZHANG T, ZHAO P, LIU Y, et al. Feature-level deeper self-attention network for sequential recommendation[C] //Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence. Macao, China: Morgan Kaufmann, 2019: 4320-4326.
[15] ZHOU K, WANG H, ZHAO W X, et al. S3-rec: self-supervised learning for sequential recommendation with mutual information maximization[C] //Proceedings of the 29th ACM International Conference on Information & Knowledge Management. Virtual Event, Ireland: ACM, 2020: 1893-1902.
[16] XIE X, SUN F, LIU Z, et al. Contrastive learning for sequential recommendation[C] //2022 IEEE 38th International Conference on Data Engineering(ICDE). Kuala Lumpur, Malaysia: IEEE, 2022: 1259-1273.
[17] LIU Z, CHEN Y, LI J, et al. Contrastive self-supervised sequential recommendation with robust augmentation[J/OL].(2021-08-14)[2022-11-11]. https://arxiv.org/abs/2108.06479.
[18] QIU R, HUANG Z, YIN H, et al. Contrastive learning for representation degeneration problem in sequential recommendation[C] //Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining. Tempe, USA: ACM, 2022: 813-823.
[19] SRIVASTAVA N, HINTON G, KRIZHEVSKY A, et al. Dropout: a simple way to prevent neural networks from overfitting[J]. The Journal of Machine Learning Research, 2014, 15(1): 1929-1958.
[20] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C] //Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems. Long Beach, USA: MIT, 2017: 5998-6008.
[21] HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C] //Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE, 2016: 770-778.
[22] XIONG R, YANG Y, HE D, et al. On layer normalization in the transformer architecture[C] // Proceedings of the 37th International Conference on Machine Learning. Vienna, Austria: ACM, 2020: 10524-10533.
[23] MCAULEY J, TARGETT C, SHI Q, et al. Image-based recommendations on styles and substitutes[C] //Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval. Santiago, Chile: ACM, 2015: 43-52.
[24] HARPER F M, KONSTAN J A. The movielens datasets: history and context[J]. Acm Transactions on Interactive Intelligent Systems, 2015, 5(4): 1-19.
[25] KRICHENE W, RENDLE S. On sampled metrics for item recommendation[J]. Communications of the ACM, 2022, 65(7): 75-83.
[26] ZHAO W X, MU S, HOU Y, et al. Recbole: towards a unified, comprehensive and efficient framework for recommendation algorithms[C] //Proceedings of the 30th ACM International Conference on Information & Knowledge Management. Queensland, Australia: ACM, 2021: 4653-4664.
[27] PASZKE A, GROSS S, MASSA F, et al. Pytorch: an imperative style, high-performance deep learning library[J]. Advances in Neural Information Processing Systems, 2019: 8024-8035.
[28] KINGMA D P, BA J. Adam: a method for stochastic optimization[C] //The 3rd International Conference for Learning Representations. San Diego, USA: [s.n.] , 2015:1-15.
[1] Pengcheng ZHAO, Fuquan ZHANG, Xubing YANG, Yin WU. Optimal deployment strategy of forest fire monitoring nodes based on visualization [J]. Journal of Shandong University(Engineering Science), 2019, 49(1): 30-35.
[2] Zhenbing LIU, Xusheng FANG, Huihua YANG, Rushi LAN. The diagnosis of Alzheimer's disease classification based on multi-scale residual neutral network [J]. Journal of Shandong University(Engineering Science), 2018, 48(6): 1-7.
[3] CHEN Jiajie, WANG Jinfeng. Method for solving Choquet integral model based on ant colony algorithm [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2018, 48(3): 81-87.
[4] DAI Hongwei, YANG Yu, ZHONG Zhaoman, LI Cunhua. Improved quantum crossover immune clonal algorithm and its application [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2015, 45(2): 17-21.
[5] GUO Hui-ling, WANG Shi-tong*, YAN Xiao-bo. A novel method for face recognition based on generalized rotation invariant kernel [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2012, 42(5): 71-79.
[6] ZHOU Xiao-lin,ZENG Guang-zhou . P2P-based workflow management system [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2007, 37(5): 89-94 .
[7] WU Wan-qing, ZHOU Guo-long, WANG Qiao, ZHAO Yong-xin. Research of balanced Boolean functions with high nonlinearity based on GSA [J]. Journal of Shandong University(Engineering Science), 2022, 57(5): 74-84.
[8] HUANG Caiyun, CHEN Dewu, HE Jifu, HU Yi, WANG Nan, CHEN Pei. Detection of upper limb musculoskeletal abnormality based on improved dual path network [J]. Journal of Shandong University(Engineering Science), 2022, 52(3): 25-33.
[9] QIAN Cheng, ZHAO Gansen, LUO Haoyu. Dynamic environment oriented D2D collaborative offloading for workflow applications in mobile edge computing [J]. Journal of Shandong University(Engineering Science), 2022, 52(4): 45-53.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!