Abstract: The existing multimodal recommenders had three main problems: the potential relevance between multimodal data and interaction data had not been fully explored, leading to the weakening of key features; the accidentally caused noise unrelated to user interests was ignored; the static multimodal fusion method provided the same weight to each modality and could not dynamically perceive the change of user interests, resulting in insufficient discrimination of the learned representations. Therefore, a user and item two-sided collaborative filtering multimodal contrastive representation enhancement(TCFCRE)recommender was proposed. To address the shortcomings in combining multimodal and interaction data, TCFCRE used contrastive learning to enhance key features and mine the potential associations. Meanwhile, to reduce the impact of noise, a cross-modal user representation alignment module was designed to discover the consistency of user features and extract users' true interests. A mask matrix based on the user-item multimodal relationship was also constructed to generate an augmented view, and contrastive learning was adopted to reduce the noise impact in implicit feedback. To alleviate the problem that traditional methods ignored the importance of modalities and could not adapt to dynamic changes, a multimodal dynamic fusion module that calculated fusion weights for each representation was designed. Experiments on three public datasets demonstrated that TCFCRE had achieved significant improvements over existing solutions.

Key words: multimodal recommendation, graph neural network, representation learning, contrastive learning, representation enhancement