深度卷积神经网络嵌套fine-tune的图像美感品质评价

doi:10.6040/j.issn.1672-3961.0.2017.421

山东大学学报(工学版) ›› 2018, Vol. 48 ›› Issue (3): 60-66.doi: 10.6040/j.issn.1672-3961.0.2017.421

深度卷积神经网络嵌套fine-tune的图像美感品质评价

李雨鑫,普园媛^*,徐丹,钱文华,刘和娟

云南大学信息学院, 云南昆明 650504

收稿日期:2017-05-05 出版日期:2018-06-20 发布日期:2017-05-05
通讯作者: 普园媛(1972— ), 女, 云南晋宁人, 教授, 博士, 主要研究方向为数字图像处理, 非真实感绘制, 视觉艺术科学理解等方面的研究. E-mail:km_pyy@126.com E-mail:837506201@qq.com
作者简介:李雨鑫(1989— ), 男, 山东烟台人, 硕士研究生, 主要研究方向为数字图像处理, 深度学习, 视觉美感评价.E-mail:837506201@qq.com
基金资助:
国家自然科学基金资助项目(61163019,61271361,61462093,61761046);云南省科技厅资助项目(2014FA021,2014FB113)

Image aesthetic quality evaluation based on embedded fine-tune deep CNN

LI Yuxin, PU Yuanyuan^*, XU Dan, QIAN Wenhua, LIU Hejuan

School of Information Science and Engineering, Yunnan University, Kunming 650504, Yunnan, China

Received:2017-05-05 Online:2018-06-20 Published:2017-05-05

摘要/Abstract

摘要： 针对使用卷积神经网络对图像美感品质研究中图像数据库过小的问题,使用fine-tune的迁移学习方法,分析卷积神经网络结构和图像内容对图像美感品质评价的影响。在按图像内容进行美感品质评价研究时,针对图像数据再次减小的问题,提出连续两次fine-tune的嵌套fine-tune方法,并在数据库Photo Quality上进行试验。试验结果表明,嵌套fine-tune方法得到的美感品质评价正确率比传统提取人工设计特征方法平均高出5.36%,比两种深度学习方法分别平均高出3.35%和2.33%,有效解决了卷积神经网络在图像美感品质研究中因图像数据库过小而带来的训练问题。

关键词: 图像美感品质评价, CNN, 迁移学习, 嵌套fine-tune, 图像内容

Abstract: The image database was not big enough for using convolutional neural networks to research the image aesthetic quality. Aiming at this problem, a fine-tune transfer learning method was used to analyze the effect of convolutional neural networks architecture and image contents on image aesthetic quality evaluation. During the research of image aesthetic quality evaluation by image contents, the problem of image data decrease rose again. The embedded fine-tune method using fine-tune twice continuously was proposed to solve the problem. The experiments were performed on Photo Quality, a small image database, and got a good effect. The results indicated that the accuracy of image aesthetic quality evaluation by embedded fine-tune was an average of 5.36% higher than by traditional artificially designed feature extraction method, 3.35% and 2.33% higher than by the other two deep learning methods respectively. The embedded fine-tune deep convolutional neural networks solved the problem of small database in image aesthetic quality evaluation research effectively and accurately.

Key words: image aesthetic quality evaluation, convolutional neural networks, transfer learning, embedded fine-tune, image contents

中图分类号:

TP391

李雨鑫,普园媛,徐丹,钱文华,刘和娟. 深度卷积神经网络嵌套fine-tune的图像美感品质评价[J]. 山东大学学报(工学版), 2018, 48(3): 60-66.

LI Yuxin, PU Yuanyuan, XU Dan, QIAN Wenhua, LIU Hejuan. Image aesthetic quality evaluation based on embedded fine-tune deep CNN[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2018, 48(3): 60-66.

参考文献

[1] DATTA R, JOSHI D, LI J, et al. Studying aesthetics in photographic images using a computational approach[C] //European Conference on Computer Vision. Graz, Austria: Springer-Verlag, 2006: 288-301.
[2] LUO W, WANG X G, TANG X OU. Content-based photo quality assessment [J]. IEEE Transaction on Multimedia, 2013, 15(8): 1930-1943.
[3] SHAO J, ZHOU Y. Photo quality assessment in different categories [J]. Journal of Computational Information Systems, 2013, 9(8): 3209-3217.
[4] DHAR S, ORDONEZ V, BERG T L. High level describable attributes for predicting aesthetics and interestingness[C] // Computer Vision and Pattern Recognition. Colorado, USA: IEEE Computer Society, 2011:1657-1664.
[5] OBRADOR P, SCHMIDT-HACKENBERG L, OLIVER N. The role of image composition in image aesthetics[C] //IEEE International Conference on Image Processing. Hong Kong, China: IEEE Xplore, 2010:3185-3188.
[6] WANG C H, PU Y Y, XU D, et al. Evaluating aesthetics quality in scenery photo[C] //Proceeding of National Conference on Multimedia Technology. Beijing, China: Machinery Industry Press, 2015:141-149.
[7] WANG C, PU Y, XU D. Evaluating aesthetics qualities in portrait images[J]. Journal of Software, 2015, 26(S2): 20-28.
[8] LECUN Y, BOSER B, DENKER J, et al. Back propagation applied to handwritten zip code recognition[J]. Neural Computation, 2014, 1(4):541-551.
[9] HINTON G E, OSINDERO S, TEH Y W. A fast learning algorithm for deep belief nets[J]. Neural Computation, 2006, 18(7):1527-1544.
[10] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[C] //International Conference on Neural Information Processing Systems. Nevada, USA:Curran Associates Inc, 2012:1097-1105.
[11] GUO L, LI F, LIEW W C. Image Aesthetic Evaluation Using Parallel Deep Convolution Neural Network[C] //International Conference on Digital Image Computing: Techniques and Applications. Gold Coast, Australia: IEEE Computer Society, 2016:1-5.
[12] LU X, LIN Z, JIN H, et al. Rapid: rating pictorial aesthetics using deep learning[C] //Proceedings of the ACM International Conference on Multimedia. Orlando, USA: ACM, 2014:457-466.
[13] WANG L, OUYANG W, WANG X, et al. Visual tracking with fully convolutional networks[C] //IEEE International Conference on Computer Vision. Santiago, Chile: IEEE Computer Society, 2015:3119-3127.
[14] GIRSHICK R, DONAHUE J, DARRELL T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C] //IEEE Conference on Computer Vision and Pattern Recognition. Columbus, USA: IEEE Computer Society, 2014:580-587.
[15] SUN Y, WANG X, TANG X. Deep learning face representation from predicting 10 000 classes[C] //IEEE Conference on Computer Vision and Pattern Recognition. Hong Kong, China: IEEE, 2014:1891-1898.
[16] KARAYEV S, TRENTACOSTE M, HAN H, et al. Recognizing image style[EB/OL].(2014-7-23)[2018-3-22]. https://arxiv.org/abs/1311.3715.
[17] CHATFIELD K, SIMONYAN K, VEDALDI A, et al. Return of the devil in the details: delving deep into convolutional nets[C] //Proceeding of British Machine Vision Conference. Nottingham, UK: BMVA Press, 2014.
[18] JIA Y, SHELHAMER E, DONAHUE J, et al. Caffe: convolutional architecture for fast feature embedding[EB/OL].(2014-6-20)[2018-3-22]. https://arxiv.org/abs/1408.5093

多维度评价

Viewed

Full text

Abstract

Cited

Shared

Discussed

深度卷积神经网络嵌套fine-tune的图像美感品质评价

Image aesthetic quality evaluation based on embedded fine-tune deep CNN

PDF (PC)

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 3

多维度评价

本文评价

推荐阅读 0

[1]	杨秀林1,黄硕2*,邓苗1,张基宏1,3. 基于显著计算与自适应PCNN的图像融合方法[J]. 山东大学学报(工学版), 2014, 44(2): 35-42.
[2]	于立萍1,2,唐焕玲1,2. 基于分类一致性的迁移学习及其在行人检测中的应用[J]. 山东大学学报(工学版), 2013, 43(4): 26-31.
[3]	郑君君1,夏胜平1,李新光1,祝一薇1,刘建军1,谭立球1,2. 基于RSOM树的图像K近邻求解算法[J]. 山东大学学报(工学版), 2011, 41(2): 80-84.