JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE) ›› 2018, Vol. 48 ›› Issue (3): 60-66.doi: 10.6040/j.issn.1672-3961.0.2017.421

Previous Articles     Next Articles

Image aesthetic quality evaluation based on embedded fine-tune deep CNN

LI Yuxin, PU Yuanyuan*, XU Dan, QIAN Wenhua, LIU Hejuan   

  1. School of Information Science and Engineering, Yunnan University, Kunming 650504, Yunnan, China
  • Received:2017-05-05 Online:2018-06-20 Published:2017-05-05

PDF (PC)

336

Abstract: The image database was not big enough for using convolutional neural networks to research the image aesthetic quality. Aiming at this problem, a fine-tune transfer learning method was used to analyze the effect of convolutional neural networks architecture and image contents on image aesthetic quality evaluation. During the research of image aesthetic quality evaluation by image contents, the problem of image data decrease rose again. The embedded fine-tune method using fine-tune twice continuously was proposed to solve the problem. The experiments were performed on Photo Quality, a small image database, and got a good effect. The results indicated that the accuracy of image aesthetic quality evaluation by embedded fine-tune was an average of 5.36% higher than by traditional artificially designed feature extraction method, 3.35% and 2.33% higher than by the other two deep learning methods respectively. The embedded fine-tune deep convolutional neural networks solved the problem of small database in image aesthetic quality evaluation research effectively and accurately.

Key words: image aesthetic quality evaluation, convolutional neural networks, transfer learning, embedded fine-tune, image contents

CLC Number: 

  • TP391
[1] DATTA R, JOSHI D, LI J, et al. Studying aesthetics in photographic images using a computational approach[C] //European Conference on Computer Vision. Graz, Austria: Springer-Verlag, 2006: 288-301.
[2] LUO W, WANG X G, TANG X OU. Content-based photo quality assessment [J]. IEEE Transaction on Multimedia, 2013, 15(8): 1930-1943.
[3] SHAO J, ZHOU Y. Photo quality assessment in different categories [J]. Journal of Computational Information Systems, 2013, 9(8): 3209-3217.
[4] DHAR S, ORDONEZ V, BERG T L. High level describable attributes for predicting aesthetics and interestingness[C] // Computer Vision and Pattern Recognition. Colorado, USA: IEEE Computer Society, 2011:1657-1664.
[5] OBRADOR P, SCHMIDT-HACKENBERG L, OLIVER N. The role of image composition in image aesthetics[C] //IEEE International Conference on Image Processing. Hong Kong, China: IEEE Xplore, 2010:3185-3188.
[6] WANG C H, PU Y Y, XU D, et al. Evaluating aesthetics quality in scenery photo[C] //Proceeding of National Conference on Multimedia Technology. Beijing, China: Machinery Industry Press, 2015:141-149.
[7] WANG C, PU Y, XU D. Evaluating aesthetics qualities in portrait images[J]. Journal of Software, 2015, 26(S2): 20-28.
[8] LECUN Y, BOSER B, DENKER J, et al. Back propagation applied to handwritten zip code recognition[J]. Neural Computation, 2014, 1(4):541-551.
[9] HINTON G E, OSINDERO S, TEH Y W. A fast learning algorithm for deep belief nets[J]. Neural Computation, 2006, 18(7):1527-1544.
[10] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[C] //International Conference on Neural Information Processing Systems. Nevada, USA:Curran Associates Inc, 2012:1097-1105.
[11] GUO L, LI F, LIEW W C. Image Aesthetic Evaluation Using Parallel Deep Convolution Neural Network[C] //International Conference on Digital Image Computing: Techniques and Applications. Gold Coast, Australia: IEEE Computer Society, 2016:1-5.
[12] LU X, LIN Z, JIN H, et al. Rapid: rating pictorial aesthetics using deep learning[C] //Proceedings of the ACM International Conference on Multimedia. Orlando, USA: ACM, 2014:457-466.
[13] WANG L, OUYANG W, WANG X, et al. Visual tracking with fully convolutional networks[C] //IEEE International Conference on Computer Vision. Santiago, Chile: IEEE Computer Society, 2015:3119-3127.
[14] GIRSHICK R, DONAHUE J, DARRELL T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C] //IEEE Conference on Computer Vision and Pattern Recognition. Columbus, USA: IEEE Computer Society, 2014:580-587.
[15] SUN Y, WANG X, TANG X. Deep learning face representation from predicting 10 000 classes[C] //IEEE Conference on Computer Vision and Pattern Recognition. Hong Kong, China: IEEE, 2014:1891-1898.
[16] KARAYEV S, TRENTACOSTE M, HAN H, et al. Recognizing image style[EB/OL].(2014-7-23)[2018-3-22]. https://arxiv.org/abs/1311.3715.
[17] CHATFIELD K, SIMONYAN K, VEDALDI A, et al. Return of the devil in the details: delving deep into convolutional nets[C] //Proceeding of British Machine Vision Conference. Nottingham, UK: BMVA Press, 2014.
[18] JIA Y, SHELHAMER E, DONAHUE J, et al. Caffe: convolutional architecture for fast feature embedding[EB/OL].(2014-6-20)[2018-3-22]. https://arxiv.org/abs/1408.5093
[1] YU Li-ping1,2, TANG Huan-ling1,2. Transfer learning model based on classification consensus and  its application in pedestrian detection [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2013, 43(4): 26-31.
[2] XU Shan-shan, LIU Ying-an*, XU Sheng. Wood defects recognition based on the convolutional neural network [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2013, 43(2): 23-28.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
Copyright 2018 © Editorial office of Journal of Shandong University (Engineering Science)
Supported by Beijing Magtech Co.Ltd