深度卷积神经网络嵌套fine-tune的图像美感品质评价

doi:10.6040/j.issn.1672-3961.0.2017.421

Abstract

Abstract: The image database was not big enough for using convolutional neural networks to research the image aesthetic quality. Aiming at this problem, a fine-tune transfer learning method was used to analyze the effect of convolutional neural networks architecture and image contents on image aesthetic quality evaluation. During the research of image aesthetic quality evaluation by image contents, the problem of image data decrease rose again. The embedded fine-tune method using fine-tune twice continuously was proposed to solve the problem. The experiments were performed on Photo Quality, a small image database, and got a good effect. The results indicated that the accuracy of image aesthetic quality evaluation by embedded fine-tune was an average of 5.36% higher than by traditional artificially designed feature extraction method, 3.35% and 2.33% higher than by the other two deep learning methods respectively. The embedded fine-tune deep convolutional neural networks solved the problem of small database in image aesthetic quality evaluation research effectively and accurately.

Key words: image aesthetic quality evaluation, convolutional neural networks, transfer learning, embedded fine-tune, image contents

CLC Number:

TP391

LI Yuxin, PU Yuanyuan, XU Dan, QIAN Wenhua, LIU Hejuan. Image aesthetic quality evaluation based on embedded fine-tune deep CNN[J].JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2018, 48(3): 60-66.

References

[1] DATTA R, JOSHI D, LI J, et al. Studying aesthetics in photographic images using a computational approach[C] //European Conference on Computer Vision. Graz, Austria: Springer-Verlag, 2006: 288-301.
[2] LUO W, WANG X G, TANG X OU. Content-based photo quality assessment [J]. IEEE Transaction on Multimedia, 2013, 15(8): 1930-1943.
[3] SHAO J, ZHOU Y. Photo quality assessment in different categories [J]. Journal of Computational Information Systems, 2013, 9(8): 3209-3217.
[4] DHAR S, ORDONEZ V, BERG T L. High level describable attributes for predicting aesthetics and interestingness[C] // Computer Vision and Pattern Recognition. Colorado, USA: IEEE Computer Society, 2011:1657-1664.
[5] OBRADOR P, SCHMIDT-HACKENBERG L, OLIVER N. The role of image composition in image aesthetics[C] //IEEE International Conference on Image Processing. Hong Kong, China: IEEE Xplore, 2010:3185-3188.
[6] WANG C H, PU Y Y, XU D, et al. Evaluating aesthetics quality in scenery photo[C] //Proceeding of National Conference on Multimedia Technology. Beijing, China: Machinery Industry Press, 2015:141-149.
[7] WANG C, PU Y, XU D. Evaluating aesthetics qualities in portrait images[J]. Journal of Software, 2015, 26(S2): 20-28.
[8] LECUN Y, BOSER B, DENKER J, et al. Back propagation applied to handwritten zip code recognition[J]. Neural Computation, 2014, 1(4):541-551.
[9] HINTON G E, OSINDERO S, TEH Y W. A fast learning algorithm for deep belief nets[J]. Neural Computation, 2006, 18(7):1527-1544.
[10] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[C] //International Conference on Neural Information Processing Systems. Nevada, USA:Curran Associates Inc, 2012:1097-1105.
[11] GUO L, LI F, LIEW W C. Image Aesthetic Evaluation Using Parallel Deep Convolution Neural Network[C] //International Conference on Digital Image Computing: Techniques and Applications. Gold Coast, Australia: IEEE Computer Society, 2016:1-5.
[12] LU X, LIN Z, JIN H, et al. Rapid: rating pictorial aesthetics using deep learning[C] //Proceedings of the ACM International Conference on Multimedia. Orlando, USA: ACM, 2014:457-466.
[13] WANG L, OUYANG W, WANG X, et al. Visual tracking with fully convolutional networks[C] //IEEE International Conference on Computer Vision. Santiago, Chile: IEEE Computer Society, 2015:3119-3127.
[14] GIRSHICK R, DONAHUE J, DARRELL T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C] //IEEE Conference on Computer Vision and Pattern Recognition. Columbus, USA: IEEE Computer Society, 2014:580-587.
[15] SUN Y, WANG X, TANG X. Deep learning face representation from predicting 10 000 classes[C] //IEEE Conference on Computer Vision and Pattern Recognition. Hong Kong, China: IEEE, 2014:1891-1898.
[16] KARAYEV S, TRENTACOSTE M, HAN H, et al. Recognizing image style[EB/OL].(2014-7-23)[2018-3-22]. https://arxiv.org/abs/1311.3715.
[17] CHATFIELD K, SIMONYAN K, VEDALDI A, et al. Return of the devil in the details: delving deep into convolutional nets[C] //Proceeding of British Machine Vision Conference. Nottingham, UK: BMVA Press, 2014.
[18] JIA Y, SHELHAMER E, DONAHUE J, et al. Caffe: convolutional architecture for fast feature embedding[EB/OL].(2014-6-20)[2018-3-22]. https://arxiv.org/abs/1408.5093

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

Comments

Recommended 10

[1]	WANG Su-yu,<\sup>,AI Xing<\sup>,ZHAO Jun<\sup>,LI Zuo-li<\sup>,LIU Zeng-wen<\sup> . Milling force prediction model for highspeed end milling 3Cr2Mo steel[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2006, 36(1): 1 -5 .
[2]	LI Kan . Empolder and implement of the embedded weld control system[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2008, 38(4): 37 -41 .
[3]	KONG Xiang-zhen,LIU Yan-jun,WANG Yong,ZHAO Xiu-hua . Compensation and simulation for the deadband of the pneumatic proportional valve[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2006, 36(1): 99 -102 .
[4]	CHEN Rui, LI Hongwei, TIAN Jing. The relationship between the number of magnetic poles and the bearing capacity of radial magnetic bearing[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2018, 48(2): 81 -85 .
[5]	LI Ke,LIU Chang-chun,LI Tong-lei . Medical registration approach using improved maximization of mutual information[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2006, 36(2): 107 -110 .
[6]	JI Tao,GAO Xu/sup>,SUN Tong-jing,XUE Yong-duan/sup>,XU Bing-yin/sup> . Characteristic analysis of fault generated traveling waves in 10 Kv automatic blocking and continuous power transmission lines[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2006, 36(2): 111 -116 .
[7]	. [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(1): 27 -32 .
[8]	QIN Tong, SUN Fengrong*, WANG Limei, WANG Qinghao, LI Xincai. 3D surface reconstruction using the shape based interpolation guided by maximal discs[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2010, 40(3): 1 -5 .
[9]	WANG Li-ju,HUANG Qi-cheng,WANG Zhao-xu . [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2006, 36(6): 51 -56 .
[10]	SUN Dianzhu, ZHU Changzhi, LI Yanrui. [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(1): 84 -86 .

Image aesthetic quality evaluation based on embedded fine-tune deep CNN

PDF (PC)

Abstract

Cite this article

share this article

References

Related Articles 5

Metrics

Comments

Recommended 10

[1]	HUO Bingqiang, ZHOU Tao, LU Huiling, DONG Yali, LIU Shan. Lung tumor benign-malignant classification based on multi-modal residual neural network and NRC algorithm [J]. Journal of Shandong University(Engineering Science), 2020, 50(6): 59-67.
[2]	Jun QIN,Yuanpeng ZHANG,Yizhang JIANG,Wenlong HANG. Transfer fuzzy clustering based on self-constraint of multiple medoids [J]. Journal of Shandong University(Engineering Science), 2019, 49(2): 107-115.
[3]	Hongbin ZHANG,Diedie QIU,Renzhong WU,Tao ZHU,Jin HUA,Donghong JI. Image attribute annotation based on extreme gradient boosting algorithm [J]. Journal of Shandong University(Engineering Science), 2019, 49(2): 8-16.
[4]	YU Li-ping1,2, TANG Huan-ling1,2. Transfer learning model based on classification consensus and its application in pedestrian detection [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2013, 43(4): 26-31.
[5]	XU Shan-shan, LIU Ying-an*, XU Sheng. Wood defects recognition based on the convolutional neural network [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2013, 43(2): 23-28.