基于单目RGB摄像机的三维人脸表情跟踪算法

doi:10.6040/j.issn.1672-3961.0.2016.466

Abstract

Abstract: There were two key problems in tracking 3D facial expression using a monocular RGB camera. One was how to reconstruct 3D facial model from a single image, and the other was how to establish the dense correspondences between the different 3D models. To solve the above problems, an effective 3D facial expression tracking algorithm was proposed. The feature points were automatically detected on each input facial image frame, and the 3D models of different expressions were reconstructed by Multilinear Model according to 2D feature points. By re-projecting the 3D models into the image space, the Optic Flow and Sift Flow correspondences between images were calculated. According to the correspondences of the images, the 3D models were rectified by Laplacian mesh deformation, and the dense 3D correspondences between different expressions were established. The experimental results showed that the proposed method could create more realistic 3D expression models, and could accurately track the continuous change of the 3D expression sequence.

Key words: 3D facial expressions tracking, Multilinear Model, Laplacian mesh deformation

CLC Number:

TP391.9

HUANG Yanhui, FAN Yangyu, SU Xuhui. 3D facial expression tracking using a monocular RGB camera[J].JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2017, 47(4): 7-13.

References

[1] ZHANG Li, SNAVELY N, CURLESS B, et al. Spacetime faces: high resolution capture for modeling and animation[J]. ACM Transactions on Graphics, 2004, 23(3): 548-558.
[2] HUANG Haoda, CHAI Jinxiang, TONG Xin, et al. Leveraging motion capture and 3D scanning for high-fidelity facial performance acquisition[J]. ACM Transactions on Graphics, 2011, 30(4): 74.
[3] BEELER T, BICKEL B, BEARDSLEY P, et al. High-quality single-shot capture of facial geometry[J]. ACM Transactions on Graphics, 2010, 29(4): 157-166.
[4] BEELER T, HAHN F, BRADLEY D, et al. High-quality passive facial performance capture using anchor frames[J]. ACM Transactions on Graphics, 2011, 30(4): 75.
[5] BRADLEY D, HEIDRICH W, POPA T, et al. High resolution passive facial performance capture[J]. ACM Transactions on Graphics, 2010, 29(4): 41.
[6] 迟静, 屠长河, 陈雪. 基于法向保持的三维人脸表情跟踪[J]. 计算机辅助设计与图形学学报, 2011, 23(6): 1069-1077. CHI Jing, TU Changhe, CHEN Xue. 3D facial expression tracking based on normal preserving[J]. Journal of Computer-Aided Design & Computer Graphics, 2011, 23(6): 1069-1077.
[7] CHEN Yenlin, WU Hsiangtao, SHI Fuhao, et al. Accurate and robust 3D facial capture using a single RGBD camera[C] //Proceedings of the 2013 IEEE International Conference on Computer Vision. Sydney, Australia: IEEE, 2013: 3615-3622.
[8] LI Hao, YU Jihun, YE Yuting, et al. Realtime facial animation with on-the-fly correctives[J]. ACM Transactions on Graphics, 2013, 32(4): 42.
[9] WEISE T, BOUAZIZ S, LI Hao, et al. Realtime performance-based facial animation[J]. ACM Transactions on Graphics, 2011, 30(4): 77.
[10] 何钦政, 王运巧. 基于Kinect的人脸表情捕捉及动画模拟系统研究[J]. 图学学报, 2016, 37(3): 290-295. HE Qinzheng, WANG Yunqiao. Research on system of facial expression capture and animation simulation based on Kinect[J]. Journal of Graphics, 2016, 37(3): 290-295.
[11] 李俊龙, 章登义, 黄珺. Kinect驱动的人脸动画合成技术研究[J]. 计算机工程, 2015, 41(3):237-241. LI Junlong, ZHANG Dengyi, HUANG Jun. Research on synthesis technology of facial animation driven by Kinect[J]. Computer Engineering, 2015, 41(3): 237-241.
[12] BLANZ V, SCHERBAUM K, VETTER T, et al. Exchanging faces in images[J].Computer Graphics Forum, 2004, 23(3): 669-676.
[13] VLASIC D, BRAND M, PFISTER H, et al. Face transfer with multilinear models[J]. ACM Transactions on Graphics, 2005, 24(3): 426-433.
[14] CAO Chen, WENG Yanlin, LIN Stephen, et al. 3D shape regression for real-time facial animation[J]. ACM Transactions on Graphics, 2013, 32(4): 41.
[15] CAO Chen, WENG Yanlin, ZHOU Shun, et al. Face-warehouse: a 3d facial expression database for visual computing[J]. IEEE Transactions on Visualization and Computer Graphics, 2014, 20(3): 413-425.
[16] SHI Fuhao, WU Hsiangtao, TONG Xin, et al. Automatic acquisition of high-fidelity facial performances using monocular videos[J]. ACM Transactions on Graphics, 2014, 33(6): 222.
[17] SUPASOM S, IRA K, SEITZ M S. Total moving face reconstruction[M] //Lecture Notes in Computer Science(including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). [S.l.] : Springer International Publishing,2014,8692(4): 796-812.
[18] FYFFE G, JONES A, ALEXANDER O, et al. Driving high-resolution facial scans with video performance capture[J]. ACM Transactions on Graphics, 2014, 34(1): 8.
[19] SIBBING D, HABBECKE M, KOBBELT L. Markerless reconstruction and synthesis of dynamic facial expressions[J]. Computer Vision and Image Understanding, 2011, 115(5): 668-680.
[20] REN Shaoqing, CAO Xuding, WEI Yichen, et al. Face alignment at 3000 fps via regressing local binary features[C] //Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus, USA: IEEE, 2014: 1685-1692.
[21] 栾勐. 基于拉普拉斯变形技术的形状分析方法研究[D].杭州: 浙江大学, 2011. LUAN Meng. Shape analysis based on Laplacian deformation[D]. Hangzhou: Zhejiang University, 2011.
[22] SORKINE O, COHEN-OR D, LIPMAN Y, et al. Laplacian surface editing[C] //Proceedings of the 2004 Eurographics/ACM SIGGRAPH Symposium on Geometry processing. [S.l.] :ACM, 2004, 71:175-184.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

Comments

Recommended 0

No Suggested Reading articles found!

3D facial expression tracking using a monocular RGB camera

PDF (PC)

Abstract

Cite this article

share this article

References

Related Articles 6

Metrics

Comments

Recommended 0

[1]	SHI Wen-Hua, LIU Wei-Dong, SUN Yong-Fu. Research of 1／3 dam breach simulation and personnel evacuation scenario based on digital elevation model DEM in a quake lake [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(5): 144-148.
[2]	LI Fangjia, GAO Shangce, TANG Zheng*, Ishii Masahiro, Yamashita Kazuya. 3D similar pattern generation of snow crystals with cellular automata [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(1): 102-105.
[3]	CHEN Cheng-jun,ZHOU Yi-qi,YANG Hong-juan . Study on an approach of transformation and representation based on the SolidWorks model to the virtual assembly model [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2008, 38(1): 61-65 .
[4]	SONG Qing,LI Xiao-lei,ZHANG Cheng-jin . Optimization of a postal express mail network based on bottleneck analysis [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2007, 37(5): 29-33 .
[5]	ZHAO Wei-hua,WANG Yong,WANG Xian-lun . Space continuous path kinematic simulation of a robot based on VRML [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2007, 37(2): 8-11 .
[6]	KONG Xiang-zhen,LIU Yan-jun,WANG Yong,ZHAO Xiu-hua . Compensation and simulation for the deadband of the pneumatic proportional valve [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2006, 36(1): 99-102 .