基于单目RGB摄像机的三维人脸表情跟踪算法

doi:10.6040/j.issn.1672-3961.0.2016.466

山东大学学报(工学版) ›› 2017, Vol. 47 ›› Issue (4): 7-13.doi: 10.6040/j.issn.1672-3961.0.2016.466

基于单目RGB摄像机的三维人脸表情跟踪算法

黄炎辉,樊养余^*,苏旭辉

西北工业大学电子信息学院, 陕西西安 710129

收稿日期:2016-12-14 出版日期:2017-08-20 发布日期:2016-12-14
通讯作者: 樊养余(1960— ), 男,陕西西安人,教授, 博导,主要研究方向为虚拟现实,图像处理.E-mail:fan_yangyu@nwpu.edu.cn E-mail:yanhuihuang_nwpu@sina.com
作者简介:黄炎辉(1985— ), 男,山东泰安人,博士研究生, 主要研究方向为人脸建模,计算机视觉,图像处理.E-mail:yanhuihuang_nwpu@sina.com
基金资助:
陕西省科技统筹创新工程重点实验室资助项目(2013SZS15-K02);陕西省自然科学基金资助项目(2015JM6317)

3D facial expression tracking using a monocular RGB camera

HUANG Yanhui, FAN Yangyu^*, SU Xuhui

School of Electronics and Information, Northwestern Polytechnical University, Xi'an 710129, Shaanxi, China

Received:2016-12-14 Online:2017-08-20 Published:2016-12-14

摘要/Abstract

摘要： 利用单目摄像机对人脸表情进行三维跟踪有两个关键问题,一是从单幅图像中重建人脸的三维模型,二是建立不同三维模型之间稠密的对应关系。针对上述问题,提出一种有效的三维人脸表情跟踪算法。对输入的各帧人脸图像进行自动特征点检测,并根据2D特征点,利用Multilinear Model重建不同表情的三维模型。通过将三维模型重投影至图像空间,计算各帧图像间的Optic Flow与Sift Flow对应。根据图像间的对应关系,利用Laplacian网格变形对三维模型进行修正,建立不同表情之间三维的稠密对应。试验表明,该方法可以重建出较为真实的三维人脸表情模型,同时可以精确跟踪连续变化的三维表情序列。

关键词: 三维人脸表情跟踪, Laplacian网格变形, Multilinear Model

Abstract: There were two key problems in tracking 3D facial expression using a monocular RGB camera. One was how to reconstruct 3D facial model from a single image, and the other was how to establish the dense correspondences between the different 3D models. To solve the above problems, an effective 3D facial expression tracking algorithm was proposed. The feature points were automatically detected on each input facial image frame, and the 3D models of different expressions were reconstructed by Multilinear Model according to 2D feature points. By re-projecting the 3D models into the image space, the Optic Flow and Sift Flow correspondences between images were calculated. According to the correspondences of the images, the 3D models were rectified by Laplacian mesh deformation, and the dense 3D correspondences between different expressions were established. The experimental results showed that the proposed method could create more realistic 3D expression models, and could accurately track the continuous change of the 3D expression sequence.

Key words: 3D facial expressions tracking, Multilinear Model, Laplacian mesh deformation

中图分类号:

TP391.9

黄炎辉,樊养余,苏旭辉. 基于单目RGB摄像机的三维人脸表情跟踪算法[J]. 山东大学学报(工学版), 2017, 47(4): 7-13.

HUANG Yanhui, FAN Yangyu, SU Xuhui. 3D facial expression tracking using a monocular RGB camera[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2017, 47(4): 7-13.

参考文献

[1] ZHANG Li, SNAVELY N, CURLESS B, et al. Spacetime faces: high resolution capture for modeling and animation[J]. ACM Transactions on Graphics, 2004, 23(3): 548-558.
[2] HUANG Haoda, CHAI Jinxiang, TONG Xin, et al. Leveraging motion capture and 3D scanning for high-fidelity facial performance acquisition[J]. ACM Transactions on Graphics, 2011, 30(4): 74.
[3] BEELER T, BICKEL B, BEARDSLEY P, et al. High-quality single-shot capture of facial geometry[J]. ACM Transactions on Graphics, 2010, 29(4): 157-166.
[4] BEELER T, HAHN F, BRADLEY D, et al. High-quality passive facial performance capture using anchor frames[J]. ACM Transactions on Graphics, 2011, 30(4): 75.
[5] BRADLEY D, HEIDRICH W, POPA T, et al. High resolution passive facial performance capture[J]. ACM Transactions on Graphics, 2010, 29(4): 41.
[6] 迟静, 屠长河, 陈雪. 基于法向保持的三维人脸表情跟踪[J]. 计算机辅助设计与图形学学报, 2011, 23(6): 1069-1077. CHI Jing, TU Changhe, CHEN Xue. 3D facial expression tracking based on normal preserving[J]. Journal of Computer-Aided Design & Computer Graphics, 2011, 23(6): 1069-1077.
[7] CHEN Yenlin, WU Hsiangtao, SHI Fuhao, et al. Accurate and robust 3D facial capture using a single RGBD camera[C] //Proceedings of the 2013 IEEE International Conference on Computer Vision. Sydney, Australia: IEEE, 2013: 3615-3622.
[8] LI Hao, YU Jihun, YE Yuting, et al. Realtime facial animation with on-the-fly correctives[J]. ACM Transactions on Graphics, 2013, 32(4): 42.
[9] WEISE T, BOUAZIZ S, LI Hao, et al. Realtime performance-based facial animation[J]. ACM Transactions on Graphics, 2011, 30(4): 77.
[10] 何钦政, 王运巧. 基于Kinect的人脸表情捕捉及动画模拟系统研究[J]. 图学学报, 2016, 37(3): 290-295. HE Qinzheng, WANG Yunqiao. Research on system of facial expression capture and animation simulation based on Kinect[J]. Journal of Graphics, 2016, 37(3): 290-295.
[11] 李俊龙, 章登义, 黄珺. Kinect驱动的人脸动画合成技术研究[J]. 计算机工程, 2015, 41(3):237-241. LI Junlong, ZHANG Dengyi, HUANG Jun. Research on synthesis technology of facial animation driven by Kinect[J]. Computer Engineering, 2015, 41(3): 237-241.
[12] BLANZ V, SCHERBAUM K, VETTER T, et al. Exchanging faces in images[J].Computer Graphics Forum, 2004, 23(3): 669-676.
[13] VLASIC D, BRAND M, PFISTER H, et al. Face transfer with multilinear models[J]. ACM Transactions on Graphics, 2005, 24(3): 426-433.
[14] CAO Chen, WENG Yanlin, LIN Stephen, et al. 3D shape regression for real-time facial animation[J]. ACM Transactions on Graphics, 2013, 32(4): 41.
[15] CAO Chen, WENG Yanlin, ZHOU Shun, et al. Face-warehouse: a 3d facial expression database for visual computing[J]. IEEE Transactions on Visualization and Computer Graphics, 2014, 20(3): 413-425.
[16] SHI Fuhao, WU Hsiangtao, TONG Xin, et al. Automatic acquisition of high-fidelity facial performances using monocular videos[J]. ACM Transactions on Graphics, 2014, 33(6): 222.
[17] SUPASOM S, IRA K, SEITZ M S. Total moving face reconstruction[M] //Lecture Notes in Computer Science(including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). [S.l.] : Springer International Publishing,2014,8692(4): 796-812.
[18] FYFFE G, JONES A, ALEXANDER O, et al. Driving high-resolution facial scans with video performance capture[J]. ACM Transactions on Graphics, 2014, 34(1): 8.
[19] SIBBING D, HABBECKE M, KOBBELT L. Markerless reconstruction and synthesis of dynamic facial expressions[J]. Computer Vision and Image Understanding, 2011, 115(5): 668-680.
[20] REN Shaoqing, CAO Xuding, WEI Yichen, et al. Face alignment at 3000 fps via regressing local binary features[C] //Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus, USA: IEEE, 2014: 1685-1692.
[21] 栾勐. 基于拉普拉斯变形技术的形状分析方法研究[D].杭州: 浙江大学, 2011. LUAN Meng. Shape analysis based on Laplacian deformation[D]. Hangzhou: Zhejiang University, 2011.
[22] SORKINE O, COHEN-OR D, LIPMAN Y, et al. Laplacian surface editing[C] //Proceedings of the 2004 Eurographics/ACM SIGGRAPH Symposium on Geometry processing. [S.l.] :ACM, 2004, 71:175-184.

多维度评价

Viewed

Full text

Abstract

Cited

Shared

Discussed

基于单目RGB摄像机的三维人脸表情跟踪算法

3D facial expression tracking using a monocular RGB camera

PDF (PC)

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 6

多维度评价

本文评价

推荐阅读 0

[1]	世文学刘卫东孙永福. 基于DEM的堰塞湖1／3溃决模拟及人员撤离方案研究[J]. 山东大学学报(工学版), 2009, 39(5): 144-148.
[2]	李芳佳, 高尚策, 唐政, 石井雅博, 山下和也. 基于元胞自动化模型的三维雪花晶体近似模式的产生（英文）[J]. 山东大学学报(工学版), 2009, 39(1): 102-105.
[3]	陈成军,周以齐,杨红娟 . 基于SolidWorks模型的虚拟装配模型转换和表达方法[J]. 山东大学学报(工学版), 2008, 38(1): 61-65 .
[4]	宋青,李晓磊,张承进 . 基于瓶颈分析的邮政速递网络的优化[J]. 山东大学学报(工学版), 2007, 37(5): 29-33 .
[5]	赵伟华,王勇,王宪伦 . 基于VRML的机器人空间连续轨迹运动仿真[J]. 山东大学学报(工学版), 2007, 37(2): 8-11 .
[6]	孔祥臻,刘延俊,王勇,赵秀华 . 气动比例阀的死区补偿与仿真[J]. 山东大学学报(工学版), 2006, 36(1): 99-102 .