﻿ 基于流形的约束局部模型拟合
 文章快速检索 高级检索
 山东大学学报(工学版)  2016, Vol. 46 Issue (3): 31-36  DOI: 10.6040/j.issn.1672-3961.0.2015.279 0

### 引用本文

LIU Dakun, TAN Xiaoyang. Fitting of constrained local model based on manifold[J]. Journal of Shandong University(Engineering Science), 2016, 46(3): 31-36. DOI: 10.6040/j.issn.1672-3961.0.2015.279.

### 文章历史

Fitting of constrained local model based on manifold
LIU Dakun, TAN Xiaoyang
College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics,Nanjing 211106, Jiangsu, China
Abstract: For the sake of embedding the manifold of face shape vectors into the models of face alignment efficiently, the research was carried out based on the typical constraint local model in the face location parameter model. According to the theorem of local coordinate coding and sparse constrain, the non-rigid deformations were replaced by the adjacent facial shape vectors which were based on manifold of facial shapes. The local tangent space alignment in manifold learning was mixed with the point distribution model, and a manifold embedded constrained local model was derived. The experimental verification on the toyed dataset and two public facial databases (i.e. labeled face parts in the wild and labeled face in the wild) were fulfilled. Compared with linear reconstruction based on constrained local model method fitting, the manifold embedded constrained local model method had better accuracy.
Key words: face alignment    constrained local model    local tangent space alignment
0 引言

1 基于局部切空间排列的约束局部模型拟合 1.1 人脸形状的流形结构

CLM模型通常看作由两部分组成,首先是判别地训练局部模板用于生成每个特征点的响应图,其次是结合响应图和形状共同优化得到未知人脸图像的形状。其中,关于形状拟合部分采用的是点分布模型(point distribution model,PDM),具体的说,就是通过线性方式对非刚性形状变化建模,即

 ${{x}_{i}}==sR\left( {{{\bar{x}}}_{_{i}}}+{{\Phi }_{i}}q \right)+t,$ (1)

 图 1 关于人脸形状向量的流形结构示意图 Figure 1 Example on the manifold structure of face shape vectors

1.2 局部切空间排列

(1) 构建子流形

 \begin{align} & \underset{\tau ,{{L}_{c}}}{\mathop{min}}\,\sum\limits_{i}{\sum\limits_{j}{\|{{\tau }_{ij}}-{{\tau }_{i}}-L_{_{_{c}}}^{^{(i)}}\left( {{x}_{ij}}-{{x}_{i}} \right)\|}}\Leftrightarrow \\ & \underset{{{T}_{i}},{{L}_{c}}}{\mathop{min}}\,\sum\limits_{i}{{}}\|{{T}_{i}}-\bar{T}-L_{c}^{\left( i \right)}({{X}_{i}}-{{x}_{i}}e_{k-1}^{T}){{\|}_{F}}, \\ \end{align} (2)

 $\underset{T,{{L}_{c}}}{\mathop{min}}\,\sum\limits_{i}{\|{{T}_{i}}J-L_{_{c}}^{\left( i \right)}{{X}_{i}}{{I}_{i}}{{\|}_{F}}。}$ (3)

(2) 子流形全局对齐

 \begin{align} & \underset{T}{\mathop{min}}\,\sum\limits_{i}{\|T{{S}_{i}}-{{T}_{i}}{{\|}_{F}}}, \\ & s.t.\text{ }T{{T}^{T}}=I, \\ \end{align} (4)

 \begin{align} & \underset{T,{{L}_{c}}}{\mathop{min}}\,\sum\limits_{i}{\|T{{S}_{i}}J-L_{_{c}}^{^{\left( i \right)}}{{X}_{i}}{{I}_{i}}{{\|}_{F}}}, \\ & s.t.\text{ }T{{T}^{T}}=I。 \\ \end{align} (5)
1.3 基于流形的目标形状向量拟合

 ${{x}_{k}}=\underset{{{x}_{i}}}{\mathop{argmax}}\,\|{{x}_{0}}-{{x}_{i}}\|,i=1,2,\cdots ,$ (6)

 ${{\mu }^{M}}=L_{_{c}}^{k}\times {{x}_{0}},$ (7)

 \begin{align} & \underset{\left\{ s,q,R,t \right\}}{\mathop{min}}\,\lambda \|q{{\|}_{1}}+\|x-\mu {{\|}^{2}},\text{ } \\ & s.t.\text{ }x=sR\left( \bar{x}+\psi q \right)+t, \\ \end{align} (8)

(1) 通过式(5)给出｛xi｝的低维流形空间中样本的坐标T以及相应的子流形投影矩阵集｛Lc(i)｝。

(2) 根据式(6)(7),找出在流形空间中与目标形状坐标向量μ最近的局部子流形以及对应的局部投影矩阵Lck,从而得到目标形状在流形空间中的坐标向量μM

(3) 在流形空间中对所有的训练集形状利用k-means聚类,并用l2-范数计算μM到各类中点的距离,以此找出流形空间中与目标形状最相近的子流形簇｛s1,s2,…,sn｝,其中n为近邻子流形个数。

(4) 在原空间中找出与｛s1,s2,…,sn｝对应的训练形状向量集｛x1,x2,…,xn｝,组合成形状坐标向量基矩阵ψ。

(5) 通过式(8)稀疏重建目标形状坐标向量。

2 试验 2.1 数据库及参数设定

(1) LFPW数据库

LFPW数据库的全称为自然条件下标注的人脸部件(Labeled Face Parts in the Wild)图片数据库[9],其中的图片来自于网络,并且受到姿态、光照、表情、遮挡等各种噪声影响。每张人脸图片包含29个特征点。由于数据库网站提供的图片下载链接中存在很多无效的地址,只下载了1 000张训练图片中的833张,以及300张测试图片中的232张用于试验。

(2) LFW数据库

LFW数据库的全称为自然条件下标注的人脸(Labeled Face Parts in the Wild)图片数据库[16],包含通过网络搜集来的13233张来自于5749个人的低分辨率的人脸图像。LFW数据库主要用于人脸识别和人脸验证。为了将其用于人脸特征点位置估计,文献[17]对每张图像都标注了10个主要特征点。在试验中,为了便于度量各方法的性能,进一步增加了2个关于瞳孔的特征点位置,因此,每张图片包含12个特征点。

 图 2 试验数据库人脸图片示例 Figure 2 Example of the databases used in the experiments

2.2 模拟数据试验

 图 3 LFPW中测试集形状扰动示例 Figure 3 Examples of the perturbed test shapes in LFPW database

 $err\left( i \right)=\frac{1}{N}\sum\limits_{j}{{}}\|x_{i}^{j}-o_{i}^{j}\|,$ (9)

 图 4 模拟数据集上两种PDM的拟合结果比较 Figure 4 Results comparison of the two PDMs on the simulated data

2.3 LFPW与LFW数据库上的试验

 $NRMSE\left( i \right)=\frac{1}{N\times l}\sum\limits_{j}{\|x_{i}^{j}-o_{i}^{j}\|},$ (10)

 图 5 2个数据库上不同拟合方法特征点平均误差比较 Figure 5 Mean pixel errors comparison of the two fitting methods on databases

 图 6 两种拟合方法得到的特征点坐标差异性 Figure 6 Demonstration on the landmarks derived from the two fitting methods

2.4 试验结果讨论

(1) 在流形空间中利用稀疏表示找出相邻的训练集形状,并将这些形状投影回原空间重建目标形状向量;

(2) 直接在原空间中利用稀疏重建来拟合目标形状向量。

3 结语

 [1] 杜吉祥, 余庆, 翟传敏. 基于稀疏性约束非负矩阵分解的人脸年龄估计方法[J]. 山东大学学报(理学版),2010, 45 (7) : 65-69. DU Jixiang, YU Qing, ZHAI Chuanmin. Age estimation of facial image based on non-negative matrix factorization with sparseness constraints[J]. Journal of Shandong University (Nature Science),2010, 45 (7) : 65-69. (0) [2] 夏海英, 杜海明, 徐鲁辉, 等. 基于自适应词典学习和稀疏表示的人脸表情识别[J]. 山东大学学报(工学版),2014, 44 (1) : 45-48. XIA Haiying, DU Haiming, XU Luhui, et al. Facial expression recognition based on adaptive dictionary learning and spares representation[J]. Journal of Shandong University (Engineering Science),2014, 44 (1) : 45-48. (0) [3] SARAGIH J M, LUCEY S, COHN J F, et al. Deformable model fitting by regularized landmark mean-shift[J]. International Journal of Computer Vision,2011, 91 (2) : 200-215. (0) [4] COOTES T F, TAYLOR C J, COOPER D H, et al. Active shape models-their training and application[J]. Computer Vision and Image Understanding,1995, 61 (1) : 38-59. (0) [5] JOLLIFFE I. Principal component analysis[M]. New York: John Wiley ＆ Sons, Ltd., 2002 . (0) [6] CRISTINACCE D, COOTES T F. Feature detection and tracking with constrained local models[C]//Proceedings of the 17th British Machine Vision Conference.Edinburgh: BMVC, 2006: 929-938. (0) [7] MARTINS P, CASEIRO R, HENRIQUES J F. Likelihood-enhanced Bayesian constrained local models[C]//Proceedings of the 21st IEEE International Conference on Image Processing.Paris: ICIP, 2014: 303-307. (0) [8] ASTHANA A, ZAFEIROU S, CHENG S. Robust discriminative response map fitting with constrained local models[C]//Proceedings of the 26th IEEE Conference on Computer Vision and Pattern Recognition. Columbus: CVPR, 2013:3444-3451. (0) [9] BELHUMEUR P N, JACOBS D W, KRIEGMAN D J. Localizing parts of faces using a consensus of exemplars[J]. Pattern Analysis and Machine Intelligence, IEEE Transactions on,2013, 35 (12) : 2930-2940. (0) [10] CAO X, WEI Y, WEN F. Face alignment by explicit shape regression[J]. International Journal of Computer Vision,2014, 107 (2) : 177-190. (0) [11] YU K, ZHANG T, GONG Y. Nonlinear learning using local coordinate coding[C]//Proceedings of the 22nd Advances in neural information processing systems. Vancouver: NIPS, 2009:2223-2231. (0) [12] 杜世强, 石玉清, 王维兰, 等. 基于流形正则化判别的因子分解[J]. 山东大学学报(理学版),2013, 48 (5) : 63-69. DU Shiqiang, SHI Yuqing, WANG Weilan, et al. Manifold regularized-based discriminant concept factorization[J]. Journal of Shandong University (Nature Science),2013, 48 (5) : 63-69. (0) [13] KASINSKI A, FLOREK A, SCHMIDT A. The PUT face database[J]. Image Processing and Communications,2008, 13 (3-4) : 59-64. (0) [14] TENENBAUM J B, DE Silva V, LANGFORD J C. A global geometric framework for nonlinear dimensionality reduction[J]. Science,2000, 290 (5500) : 2319-2323. (0) [15] ZHANG Z, ZHA H. Principal manifolds and nonlinear dimensionality reduction via tangent space alignment[J]. Journal of Shanghai University:English Edition,2004, 8 (4) : 406-424. (0) [16] HUANG G B, RAMESH M, BERG T. Labeled faces in the wild: a database for studying face recognition in unconstrained environments[R]. Amherst, USA: University of Massachusetts, 2007: 7-49. (0) [17] DANTONE M, GALL J, FANELLI G. Real-time facial feature detection using conditional regression forests[C]//Proceedings of the 25th IEEE Conference on Computer Vision and Pattern Recognition. Rhode Island, USA: CVPR, 2012: 2578-2585. (0)