您的位置:山东大学 -> 科技期刊社 -> 《山东大学学报(工学版)》

山东大学学报 (工学版) ›› 2020, Vol. 50 ›› Issue (2): 17-26.doi: 10.6040/j.issn.1672-3961.0.2019.418

• 机器学习与数据挖掘 • 上一篇    下一篇

基于校正神经网络的视频追踪算法

陈宁宁(),赵建伟*(),周正华   

  1. 中国计量大学理学院, 浙江 杭州 310018
  • 收稿日期:2019-03-28 出版日期:2020-04-20 发布日期:2020-04-16
  • 通讯作者: 赵建伟 E-mail:853078476@qq.com;zhaojw@amss.ac.cn
  • 作者简介:陈宁宁(1994—),男,陕西汉中人,硕士研究生,主要研究方向为机器学习与视频追踪. E-mail: 853078476@qq.com
  • 基金资助:
    浙江省自然科学基金资助项目(LY18F020018);浙江省自然科学基金资助项目(LSY19F020001);国家自然科学基金资助项目(61571410)

Visual tracking algorithm based on verifying networks

Ningning CHEN(),Jianwei ZHAO*(),Zhenghua ZHOU   

  1. School of Science, China Jiliang University, Hangzhou 310018, Zhejiang, China
  • Received:2019-03-28 Online:2020-04-20 Published:2020-04-16
  • Contact: Jianwei ZHAO E-mail:853078476@qq.com;zhaojw@amss.ac.cn
  • Supported by:
    浙江省自然科学基金资助项目(LY18F020018);浙江省自然科学基金资助项目(LSY19F020001);国家自然科学基金资助项目(61571410)

摘要:

针对现有基于深度学习的视频追踪算法关注深层特征而忽略浅层特征以及追踪网络没有对每帧追踪结果进行检测的问题,提出基于校正神经网络的视频追踪算法。该算法包含追踪网络和校正网络。在追踪网络中,考虑到深层特征和浅层边缘特征的融合,设计一个多输入的残差网络,学习目标和对应的高斯响应图之间的关系,从而获得目标对象的位置信息。在校正网络中,设计浅层链式判别网络,将两个网络的追踪结果进行比较,根据比较结果对追踪网络进行在线更新。本算法既考虑了深层特征,又避免了细节信息的丢失;同时,对追踪结果进行评判,防止网络更新中延续错误信息。对比试验说明本研究所提的追踪算法比现有的一些追踪方法取得更好的追踪结果。

关键词: 视频追踪, 深度神经网络, 校正网络, 浅层特征, 深度特征

Abstract:

In order to solve the problem that the existing deep learning based visual tracking algorithms paid attention to the deep features but neglected the shallow features, and the tracking network did not evaluate the tracking results, a visual tracking algorithm based on verifying network was proposed. The proposed algorithm consisted of tracking network and verifying network. In the tracking network, considering the fusion of deep features and shallow edge features, a multi-input residual network was designed to learn the relationship between the target and its corresponding Gaussian response map to obtain the position information of the target. In the verifying network, a shallow chain discriminate network was designed, and this paper compared the tracking results of tracking network and verifying network, and updated the tracking network according to the compared results. Therefore, the proposed algorithm not only took the deep features into account, but also avoided the loss of detail information. Furthermore, the tracking results were evaluated to prevent the continuation of error messages in the update. The experimental results illustrated that the proposed tracking algorithm achieved better tracking results than some other existing tracking methods.

Key words: visual tracking, deep neural network, verifying network, shallow feature, deep feature

中图分类号: 

  • TP391

图1

追踪器的框架流程图"

图2

追踪网络MRNet的结构图"

图3

校正网络PCNet的结构图"

表1

ω的选择对成功率和距离精度的影响"

ω 成功率 距离精度
0 0.668 0.830
1 0.669 0.831
2 0.686 0.853
3 0.681 0.847
4 0.660 0.832

图4

OTB-2013数据集上排名前14的追踪算法的综合距离精度图和成功重叠率图"

表2

OTB-2013数据集上各挑战属性的精确度排名前5的追踪算法"

排名 IV SV OCC DEF MB FM IPR OPR OV BC LR
1 OUR
(0.828)
OUR
(0.854)
OUR
(0.835)
OUR
(0.851)
OUR
(0.795)
OUR
(0.787)
OUR
(0.831)
OUR
(0.844)
OUR
(0.798)
OUR
(0.843)
OUR
(0.822)
2 CNN-
SVM
(0.716)
DeepS
RDCF
(0.750)
DeepS
RDCF
(0.778)
DLS-
SVM
(0.807)
DeepS
RDCF
(0.724)
DeepS
RDCF
(0.705)
CNN-
SVM
(0.754)
DeepS
RDCF
(0.770)
DCFNet
(0.767)
RPT
(0.753)
SiamFC
(0.669)
3 RPT
(0.713)
CNN-
SVM
(0.744)
DCFNet
(0.767)
ACFN-
selNet
(0.805)
RPT
(0.706)
MEEM
(0.680)
STAPL
E_CA
(0.742)
ACFN-
selNet
(0.761)
ACFN-
selNet
(0.715)
DLS
SVM
(0.730)
CNN-
SVM
(0.612)
4 STAPL
E_CA
(0.710)
SiamFC
(0.730)
SAMF
(0.765)
CNN-
SVM
(0.797)
SRDCF
(0.701)
RPT
(0.673)
DeepS
RDCF
(0.730)
DLS SVM
(0.758)
DeepS
RDCF
(0.713)
SRDCF
(0.721)
DCFNet
(0.590)
5 DeepS
RDCF
(0.703)
DCFNet
(0.726)
SRDCF
(0.760)
STAPL
E_CA
(0.767)
DLS SVM
(0.686)
SiamFC
(0.671)
DLS SVM
(0.725)
CNN-
SVM
(0.756)
SiamFC
(0.708)
DeepS
RDCF
(0.720)
DLS
SVM
(0.544)

表3

OTB-2013数据集上各挑战属性的成功率排名前5的追踪算法"

排名 IV SV OCC DEF MB FM IPR OPR OV BC LR
1 OUR
(0.673)
OUR
(0.670)
OUR
(0.678)
OUR
(0.688)
OUR
(0.664)
OUR
(0.649)
OUR
(0.656)
OUR
(0.674)
OUR
(0.708)
OUR
(0.676)
OUR
(0.662)
2 DCFNet
(0.596)
DeepS
RDCF
(0.628)
DCFNet
(0.645)
CNN-
SVM
(0.640)
DeepS
RDCF
(0.625)
DeepS
RDCF
(0.608)
STAPLE_
CA
(0.601)
DeepS
RDCF
(0.630)
DCFNet
(0.690)
RPT
(0.614)
SiamFC
(0.566)
3 STAPL
E_CA
(0.596)
DCFNet
(0.619)
DeepS
RDCF
(0.628)
SRDCF
(0.635)
SRDCF
(0.601)
SRDCF
(0.569)
DeepS
RDCF
(0.596)
DCFNet
(0.612)
SiamFC
(0.635)
CNN-
SVM
(0.593)
DCFNet
(0.496)
4 DeepS
RDCF
(0.589)
SiamFC
(0.603)
SRDCF
(0.627)
STAPL
E_CA
(0.632)
DLS SVM
(0.578)
STAPL
E_CA
(0.566)
SiamFC
(0.582)
SRDCF
(0.599)
DeepS
RDCF
(0.619)
DLS SVM
(0.592)
CNN-
SVM
(0.461)
5 SRDCF
(0.576)
SRDCF
(0.587)
SAMF
(0.612)
DLS SVM
(0.632)
RPT
(0.576)
RPT
(0.561)
STAPLE
(0.580)
STAPL
E_CA
(0.594)
ACFN-
selNet
(0.609)
DeepS
RDCF
(0.591)
STAPLE
(0.438)

表4

VOT2016数据集上15个追踪器的成功率和距离精度的实验结果"

追踪器 距离精度 成功率
综合 CM IC MC OCC SC 综合 CM IC MC OCC SC
OUR 0.668 0.663 0.663 0.663 0.663 0.668 0.502 0.500 0.500 0.500 0.492 0.502
DeepSTRCF 0.595 0.589 0.589 0.589 0.585 0.595 0.460 0.454 0.454 0.454 0.447 0.460
CFWCR 0.586 0.579 0.579 0.579 0.576 0.586 0.453 0.448 0.448 0.448 0.444 0.453
STRCF 0.548 0.541 0.541 0.541 0.538 0.548 0.418 0.412 0.412 0.412 0.405 0.418
CREST 0.526 0.519 0.519 0.519 0.519 0.526 0.423 0.418 0.418 0.418 0.412 0.423
BACF 0.464 0.456 0.456 0.456 0.448 0.464 0.366 0.360 0.360 0.360 0.351 0.366
DCFNet 0.454 0.445 0.445 0.445 0.437 0.454 0.367 0.360 0.360 0.360 0.351 0.367
MRNet 0.448 0.440 0.440 0.440 0.437 0.448 0.360 0.354 0.354 0.354 0.346 0.360
SAMF 0.427 0.423 0.423 0.423 0.407 0.427 0.337 0.335 0.335 0.335 0.322 0.337
SRDCF 0.419 0.410 0.410 0.410 0.402 0.419 0.325 0.317 0.317 0.317 0.311 0.325
KCF 0.367 0.358 0.358 0.358 0.355 0.367 0.276 0.270 0.270 0.270 0.265 0.276
CSK 0.318 0.308 0.308 0.308 0.318 0.318 0.241 0.233 0.233 0.233 0.238 0.241
DSST 0.294 0.283 0.283 0.283 0.305 0.294 0.219 0.210 0.210 0.210 0.227 0.219
DFT 0.260 0.254 0.254 0.254 0.264 0.260 0.213 0.208 0.208 0.208 0.212 0.213
CT 0.233 0.234 0.234 0.234 0.239 0.233 0.195 0.197 0.197 0.197 0.195 0.195

图5

本研究算法与其它6种算法在视频序列上的追踪效果"

1 ISARD Michael , BLAKE Andrew . Condensation-conditional density propagation for visual tracking[J]. International Journal of Computer Vision, 1998, 29 (1): 5- 28.
2 YANG Changjiang, DURAISWAMI Ramani, DAVIS Larry. Fast multiple object tracking via a hierarchical particle filter[C]// Proceedings of the IEEE International Conference on Computer Vision. Beijing, China: IEEE, 2005: 212-219.
3 KHAN Zia, BALCH Tucker, DELLAERT Frank. A rao-blackwellized particle filter for eigen tracking [C]// Proceedings of the Computer Vision and Pattern Recognition. Washington, American: IEEE, 2004: 980-986.
4 ZHOU Shaohua Kevin , CHELLAPPA Rama , MOGHAD-DAM Baback . Visual tracking and recognition using appearance-adaptive models in particle filters[J]. IEEE Transactions on Image Processing, 2004, 13 (11): 1491- 1506.
doi: 10.1109/TIP.2004.836152
5 PEREZ P, HUE C, VERMAAK J, et al. Color-based probabilistic tracking[C] //Proceedings of the European Conference on Computer Vision. Copenhagen, Denmark: Springer, 2002: 661-675.
6 BOLME David S, BEVERRIDGE J R, DRAPER Bruce A, et al. Visual object tracking using adaptive correlation filters[C]//Proceedings of the Computer Vision and Pattern Recognition. San Francisco, American: IEEE, 2010: 2544-2550.
7 DANELLJAN Martin , HAGER Gustav , KHAN Fahad Shahbaz , et al. Accurate scale estimation for robust visual tracking[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39 (8): 1561- 1575.
doi: 10.1109/TPAMI.2016.2609928
8 HENRIQUES Joao F , CASEIRO Rui , MARTINS Pedro , et al. High speed tracking with kernelized correlation filters[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37 (3): 583- 596.
doi: 10.1109/TPAMI.2014.2345390
9 DANELLJAN Martin, HAGER Gustav, KHAN Fahad Shahbaz, et al. Learning spatially regularized correlation filters for visual tracking[C]//Proceedings of the International Conference on Computer Vision. Santiago, Chile: IEEE, 2015: 4310-4318.
10 YANG Li, ZHU Jianke. A scale adaptive kernel correlation filter tracker with feature Integration[C]// Proceedings of European Conference on Computer Vision Workshops. Zurich, Switzerland: Springer, 2014: 254-265.
11 HENRIQUES Joao F, CASEIRO Rui, MARTINS Pedro, et al. Exploiting the circulant structure of tracking-by-detection with kernels[C] //Proceedings of the European Conference on Computer Vision. Florence, Italy: Springer, 2012: 702-715.
12 GALOOGAHI Hamed Kiani, FAGG Ashton, LUCEY Simon. Learning background-aware correlation filters for visual tracking[C]// Proceedings of the International Conference on Computer Vision. Venice, Italy: IEEE, 2017: 1144-1152.
13 HONG Zhibin, CHEN Zhe, WANG Chaohui, et al. Multi-store tracker (muster): a cognitive psychology inspired approach to object tracking[C]// Proceedings of the Computer Vision and Pattern Recognition. Boston, American: IEEE, 2015: 749-758.
14 HONG Seunghoon, YOU Tackgeun, KWAK Suha, et al. Online tracking by learning discriminative saliency map with convolutional neural network[C] // Proceedings of International Conference on Machine Learning. Lille, France: IEEE, 2015: 597-606.
15 WANG Lijun, OUYANG Wangli, WANG Xiaogang, et al. Visual tracking with fully convolutional networks[C]//Proceedings of the International Conference on Computer Vision. Santiago, Chile: IEEE, 2015: 3119-3127.
16 TAO Ran, GAVVES Efstratios, SMEULDERS Arnold W M. Siamese instance search for tracking[C]// Proceedings of the Computer Vision and Pattern Recognition. Las Vegas, American: IEEE, 2016: 1420-1429.
17 WANG Qiang, GAO Jin, XING Junliang, et al. DCFNet: discriminant correlation filters network for visual tracking [EB/OL]. https://arxiv.org/abs/1704.04057v1.
18 BERTINETTO Luca, VALMADRE Jack, HENRIQUES Joao F, et al. Fully convolutional siamese networks for object tracking[C]// Proceedings of the European Conference on Computer Vision. Amsterdam, The Netherlands: Springer, 2016: 850-865.
19 JIANG Chenru , XIAO Jimin , XIE Yanchun , et al. Siamese network ensemble for visual tracking[J]. Neurocomputing, 2018, 275, 2892- 2903.
doi: 10.1016/j.neucom.2017.10.043
20 DANELLJAN Martin, HAGER Gustav, KHAN Fahad Shahbaz. Convolutional features for correlation filter based visual tracking[C]//Proceedings of the International Conference on Computer Vision. Santiago, Chile: IEEE, 2015: 621-629.
21 SIMONYAN Karen, ZISSERMAN Andrew. Very deep convolutional networks for large-scale image recognition [EB/OL]. https://arxiv.org/abs/1409.1556.
22 WU Yi, LIM Jongwoo, YANG Ming-Hsuan. Online object tracking: a benchmark[C] // Proceedings of the Computer Vision and Pattern Recognition. Portland, American: IEEE, 2013: 2411-2418.
23 KRISTAN Matej, LEONARDIS Ales, MATAS J Felsberg, et al. The visual object tracking vot2016 challenge results[C]// In Proceedings of European Conference on Computer Vision Workshops. Washington, American: IEEE, 2016: 777-823.
24 VEDALDI Andrea, LENC Karel. Matconvnet: convolutional neural networks for matlab[C]//Proceedings of the ACM International Conference on Multimedia. New York, American: ACM, 2015: 689-692.
25 NING Jifeng, YANG Jimei, JIANG Shaojie, et al. Object tracking via dual linear structured SVM and explicit feature map [C] //Proceedings of the Computer Vision and Pattern Recognition. Las Vegas, American: IEEE, 2016: 4266-4274.
26 CHOI Jongwon, CHANG Hyung-Jin, YUN Sangdoo, et al. Attentional correlation filter network for adaptive visual tracking[C]//IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, American: IEEE, 2017: 4828-4837.
27 ZHANG Jianming, MA Shugao, SCLAROFF Stan. MEEM: robust tracking via multiple experts using entropy minimization[C]// Proceedings of the European Conference on Computer Vision. Zurich, Switzerland: Springer, 2014: 188-203.
28 LI Yang, ZHU Jianke, HOI Steven CH. Reliable patch trackers: robust visual tracking by exploiting reliable patches[C]//Proceedings of the Computer Vision and Pattern Recognition. Boston, American: IEEE, 2015: 353-361.
29 LAURA Sevilla-Lara, ERIK Learned-Miller. Distribution fields for tracking[C]// IEEE Conference on Computer Vision and Pattern Recognition. Providence, American: IEEE, 2012: 1910-1917.
30 ZHANG Kaihua, ZHANG Lei, YANG Ming-Hsuan. Real-time compressive tracking[C] // Proceedings of the European Conference on Computer Vision. Florence, Italy: Springer, 2012: 866-879.
31 BERTINETTO Luca, VALMADRE Jack, GOLODETZ Stuart, et al. Staple: complementary learners for real-time tracking[C]// Proceedings of the Computer Vision and Pattern Recognition. Las Vegas, American: IEEE, 2016: 1401-1409.
32 MUELLER Matthias, SMITH Neil, GHANEM Bernard. Context-aware correlation filter tracking[C]// Procee-dings of the Computer Vision and Pattern Recognition. Honolulu, American: IEEE, 2017: 1396-1404.
33 HE Zhiqun, FAN Yingruo, ZHUANG Junfei, et al. Correlation filters with weighted convolution responses[C]// Proceedings of the International Conference on Computer Vision. Venice, Italy: IEEE, 2017: 1992-2000.
34 LI Feng, CHENG Tian, ZUO Wangmeng, et al. Learning spatial-tem poral regularized correlation filters for visual tracking[C] // Proceedings of the Computer Vision and Pattern Recognition. Salt Lake City, American: IEEE, 2018: 4904-4913.
35 SONG Yibing, MA Chao, GONG Lijun, et al. CREST: convolutional residual learning for visual tracking [C]// Proceedings of the International Conference on Computer Vision. Venice, Italy: IEEE, 2017: 2555-2564.
[1] 陈艳平,冯丽,秦永彬,黄瑞章. 一种基于深度神经网络的句法要素识别方法[J]. 山东大学学报 (工学版), 2020, 50(2): 44-49.
[2] 沈冬冬,周风余,栗梦媛,王淑倩,郭仁和. 基于集成深度神经网络的室内无线定位[J]. 山东大学学报 (工学版), 2018, 48(5): 95-102.
[3] 唐乐爽,田国会,黄彬. 一种基于DSmT推理的物品融合识别算法[J]. 山东大学学报(工学版), 2018, 48(1): 50-56.
[4] 刘帆,陈泽华,柴晶. 一种基于深度神经网络模型的多聚焦图像融合方法[J]. 山东大学学报(工学版), 2016, 46(3): 7-13.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
[1] 李 侃 . 嵌入式相贯线焊接控制系统开发与实现[J]. 山东大学学报(工学版), 2008, 38(4): 37 -41 .
[2] 施来顺,万忠义 . 新型甜菜碱型沥青乳化剂的合成与性能测试[J]. 山东大学学报(工学版), 2008, 38(4): 112 -115 .
[3] 余嘉元1 , 田金亭1 , 朱强忠2 . 计算智能在心理学中的应用[J]. 山东大学学报(工学版), 2009, 39(1): 1 -5 .
[4] 刘忠国,张晓静,刘伯强,刘常春 . 视觉刺激间隔对大脑诱发电位的影响[J]. 山东大学学报(工学版), 2006, 36(3): 34 -38 .
[5] 龚毅光,龚异光,白俊杰,王宁生 . 基于SCA规范的DNC系统及其实现[J]. 山东大学学报(工学版), 2008, 38(1): 5 -8 .
[6] 王凯,孙奉仲,赵元宾,高明,高山 . 自然通风冷却塔进风口流场模型的建立及计算[J]. 山东大学学报(工学版), 2008, 38(1): 13 -17 .
[7] 张宁 李术才 李明田 杨磊. 新型岩石相似材料的研制[J]. 山东大学学报(工学版), 2009, 39(4): 149 -154 .
[8] 贾秀芹,刘允刚 . 一类更一般Lipschitz非线性系统的观测器设计[J]. 山东大学学报(工学版), 2007, 37(2): 113 -120 .
[9] 邱道宏1,张乐文1,崔伟2,苏茂鑫1,孙怀凤1. 基于趋势检查法的遗传神经网络模型及工程应用[J]. 山东大学学报(工学版), 2010, 40(3): 113 -118 .
[10] 何文杰 ,何伟超,孙权森. 压缩感知重构算法的并行化及GPU加速[J]. 山东大学学报(工学版), 2018, 48(3): 110 -114 .