山东大学学报 (工学版) ›› 2023, Vol. 53 ›› Issue (2): 61-69.doi: 10.6040/j.issn.1672-3961.0.2022.135
余明骏1,刁红军1,凌兴宏1,2,3*
YU Mingjun1, DIAO Hongjun1, LING Xinghong1,2,3*
摘要： 针对现有多目标跟踪方法易受到遮挡、运动模糊等问题干扰的情况,提出基于轨迹掩膜的在线多目标跟踪方法(online multi-object tracking method based on trajectory mask, OMTMTM)。提出轨迹掩膜生成算法,利用前一帧跟踪轨迹结果生成轨迹掩膜,设计轨迹掩膜网络对轨迹掩膜提取多维度特征,包含目标可见区域的估计值、大致位置及形状等信息;将该特征与基础骨干网络提取的原始图像特征融合后进行多目标检测跟踪。OMTMTM的目标跟踪器具备先验判断能力,可实现遮挡情况下的准确跟踪;OMTMTM利用目标跟踪轨迹的时空信息,恢复出部分漏检或低置信待检目标,使轨迹掩膜更加合理,有利于后续跟踪。对OMTMTM的性能进行多维度评估,并结合基线模型进行对比分析。试验结果表明,OMTMTM具有先进的多目标跟踪性能。
中图分类号:
| [1] | LAN Long, WANG Xinchao, HUA Gang, et al. Semi-online multi-people tracking by re-identification[J]. International Journal of Computer Vision, 2020, 128(7): 1-19. |
| [2] | XU Y H, OSEP A, BAN Y T, et al. How to train your deep multi-object tracker[C] //Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle, USA: IEEE, 2020: 6787-6796. |
| [3] | LIU Qiankun, CHU Qi, LIU Bin, et al. GSM: graph similarity model for multi-object tracking[C] //Proceedings of the International Joint Conference on Artificial Intelligence. Yokohama, Japan: Morgan Kaufmann, 2020: 530-536. |
| [4] | 侯建华,张国帅,项俊. 基于深度学习的多目标跟踪关联模型设计[J]. 自动化学报, 2020, 46(12): 2690-2700. HOU Jianhua, ZHANG Guoshuai, XIANG Jun. Designing affinity model for multiple object tracking based on deep learning[J]. Acta Automatica Sinica, 2020, 46(12): 2690-2700. |
| [5] | 朱珠. 卷积神经网络的多目标跟踪系统[J]. 网络空间安全, 2018, 9(11):68-71. ZHU Zhu. Multi-object tracking system by convolution neural networks[J]. Cyberspace Security, 2018, 9(11):68-71. |
| [6] | 孙金萍, 丁恩杰, 鲍蓉, 等. 多特征融合的长时间目标跟踪算法[J]. 南京大学学报(自然科学版), 2021, 57(2): 217-226. SUN Jinping, DING Enjie, BAO Rong, et al. Long-term object tracking algorithm based on multi-feature fusion[J]. Journal of Nanjing University(Natural Sciences), 2021, 57(2): 217-226. |
| [7] | BEWLEY A, GE Z Y, OTT L, et al. Simple online and realtime tracking[C] //Proceedings of the 2016 IEEE International Conference on Image Processing. Arizona, USA: IEEE, 2016: 3464-3468. |
| [8] | WOJKE N, BEWLEY A, PAULUS D. Simple online and realtime tracking with a deep association metric[C] //Proceedings of the 2017 IEEE International Conference on Image Processing. Beijing, China: IEEE, 2017: 3645-3649. |
| [9] | YIN Junbo, WANG Wenguan, MENG Qinghao, et al. A unified object motion and affinity model for online multi-object tracking[C] //Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle, USA: IEEE, 2020: 6768-6777. |
| [10] | HENSCHEL R, ZOU Y Z, ROSENHAHN B. Multiple people tracking using body and joint detections[C] //Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Long Beach, USA: IEEE, 2019: 770-779. |
| [11] | TANG Peng, WANG Chunyu, WANG Xinggang, et al. Object detection in videos by high quality object linking[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 42(5): 1272-1278. |
| [12] | LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection[C] //Proceedings of the IEEE International Conference on Computer Vision. Venice, Italy: IEEE, 2017: 2980-2988. |
| [13] | LU Z C, RATHOD V, VOTEL R, et al. Retinatrack: online single stage joint detection and tracking[C] //Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Seattle, USA: IEEE, 2020: 14668-14678. |
| [14] | PENG Jinlong, WANG Chang'an, WAN Fangbin, et al. Chained-tracker: chaining paired attentive regression results for end-to-end joint multiple-object detection and tracking[C] //Proceedings of the European Conference on Computer Vision. Berlin, Germany: Springer, 2020: 145-161. |
| [15] | ZHOU X Y, KOLTUN V, KRAHENBUHL P. Tracking objects as points[C] //Proceedings of the European Conference on Computer Vision.Berlin, Germany: Springer, 2020: 474-490. |
| [16] | ZHANG Yifu, WANG Chunyu, WANG Xinggang, et al. Fairmot: on the fairness of detection and re-identification in multiple object tracking[J]. International Journal of Computer Vision, 2021, 129(11): 3069-3087. |
| [17] | ZHENG Linyu, TANG Ming, CHEN Yingying, et al. Improving multiple object tracking with single object tracking[C] //Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville, USA: IEEE, 2021: 2453-2462. |
| [18] | FARHADI A,REDMON J. Yolov3: an incremental improvement[EB/OL].(2018-04-08)[2022-04-11]. https://arxiv.org/abs/1804.02767. |
| [19] | WANG C Y, BOCHKOVSKIY A, LIAO H Y M. Scaled-yolov4: scaling cross stage partial network[C] //Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville, USA: IEEE, 2021: 13029-13038. |
| [20] | LIANG Chao, ZHANG Zhipeng, LU Yi, et al. Rethinking the competition between detection and reid in multi-object tracking[J]. IEEE Transactions on Image Processing, 2022, 31: 3182-3196. |
| [21] | GE Zheng, LIU Songtao, WANG Feng, et al. YOLOX: exceeding yolo series in 2021[EB/OL].(2021-07-18)[2022-04-11]. https://arxiv.org/abs/2107.08430. |
| [22] | WANG C Y, LIAO H Y M, WU Y H, et al. CSPNet: a new backbone that can enhance learning capability of CNN[C] //Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Seattle, USA: IEEE, 2020: 390-391. |
| [23] | LIU Shu, QI Lu, QIN Haifang, et al. Path aggregation network for instance segmentation[C] //Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE, 2018: 8759-8768. |
| [24] | ZHANG H Y, CISSE M, DAUPHIN Y N, et al. Mixup: beyond empirical risk minimization[EB/OL].(2017-10-25)[2022-04-11]. https://arxiv.org/abs/1710.09412. |
| [25] | GE Zheng, LIU Songtao, LI Zeming, et al. OTA: optimal transport assignment for object detection[C] //Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville, USA: IEEE, 2021: 303-312. |
| [26] | 刘彩虹, 张磊, 黄华. 交通路口监控视频跨视域多目标跟踪的可视化[J]. 计算机学报, 2018, 41(1): 221-235. LIU Caihong, ZHANG Lei, HUANG Hua. Visualization of cross-view multi-object tracking for surveillance videos in crossroad[J]. Chinese Journal of Computers, 2018, 41(1): 221-235. |
| [27] | 张子龙, 王永雄. 基于卡尔曼滤波的SiamRPN目标跟踪方法[J]. 智能计算机与应用, 2020, 10(3): 44-50. ZHANG Zilong, WANG Yongxiong. SiamRPN network for object tracking based on Kalman filter[J]. Intelligent Computer and Applications, 2020, 10(3): 44-50. |
| [28] | CHEN Long, AI Haizhou, ZHUANG Zijie, et al. Real-time multiple people tracking with deeply learned candidate selection and person re-identification[C] //Proceedings of the 2018 IEEE International Conference on Multimedia and Expo. San Diego, USA: IEEE, 2018: 1-6. |
| [29] | PANG Jiangmiao, QIU Linlu, LI Xia, et al. Quasi-dense similarity learning for multiple object tracking[C] //Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville, USA: IEEE, 2021: 164-173. |
| [30] | ZHANG Yifu, SUN Peize, JIANG Yi, et al. ByteTrack: multi-object tracking by associating every detection box[C] // Proceedings of the European Conference on Computer Vision. Berlin, Germany: Springer, 2022: 1-21. |
| [31] | REZATOFIGHI H, TSOI N, GWAK J Y, et al. Generalized intersection over union: a metric and a loss for bounding box regression[C] //Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach, USA: IEEE, 2019: 658-666. |
| [32] | BERNARDIN K, STIEFELHAGEN R. Evaluating multiple object tracking performance: the clear mot metrics[J]. EURASIP Journal on Image and Video Processing, 2008, 2008(1): 1-10. |
| [33] | LUITEN J, OSEP A, DEMDPRFER P, et al. Hota: a higher order metric for evaluating multi-object tracking[J]. International Journal of Computer Vision, 2021, 129(2): 548-578. |
| [34] | WANG Y X, KITANI K, WENG X S. Joint object detection and multi-object tracking with graph neural networks[C] //Proceedings of the 2021 IEEE International Conference on Robotics and Automation. Xi'an, China: IEEE, 2021: 13708-13715. |
| [35] | WANG Qiang, ZHENG Yun, PAN Pan, et al. Multiple object tracking with correlation learning[C] //Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville, USA: IEEE, 2021: 3876-3886. |
| [1] | 刘丁菠,刘学艳,于东然,杨博,李伟. 面向小样本目标检测任务的自适应特征重构算法[J]. 山东大学学报 (工学版), 2022, 52(6): 115-122. |
| [2] | 孟令灿,聂秀山,张雪. 基于遮挡目标去除的公交车拥挤度分类算法[J]. 山东大学学报 (工学版), 2022, 52(4): 83-88. |
| [3] | 刘新锋, 张旖旎,徐惠三,宋玲,陈梦雅. 基于随机森林和专家系统的分布式光伏电站阴影遮挡诊断[J]. 山东大学学报 (工学版), 2021, 51(2): 98-104. |
| [4] | 万鹏. 基于F-PointNet的3D点云数据目标检测[J]. 山东大学学报 (工学版), 2019, 49(5): 98-104. |
| [5] | 江珊珊,杨静,范丽亚. 基于PDEs的图像特征提取方法[J]. 山东大学学报(工学版), 2018, 48(4): 27-36. |
| [6] | 惠开发,成科扬,詹永照. 基于改进ViBe算法的视频浓缩[J]. 山东大学学报(工学版), 2017, 47(3): 43-48. |
| [7] | 马帅依凡,赵子健. 基于人工标记的手术导航仪[J]. 山东大学学报(工学版), 2017, 47(3): 63-68. |
| [8] | 刘英霞,王希常,唐晓丽,常发亮. 基于小波域特征和贝叶斯估计的目标检测算法[J]. 山东大学学报(工学版), 2017, 47(2): 63-70. |
| [9] | 邱晓欣1,2,张文强1,2*,秦晋贤1,2,杜正阳1,2,张德峰1,2. 恶劣环境下多目标实时跟踪算法研究[J]. 山东大学学报(工学版), 2014, 44(2): 21-27. |
| [10] | 杨健梅1,黄添强1,2*,江伟坚1. 基于人脸色温的拼接图像篡改检测[J]. 山东大学学报(工学版), 2013, 43(5): 24-30. |
| [11] | 张国华,谭晓阳*,陈松灿. 基于视频流的复杂场景公车人头对象计数研究[J]. 山东大学学报(工学版), 2013, 43(4): 39-45. |
| [12] | 王秀芬,王汇源,王松. 基于背景差分法和显著性图的海底目标检测方法[J]. 山东大学学报(工学版), 2011, 41(1): 12-16. |
| [13] | 乔伟1,王汇源1,2,吴晓娟1,刘鹏威1. 基于混沌动力学模型的群体目标检测与分类[J]. 山东大学学报(工学版), 2010, 40(2): 19-23. |
| [14] | 方 挺,杨 忠,沈春林 . 无人机编队视频序列中的多目标精确跟踪[J]. 山东大学学报(工学版), 2008, 38(4): 22-26 . |
| [15] | 吕行,史忠科 . DirectShow框架下实时运动目标检测与跟踪方法的研究与应用[J]. 山东大学学报(工学版), 2007, 37(6): 5-9 . |
|
