基于层次化双重注意力网络的乳腺多模态图像分类

doi:10.6040/j.issn.1672-3961.0.2021.603

摘要/Abstract

摘要： 为解决现有多模态图像融合方法忽略临床先验知识的利用,且多模态之间的信息交互不充分等问题,提出基于层次化双重注意力网络的乳腺多模态图像分类方法,引入新的先验学习模块,有效挖掘和利用临床先验,提升单模态特征的区分性。设计层次化的双重注意力模块,利用注意力机制同时增强全局模态间通道特征和局部模态内特征的区分性信息,增强模态间的信息交互,进一步提升多模态融合的分类性能。试验结果表明,与其他方法对比,提出的模型能够取得更好的性能,在受试者工作特征曲线下面积、准确性、特异性和灵敏度分别达到为82.5%、83.3%、80.0%和85.0%。结果证明建立层次化双重注意力网络预测乳腺肿瘤良恶性可行。

关键词: 乳腺肿瘤, 多模态融合, 先验知识, 层次化双重注意力网络, 深度学习

中图分类号:

TP391.4

杨霄,袭肖明,李维翠,杨璐. 基于层次化双重注意力网络的乳腺多模态图像分类[J]. 山东大学学报 (工学版), 2022, 52(3): 34-41.

YANG Xiao, XI Xiaoming, LI Weicui, YANG Lu. Hierarchical dual attention network for breast multi-modality image classification[J]. Journal of Shandong University(Engineering Science), 2022, 52(3): 34-41.

参考文献

[1] HARRIS J R, LIPPMAN M E, VERONESI U, et al. Breast cancer[J]. New England Journal of Medicine, 1992, 327(5): 319-328.
[2] CAO Wei, CHEN Hongda, YU Yiwen, et al. Changing profiles of cancer burden worldwide and in China: a secondary analysis of the global cancer statistics 2020[J]. Chinese Medical Journal, 2021, 134(7): 783-791.
[3] 付子文, 杨华.MRI 增强扫描联合 DWI 检测在乳腺癌诊断中的应用[J]. 影像科学与光化学, 2020, 38(3): 540-545. FU Ziwen, YANG Hua. Application of MRI enhanced scan combined with DWI detection in diagnosis of breast cancer[J]. Imaging Science and Photochemistry, 2020, 38(3): 540-545.
[4] 中华医学会放射学分会乳腺专业委员会专家组. 乳腺磁共振检查及诊断规范专家共识[J]. 肿瘤影像学,2018, 26(4): 241-249. Chinese Medical Association Radiology Subcommittee Expert Group of Breast Professional Committee. Expert consensus on breast magnetic resonance examination and diagnostic criteria[J]. Tumor Imaging, 2018, 26(4): 241-249.
[5] IANNETTI G, APKARIAN A V. fMRI and human pain perception[M]. New York, USA: Oxford University Press, 2018: 499-516.
[6] 汪国宴, 冯平勇, 宋振虎, 等. 磁共振全身弥散加权成像的研究进展[J]. 临床荟萃, 2011, 26(22): 2018-2021. WANG Guoyan, FENG Pingyong, SONG Zhenhu, et al.Research progress of whole-body diffusion weighted magnetic resonance imaging[J]. Clinical Focus, 2011, 26(22): 2018-2021.
[7] BOLLINENI V R, KRAMER G, LIU Y, et al. A literature review of the association between diffusion-weighted MRI derived apparent diffusion coefficient and tumour aggressiveness in pelvic cancer[J]. Cancer Treatment Reviews, 2015, 41(6):496-502.
[8] WANG Haifeng, ZHENG Bichen, YOON S W, et al. A support vector machine-based ensemble algorithm for breast cancer diagnosis[J]. European Journal of Operational Research, 2018, 267(2): 687-699.
[9] KOSTOPOULOS S, SIDIROPOULOS K, GLOTSOS D, et al. Multimodality imaging and state-of-art GPU technology in discriminating benign from malignant breast lesions on real time decision support system[C] //Journal of Physics: Conference Series. Bristol, UK:IOP Publishing, 2014, 490(1): 012137.
[10] RAGAB D A, SHARKAS M, ATTALLAH O. Breast cancer diagnosis using an efficient CAD system based on multiple classifiers[J]. Diagnostics, 2019, 9(4): 165.
[11] CONCEICAO R C, CAPOTE R M, OLIVEIRA B L, et al. Imaging and classification of breast cancer with multimodal PEM-UWB techniques[C] //2013 International Conference on Electromagnetics in Advanced Applications(ICEAA). Piscataway, USA: IEEE Press, 2013: 421-424.
[12] LESNIAK J M, SCHIE G, TANNER C, et al. Multimodal classification of breast masses in mammography and MRI using unimodal feature selection and decision fusion[C] //International Workshop on Digital Mammography. Heidelberg, Germany: Springer-Verlag Press, 2012: 88-95.
[13] ZU Chen, JIE Biao, CHEN Songcan, et al. Label-alignment-based multi-task feature selection for multimodal classification of brain disease[C] //Machine Learning and Interpretation in Neuroimaging.Cham, Switzerland: Springer International Publishing, 2013: 51-59.
[14] MHIRI I, NEBLI A, MAHJOUB M A, et al. Non-isomorphic inter-modality graph alignment and synthesis for holistic brain mapping[C] //International Conference on Information Processing in Medical Imaging. Cham, Switzerland: Springer International Publishing, 2021: 203-215.
[15] RUECKERT D, ALJABAR P. Nonrigid registration of medical images: theory, methods, and applications [applications corner] [J]. IEEE Signal Processing Magazine, 2010, 27(4): 113-119.
[16] YALCIN A, REKIK I. A diagnostic unified classification model for classifying multi-sized and multi-modal brain graphs using graph alignment[J]. Journal of Neuroscience Methods, 2021, 348: 109014.
[17] RONNEBERGER O, FISCHER P, BROX T. U-net: convolutional networks for biomedical image segmentation[C] //International Conference on Medical Image Computing and Computer-assisted Intervention. Cham, Switzerland: Springer International Publishing, 2015: 234-241.
[18] FAN Fanda, HUANG Yunyou, WANG Lei, et al. A semantic-based medical image fusion approach[J/OL].(2019-12-11)[2022-03-16]. https://arxiv.org/abs/1906.00225.
[19] SIMONYAN K, ZISSERMAN A. Very deep convol-utional networks for large-scale image recognition[J/OL].(2015-04-10)[2022-03-16]. https://arxiv.org/abs/1409.1556.
[20] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C] //Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, USA: IEEE Press, 2016: 770-778.
[21] HUANG Gao, LIU Zhuang, MAATEN L, et al. Densely connected convolutional networks[C] //Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, USA:IEEE Press, 2017: 4700-4708.
[22] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. Imagenet classification with deep convolutional neural networks[J]. Advances in Neural Information Processing Systems, 2012, 25: 1097-1105.
[23] ZHANG Li, XIANG Tao, GONG Shaogang. Learning a deep embedding model for zero-shot learning[C] //Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, USA: IEEE Press, 2017: 2021-2030.
[24] LIU Xingbo, NIE Xiushan, TENG Junya, et al. Single-shot semantic matching network for moment localization in videos[J]. ACM Transactions on Multimedia Computing, Communications, and Applications(TOMM), 2021, 17(3): 1-14.
[25] ZHANG Yu, LIU Yu, SUN Peng, et al. IFCNN: a general image fusion framework based on convolutional neural network[J]. Information Fusion, 2020, 54: 99-118.

相关文章 15

[1]	王心哲,邓棋文,王际潮,范剑超. 深度语义分割MRF模型的海洋筏式养殖信息提取[J]. 山东大学学报 (工学版), 2022, 52(2): 89-98.
[2]	蒋桐雨,陈帆,和红杰. 基于非对称U型金字塔重建的轻量级人脸超分辨率网络[J]. 山东大学学报 (工学版), 2022, 52(1): 1-8, 18.
[3]	吴建清,宋修广. 同步定位与建图技术发展综述[J]. 山东大学学报 (工学版), 2021, 51(5): 16-31.
[4]	柴庆发,孙守晶,邱吉福,陈明,魏振,丛伟. 气象灾害条件下电网应急物资预测方法[J]. 山东大学学报 (工学版), 2021, 51(3): 76-83.
[5]	杨修远,彭韬,杨亮,林鸿飞. 基于知识蒸馏的自适应多领域情感分析[J]. 山东大学学报 (工学版), 2021, 51(3): 15-21.
[6]	廖锦萍,莫毓昌,YAN Ke. 基于C-LSTM的短期用电预测模型和应用[J]. 山东大学学报 (工学版), 2021, 51(2): 90-97.
[7]	刘帅,王磊,丁旭涛. 基于Bi-LSTM的脑电情绪识别[J]. 山东大学学报 (工学版), 2020, 50(4): 35-39.
[8]	蔡国永,贺歆灏,储阳阳. 基于空间注意力和卷积神经网络的视觉情感分析[J]. 山东大学学报 (工学版), 2020, 50(4): 8-13.
[9]	袁高腾,刘毅慧,黄伟,胡兵. 基于Gabor特征的乳腺肿瘤MR图像分类识别模型[J]. 山东大学学报 (工学版), 2020, 50(3): 15-23.
[10]	李春阳,李楠,冯涛,王朱贺,马靖凯. 基于深度学习的洗衣机异常音检测[J]. 山东大学学报 (工学版), 2020, 50(2): 108-117.
[11]	陈德蕾,王成,陈建伟,吴以茵. 基于门控循环单元与主动学习的协同过滤推荐算法[J]. 山东大学学报 (工学版), 2020, 50(1): 21-27,48.
[12]	张继,金翠,王洪元,陈首兵. 基于奇异值分解行人对齐网络的行人重识别[J]. 山东大学学报 (工学版), 2019, 49(5): 91-97.
[13]	万鹏. 基于F-PointNet的3D点云数据目标检测[J]. 山东大学学报 (工学版), 2019, 49(5): 98-104.
[14]	梁志祥,刘晓明,牟颖,刘玉田. 基于深度学习的新能源爬坡事件预测方法[J]. 山东大学学报 (工学版), 2019, 49(5): 24-28.
[15]	刘玉田, 孙润稼, 王洪涛, 顾雪平. 人工智能在电力系统恢复中的应用综述[J]. 山东大学学报 (工学版), 2019, 49(5): 1-8.

多维度评价

Viewed

Full text

Abstract

Cited

Shared

Discussed