山东大学学报 (工学版) ›› 2022, Vol. 52 ›› Issue (3): 34-41.doi: 10.6040/j.issn.1672-3961.0.2021.603
杨霄1,袭肖明1*,李维翠2,杨璐1
YANG Xiao1, XI Xiaoming1*, LI Weicui2, YANG Lu1
摘要： 为解决现有多模态图像融合方法忽略临床先验知识的利用,且多模态之间的信息交互不充分等问题,提出基于层次化双重注意力网络的乳腺多模态图像分类方法,引入新的先验学习模块,有效挖掘和利用临床先验,提升单模态特征的区分性。设计层次化的双重注意力模块,利用注意力机制同时增强全局模态间通道特征和局部模态内特征的区分性信息,增强模态间的信息交互,进一步提升多模态融合的分类性能。试验结果表明,与其他方法对比,提出的模型能够取得更好的性能,在受试者工作特征曲线下面积、准确性、特异性和灵敏度分别达到为82.5%、83.3%、80.0%和85.0%。结果证明建立层次化双重注意力网络预测乳腺肿瘤良恶性可行。
中图分类号:
| [1] HARRIS J R, LIPPMAN M E, VERONESI U, et al. Breast cancer[J]. New England Journal of Medicine, 1992, 327(5): 319-328. [2] CAO Wei, CHEN Hongda, YU Yiwen, et al. Changing profiles of cancer burden worldwide and in China: a secondary analysis of the global cancer statistics 2020[J]. Chinese Medical Journal, 2021, 134(7): 783-791. [3] 付子文, 杨华.MRI 增强扫描联合 DWI 检测在乳腺癌诊断中的应用[J]. 影像科学与光化学, 2020, 38(3): 540-545. FU Ziwen, YANG Hua. Application of MRI enhanced scan combined with DWI detection in diagnosis of breast cancer[J]. Imaging Science and Photochemistry, 2020, 38(3): 540-545. [4] 中华医学会放射学分会乳腺专业委员会专家组. 乳腺磁共振检查及诊断规范专家共识[J]. 肿瘤影像学,2018, 26(4): 241-249. Chinese Medical Association Radiology Subcommittee Expert Group of Breast Professional Committee. Expert consensus on breast magnetic resonance examination and diagnostic criteria[J]. Tumor Imaging, 2018, 26(4): 241-249. [5] IANNETTI G, APKARIAN A V. fMRI and human pain perception[M]. New York, USA: Oxford University Press, 2018: 499-516. [6] 汪国宴, 冯平勇, 宋振虎, 等. 磁共振全身弥散加权成像的研究进展[J]. 临床荟萃, 2011, 26(22): 2018-2021. WANG Guoyan, FENG Pingyong, SONG Zhenhu, et al.Research progress of whole-body diffusion weighted magnetic resonance imaging[J]. Clinical Focus, 2011, 26(22): 2018-2021. [7] BOLLINENI V R, KRAMER G, LIU Y, et al. A literature review of the association between diffusion-weighted MRI derived apparent diffusion coefficient and tumour aggressiveness in pelvic cancer[J]. Cancer Treatment Reviews, 2015, 41(6):496-502. [8] WANG Haifeng, ZHENG Bichen, YOON S W, et al. A support vector machine-based ensemble algorithm for breast cancer diagnosis[J]. European Journal of Operational Research, 2018, 267(2): 687-699. [9] KOSTOPOULOS S, SIDIROPOULOS K, GLOTSOS D, et al. Multimodality imaging and state-of-art GPU technology in discriminating benign from malignant breast lesions on real time decision support system[C] //Journal of Physics: Conference Series. Bristol, UK:IOP Publishing, 2014, 490(1): 012137. [10] RAGAB D A, SHARKAS M, ATTALLAH O. Breast cancer diagnosis using an efficient CAD system based on multiple classifiers[J]. Diagnostics, 2019, 9(4): 165. [11] CONCEICAO R C, CAPOTE R M, OLIVEIRA B L, et al. Imaging and classification of breast cancer with multimodal PEM-UWB techniques[C] //2013 International Conference on Electromagnetics in Advanced Applications(ICEAA). Piscataway, USA: IEEE Press, 2013: 421-424. [12] LESNIAK J M, SCHIE G, TANNER C, et al. Multimodal classification of breast masses in mammography and MRI using unimodal feature selection and decision fusion[C] //International Workshop on Digital Mammography. Heidelberg, Germany: Springer-Verlag Press, 2012: 88-95. [13] ZU Chen, JIE Biao, CHEN Songcan, et al. Label-alignment-based multi-task feature selection for multimodal classification of brain disease[C] //Machine Learning and Interpretation in Neuroimaging.Cham, Switzerland: Springer International Publishing, 2013: 51-59. [14] MHIRI I, NEBLI A, MAHJOUB M A, et al. Non-isomorphic inter-modality graph alignment and synthesis for holistic brain mapping[C] //International Conference on Information Processing in Medical Imaging. Cham, Switzerland: Springer International Publishing, 2021: 203-215. [15] RUECKERT D, ALJABAR P. Nonrigid registration of medical images: theory, methods, and applications [applications corner] [J]. IEEE Signal Processing Magazine, 2010, 27(4): 113-119. [16] YALCIN A, REKIK I. A diagnostic unified classification model for classifying multi-sized and multi-modal brain graphs using graph alignment[J]. Journal of Neuroscience Methods, 2021, 348: 109014. [17] RONNEBERGER O, FISCHER P, BROX T. U-net: convolutional networks for biomedical image segmentation[C] //International Conference on Medical Image Computing and Computer-assisted Intervention. Cham, Switzerland: Springer International Publishing, 2015: 234-241. [18] FAN Fanda, HUANG Yunyou, WANG Lei, et al. A semantic-based medical image fusion approach[J/OL].(2019-12-11)[2022-03-16]. https://arxiv.org/abs/1906.00225. [19] SIMONYAN K, ZISSERMAN A. Very deep convol-utional networks for large-scale image recognition[J/OL].(2015-04-10)[2022-03-16]. https://arxiv.org/abs/1409.1556. [20] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C] //Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, USA: IEEE Press, 2016: 770-778. [21] HUANG Gao, LIU Zhuang, MAATEN L, et al. Densely connected convolutional networks[C] //Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, USA:IEEE Press, 2017: 4700-4708. [22] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. Imagenet classification with deep convolutional neural networks[J]. Advances in Neural Information Processing Systems, 2012, 25: 1097-1105. [23] ZHANG Li, XIANG Tao, GONG Shaogang. Learning a deep embedding model for zero-shot learning[C] //Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, USA: IEEE Press, 2017: 2021-2030. [24] LIU Xingbo, NIE Xiushan, TENG Junya, et al. Single-shot semantic matching network for moment localization in videos[J]. ACM Transactions on Multimedia Computing, Communications, and Applications(TOMM), 2021, 17(3): 1-14. [25] ZHANG Yu, LIU Yu, SUN Peng, et al. IFCNN: a general image fusion framework based on convolutional neural network[J]. Information Fusion, 2020, 54: 99-118. |
| [1] | 李常刚,李宝亮,曹永吉,王佳颖. 人工智能在电力系统潮流计算中的应用综述及展望[J]. 山东大学学报 (工学版), 2025, 55(5): 1-17. |
| [2] | 周群颖,隋家成,张继,王洪元. 基于自监督卷积和无参数注意力机制的工业品表面缺陷检测[J]. 山东大学学报 (工学版), 2025, 55(4): 40-47. |
| [3] | 薛冰冰,王勇,杨维浩,王川,于迪,王旭. 基于ETC收费数据的高速公路交通流数据修复及实时预测[J]. 山东大学学报 (工学版), 2025, 55(3): 58-71. |
| [4] | 董明书,陈俐企,马川义,张珠皓,孙仁娟,管延华,庄培芝. 沥青路面内部裂缝雷达图像智能判识算法研究[J]. 山东大学学报 (工学版), 2025, 55(3): 72-79. |
| [5] | 常新功,苏敏惠,周志刚. 基于进化集成的图神经网络解释方法[J]. 山东大学学报 (工学版), 2024, 54(4): 1-12. |
| [6] | 索大翔,李波. 基于Gromov-Wasserstein最优传输的输电线路小目标检测方法[J]. 山东大学学报 (工学版), 2024, 54(3): 22-29. |
| [7] | 宋辉,张轶哲,张功萱,孟元. 基于类权重和最小化预测熵的测试时集成方法[J]. 山东大学学报 (工学版), 2024, 54(3): 36-43. |
| [8] | 刘新,刘冬兰,付婷,王勇,常英贤,姚洪磊,罗昕,王睿,张昊. 基于联邦学习的时间序列预测算法[J]. 山东大学学报 (工学版), 2024, 54(3): 55-63. |
| [9] | 聂秀山,巩蕊,董飞,郭杰,马玉玲. 短视频场景分类方法综述[J]. 山东大学学报 (工学版), 2024, 54(3): 1-11. |
| [10] | 李璐,张志军,范钰敏,王星,袁卫华. 面向冷启动用户的元学习与图转移学习序列推荐[J]. 山东大学学报 (工学版), 2024, 54(2): 69-79. |
| [11] | 高泽文,王建,魏本征. 基于混合偏移轴向自注意力机制的脑胶质瘤分割算法[J]. 山东大学学报 (工学版), 2024, 54(2): 80-89. |
| [12] | 陈成,董永权,贾瑞,刘源. 基于交互序列特征相关性的可解释知识追踪[J]. 山东大学学报 (工学版), 2024, 54(1): 100-108. |
| [13] | 李家春,李博文,常建波. 一种高效且轻量的RGB单帧人脸反欺诈模型[J]. 山东大学学报 (工学版), 2023, 53(6): 1-7. |
| [14] | 王旭晴,魏伟波,杨光宇,宋金涛,吕婷,潘振宽. 基于算法展开的图像盲去模糊深度学习网络[J]. 山东大学学报 (工学版), 2023, 53(6): 35-46. |
| [15] | 王碧瑶,韩毅,崔航滨,刘毅超,任铭然,高维勇,陈姝廷,刘嘉巍,崔洋. 基于图像的道路语义分割检测方法[J]. 山东大学学报 (工学版), 2023, 53(5): 37-47. |
|
