基于联合图像块聚类自适应字典学习的多模态医学图像融合

doi:10.11896/j.issn.1002-137X.2019.07.036

计算机科学 ›› 2019, Vol. 46 ›› Issue (7): 238-245.doi: 10.11896/j.issn.1002-137X.2019.07.036

基于联合图像块聚类自适应字典学习的多模态医学图像融合

王丽芳,史超宇,蔺素珍,秦品乐,高媛

(中北大学山西省生物医学成像与影像大数据重点实验室太原030051)

收稿日期:2018-06-04 出版日期:2019-07-15 发布日期:2019-07-15
作者简介:王丽芳(1977-),女,博士,副教授,CCF会员,主要研究方向为机器视觉、大数据处理、医学图像处理,E-mail:wsm2004@nuc.edu.cn(通信作者);史超宇(1993-),男,硕士生,主要研究方向为医学图像融合、机器学习;蔺素珍(1966-),女,博士,教授,主要研究方向为图像处理、文物虚拟修复;秦品乐(1978-),男,博士,副教授,主要研究方向为机器视觉、大数据处理、三维重建;高媛(1972-),女,硕士,副教授,主要研究方向为机器视觉、大数据处理、三维重建。
基金资助:
山西省青年基金项目(201601D021080),中北大学研究生科技立项自然科学项目(20171441)资助

Multi-modal Medical Image Fusion Based on Joint Patch Clustering of Adaptive Dictionary Learning

ANG Li-fang,SHI Chao-yu,LIN Su-zhen,QIN Pin-le,GAO Yuan

(The Key Laboratory of Biomedical Imaging and Imaging on Big Data,North University of China,Taiyuan 030051,China)

Received:2018-06-04 Online:2019-07-15 Published:2019-07-15

摘要/Abstract

摘要： 针对多模态医学图像融合中过完备自适应字典存在的大量冗余信息会导致图像重建质量不佳的问题,文中提出了基于联合图像块聚类自适应字典学习的多模态医学图像融合方法(JCPD)。该方法首先计算图像块的欧氏距离,通过比较设定的阈值和图像块的最小距离来剔除冗余图像块,减少冗余图像块的数量。然后,使用局部调制核回归(SKR)提取图像块的局部梯度信息作为聚类中心,将具有相同局部梯度信息的两种模态的图像块进行联合图像块聚类。在联合图像块聚类的基础上使用改进的K-SVD算法对图像块聚类形成的类簇进行训练得到子字典,并将子字典合并成自适应字典。最后,在自适应字典的作用下用正交匹配追踪算法(OMP)计算得到稀疏表示系数,再使用“2范数最大”的规则融合稀疏系数,之后通过重建得到融合图像。实验表明,与2种基于多尺度变换的方法和6种基于稀疏表示的方法相比,所提方法在保证字典信息的完整性和字典的紧凑性基础上使得融合的图像清晰度更高、对比度更强,便于临床诊断和辅助治疗。

关键词: 多模态, 图像块聚类, 稀疏表示, 医学图像融合, 自适应字典学习

Abstract: In view of the poor image reconstruction quality problem caused by a large amount of redundant information existing in over complete adaptive dictionaries in medical image fusion,this paper proposed a multi-modal medical image fusion method based on joint image patch clustering and adaptive dictionary learning.First,this method calculates the Euclidean distance of image patches and reduces redundant image patches by comparing the cut-off threshold and the minimum distance of image patches.Then,it extracts the local gradient information of image patches as the clustering center by local regression weight of steering kernel (SKR),and combines the two different modal image patches with the same local gradient information for image patch clustering.On the basis of joint image patch clustering,it uses the improved K-SVD algorithm to train the clusters formed by image patch clustering to get sub-dictionaries,and merges the sub-dictionaries into an adaptive dictionary.Finally,the sparse representation coefficients can be obtained by the orthogonal matching tracking algorithm (OMP) and the adaptive dictionary,and they are fused with the rule of “2-norm max”.Through the reconstruction,this paper obtained the fused image.Compared with two methods based on multi-scale transform and six methods based on sparse representation,experimental results show that the proposed method can construct a compact and informative dictionary,and endow the fused image with higher clarity and stronger contrast to facilitate clinical diagnosis and adjuvant treatment.

Key words: Adaptive dictionary lear-ning, Image patch clustering, Medical image fusion, Multi-model, Spares representation

中图分类号:

TP391.41

王丽芳, 史超宇, 蔺素珍, 秦品乐, 高媛. 基于联合图像块聚类自适应字典学习的多模态医学图像融合[J]. 计算机科学, 2019, 46(7): 238-245. https://doi.org/10.11896/j.issn.1002-137X.2019.07.036

ANG Li-fang, SHI Chao-yu, LIN Su-zhen, QIN Pin-le, GAO Yuan. Multi-modal Medical Image Fusion Based on Joint Patch Clustering of Adaptive Dictionary Learning[J]. Computer Science, 2019, 46(7): 238-245. https://doi.org/10.11896/j.issn.1002-137X.2019.07.036

参考文献

[1]ZHANG Q,LIU Y,BLUM R S,et al.Sparse Representation based Multi-sensor Image Fusion for Multi-focus and Multi-modality Images:A Review[J].Information Fusion,2017,40(2018):57-75.
[2]LIU Y.Research on methods for pixel-level multi-source image fusion[D].Hefei:University of Science and Technology of China,2016.(in Chinese)
刘羽.像素级多源图像融合方法研究[D].合肥:中国科学技术大学,2016.
[3]ASIF U,BENNAMOUN M,SOHEL F.A model-free approach for the segmentation of unknown objects[C]∥IEEE/RSJ International Conference on Intelligent Robots and Systems.IEEE,2014:4914-4921.
[4]ZHU P,ZHU W,HU Q,et al.Subspace clustering guided unsupervised feature selection[J].Pattern Recognition,2017,66(C):364-374.
[5]YIN H,LI Y,CHAI Y,et al.A novel sparse-representation- based multi-focus image fusion approach[J].Neurocomputing,2016,216(C):216-229.
[6]ZONG J,QIU T.Medical image fusionbased on sparse representation of classifiedimage patches[J].Biomedical Signal Proces-sing and Control,2017,34(6):195-205.
[7]ZHU Z,QI G,CHAI Y,et al.A Geometric Dictionary Learning Based Approach for Fluorescence Spectroscopy Image Fusion[J].Applied Sciences,2017,7(2):161-178.
[8]WANG K,QI G,ZHU Z,et al.A Novel Geometric Dictionary Construction Approach for Sparse Representation Based Image Fusion[J].Entropy,2017,19(7):306-324.
[9]YIN F,GAO W,SONG Z.Medical Image Fusion Based on Feature Extraction and Sparse Representation[J].International journal of biomedical imaging,2017(5):11-34.
[10]KIM M,HAN D K,KO H.Joint patch clustering-based dictio- nary learning for multimodal image fusion[J].Information Fusion,2015,27(C):198-214.
[11]ZHU Z,CHAI Y,YIN H,et al.A novel dictionary learning approach for multi-modality medical image fusion[J].Neurocomputing,2016,214:471-482.
[12]TAKEDA H,FARSIU S,MILANFAR P.Kernel regression for image processing and reconstruction[J].IEEE Transactions on Image Processing,2007,16(2):349-366.
[13]AGARWAL A,ANANDKUMAR A,NETRAPALLI P.A Clustering Approach to Learning Sparsely Used Overcomplete Dictionaries[J].IEEE Transactions on Information Theory,2016,63(1):575-592.
[14]YEGANLI F,NAZZAL M,OZKARAMANLI H.Image super-resolution via sparse representation over multiple learned dictionaries based on edge sharpness and gradient phase angle[J].Signal Image & Video Processing,2015,9(1):285-293.
[15]ZHANG Q,LIU Y,BLUM R S,et al.Sparse Representation based Multi-sensor Image Fusion for Multi-focus and Multi-modality Images:A Review[J].Information Fusion,2017,9(1):25-33.
[16]LIU J W,CUI L P,LUO X L.Structured sparse models[J].Chi- nese Journal of Computers,2017,40(6):1309-1337.(in Chinese)
刘建伟,崔立鹏,罗雄麟.结构稀疏模型[J].计算机学报,2017,40(6):1309-1337.
[17]JAVED U,RIAZ M M,GHAFOOR A,et al.MRI and PET ima ge fusion using fuzzy logic and image local features[J].The Scientific World Journal,2014,2014(2014):8-36.
[18]SUN J G,LIU J,ZHAO L Y.Clustering algorithms research [J].Journal of Software,2008,19(1):48-61.(in Chinese)
孙吉贵,刘杰,赵连宇.聚类算法研究[J].软件学报,2008,19(1):48-61.
[19]ZHANG X L,LI X F,LI J.Validation and Correlation Analysis of Metrics for Evaluating Performance of Image Fusion[J].Acta automatica scinica,2014,40(2):306-315.(in Chinese)
张小利,李雄飞,李军.融合图像质量评价指标的相关性分析及性能评估[J].自动化学报,2014,40(2):306-315.
[20]LIU Z,BLASCH E,XUE Z,et al.Objective Assessment of Multiresolution Image Fusion Algorithms for Context Enhancement in Night Vision:A Comparative Study[J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2011,34(1):94-109.
[21]JOHNSON K A,BECKER J A.The whole brain atlas [EB/OL].[2016-10-09].http:// www.med.harvard.edu /aanlib/home.html.

相关文章 15

[1]	聂秀山, 潘嘉男, 谭智方, 刘新放, 郭杰, 尹义龙. 基于自然语言的视频片段定位综述 Overview of Natural Language Video Localization 计算机科学, 2022, 49(9): 111-122. https://doi.org/10.11896/jsjkx.220500130
[2]	周旭, 钱胜胜, 李章明, 方全, 徐常胜. 基于对偶变分多模态注意力网络的不完备社会事件分类方法 Dual Variational Multi-modal Attention Network for Incomplete Social Event Classification 计算机科学, 2022, 49(9): 132-138. https://doi.org/10.11896/jsjkx.220600022
[3]	常炳国, 石华龙, 常雨馨. 基于深度学习的黑色素瘤智能诊断多模型算法 Multi Model Algorithm for Intelligent Diagnosis of Melanoma Based on Deep Learning 计算机科学, 2022, 49(6A): 22-26. https://doi.org/10.11896/jsjkx.210500197
[4]	李浩东, 胡洁, 范勤勤. 基于并行分区搜索的多模态多目标优化及其应用 Multimodal Multi-objective Optimization Based on Parallel Zoning Search and Its Application 计算机科学, 2022, 49(5): 212-220. https://doi.org/10.11896/jsjkx.210300019
[5]	赵亮, 张洁, 陈志奎. 基于双图正则化的自适应多模态鲁棒特征学习 Adaptive Multimodal Robust Feature Learning Based on Dual Graph-regularization 计算机科学, 2022, 49(4): 124-133. https://doi.org/10.11896/jsjkx.210300078
[6]	刘创, 熊德意. 多语言问答研究综述 Survey of Multilingual Question Answering 计算机科学, 2022, 49(1): 65-72. https://doi.org/10.11896/jsjkx.210900003
[7]	陈志毅, 隋杰. 基于DeepFM和卷积神经网络的集成式多模态谣言检测方法 DeepFM and Convolutional Neural Networks Ensembles for Multimodal Rumor Detection 计算机科学, 2022, 49(1): 101-107. https://doi.org/10.11896/jsjkx.201200007
[8]	袁景凌, 丁远远, 盛德明, 李琳. 基于视觉方面注意力的图像文本情感分析模型 Image-Text Sentiment Analysis Model Based on Visual Aspect Attention 计算机科学, 2022, 49(1): 219-224. https://doi.org/10.11896/jsjkx.201000074
[9]	张晓宇, 王彬, 安卫超, 阎婷, 相洁. 基于融合损失函数的3D U-Net++脑胶质瘤分割网络 Glioma Segmentation Network Based on 3D U-Net＋+ with Fusion Loss Function 计算机科学, 2021, 48(9): 187-193. https://doi.org/10.11896/jsjkx.200800099
[10]	周新民, 胡宜桂, 刘文洁, 孙荣俊. 基于多模态多层级数据融合方法的城市功能识别研究 Research on Urban Function Recognition Based on Multi-modal and Multi-level Data Fusion Method 计算机科学, 2021, 48(9): 50-58. https://doi.org/10.11896/jsjkx.210500220
[11]	孙圣姿, 郭炳晖, 杨小博. 用于多模态语义分析的嵌入共识自动编码器 Embedding Consensus Autoencoder for Cross-modal Semantic Analysis 计算机科学, 2021, 48(7): 93-98. https://doi.org/10.11896/jsjkx.200600003
[12]	张少钦, 杜圣东, 张晓博, 李天瑞. 融合多模态信息的社交网络谣言检测方法 Social Rumor Detection Method Based on Multimodal Fusion 计算机科学, 2021, 48(5): 117-123. https://doi.org/10.11896/jsjkx.200400057
[13]	武阿明, 姜品, 韩亚洪. 基于视觉和语言的跨媒体问答与推理研究综述 Survey of Cross-media Question Answering and Reasoning Based on Vision and Language 计算机科学, 2021, 48(3): 71-78. https://doi.org/10.11896/jsjkx.201100176
[14]	王树徽, 闫旭, 黄庆明. 跨媒体分析与推理技术研究综述 Overview of Research on Cross-media Analysis and Reasoning Technology 计算机科学, 2021, 48(3): 79-86. https://doi.org/10.11896/jsjkx.210200086
[15]	钱胜胜, 张天柱, 徐常胜. 多媒体社会事件分析综述 Survey of Multimedia Social Events Analysis 计算机科学, 2021, 48(3): 97-112. https://doi.org/10.11896/jsjkx.210200023

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

基于联合图像块聚类自适应字典学习的多模态医学图像融合

Multi-modal Medical Image Fusion Based on Joint Patch Clustering of Adaptive Dictionary Learning

PDF (PC)

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

Metrics

本文评价

推荐阅读 0