Computer Science ›› 2019, Vol. 46 ›› Issue (7): 238-245.doi: 10.11896/j.issn.1002-137X.2019.07.036

• Graphics, Image & Pattern Recognition • Previous Articles     Next Articles

Multi-modal Medical Image Fusion Based on Joint Patch Clustering of Adaptive Dictionary Learning

ANG Li-fang,SHI Chao-yu,LIN Su-zhen,QIN Pin-le,GAO Yuan   

  1. (The Key Laboratory of Biomedical Imaging and Imaging on Big Data,North University of China,Taiyuan 030051,China)
  • Received:2018-06-04 Online:2019-07-15 Published:2019-07-15

Abstract: In view of the poor image reconstruction quality problem caused by a large amount of redundant information existing in over complete adaptive dictionaries in medical image fusion,this paper proposed a multi-modal medical image fusion method based on joint image patch clustering and adaptive dictionary learning.First,this method calculates the Euclidean distance of image patches and reduces redundant image patches by comparing the cut-off threshold and the minimum distance of image patches.Then,it extracts the local gradient information of image patches as the clustering center by local regression weight of steering kernel (SKR),and combines the two different modal image patches with the same local gradient information for image patch clustering.On the basis of joint image patch clustering,it uses the improved K-SVD algorithm to train the clusters formed by image patch clustering to get sub-dictionaries,and merges the sub-dictionaries into an adaptive dictionary.Finally,the sparse representation coefficients can be obtained by the orthogonal matching tracking algorithm (OMP) and the adaptive dictionary,and they are fused with the rule of “2-norm max”.Through the reconstruction,this paper obtained the fused image.Compared with two methods based on multi-scale transform and six methods based on sparse representation,experimental results show that the proposed method can construct a compact and informative dictionary,and endow the fused image with higher clarity and stronger contrast to facilitate clinical diagnosis and adjuvant treatment.

Key words: Multi-model, Medical image fusion, Spares representation, Image patch clustering, Adaptive dictionary lear-ning

CLC Number: 

  • TP391.41
[1] ZHANG Q,LIU Y,BLUM R S,et al.Sparse Representation based Multi-sensor Image Fusion for Multi-focus and Multi-modality Images:A Review[J].Information Fusion,2017,40(2018):57-75.
[2] LIU Y.Research on methods for pixel-level multi-source image fusion[D].Hefei:University of Science and Technology of China,2016.(in Chinese)刘羽.像素级多源图像融合方法研究[D].合肥:中国科学技术大学,2016.
[3] ASIF U,BENNAMOUN M,SOHEL F.A model-free approach for the segmentation of unknown objects[C]∥IEEE/RSJ International Conference on Intelligent Robots and Systems.IEEE,2014:4914-4921.
[4] ZHU P,ZHU W,HU Q,et al.Subspace clustering guided unsupervised feature selection[J].Pattern Recognition,2017,66(C):364-374.
[5] YIN H,LI Y,CHAI Y,et al.A novel sparse-representation- based multi-focus image fusion approach[J].Neurocomputing,2016,216(C):216-229.
[6] ZONG J,QIU T.Medical image fusionbased on sparse representation of classifiedimage patches[J].Biomedical Signal Proces-sing and Control,2017,34(6):195-205.
[7] ZHU Z,QI G,CHAI Y,et al.A Geometric Dictionary Learning Based Approach for Fluorescence Spectroscopy Image Fusion[J].Applied Sciences,2017,7(2):161-178.
[8] WANG K,QI G,ZHU Z,et al.A Novel Geometric Dictionary Construction Approach for Sparse Representation Based Image Fusion[J].Entropy,2017,19(7):306-324.
[9] YIN F,GAO W,SONG Z.Medical Image Fusion Based on Feature Extraction and Sparse Representation[J].International journal of biomedical imaging,2017(5):11-34.
[10] KIM M,HAN D K,KO H.Joint patch clustering-based dictio- nary learning for multimodal image fusion[J].Information Fusion,2015,27(C):198-214.
[11] ZHU Z,CHAI Y,YIN H,et al.A novel dictionary learning approach for multi-modality medical image fusion[J].Neurocomputing,2016,214:471-482.
[12] TAKEDA H,FARSIU S,MILANFAR P.Kernel regression for image processing and reconstruction[J].IEEE Transactions on Image Processing,2007,16(2):349-366.
[13] AGARWAL A,ANANDKUMAR A,NETRAPALLI P.A Clustering Approach to Learning Sparsely Used Overcomplete Dictionaries[J].IEEE Transactions on Information Theory,2016,63(1):575-592.
[14] YEGANLI F,NAZZAL M,OZKARAMANLI H.Image super-resolution via sparse representation over multiple learned dictionaries based on edge sharpness and gradient phase angle[J].Signal Image & Video Processing,2015,9(1):285-293.
[15] ZHANG Q,LIU Y,BLUM R S,et al.Sparse Representation based Multi-sensor Image Fusion for Multi-focus and Multi-modality Images:A Review[J].Information Fusion,2017,9(1):25-33.
[16] LIU J W,CUI L P,LUO X L.Structured sparse models[J].Chi- nese Journal of Computers,2017,40(6):1309-1337.(in Chinese)刘建伟,崔立鹏,罗雄麟.结构稀疏模型[J].计算机学报,2017,40(6):1309-1337.
[17] JAVED U,RIAZ M M,GHAFOOR A,et al.MRI and PET ima ge fusion using fuzzy logic and image local features[J].The Scientific World Journal,2014,2014(2014):8-36.
[18] SUN J G,LIU J,ZHAO L Y.Clustering algorithms research [J].Journal of Software,2008,19(1):48-61.(in Chinese)孙吉贵,刘杰,赵连宇.聚类算法研究[J].软件学报,2008,19(1):48-61.
[19] ZHANG X L,LI X F,LI J.Validation and Correlation Analysis of Metrics for Evaluating Performance of Image Fusion[J].Acta automatica scinica,2014,40(2):306-315.(in Chinese)张小利,李雄飞,李军.融合图像质量评价指标的相关性分析及性能评估[J].自动化学报,2014,40(2):306-315.
[20] LIU Z,BLASCH E,XUE Z,et al.Objective Assessment of Multiresolution Image Fusion Algorithms for Context Enhancement in Night Vision:A Comparative Study[J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2011,34(1):94-109.
[21] JOHNSON K A,BECKER J A.The whole brain atlas [EB/OL].[2016-10-09].http:// /aanlib/home.html.
[1] LI Chang-xing, LEI Liu, ZHANG Xiao-lu. Brain CT and MRI Image Fusion Based on Morphological Image Enhancement and PCNN [J]. Computer Science, 2020, 47(10): 194-199.
[2] LI Heng-chao, LIN Hong-fei, YANG Liang, XU Bo, WEI Xiao-cong, ZHANG Shao-wu and Gulziya ANIWAR. Two-level Stacking Algorithm Framework for Building User Portrait [J]. Computer Science, 2018, 45(1): 157-161.
[3] . Method of Medical Image Fusion Based on Nonsubsampled Contourlet Transform [J]. Computer Science, 2013, 40(3): 310-313.
[4] . Medical Image Fusion Method Based on Lifting Wavelet Transform [J]. Computer Science, 2011, 38(12): 266-268.
Full text



[1] LEI Li-hui and WANG Jing. Parallelization of LTL Model Checking Based on Possibility Measure[J]. Computer Science, 2018, 45(4): 71 -75 .
[2] SUN Qi, JIN Yan, HE Kun and XU Ling-xuan. Hybrid Evolutionary Algorithm for Solving Mixed Capacitated General Routing Problem[J]. Computer Science, 2018, 45(4): 76 -82 .
[3] ZHANG Jia-nan and XIAO Ming-yu. Approximation Algorithm for Weighted Mixed Domination Problem[J]. Computer Science, 2018, 45(4): 83 -88 .
[4] WU Jian-hui, HUANG Zhong-xiang, LI Wu, WU Jian-hui, PENG Xin and ZHANG Sheng. Robustness Optimization of Sequence Decision in Urban Road Construction[J]. Computer Science, 2018, 45(4): 89 -93 .
[5] SHI Wen-jun, WU Ji-gang and LUO Yu-chun. Fast and Efficient Scheduling Algorithms for Mobile Cloud Offloading[J]. Computer Science, 2018, 45(4): 94 -99 .
[6] ZHOU Yan-ping and YE Qiao-lin. L1-norm Distance Based Least Squares Twin Support Vector Machine[J]. Computer Science, 2018, 45(4): 100 -105 .
[7] LIU Bo-yi, TANG Xiang-yan and CHENG Jie-ren. Recognition Method for Corn Borer Based on Templates Matching in Muliple Growth Periods[J]. Computer Science, 2018, 45(4): 106 -111 .
[8] GENG Hai-jun, SHI Xin-gang, WANG Zhi-liang, YIN Xia and YIN Shao-ping. Energy-efficient Intra-domain Routing Algorithm Based on Directed Acyclic Graph[J]. Computer Science, 2018, 45(4): 112 -116 .
[9] CUI Qiong, LI Jian-hua, WANG Hong and NAN Ming-li. Resilience Analysis Model of Networked Command Information System Based on Node Repairability[J]. Computer Science, 2018, 45(4): 117 -121 .
[10] WANG Zhen-chao, HOU Huan-huan and LIAN Rui. Path Optimization Scheme for Restraining Degree of Disorder in CMT[J]. Computer Science, 2018, 45(4): 122 -125 .