Abstract: Under unfavorable circumstances,the fused image of the infrared and visible images sometimes suffers from low brightness and insufficient details.Therefore,a novel infrared and visible deep unfolding image fusion network based on joint enhancement image pairs is proposed.To increase input information,both the original infrared/visible image pair and their enhancement pair are used as deep network's input.Firstly,an iterative residual unfolding convolutional network based on deep residual unfolding module is developed to obtain the background features or detail features according to different initialization network parameters.Then,concatenate operation and up-down sampling pair are introduced to the convolutional feature fusion network,where features of the corresponding enhancement image pairs can be added to fusion task and the discrepant features of raw images are maximumly retained.Meanwhile,the loss function is optimized to obtain better results.Numerous experiments on multiple datasets demonstrate that the proposed method can get competitive fusion images both in terms of subjective evaluation and objective metrics,and have better performance under low light environments.

Key words: Image fusion, Deep algorithm unrolling network, Image enhancement, Feature extraction, Feature fusion