Abstract: Medical images play a vital role in medical diagnosis.Accurately described text reports are essential for understanding images and subsequent disease diagnosis.In recent years,the generation of standardized reports based on modeling methods has become a research hotspot in the field of medical imaging report generation.However,due to the data deviation problem caused by the large gap between positive and negative samples,the content of the generated report generally tends to describe the normal situation.This limitation creates challenges in accurately capturing abnormal information.To address this issue,this paper proposes a novel approach based on diversified label matrix for medical report generation.This method utilizes a diverse label matrix to perform differential learning on different diseases and generate diverse medical reports.Additionally,a text-matrix feature loss function is designed to optimize the diverse label matrix,enhancing its effectiveness.Furthermore,the Transformer network is enhanced by incorporating a feature intersection module.This module strengthens the mapping between images and text,and improves accuracy in disease description.Experimental results on the two datasets of IU-X-Ray and MIMIC-CXR show that,the proposed method achieves the best results in multiple indicators,such as BLEU and METEOR,compared with the current mainstream methods.

Key words: Deep learning, Medical report generation, Attention mechanism, Image-Text generation, Multi-modal