Abstract: Large language model(LLM) has demonstrated remarkable generation and reasoning capabilities in terms of conversation,reasoning,and knowledge retention,but they still suffer from numerous limitations,such as generating answers containing hallucinations,relying on outdated parameterized knowledge,and poor model interpretation.RAG(Retrieval Augmented Generation) addresses these issues by integrating non-parametric data stores.However,directly integrating information retrieval or training these components end-to-end often leads to suboptimal results or computational inefficiencies.To this end,this paper proposes the LSQ-RAG framework,which enhances the contextual understanding capabilities of the LLM with an LLM-supervised ranker and improves the quality and accuracy of the provided passages.LSQ-RAG fine-tunes the LLM to follow instructions and discriminatively use the provided information.Subsequently,it leverages the fine-tuned LLM to generate ranking scores,which serve as a supervisory signal for training the ranker.By leveraging the power of the LLM,the proposed approach eliminates the reliance on manual annotations in ranker training while achieving higher performance.Experimental results demonstrate that LSQ-RAG outperforms existing retrieval-augmented LLMs on open-domain QA and fact-checking tasks,while exhibiting consis-tent performance improvements when applied to different LLMs,highlighting its versatility and effectiveness.

Key words: LSQ-RAG framework, Retrieval-enhanced generation, Instruction fine-tuning, Ranker, Large language model