BIDER is a method that refines retrieval documents into Key Supporting Evidence (KSE) to improve the performance of retrieval-augmented large language models (LLMs). The approach involves three stages: knowledge synthesis, supervised fine-tuning, and preference alignment. Knowledge synthesis involves extracting, refining, and cleaning nuggets from retrieved documents to identify essential information. Supervised fine-tuning trains a model to map retrieved documents to KSE, while preference alignment uses reinforcement learning to align the model with the preferences of downstream LLMs. Evaluations on five datasets show that BIDER improves LLM answer quality by 7% while reducing input content length by 80%, outperforming existing methods. The proposed KSE simulation effectively equips LLMs with essential information for accurate question answering. BIDER addresses the issue of knowledge inconsistency between retrieval results and the knowledge required by LLMs for answering questions. The method reduces noise in retrieved documents, enhances the quality of generated answers, and improves the efficiency of retrieval-augmented LLMs. The approach is evaluated on three types of knowledge-intensive tasks: open-domain QA, dialogue generation, and fact verification. Results show that BIDER achieves better generation performance while reducing input information length by 80%, effectively condensing retrieved documents. The method is robust under various text retrieval quality conditions and demonstrates significant improvements in answer quality. The main contributions of this work include proposing a three-step knowledge synthesis method to generate oracle KSE, introducing a method to refine retrieval documents into KSE, and training the refiner model using supervised distillation and preference alignment techniques to enhance RAG performance during inference.BIDER is a method that refines retrieval documents into Key Supporting Evidence (KSE) to improve the performance of retrieval-augmented large language models (LLMs). The approach involves three stages: knowledge synthesis, supervised fine-tuning, and preference alignment. Knowledge synthesis involves extracting, refining, and cleaning nuggets from retrieved documents to identify essential information. Supervised fine-tuning trains a model to map retrieved documents to KSE, while preference alignment uses reinforcement learning to align the model with the preferences of downstream LLMs. Evaluations on five datasets show that BIDER improves LLM answer quality by 7% while reducing input content length by 80%, outperforming existing methods. The proposed KSE simulation effectively equips LLMs with essential information for accurate question answering. BIDER addresses the issue of knowledge inconsistency between retrieval results and the knowledge required by LLMs for answering questions. The method reduces noise in retrieved documents, enhances the quality of generated answers, and improves the efficiency of retrieval-augmented LLMs. The approach is evaluated on three types of knowledge-intensive tasks: open-domain QA, dialogue generation, and fact verification. Results show that BIDER achieves better generation performance while reducing input information length by 80%, effectively condensing retrieved documents. The method is robust under various text retrieval quality conditions and demonstrates significant improvements in answer quality. The main contributions of this work include proposing a three-step knowledge synthesis method to generate oracle KSE, introducing a method to refine retrieval documents into KSE, and training the refiner model using supervised distillation and preference alignment techniques to enhance RAG performance during inference.