RaFe is a framework for training query rewriting models without requiring annotations. It leverages a publicly available reranker to provide feedback aligned with the rewriting objectives. By using ranking scores from the reranker, RaFe conducts feedback training on the query rewriting model. The framework consists of two stages: initial supervised fine-tuning and feedback training. RaFe supports both offline and online feedback training. Experimental results show that RaFe outperforms baselines in query rewriting tasks. The method does not require annotated labels or domain-specific scores, ensuring generalizability. RaFe is validated on cross-lingual datasets and demonstrates effectiveness in improving query rewriting. The framework utilizes reranker feedback to enhance the performance of query rewriting models, achieving significant improvements in retrieval and question-answering metrics. RaFe is shown to be effective in both Substitute and Expand settings, with the best performance in the Expand-Ranked setting. The method is compared with other feedback types, and RaFe outperforms them in terms of performance and cost efficiency. The results indicate that RaFe is a promising approach for improving query rewriting in RAG systems.RaFe is a framework for training query rewriting models without requiring annotations. It leverages a publicly available reranker to provide feedback aligned with the rewriting objectives. By using ranking scores from the reranker, RaFe conducts feedback training on the query rewriting model. The framework consists of two stages: initial supervised fine-tuning and feedback training. RaFe supports both offline and online feedback training. Experimental results show that RaFe outperforms baselines in query rewriting tasks. The method does not require annotated labels or domain-specific scores, ensuring generalizability. RaFe is validated on cross-lingual datasets and demonstrates effectiveness in improving query rewriting. The framework utilizes reranker feedback to enhance the performance of query rewriting models, achieving significant improvements in retrieval and question-answering metrics. RaFe is shown to be effective in both Substitute and Expand settings, with the best performance in the Expand-Ranked setting. The method is compared with other feedback types, and RaFe outperforms them in terms of performance and cost efficiency. The results indicate that RaFe is a promising approach for improving query rewriting in RAG systems.