RaFe: Ranking Feedback Improves Query Rewriting for RAG **Abstract:** As Large Language Models (LLMs) and Retrieval Augmentation Generation (RAG) techniques have evolved, query rewriting has been widely incorporated into RAG systems for downstream tasks like open-domain QA. Many works have attempted to use small models with reinforcement learning instead of costly LLMs to improve query rewriting. However, current methods often require annotations or pre-designed rewards, lacking generalization and failing to utilize signals tailored for query rewriting. This paper proposes RaFe, a framework for training query rewriting models without annotations. By leveraging a publicly available reranker, RaFe provides feedback aligned with the rewriting objectives. Experimental results demonstrate that RaFe can achieve better performance than baselines. **Introduction:** Query rewriting is crucial for enhancing RAG systems by expanding retrieved documents. While many methods use LLMs to generate rewrites, practical applications often prefer smaller models to avoid costly LLM usage. Reinforcement learning with feedback is a common solution, but it often relies on annotated labels or pre-designed rewards. RaFe addresses these issues by using feedback from a reranker, which scores and sorts retrieved documents based on the query. This feedback is aligned with the query rewriting objective, making it effective and generalizable. **Method:** RaFe consists of two stages: initial supervised fine-tuning and feedback training. The initial model is trained using standard supervised fine-tuning, and then feedback training is conducted using ranking scores from the reranker. RaFe supports both offline and online RL feedback training. Empirical results show that RaFe can drive the training of the query rewriting model, demonstrating its effectiveness and generalizability. **Experimental Setup:** Experiments are conducted on cross-lingual datasets using Open-Domain Question Answering (ODQA). The evaluation settings include SUBSTITUTE and Expand, where the former uses documents retrieved by rewritten queries, and the latter uses both original and rewritten queries. Results show that RaFe outperforms baselines in most settings, especially in the EXPAND-Ranked setting, where it achieves significant improvements. **Conclusion:** RaFe is a novel framework for query rewriting that leverages reranker feedback, improving the retrieval of relevant documents. Experimental results validate its effectiveness and generalizability across cross-lingual datasets. Future work includes joint training of rerankers and rewrite models to further enhance RAG performance.RaFe: Ranking Feedback Improves Query Rewriting for RAG **Abstract:** As Large Language Models (LLMs) and Retrieval Augmentation Generation (RAG) techniques have evolved, query rewriting has been widely incorporated into RAG systems for downstream tasks like open-domain QA. Many works have attempted to use small models with reinforcement learning instead of costly LLMs to improve query rewriting. However, current methods often require annotations or pre-designed rewards, lacking generalization and failing to utilize signals tailored for query rewriting. This paper proposes RaFe, a framework for training query rewriting models without annotations. By leveraging a publicly available reranker, RaFe provides feedback aligned with the rewriting objectives. Experimental results demonstrate that RaFe can achieve better performance than baselines. **Introduction:** Query rewriting is crucial for enhancing RAG systems by expanding retrieved documents. While many methods use LLMs to generate rewrites, practical applications often prefer smaller models to avoid costly LLM usage. Reinforcement learning with feedback is a common solution, but it often relies on annotated labels or pre-designed rewards. RaFe addresses these issues by using feedback from a reranker, which scores and sorts retrieved documents based on the query. This feedback is aligned with the query rewriting objective, making it effective and generalizable. **Method:** RaFe consists of two stages: initial supervised fine-tuning and feedback training. The initial model is trained using standard supervised fine-tuning, and then feedback training is conducted using ranking scores from the reranker. RaFe supports both offline and online RL feedback training. Empirical results show that RaFe can drive the training of the query rewriting model, demonstrating its effectiveness and generalizability. **Experimental Setup:** Experiments are conducted on cross-lingual datasets using Open-Domain Question Answering (ODQA). The evaluation settings include SUBSTITUTE and Expand, where the former uses documents retrieved by rewritten queries, and the latter uses both original and rewritten queries. Results show that RaFe outperforms baselines in most settings, especially in the EXPAND-Ranked setting, where it achieves significant improvements. **Conclusion:** RaFe is a novel framework for query rewriting that leverages reranker feedback, improving the retrieval of relevant documents. Experimental results validate its effectiveness and generalizability across cross-lingual datasets. Future work includes joint training of rerankers and rewrite models to further enhance RAG performance.