This survey explores retrieval-based in-context learning (RetICL) for language models, focusing on how retrieval of tailored demonstrations improves performance compared to fixed or randomly selected ones. RetICL leverages retrieval systems to dynamically select demonstrations relevant to the input query, enhancing efficiency, scalability, and reducing biases. The survey discusses various design choices for retrieval models, training procedures, and inference algorithms, highlighting the importance of relevance and usefulness of retrieved demonstrations. RetICL offers several advantages over traditional fine-tuning methods, including reduced computational cost, no parameter changes, and better generalization. It also avoids issues like overfitting and requires fewer resources. The effectiveness of RetICL depends on factors such as the quality, quantity, and ordering of demonstrations. Different retrieval objectives, such as similarity and diversity, are explored, along with strategies like clustering, iterative retrieval, and using diverse demonstrations. The survey also examines various retrieval corpus types, including in-domain, mix-domain, cross-domain, and unlabelled queries with automatically generated answers. Off-the-shelf retrievers like BM25 and sentence embedding models are evaluated, while fine-tuned retrievers show superior performance but require more data and computational resources. The survey highlights the importance of training objectives, such as list-wise ranking loss, InfoNCE loss, and diversity training, in improving retrieval effectiveness. Applications of RetICL span natural language understanding, reasoning, knowledge-based QA, and text generation. It has shown significant improvements in tasks like sentiment analysis, reading comprehension, and mathematical reasoning. The survey concludes that while off-the-shelf retrievers are effective, fine-tuned retrievers offer better performance, though at a higher cost. Overall, RetICL represents a promising approach for enhancing language model performance through dynamic, context-sensitive demonstration retrieval.This survey explores retrieval-based in-context learning (RetICL) for language models, focusing on how retrieval of tailored demonstrations improves performance compared to fixed or randomly selected ones. RetICL leverages retrieval systems to dynamically select demonstrations relevant to the input query, enhancing efficiency, scalability, and reducing biases. The survey discusses various design choices for retrieval models, training procedures, and inference algorithms, highlighting the importance of relevance and usefulness of retrieved demonstrations. RetICL offers several advantages over traditional fine-tuning methods, including reduced computational cost, no parameter changes, and better generalization. It also avoids issues like overfitting and requires fewer resources. The effectiveness of RetICL depends on factors such as the quality, quantity, and ordering of demonstrations. Different retrieval objectives, such as similarity and diversity, are explored, along with strategies like clustering, iterative retrieval, and using diverse demonstrations. The survey also examines various retrieval corpus types, including in-domain, mix-domain, cross-domain, and unlabelled queries with automatically generated answers. Off-the-shelf retrievers like BM25 and sentence embedding models are evaluated, while fine-tuned retrievers show superior performance but require more data and computational resources. The survey highlights the importance of training objectives, such as list-wise ranking loss, InfoNCE loss, and diversity training, in improving retrieval effectiveness. Applications of RetICL span natural language understanding, reasoning, knowledge-based QA, and text generation. It has shown significant improvements in tasks like sentiment analysis, reading comprehension, and mathematical reasoning. The survey concludes that while off-the-shelf retrievers are effective, fine-tuned retrievers offer better performance, though at a higher cost. Overall, RetICL represents a promising approach for enhancing language model performance through dynamic, context-sensitive demonstration retrieval.