This paper advocates for retrieval-augmented language models (RAGs) as the next generation of language models, arguing that they can address key limitations of parametric LMs, such as factual errors, lack of verifiability, and difficulty in adapting to new data distributions. RAGs incorporate external datastores during inference, allowing them to be more reliable, adaptable, and attributable. Despite their potential, RAGs have not been widely adopted due to challenges in retrieval effectiveness, limited interaction between retrieval and LM components, and insufficient infrastructure for efficient training and inference. The paper outlines a roadmap for developing general-purpose RAGs, emphasizing the need to reconsider datastores and retrievers, improve retriever-LM interaction, and invest in infrastructure for efficient training and inference. It identifies three main challenges: (C1) limitations in retrievers and datastores, (C2) limited interactions between retrievers and LMs, and (C3) lack of specialized infrastructure for RAGs. RAGs can reduce factual errors, provide better attributions, enable flexible sequence opt-in and out, improve adaptability, and be more parameter-efficient than parametric LMs. However, current RAGs face challenges in retrieving helpful text, shallow interactions between retrieval and LM components, and scalability issues. The paper suggests future research directions, including rethinking retrieval and datastores, enhancing retriever-LM interactions, and building better systems and infrastructure for scaling and adaptation. The paper concludes that RAGs have the potential to address fundamental limitations of parametric LMs, but their adoption remains limited due to technical challenges. The authors call for collaborative interdisciplinary efforts to advance RAGs and overcome these challenges.This paper advocates for retrieval-augmented language models (RAGs) as the next generation of language models, arguing that they can address key limitations of parametric LMs, such as factual errors, lack of verifiability, and difficulty in adapting to new data distributions. RAGs incorporate external datastores during inference, allowing them to be more reliable, adaptable, and attributable. Despite their potential, RAGs have not been widely adopted due to challenges in retrieval effectiveness, limited interaction between retrieval and LM components, and insufficient infrastructure for efficient training and inference. The paper outlines a roadmap for developing general-purpose RAGs, emphasizing the need to reconsider datastores and retrievers, improve retriever-LM interaction, and invest in infrastructure for efficient training and inference. It identifies three main challenges: (C1) limitations in retrievers and datastores, (C2) limited interactions between retrievers and LMs, and (C3) lack of specialized infrastructure for RAGs. RAGs can reduce factual errors, provide better attributions, enable flexible sequence opt-in and out, improve adaptability, and be more parameter-efficient than parametric LMs. However, current RAGs face challenges in retrieving helpful text, shallow interactions between retrieval and LM components, and scalability issues. The paper suggests future research directions, including rethinking retrieval and datastores, enhancing retriever-LM interactions, and building better systems and infrastructure for scaling and adaptation. The paper concludes that RAGs have the potential to address fundamental limitations of parametric LMs, but their adoption remains limited due to technical challenges. The authors call for collaborative interdisciplinary efforts to advance RAGs and overcome these challenges.