**LitLLM: A Toolkit for Scientific Literature Review** **Authors:** Shubham Agarwal, Issam H. Laradj, Laurent Charlin, Christopher Pal **Institutional Affiliations:** ServiceNow Research, Mila - Quebec AI Institute, HEC Montreal, UBC, Canada CIFAR AI Chair **Abstract:** Conducting literature reviews is crucial for understanding research limitations and building on existing work. However, it is a tedious task, making automatic literature review generation appealing. Existing methods using Large Language Models (LLMs) often hallucinate and ignore recent research. To address these limitations, the authors propose LitLLM, a toolkit based on Retrieval Augmented Generation (RAG) principles. The system first summarizes user-provided abstracts into keywords using an off-the-shelf LLM, then retrieves relevant papers, re-ranks them based on the user-provided abstract, and generates the related work section. This approach significantly reduces the time and effort required for literature reviews, making LitLLM an efficient alternative. The open-source toolkit is available at <https://github.com/shubhamagarwal192/LitLLM> and Huggingface space (<https://huggingface.co/spaces/shubhamagarwal192/LitLLM>). **Introduction:** Scientists use NLP systems like search engines to find relevant papers. Recent advances in LLMs have led to systems that assist with literature reviews, such as Explainpaper and Writefull. However, these systems often hallucinate and generate non-factual content. LitLLM addresses this by using RAG techniques to condition the generated related work on factual content and avoid hallucinations. **Related Work:** LLMs have shown significant capabilities in storing factual knowledge and achieving state-of-the-art results on NLP tasks. However, they face challenges like hallucination and outdated knowledge. RAG techniques enhance the accuracy and credibility of LLMs by incorporating external database knowledge. **Pipeline:** The LitLLM system retrieves relevant papers using the Semantic Scholar API, re-ranks them based on the user-provided abstract, and generates the related work section using an LLM. The system supports zero-shot and plan-based generation, with sentence plans guiding the LLM to generate controllable outputs. **Implementation Details:** The system is built using Gradio and queries the Semantic Scholar API. It uses OpenAI's GPT-3.5-turbo and GPT-4 models for LLM-based generation. The modular pipeline can be adapted to include next-generation LLMs and other domains. **User Experience:** A preliminary study with 5 researchers showed that zero-shot generation provides valuable insights, while plan-based generation is more accessible and tailored for specific research papers. **Conclusion and Future Work:** LitLLM is a promising tool for generating literature reviews from abstracts using LLMs. Future work will explore multi-API academic search and**LitLLM: A Toolkit for Scientific Literature Review** **Authors:** Shubham Agarwal, Issam H. Laradj, Laurent Charlin, Christopher Pal **Institutional Affiliations:** ServiceNow Research, Mila - Quebec AI Institute, HEC Montreal, UBC, Canada CIFAR AI Chair **Abstract:** Conducting literature reviews is crucial for understanding research limitations and building on existing work. However, it is a tedious task, making automatic literature review generation appealing. Existing methods using Large Language Models (LLMs) often hallucinate and ignore recent research. To address these limitations, the authors propose LitLLM, a toolkit based on Retrieval Augmented Generation (RAG) principles. The system first summarizes user-provided abstracts into keywords using an off-the-shelf LLM, then retrieves relevant papers, re-ranks them based on the user-provided abstract, and generates the related work section. This approach significantly reduces the time and effort required for literature reviews, making LitLLM an efficient alternative. The open-source toolkit is available at <https://github.com/shubhamagarwal192/LitLLM> and Huggingface space (<https://huggingface.co/spaces/shubhamagarwal192/LitLLM>). **Introduction:** Scientists use NLP systems like search engines to find relevant papers. Recent advances in LLMs have led to systems that assist with literature reviews, such as Explainpaper and Writefull. However, these systems often hallucinate and generate non-factual content. LitLLM addresses this by using RAG techniques to condition the generated related work on factual content and avoid hallucinations. **Related Work:** LLMs have shown significant capabilities in storing factual knowledge and achieving state-of-the-art results on NLP tasks. However, they face challenges like hallucination and outdated knowledge. RAG techniques enhance the accuracy and credibility of LLMs by incorporating external database knowledge. **Pipeline:** The LitLLM system retrieves relevant papers using the Semantic Scholar API, re-ranks them based on the user-provided abstract, and generates the related work section using an LLM. The system supports zero-shot and plan-based generation, with sentence plans guiding the LLM to generate controllable outputs. **Implementation Details:** The system is built using Gradio and queries the Semantic Scholar API. It uses OpenAI's GPT-3.5-turbo and GPT-4 models for LLM-based generation. The modular pipeline can be adapted to include next-generation LLMs and other domains. **User Experience:** A preliminary study with 5 researchers showed that zero-shot generation provides valuable insights, while plan-based generation is more accessible and tailored for specific research papers. **Conclusion and Future Work:** LitLLM is a promising tool for generating literature reviews from abstracts using LLMs. Future work will explore multi-API academic search and