Web Application for Retrieval-Augmented Generation: Implementation and Testing

Web Application for Retrieval-Augmented Generation: Implementation and Testing

4 April 2024 | Irina Radeva, Ivan Popchev, Lyubka Doukovska, Miroslava Dimitrova
The paper presents the implementation and testing of a web-based application, PaSSER, which integrates retrieval-augmented generation (RAG) with open-source large language models (LLMs) such as Mistral:7b, Llama2:7b, and Orca2:7b. The application uses various evaluation metrics, including METEOR, ROUGE, BLEU, perplexity, cosine similarity, Pearson correlation, and F1 score, to assess LLM performance in the context of smart agriculture. Two tests were conducted: one evaluated LLM performance across different hardware configurations, while the other determined which model provided the most accurate and contextually relevant responses within RAG. The results showed that GPUs are essential for fast text generation, with Orca2:7b on Mac M1 being the fastest and Mistral:7b performing well on the 446 question-answer dataset. The paper discusses the integration of blockchain with LLMs to manage and store assessment results within a blockchain environment. The application allows adaptive testing of different scenarios and integrates three leading LLMs that do not require significant computational resources. The application is open-source, promoting transparency and collaborative improvement. It provides a detailed guide on installing and configuring the application, datasets for testing, and results of experimental evaluations. The paper contributes to RAG research by providing a practical framework, demonstrating the integration of RAG with blockchain, offering insights into model selection, and highlighting the application of RAG in smart agriculture. The application also enables the use of arbitrary open-source LLMs with more parameters, given appropriate computational resources. The paper outlines future developments in leveraging other LLMs, fine-tuning approaches, and further integration with blockchain and IPFS.The paper presents the implementation and testing of a web-based application, PaSSER, which integrates retrieval-augmented generation (RAG) with open-source large language models (LLMs) such as Mistral:7b, Llama2:7b, and Orca2:7b. The application uses various evaluation metrics, including METEOR, ROUGE, BLEU, perplexity, cosine similarity, Pearson correlation, and F1 score, to assess LLM performance in the context of smart agriculture. Two tests were conducted: one evaluated LLM performance across different hardware configurations, while the other determined which model provided the most accurate and contextually relevant responses within RAG. The results showed that GPUs are essential for fast text generation, with Orca2:7b on Mac M1 being the fastest and Mistral:7b performing well on the 446 question-answer dataset. The paper discusses the integration of blockchain with LLMs to manage and store assessment results within a blockchain environment. The application allows adaptive testing of different scenarios and integrates three leading LLMs that do not require significant computational resources. The application is open-source, promoting transparency and collaborative improvement. It provides a detailed guide on installing and configuring the application, datasets for testing, and results of experimental evaluations. The paper contributes to RAG research by providing a practical framework, demonstrating the integration of RAG with blockchain, offering insights into model selection, and highlighting the application of RAG in smart agriculture. The application also enables the use of arbitrary open-source LLMs with more parameters, given appropriate computational resources. The paper outlines future developments in leveraging other LLMs, fine-tuning approaches, and further integration with blockchain and IPFS.
Reach us at info@study.space
[slides] Web Application for Retrieval-Augmented Generation%3A Implementation and Testing | StudySpace