The paper introduces MiniGPT-4, a novel vision-language model that aligns a frozen visual encoder with an advanced large language model (LLM), Vicuna, using a single projection layer. This approach aims to enhance multi-modal capabilities, similar to those demonstrated by GPT-4, such as detailed image description generation and website creation from handwritten text. The study reveals that properly aligning visual features with an advanced LLM can achieve advanced multi-modal abilities. MiniGPT-4 is trained in two stages: initially on a large collection of aligned image-text pairs and then fine-tuned with a curated dataset to improve generation reliability and usability. The model demonstrates advanced capabilities, including detailed image descriptions, website creation, cooking recipe generation, and poem writing. The paper also discusses the limitations of MiniGPT-4, such as hallucination and spatial information understanding, and suggests future research directions. The code, pre-trained model, and dataset are available online.The paper introduces MiniGPT-4, a novel vision-language model that aligns a frozen visual encoder with an advanced large language model (LLM), Vicuna, using a single projection layer. This approach aims to enhance multi-modal capabilities, similar to those demonstrated by GPT-4, such as detailed image description generation and website creation from handwritten text. The study reveals that properly aligning visual features with an advanced LLM can achieve advanced multi-modal abilities. MiniGPT-4 is trained in two stages: initially on a large collection of aligned image-text pairs and then fine-tuned with a curated dataset to improve generation reliability and usability. The model demonstrates advanced capabilities, including detailed image descriptions, website creation, cooking recipe generation, and poem writing. The paper also discusses the limitations of MiniGPT-4, such as hallucination and spatial information understanding, and suggests future research directions. The code, pre-trained model, and dataset are available online.