FGAIF: Aligning Large Vision-Language Models with Fine-grained AI Feedback

FGAIF: Aligning Large Vision-Language Models with Fine-grained AI Feedback

05/2025 | Liqiang Jing, Xinya Du
The article presents a novel method called Fine-Grained Artificial Intelligence Feedback (FGAIF) to improve the alignment of Large Vision-Language Models (LVLMs) by addressing hallucination issues in their responses. Hallucinations in LVLMs occur due to misalignment between visual and textual modalities, leading to errors in object existence, attributes, and relationships. FGAIF addresses these issues through three key steps: AI-based feedback collection, fine-grained reward model training, and reinforcement learning with fine-grained rewards. The method uses AI to generate detailed feedback on hallucination types in LVLM responses, trains specialized reward models to provide dense feedback, and integrates this feedback into the Proximal Policy Optimization (PPO) algorithm for fine-tuning LVLMs. This approach allows for more precise alignment between visual and textual modalities, reducing hallucinations and improving the accuracy of generated responses. The method is evaluated on various benchmarks, demonstrating superior performance compared to existing methods, particularly in terms of hallucination mitigation. The results show that FGAIF effectively reduces hallucinations even with fewer parameters, making it a promising solution for enhancing the reliability of LVLMs. The study also highlights the importance of fine-grained feedback in improving the performance of LVLMs and suggests that further research into AI-based feedback could lead to more robust and accurate models.The article presents a novel method called Fine-Grained Artificial Intelligence Feedback (FGAIF) to improve the alignment of Large Vision-Language Models (LVLMs) by addressing hallucination issues in their responses. Hallucinations in LVLMs occur due to misalignment between visual and textual modalities, leading to errors in object existence, attributes, and relationships. FGAIF addresses these issues through three key steps: AI-based feedback collection, fine-grained reward model training, and reinforcement learning with fine-grained rewards. The method uses AI to generate detailed feedback on hallucination types in LVLM responses, trains specialized reward models to provide dense feedback, and integrates this feedback into the Proximal Policy Optimization (PPO) algorithm for fine-tuning LVLMs. This approach allows for more precise alignment between visual and textual modalities, reducing hallucinations and improving the accuracy of generated responses. The method is evaluated on various benchmarks, demonstrating superior performance compared to existing methods, particularly in terms of hallucination mitigation. The results show that FGAIF effectively reduces hallucinations even with fewer parameters, making it a promising solution for enhancing the reliability of LVLMs. The study also highlights the importance of fine-grained feedback in improving the performance of LVLMs and suggests that further research into AI-based feedback could lead to more robust and accurate models.
Reach us at info@study.space