The paper evaluates the performance of Vision-Language Models (VLMs) on Raven's Progressive Matrices (RPMs), a challenging benchmark for visual deductive reasoning. Despite the impressive capabilities of VLMs in text-based reasoning, they fall short in visual deductive tasks. The study uses three diverse datasets—Mensa IQ test, IntelligenceTest, and RAVEN—to assess VLMs' abilities to perform multi-hop relational and deductive reasoning using visual clues alone. Key findings include: 1. **Performance Gaps**: VLMs perform poorly on RPM tasks, with accuracy comparable to random guessing, indicating a significant gap compared to human performance. 2. **Perception Challenges**: VLMs struggle with perceiving and understanding complex patterns, leading to errors in description and reasoning. 3. **Strategies and Limitations**: Standard strategies effective in text-based LLMs, such as in-context learning and self-consistency, do not translate well to visual reasoning tasks. 4. **Hypothesis Verification**: VLMs have difficulty formulating and verifying hypotheses, often generating nonsensical rationales. 5. **Prompt Format Impact**: The format of prompts significantly affects VLM performance, with structured prompts improving accuracy. 6. **Oracle Descriptions**: Providing oracle text descriptions improves VLM performance, but removing visual cues degrades it, suggesting the importance of visual information. The study highlights the need for further research to enhance VLMs' visual deductive reasoning capabilities, particularly in improving perception and hypothesis verification.The paper evaluates the performance of Vision-Language Models (VLMs) on Raven's Progressive Matrices (RPMs), a challenging benchmark for visual deductive reasoning. Despite the impressive capabilities of VLMs in text-based reasoning, they fall short in visual deductive tasks. The study uses three diverse datasets—Mensa IQ test, IntelligenceTest, and RAVEN—to assess VLMs' abilities to perform multi-hop relational and deductive reasoning using visual clues alone. Key findings include: 1. **Performance Gaps**: VLMs perform poorly on RPM tasks, with accuracy comparable to random guessing, indicating a significant gap compared to human performance. 2. **Perception Challenges**: VLMs struggle with perceiving and understanding complex patterns, leading to errors in description and reasoning. 3. **Strategies and Limitations**: Standard strategies effective in text-based LLMs, such as in-context learning and self-consistency, do not translate well to visual reasoning tasks. 4. **Hypothesis Verification**: VLMs have difficulty formulating and verifying hypotheses, often generating nonsensical rationales. 5. **Prompt Format Impact**: The format of prompts significantly affects VLM performance, with structured prompts improving accuracy. 6. **Oracle Descriptions**: Providing oracle text descriptions improves VLM performance, but removing visual cues degrades it, suggesting the importance of visual information. The study highlights the need for further research to enhance VLMs' visual deductive reasoning capabilities, particularly in improving perception and hypothesis verification.