This paper introduces Stacked Attention Networks (SANs) for image question answering (QA), which use multiple layers of attention to perform multi-step reasoning to answer natural language questions based on images. SANs use semantic representations of questions as queries to search for relevant image regions. The model architecture includes an image model, a question model, and a stacked attention model. The image model uses a CNN to extract high-level image features, while the question model uses either an LSTM or a CNN to extract semantic features of the question. The stacked attention model iteratively refines the query by combining the question vector with retrieved image vectors, progressively narrowing down to the relevant regions for the answer. Experiments on four image QA datasets (DAQUAR-ALL, DAQUAR-REDUCED, COCO-QA, and VQA) show that SANs significantly outperform previous state-of-the-art approaches. The model's ability to progressively focus attention on relevant visual clues is demonstrated through visualization of attention layers, which show how the model narrows down to the correct answer region. The SANs with two attention layers achieve the best performance across all datasets, outperforming baselines by significant margins. The model's effectiveness is further confirmed by error analysis, which highlights the model's ability to focus on the correct regions and handle ambiguous answers. The results demonstrate that using multiple attention layers significantly improves the performance of image QA systems.This paper introduces Stacked Attention Networks (SANs) for image question answering (QA), which use multiple layers of attention to perform multi-step reasoning to answer natural language questions based on images. SANs use semantic representations of questions as queries to search for relevant image regions. The model architecture includes an image model, a question model, and a stacked attention model. The image model uses a CNN to extract high-level image features, while the question model uses either an LSTM or a CNN to extract semantic features of the question. The stacked attention model iteratively refines the query by combining the question vector with retrieved image vectors, progressively narrowing down to the relevant regions for the answer. Experiments on four image QA datasets (DAQUAR-ALL, DAQUAR-REDUCED, COCO-QA, and VQA) show that SANs significantly outperform previous state-of-the-art approaches. The model's ability to progressively focus attention on relevant visual clues is demonstrated through visualization of attention layers, which show how the model narrows down to the correct answer region. The SANs with two attention layers achieve the best performance across all datasets, outperforming baselines by significant margins. The model's effectiveness is further confirmed by error analysis, which highlights the model's ability to focus on the correct regions and handle ambiguous answers. The results demonstrate that using multiple attention layers significantly improves the performance of image QA systems.