Scene Graph Generation by Iterative Message Passing

Scene Graph Generation by Iterative Message Passing

12 Apr 2017 | Danfei Xu, Yuke Zhu, Christopher B. Choy, Li Fei-Fei
This paper proposes a novel end-to-end model for generating scene graphs from images. Scene graphs are visually grounded graphical structures that represent objects and their relationships in an image. The model uses iterative message passing between a pair of bipartite sub-graphs of the scene graph to iteratively refine predictions using recurrent neural networks (RNNs). The model takes an image as input and outputs a scene graph that includes object categories, their bounding boxes, and semantic relationships between pairs of objects. The model is evaluated on a new scene graph dataset based on the Visual Genome dataset and on the NYU Depth v2 dataset for support relation inference. The results show that the model significantly outperforms previous methods for generating scene graphs and inferring support relations. The model is able to capture contextual information and improve the accuracy of relationship prediction. The model is also effective on both sparsely and densely labeled relationships. The model is able to generate semantically accurate scene graphs and outperforms previous work in terms of performance. The model is able to generalize to other types of relationships and is effective in both visual and other problem domains. The model is based on a primal-dual formulation that allows for efficient inference by passing messages between two sub-graphs. The model uses a message pooling function to dynamically aggregate hidden states into messages. The model is trained using a combination of cross entropy loss for object class and relationship predicate prediction, and an L1 loss for bounding box offsets. The model is evaluated on multiple datasets and shows significant improvements in performance compared to previous methods. The model is able to capture contextual information and improve the accuracy of relationship prediction. The model is able to generate semantically accurate scene graphs and outperforms previous work in terms of performance. The model is able to generalize to other types of relationships and is effective in both visual and other problem domains.This paper proposes a novel end-to-end model for generating scene graphs from images. Scene graphs are visually grounded graphical structures that represent objects and their relationships in an image. The model uses iterative message passing between a pair of bipartite sub-graphs of the scene graph to iteratively refine predictions using recurrent neural networks (RNNs). The model takes an image as input and outputs a scene graph that includes object categories, their bounding boxes, and semantic relationships between pairs of objects. The model is evaluated on a new scene graph dataset based on the Visual Genome dataset and on the NYU Depth v2 dataset for support relation inference. The results show that the model significantly outperforms previous methods for generating scene graphs and inferring support relations. The model is able to capture contextual information and improve the accuracy of relationship prediction. The model is also effective on both sparsely and densely labeled relationships. The model is able to generate semantically accurate scene graphs and outperforms previous work in terms of performance. The model is able to generalize to other types of relationships and is effective in both visual and other problem domains. The model is based on a primal-dual formulation that allows for efficient inference by passing messages between two sub-graphs. The model uses a message pooling function to dynamically aggregate hidden states into messages. The model is trained using a combination of cross entropy loss for object class and relationship predicate prediction, and an L1 loss for bounding box offsets. The model is evaluated on multiple datasets and shows significant improvements in performance compared to previous methods. The model is able to capture contextual information and improve the accuracy of relationship prediction. The model is able to generate semantically accurate scene graphs and outperforms previous work in terms of performance. The model is able to generalize to other types of relationships and is effective in both visual and other problem domains.
Reach us at info@study.space
[slides] Scene Graph Generation by Iterative Message Passing | StudySpace