This paper introduces a model called DT-RNN (Dependency Tree Recursive Neural Network) that learns to map sentences and images into a common embedding space to enable retrieval of one from the other. The model uses dependency trees to embed sentences into a vector space, allowing it to retrieve images described by those sentences. Unlike previous RNN-based models that use constituency trees, DT-RNNs focus on the action and agents in a sentence, making them more robust to changes in word order and syntactic structure. DT-RNNs outperform other models, including kernelized CCA and a bag-of-words baseline, on tasks of finding an image that fits a sentence description and vice versa. They also produce more similar representations for sentences that describe the same image. The model is trained on a dataset of 1000 images, each with 5 descriptions. The DT-RNN learns to map sentences into a 50-dimensional space and images into a 4096-dimensional space. The model then maps both sentences and images into a shared space to learn joint representations. The model uses a max-margin objective function to train the joint representations, which encourages correct pairs of image-sentence vectors to have high inner products and incorrect pairs to have low inner products. The DT-RNN is compared to other models, including a constituency tree RNN, a standard recurrent neural network, and a bag-of-words baseline. The model outperforms these models in tasks of finding images that match a sentence description and vice versa. The model also performs better than kernelized canonical correlation analysis (kCCA) in tasks of describing images with sentences. The DT-RNN is evaluated on three tasks: sentence similarity, image search with query sentences, and describing images by finding suitable sentences. The model outperforms other models in all tasks. The model is also compared to other models using squared error loss and Euclidean distance in the multimodal space, and it performs better when trained with a max-margin loss and inner products. The model is able to capture the meaning of sentences in terms of their similarity to a "visual representation" of the textual description. The model is more robust to changes in word order and syntactic structure, and it focuses on the action and agents in a sentence. The model is able to capture the important elements of a sentence, such as the main action or verb and its subject and object, which are merged last in the dependency tree. This allows the final sentence representation to be more robust to less important adjectival modifiers and word order changes. The model is also able to capture the semantic relationships between words, which are used to compute the composition function for the DT-RNN. The model is able to capture the important relationships between words, such as nominal subject, possession modifier, passive auxiliary, and preposition. The model is able to capture the importance of nouns and their spatial prepositions and adjectives. The model is able to capture the meaning of sentences inThis paper introduces a model called DT-RNN (Dependency Tree Recursive Neural Network) that learns to map sentences and images into a common embedding space to enable retrieval of one from the other. The model uses dependency trees to embed sentences into a vector space, allowing it to retrieve images described by those sentences. Unlike previous RNN-based models that use constituency trees, DT-RNNs focus on the action and agents in a sentence, making them more robust to changes in word order and syntactic structure. DT-RNNs outperform other models, including kernelized CCA and a bag-of-words baseline, on tasks of finding an image that fits a sentence description and vice versa. They also produce more similar representations for sentences that describe the same image. The model is trained on a dataset of 1000 images, each with 5 descriptions. The DT-RNN learns to map sentences into a 50-dimensional space and images into a 4096-dimensional space. The model then maps both sentences and images into a shared space to learn joint representations. The model uses a max-margin objective function to train the joint representations, which encourages correct pairs of image-sentence vectors to have high inner products and incorrect pairs to have low inner products. The DT-RNN is compared to other models, including a constituency tree RNN, a standard recurrent neural network, and a bag-of-words baseline. The model outperforms these models in tasks of finding images that match a sentence description and vice versa. The model also performs better than kernelized canonical correlation analysis (kCCA) in tasks of describing images with sentences. The DT-RNN is evaluated on three tasks: sentence similarity, image search with query sentences, and describing images by finding suitable sentences. The model outperforms other models in all tasks. The model is also compared to other models using squared error loss and Euclidean distance in the multimodal space, and it performs better when trained with a max-margin loss and inner products. The model is able to capture the meaning of sentences in terms of their similarity to a "visual representation" of the textual description. The model is more robust to changes in word order and syntactic structure, and it focuses on the action and agents in a sentence. The model is able to capture the important elements of a sentence, such as the main action or verb and its subject and object, which are merged last in the dependency tree. This allows the final sentence representation to be more robust to less important adjectival modifiers and word order changes. The model is also able to capture the semantic relationships between words, which are used to compute the composition function for the DT-RNN. The model is able to capture the important relationships between words, such as nominal subject, possession modifier, passive auxiliary, and preposition. The model is able to capture the importance of nouns and their spatial prepositions and adjectives. The model is able to capture the meaning of sentences in