LSTMs are sequence models that do not have explicit structural representations. This study investigates whether LSTMs can learn syntax-sensitive dependencies, focusing on English subject-verb agreement. The research uses number prediction, grammaticality judgments, and language modeling tasks to evaluate LSTM performance. In strongly supervised settings, LSTMs achieved high accuracy but made more errors when sequential and structural information conflicted. Language modeling showed higher error rates, indicating that it is insufficient for capturing syntax-sensitive dependencies. The study concludes that LSTMs can learn some grammatical structure with targeted supervision, but stronger architectures may be needed to reduce errors. Supervised signals are essential for learning syntax-sensitive dependencies, and language modeling alone is not sufficient. The results suggest that explicit supervision is necessary for learning such dependencies, and that language modeling objectives should be supplemented with direct supervision for tasks requiring syntax-sensitive dependencies. The study also shows that LSTMs can capture some syntactic knowledge, but are overly reliant on function words. The results highlight the importance of syntactic cues in resolving agreement dependencies and suggest that more direct supervision is needed for accurate learning. The study also compares LSTMs to other models, showing that they can learn some syntax-sensitive dependencies but are not as effective as supervised models. The results indicate that LSTMs can learn some syntax-sensitive dependencies, but more expressive architectures may be needed for accurate learning. The study concludes that LSTMs can learn to approximate structure-sensitive dependencies with explicit supervision, but more expressive architectures may be necessary to eliminate errors. The results also suggest that language modeling alone is not sufficient for learning structure-sensitive dependencies, and that a joint training objective can be used to supplement language models on tasks requiring syntax-sensitive dependencies.LSTMs are sequence models that do not have explicit structural representations. This study investigates whether LSTMs can learn syntax-sensitive dependencies, focusing on English subject-verb agreement. The research uses number prediction, grammaticality judgments, and language modeling tasks to evaluate LSTM performance. In strongly supervised settings, LSTMs achieved high accuracy but made more errors when sequential and structural information conflicted. Language modeling showed higher error rates, indicating that it is insufficient for capturing syntax-sensitive dependencies. The study concludes that LSTMs can learn some grammatical structure with targeted supervision, but stronger architectures may be needed to reduce errors. Supervised signals are essential for learning syntax-sensitive dependencies, and language modeling alone is not sufficient. The results suggest that explicit supervision is necessary for learning such dependencies, and that language modeling objectives should be supplemented with direct supervision for tasks requiring syntax-sensitive dependencies. The study also shows that LSTMs can capture some syntactic knowledge, but are overly reliant on function words. The results highlight the importance of syntactic cues in resolving agreement dependencies and suggest that more direct supervision is needed for accurate learning. The study also compares LSTMs to other models, showing that they can learn some syntax-sensitive dependencies but are not as effective as supervised models. The results indicate that LSTMs can learn some syntax-sensitive dependencies, but more expressive architectures may be needed for accurate learning. The study concludes that LSTMs can learn to approximate structure-sensitive dependencies with explicit supervision, but more expressive architectures may be necessary to eliminate errors. The results also suggest that language modeling alone is not sufficient for learning structure-sensitive dependencies, and that a joint training objective can be used to supplement language models on tasks requiring syntax-sensitive dependencies.