This paper investigates whether large language models (LLMs) can learn languages that are impossible for humans to learn. The authors create synthetic impossible languages by altering English data with unnatural word orders and grammar rules. These languages span an impossibility continuum, ranging from inherently impossible languages like random word shuffles to those that may seem intuitively impossible but are often considered so in linguistics, particularly those with rules based on counting word positions. The authors evaluate the learning capacity of GPT-2 small models on these languages, finding that they struggle to learn impossible languages compared to English. This challenges the claim that LLMs can learn both possible and impossible languages equally well. The study suggests that LLMs may not be as effective at learning languages that are inherently impossible for humans, and that further research is needed to understand how LLMs can be used as tools for cognitive and typological investigations. The authors also highlight the importance of considering the differences between models and humans when evaluating LLMs' linguistic capabilities. The study provides new experimental evidence that challenges the claims made by Chomsky and others regarding LLMs' ability to learn both possible and impossible languages. The results suggest that LLMs may not be as effective at learning languages that are inherently impossible for humans, and that further research is needed to understand how LLMs can be used as tools for cognitive and typological investigations.This paper investigates whether large language models (LLMs) can learn languages that are impossible for humans to learn. The authors create synthetic impossible languages by altering English data with unnatural word orders and grammar rules. These languages span an impossibility continuum, ranging from inherently impossible languages like random word shuffles to those that may seem intuitively impossible but are often considered so in linguistics, particularly those with rules based on counting word positions. The authors evaluate the learning capacity of GPT-2 small models on these languages, finding that they struggle to learn impossible languages compared to English. This challenges the claim that LLMs can learn both possible and impossible languages equally well. The study suggests that LLMs may not be as effective at learning languages that are inherently impossible for humans, and that further research is needed to understand how LLMs can be used as tools for cognitive and typological investigations. The authors also highlight the importance of considering the differences between models and humans when evaluating LLMs' linguistic capabilities. The study provides new experimental evidence that challenges the claims made by Chomsky and others regarding LLMs' ability to learn both possible and impossible languages. The results suggest that LLMs may not be as effective at learning languages that are inherently impossible for humans, and that further research is needed to understand how LLMs can be used as tools for cognitive and typological investigations.