The paper proposes a novel approach to generating stable inorganic materials using fine-tuned large language models (LLMs). By encoding crystal structures as text and combining them with text instructions, the authors train LLMs to generate stable materials. The method is simple and effective, achieving around 90% of sampled structures that obey physical constraints on atom positions and charges. Using both learned machine learning potentials and gold-standard density functional theory (DFT) calculations, the strongest model (fine-tuned LLaMA-2 70B) generates materials predicted to be metastable at about twice the rate of a competing diffusion model (CD-VAE). The flexibility of LLMs allows for unconditional generation of stable materials, text-conditioned generation, and structural infilling. The study also explores the ability of LLMs to capture key symmetries of crystal structures, showing that larger models learn invariances more effectively. The approach opens up new possibilities for multitasking and multimodal training in materials science, making it a promising tool for scientific discovery and materials design.The paper proposes a novel approach to generating stable inorganic materials using fine-tuned large language models (LLMs). By encoding crystal structures as text and combining them with text instructions, the authors train LLMs to generate stable materials. The method is simple and effective, achieving around 90% of sampled structures that obey physical constraints on atom positions and charges. Using both learned machine learning potentials and gold-standard density functional theory (DFT) calculations, the strongest model (fine-tuned LLaMA-2 70B) generates materials predicted to be metastable at about twice the rate of a competing diffusion model (CD-VAE). The flexibility of LLMs allows for unconditional generation of stable materials, text-conditioned generation, and structural infilling. The study also explores the ability of LLMs to capture key symmetries of crystal structures, showing that larger models learn invariances more effectively. The approach opens up new possibilities for multitasking and multimodal training in materials science, making it a promising tool for scientific discovery and materials design.