The paper introduces CrystalFormer, an autoregressive transformer model designed for generating crystalline materials with controlled space group symmetry. The model leverages the discrete and sequential nature of Wyckoff positions to simplify the crystal space, making it more efficient for data and compute-intensive generative modeling. CrystalFormer learns to predict the species and locations of symmetry-inequivalent atoms in the unit cell, demonstrating advantages in tasks such as symmetric structure initialization and element substitution compared to conventional methods. The model also shows promise in property-guided materials design, allowing for systematic exploration of crystalline materials. The authors analyze how CrystalFormer encodes chemical intuition by compressing the material dataset, highlighting its potential as a foundational model for crystalline materials space. The simplicity, generality, and flexibility of CrystalFormer position it as a promising architecture for materials modeling and discovery.The paper introduces CrystalFormer, an autoregressive transformer model designed for generating crystalline materials with controlled space group symmetry. The model leverages the discrete and sequential nature of Wyckoff positions to simplify the crystal space, making it more efficient for data and compute-intensive generative modeling. CrystalFormer learns to predict the species and locations of symmetry-inequivalent atoms in the unit cell, demonstrating advantages in tasks such as symmetric structure initialization and element substitution compared to conventional methods. The model also shows promise in property-guided materials design, allowing for systematic exploration of crystalline materials. The authors analyze how CrystalFormer encodes chemical intuition by compressing the material dataset, highlighting its potential as a foundational model for crystalline materials space. The simplicity, generality, and flexibility of CrystalFormer position it as a promising architecture for materials modeling and discovery.