High-throughput inorganic materials prediction and autonomous synthesis face significant challenges. The discovery of new materials is complex, requiring careful analysis of solid-state chemistry principles and addressing pitfalls in materials discovery. A recent study by Szymanski et al. reported the autonomous discovery of 43 novel materials, but analysis revealed errors in their characterization. These errors led to the conclusion that no new materials were discovered. Key issues include the lack of reliability in automated Rietveld analysis of powder x-ray diffraction data and the neglect of disorder in materials predictions. Many of the claimed successful materials are likely known compositionally disordered versions of predicted ordered compounds. The paper highlights the importance of accurate characterization and the role of disorder in materials prediction. It also discusses the challenges of defining a "new" inorganic material, the need for rigorous testing to confirm novelty, and the importance of compositional and structural information in validating new materials. The analysis of the A-lab data set reveals errors in the characterization of many materials, including incorrect fits, changes in predicted structures to match observed diffraction patterns, and lack of evidence for cation order. The paper emphasizes the need for improved AI-assisted materials characterization and a better understanding of disorder in materials prediction. The study concludes that many of the materials synthesized by A-lab are not new but are known disordered compounds, highlighting the importance of accurate characterization in materials discovery.High-throughput inorganic materials prediction and autonomous synthesis face significant challenges. The discovery of new materials is complex, requiring careful analysis of solid-state chemistry principles and addressing pitfalls in materials discovery. A recent study by Szymanski et al. reported the autonomous discovery of 43 novel materials, but analysis revealed errors in their characterization. These errors led to the conclusion that no new materials were discovered. Key issues include the lack of reliability in automated Rietveld analysis of powder x-ray diffraction data and the neglect of disorder in materials predictions. Many of the claimed successful materials are likely known compositionally disordered versions of predicted ordered compounds. The paper highlights the importance of accurate characterization and the role of disorder in materials prediction. It also discusses the challenges of defining a "new" inorganic material, the need for rigorous testing to confirm novelty, and the importance of compositional and structural information in validating new materials. The analysis of the A-lab data set reveals errors in the characterization of many materials, including incorrect fits, changes in predicted structures to match observed diffraction patterns, and lack of evidence for cation order. The paper emphasizes the need for improved AI-assisted materials characterization and a better understanding of disorder in materials prediction. The study concludes that many of the materials synthesized by A-lab are not new but are known disordered compounds, highlighting the importance of accurate characterization in materials discovery.