This article explores the application of large language models (LLMs), specifically GPT-3, in chemistry and materials science. The authors demonstrate that GPT-3, trained on vast amounts of internet text, can be fine-tuned to solve various chemical and materials science tasks, including predicting molecular properties, material properties, and reaction yields. Surprisingly, the fine-tuned GPT-3 often performs comparably to or even outperforms conventional machine learning models, especially in low-data scenarios. The model's ability to perform inverse design by simply inverting questions is also highlighted. The ease of use and high performance of GPT-3, particularly for small datasets, suggest a new approach to using machine learning in these fields. The study includes benchmarking on datasets spanning molecules, materials, and reactions, and discusses the representation sensitivity of the model. Additionally, the authors explore the model's ability to extrapolate beyond its training data, demonstrating its potential for more advanced applications such as designing molecules with specific properties. Overall, the findings highlight the potential of LLMs in chemistry and materials science, offering a powerful tool for researchers to leverage the collective knowledge encoded in these models.This article explores the application of large language models (LLMs), specifically GPT-3, in chemistry and materials science. The authors demonstrate that GPT-3, trained on vast amounts of internet text, can be fine-tuned to solve various chemical and materials science tasks, including predicting molecular properties, material properties, and reaction yields. Surprisingly, the fine-tuned GPT-3 often performs comparably to or even outperforms conventional machine learning models, especially in low-data scenarios. The model's ability to perform inverse design by simply inverting questions is also highlighted. The ease of use and high performance of GPT-3, particularly for small datasets, suggest a new approach to using machine learning in these fields. The study includes benchmarking on datasets spanning molecules, materials, and reactions, and discusses the representation sensitivity of the model. Additionally, the authors explore the model's ability to extrapolate beyond its training data, demonstrating its potential for more advanced applications such as designing molecules with specific properties. Overall, the findings highlight the potential of LLMs in chemistry and materials science, offering a powerful tool for researchers to leverage the collective knowledge encoded in these models.