This paper investigates the effectiveness of supervised fine-tuning (SFT) as a method for injecting new knowledge into large language models (LLMs), focusing on recent sporting events. The study compares two dataset generation strategies—token-based and fact-based scaling—to create training data that helps LLMs learn new information. The experiments on GPT-4 demonstrate that while token-based scaling improves question-answering (Q&A) accuracy, it may not uniformly cover new knowledge. Fact-based scaling, on the other hand, provides a more systematic approach to ensure even coverage of all facts. The paper presents a novel dataset generation process that leads to more effective knowledge ingestion through SFT, resulting in significant performance improvements in Q&A tasks related to out-of-domain knowledge. The study contributes to understanding domain adaptation for LLMs and highlights the potential of SFT in enhancing the factuality of LLM responses in specific knowledge domains. The research also explores the trade-offs between direct training and retrieval-augmented generation (RAG), and the impact of hyperparameter tuning on model performance. The findings show that fact-based scaling outperforms token-based scaling in terms of coverage and effectiveness, with SFT models achieving performance close to RAG baselines. The study emphasizes the importance of fact coverage in knowledge injection and the need for more targeted approaches to incorporate new information into LLMs. The results suggest that SFT can be a practical method for domain adaptation in LLMs, particularly when combined with fact-based dataset generation. The paper also highlights the limitations of current methods and the need for further research into optimizing SFT for knowledge injection.This paper investigates the effectiveness of supervised fine-tuning (SFT) as a method for injecting new knowledge into large language models (LLMs), focusing on recent sporting events. The study compares two dataset generation strategies—token-based and fact-based scaling—to create training data that helps LLMs learn new information. The experiments on GPT-4 demonstrate that while token-based scaling improves question-answering (Q&A) accuracy, it may not uniformly cover new knowledge. Fact-based scaling, on the other hand, provides a more systematic approach to ensure even coverage of all facts. The paper presents a novel dataset generation process that leads to more effective knowledge ingestion through SFT, resulting in significant performance improvements in Q&A tasks related to out-of-domain knowledge. The study contributes to understanding domain adaptation for LLMs and highlights the potential of SFT in enhancing the factuality of LLM responses in specific knowledge domains. The research also explores the trade-offs between direct training and retrieval-augmented generation (RAG), and the impact of hyperparameter tuning on model performance. The findings show that fact-based scaling outperforms token-based scaling in terms of coverage and effectiveness, with SFT models achieving performance close to RAG baselines. The study emphasizes the importance of fact coverage in knowledge injection and the need for more targeted approaches to incorporate new information into LLMs. The results suggest that SFT can be a practical method for domain adaptation in LLMs, particularly when combined with fact-based dataset generation. The paper also highlights the limitations of current methods and the need for further research into optimizing SFT for knowledge injection.