This paper introduces DIGGER, a framework for detecting and assessing the presence of potentially copyrighted content in the training datasets of large language models (LLMs). The framework provides a confidence estimation for the likelihood of each content sample's inclusion. The authors conduct a series of simulated experiments to validate their approach, demonstrating its effectiveness in identifying and addressing instances of content misuse in LLM training processes. They also investigate the presence of recognizable quotes from famous literary works within these datasets. The study highlights the need for more transparent and responsible data management practices in the development of LLMs to ensure the ethical use of copyrighted materials. The paper explores the impact of fine-tuning on sample loss and the use of sample loss to determine if a material has been learned by an LLM. Through experiments, they find that the more frequently an LLM is trained on a specific sample, the more optimized its performance becomes, as indicated by the reduced loss. They also investigate the effectiveness of using sample loss to identify whether a material has been learned by an LLM, finding that the loss difference between learned and unlearned samples can be used to determine this. The authors propose a method to calculate confidence scores based on the sample loss differences between the target dataset and the baseline model. They use the Wasserstein distance to quantify the dissimilarity between two distributions, which is then used to calibrate the vanilla-tuned loss distribution. This calibrated distribution helps in deriving a confidence score, which estimates the likelihood of the target content being part of the training materials in the initial LLM. The study also evaluates the effectiveness of DIGGER in real-world scenarios, demonstrating its robustness in identifying prior training on the target LLM. The results show that DIGGER achieves high accuracy and recall in controlled environments and real-world settings, affirming its value in identifying particular training constituents within state-of-the-art LLMs. The authors conclude that DIGGER provides a robust and effective method for detecting the presence of copyrighted content in LLM training datasets.This paper introduces DIGGER, a framework for detecting and assessing the presence of potentially copyrighted content in the training datasets of large language models (LLMs). The framework provides a confidence estimation for the likelihood of each content sample's inclusion. The authors conduct a series of simulated experiments to validate their approach, demonstrating its effectiveness in identifying and addressing instances of content misuse in LLM training processes. They also investigate the presence of recognizable quotes from famous literary works within these datasets. The study highlights the need for more transparent and responsible data management practices in the development of LLMs to ensure the ethical use of copyrighted materials. The paper explores the impact of fine-tuning on sample loss and the use of sample loss to determine if a material has been learned by an LLM. Through experiments, they find that the more frequently an LLM is trained on a specific sample, the more optimized its performance becomes, as indicated by the reduced loss. They also investigate the effectiveness of using sample loss to identify whether a material has been learned by an LLM, finding that the loss difference between learned and unlearned samples can be used to determine this. The authors propose a method to calculate confidence scores based on the sample loss differences between the target dataset and the baseline model. They use the Wasserstein distance to quantify the dissimilarity between two distributions, which is then used to calibrate the vanilla-tuned loss distribution. This calibrated distribution helps in deriving a confidence score, which estimates the likelihood of the target content being part of the training materials in the initial LLM. The study also evaluates the effectiveness of DIGGER in real-world scenarios, demonstrating its robustness in identifying prior training on the target LLM. The results show that DIGGER achieves high accuracy and recall in controlled environments and real-world settings, affirming its value in identifying particular training constituents within state-of-the-art LLMs. The authors conclude that DIGGER provides a robust and effective method for detecting the presence of copyrighted content in LLM training datasets.