Adaptive Ensembles of Fine-Tuned Transformers for LLM-Generated Text Detection

Adaptive Ensembles of Fine-Tuned Transformers for LLM-Generated Text Detection

20 Mar 2024 | Zhixin Lai, Xuesheng Zhang, Suiyao Chen
This paper presents an adaptive ensemble approach for detecting text generated by large language models (LLMs). The study evaluates five transformer-based classifiers on both in-distribution and out-of-distribution datasets to assess their performance and generalization ability. Results show that single transformer-based classifiers perform well on in-distribution data but struggle with out-of-distribution data. To improve generalization, the researchers combined individual classifiers using adaptive ensemble algorithms, which significantly improved detection accuracy. On an in-distribution test set, the average accuracy increased from 91.8% to 99.2%, and on an out-of-distribution test set, it increased from 62.9% to 72.5%. The results demonstrate the effectiveness and generalization ability of adaptive ensemble algorithms in detecting LLM-generated text. The study also compares non-adaptive and adaptive ensemble methods. The adaptive ensemble method outperformed both single classifiers and non-adaptive ensembles, achieving the highest F1 scores for both human and LLM-generated texts. The adaptive ensemble method also showed better generalization on out-of-distribution data, with the neural network ensemble achieving the highest accuracy of 0.736. The results indicate that adaptive ensemble methods significantly enhance the performance and generalization ability of LLM-generated text detection. The paper concludes that adaptive ensemble methods are a robust and effective approach for detecting LLM-generated text.This paper presents an adaptive ensemble approach for detecting text generated by large language models (LLMs). The study evaluates five transformer-based classifiers on both in-distribution and out-of-distribution datasets to assess their performance and generalization ability. Results show that single transformer-based classifiers perform well on in-distribution data but struggle with out-of-distribution data. To improve generalization, the researchers combined individual classifiers using adaptive ensemble algorithms, which significantly improved detection accuracy. On an in-distribution test set, the average accuracy increased from 91.8% to 99.2%, and on an out-of-distribution test set, it increased from 62.9% to 72.5%. The results demonstrate the effectiveness and generalization ability of adaptive ensemble algorithms in detecting LLM-generated text. The study also compares non-adaptive and adaptive ensemble methods. The adaptive ensemble method outperformed both single classifiers and non-adaptive ensembles, achieving the highest F1 scores for both human and LLM-generated texts. The adaptive ensemble method also showed better generalization on out-of-distribution data, with the neural network ensemble achieving the highest accuracy of 0.736. The results indicate that adaptive ensemble methods significantly enhance the performance and generalization ability of LLM-generated text detection. The paper concludes that adaptive ensemble methods are a robust and effective approach for detecting LLM-generated text.
Reach us at info@study.space
Understanding Adaptive_Ensembles_of_Fine-Tuned_Transformers_for_LLM-Generated_Text_Detection