Is ChatGPT Transforming Academics' Writing Style?

Is ChatGPT Transforming Academics' Writing Style?

12 Apr 2024 | Mingmeng Geng, Roberto Trotta
This study examines the impact of ChatGPT on academic writing, particularly in the context of abstracts from arXiv papers. By analyzing one million abstracts submitted from May 2018 to January 2024, the authors assess the textual density of ChatGPT's writing style through a statistical analysis of word frequency changes. The model is calibrated and validated using a mixture of real abstracts and ChatGPT-modified abstracts (simulated data) after careful noise analysis. Key findings include: - ChatGPT is increasingly influencing arXiv abstracts, especially in computer science, where the fraction of ChatGPT-revised abstracts is estimated to be approximately 35%. - The frequency of certain words, such as "are" and "is," has dropped by more than 10% in 2023, while the frequency of other words like "significant" has significantly increased. - The impact of ChatGPT is more pronounced in computer science, with a density of ChatGPT-style texts of around 35% in the most recent time period. - The study also explores the robustness of the approach by evaluating different prompts and comparing results from real data and simulated data. The authors conclude with an analysis of both the positive and negative aspects of ChatGPT's penetration into academic writing styles, highlighting the need for adaptation and the potential benefits and risks associated with its increasing use.This study examines the impact of ChatGPT on academic writing, particularly in the context of abstracts from arXiv papers. By analyzing one million abstracts submitted from May 2018 to January 2024, the authors assess the textual density of ChatGPT's writing style through a statistical analysis of word frequency changes. The model is calibrated and validated using a mixture of real abstracts and ChatGPT-modified abstracts (simulated data) after careful noise analysis. Key findings include: - ChatGPT is increasingly influencing arXiv abstracts, especially in computer science, where the fraction of ChatGPT-revised abstracts is estimated to be approximately 35%. - The frequency of certain words, such as "are" and "is," has dropped by more than 10% in 2023, while the frequency of other words like "significant" has significantly increased. - The impact of ChatGPT is more pronounced in computer science, with a density of ChatGPT-style texts of around 35% in the most recent time period. - The study also explores the robustness of the approach by evaluating different prompts and comparing results from real data and simulated data. The authors conclude with an analysis of both the positive and negative aspects of ChatGPT's penetration into academic writing styles, highlighting the need for adaptation and the potential benefits and risks associated with its increasing use.
Reach us at info@study.space