The use of large language models (LLMs), particularly ChatGPT, in academic writing has seen a rapid rise, prompting scrutiny into its prevalence and implications for research integrity. A recent study delved into this phenomenon, focusing on changes in writing style and vocabulary in biomedical research abstracts from PubMed between 2010 and 2024.
You may be interested：OpenAI’s anticipated release of the GPT-4o voice assistant, branded as “advanced voice mode” for ChatGPT, has been postponed due to safety considerations
Traditionally, quantifying LLM-generated text in academic literature involved methods such as LLM detectors and word frequency analysis. However, a novel approach introduced in this research examines excess word usage, inspired by methodologies used to track excess mortality during crises like the COVID-19 pandemic. By analyzing over 14 million PubMed abstracts, researchers identified words that showed significant increases in frequency post-ChatGPT release, suggesting LLM involvement.
Key findings revealed notable spikes in stylistic terms like “delves,” “showcasing,” and “underscores,” indicating substantial LLM influence. Measures like the excess frequency gap and ratio provided insights into the extent of LLM usage, estimating that at least 10% of 2024 biomedical abstracts were LLM-assisted. This estimate varied across disciplines, countries, and journals, with some fields showing higher rates, such as computation papers from China reaching up to 35%.
The study underscores the transformative impact of LLMs on academic writing, posing critical questions about research authenticity and the evolving landscape of scholarly communication. As LLMs continue to shape scientific discourse, understanding their role becomes increasingly vital for maintaining research integrity and evaluating academic contributions in the digital age.
Related topics:
The Vital Difference Between Machine Learning And Generative AI
Klarity Intelligence Secures $70 Million to Automate Document Review
ChatGPT-4 passes the “Turing Test” Scientists: AI intelligence is comparable to humans