The librarian Andrew Gray has made a “very surprising” discovery. It has analyzed five million scientific studies published last year and has detected a sudden rise in the use of certain words, such as meticulously (up 137%), intricate (117%), commendable (83%) and meticulous (59%) , in their English versions. Gray, of University College London, can only find one explanation: tens of thousands of researchers are using ChatGPT – or other similar language generation programs with artificial intelligence – to write their studies or at least “polish” them.
There are blatant examples. A team of Chinese scientists published a study on lithium batteries on February 17. The work, published in a specialized magazine from the Elsevier publishing house, began like this: “Of course, here is a possible introduction for your topic: Lithium batteries are promising candidates for…”. The authors apparently asked ChatGPT for an introduction and copied it as is. Another article in another Elsevier journal, signed by Israeli researchers on March 8, includes the text: “In summary, the management of bilateral iatrogenic injuries I’m sorry, but I do not have access to real-time information or patient-specific data, since I am an artificial intelligence language model.” And, a couple of months ago, three Chinese scientists published a delirious drawing of a rat with a kind of giant penis, an image generated with artificial intelligence for a study on sperm precursor cells.
Andrew Gray estimates that at least 60,000 scientific studies (more than 1% of those analyzed in 2023) were written with the help of ChatGPT — a tool launched at the end of 2022 — or similar. “I think extreme cases of someone writing an entire study with ChatGPT are rare,” says Gray, a 41-year-old Scottish librarian. In his opinion, in most cases artificial intelligence is used appropriately to “polish” the text – identify typos or facilitate translation into English – but there is a large gray scale, in which some scientists take advantage of the assistance of ChatGPT further, without verifying the results. “Right now it is impossible to know how big that gray area is, because scientific journals do not require authors to declare the use of ChatGPT, there is very little transparency,” he laments.
Artificial intelligence language models use certain words disproportionately, as demonstrated by James Zou’s team at Stanford University (USA). These tend to be terms with positive connotations, such as commendable, meticulous, intricate, innovative, and versatile. Zou and his colleagues warned in March that the reviewers of scientific studies themselves are using these programs to write their evaluations, prior to the publication of the works. The Stanford group analyzed reviews of studies presented at two international artificial intelligence conferences and found that the probability of the word meticulous appearing had increased by 35 times.
Zou’s team, on the other hand, did not detect significant traces of ChatGPT in the corrections made in the prestigious journals of the Nature group. The use of ChatGPT was associated with poorer quality reviews. “I find it really worrying,” explains Gray. “If we know that using these tools to write reviews produces lower quality results, we must reflect on how they are being used to write studies and what that implies,” says the librarian at University College London. A year after the launch of ChatGPT, one in three scientists acknowledged that they used it to write their studies, according to a survey by the magazine Nature.
Gray’s analysis shows that the word intricate appeared in 109,000 studies in 2023, more than double the average of 50,000 in previous years. The term meticulously went from about 12,300 jobs in 2022 to more than 28,000 in 2023. Commendably, from 6,500 to almost 12,000. The researcher jokes that his colleagues have congratulated him on the meticulousness of his report, still a draft pending publication in a specialized journal.
Very few studies report the use of artificial intelligence in its preparation. Gray warns that “a vicious circle” can be generated, in which subsequent versions of ChatGPT are trained with scientific articles written by the old versions, giving rise to increasingly commendable, intricate, meticulous and, above all, insubstantial studies. .
Documentation professor Ángel María Delgado Vázquez highlights the Anglo-Saxon point of view of the new analysis. “Researchers who do not speak native English are using ChatGPT a lot, as an aid to writing and to improve the English language,” says the researcher, from the Pablo de Olavide University, in Seville. “In my environment, people are using ChatGPT mainly for a first translation, or even to stay directly with that translation,” he notes. The Spanish professor misses an analysis of the origin of authors who use unusual terms.
Another of the favorite words of artificial intelligence in English is delve (investigate or delve deeper). Researcher Jeremy Nguyen, from Swinburne University of Technology (Australia), has calculated that delve already appears by more than 0.5% of medical studies, when before ChatGPT it did not reach 0.04%. Are thousands of authors that suddenly investigate or delve deeper. If you ask the Spanish version of ChatGPT 3.5 what words it uses more often than usual, it responds with examples like intricate, detailed, praiseworthy, and careful.
Librarian Andrew Gray warns of the risk of humanity becoming infected with this new, meticulously artificial language. Nguyen himself It has been recognized on social network On April 8, the official ChatGPT account on X entered the rag: “I love going deeper, what am I going to do?”