Large language models (LLM) such as ChatGPT o DeepSeek They may not reliably recognize users’ incorrect beliefs, says a study published today by Nature Machine Intelligence.
LLMs are an increasingly popular tool in high-stakes fields such as medicine, law and science, where the ability to distinguish between personal belief and factual knowledge is crucial.
The research, led by Stanford University, analyzed how 24 LLMs, including DeepSeek and GPT-4o, They responded to personal facts and beliefs in 13,000 questions.
Study results highlight the need to cautiously use LLM results in high-stakes decisions
For example, for mental health clinicians, recognizing a patient’s erroneous beliefs is often important for diagnosis and treatment. Without this ability, “LLMs can support erroneous decisions and contribute to the spread of false information”write the authors.
The researchers asked the AIs to verify true or false factual data. The newer LLMs achieved an average accuracy of 91.1% or 91.5%, respectively, while the older models achieved an average accuracy of 84.8% or 71.5%, respectively.
When asked to respond to a belief in the first person (I think…), The authors observed that LLMs were less likely to recognize a false belief compared to a true one.
Thus, the most recent models (released after GPT-4o in May 2024, including this one) were, on average, 34.3% less likely to recognize a false first-person belief compared to a true first-person belief.
Older models were, on average, 38.6% less likely to recognize false first-person beliefs compared to true first-person beliefs.
The authors, cited by the magazine, point out that the LLMs resorted to correcting the user’s data instead of acknowledging their belief.
By recognizing the beliefs of third parties (Mary believes that…), most recent LLMs experienced a 1.6% to 4.6% reduction in accuracywhile the oldest ones experienced a decrease of 15.5%.
The authors conclude that LLMs must be able to successfully distinguish the nuances of facts and beliefs and whether they are true or false, to effectively respond to user queries and prevent the spread of misinformation.