New study shows radiologists shouldn’t completely trust AI to make diagnoses

In diagnosing diseases, doctors have recently often trusted artificial intelligence without checking the correctness of its answers. Sometimes this leads to an error in diagnosis.

A group of American researchers decided to study how the type of recommendations offered by artificial intelligence, as well as their accuracy, affect the diagnostic process. For the experiment, they invited 220 doctors to analyze chest X-rays using AI-generated recommendations. The study included radiologists, as well as internal medicine and emergency medicine specialists. Doctors could accept, adjust or reject the artificial intelligence’s suggestions.

The study found that doctors often blindly trusted AI answers without double-checking them themselves. When artificial intelligence provided correct recommendations, diagnostic accuracy increased to 92.8%. However, in the case of incorrect AI diagnoses, diagnostic accuracy decreased to 23.6%.

“When we rely too much on what the computer tells us, it becomes a problem because AI is not always right. I think as radiologists using AI, we need to be aware of these pitfalls and be mindful of our diagnostic models and training.” , says Paul H. Yee, MD, one of the study’s authors.