AI cloned voices are easier to understand than human voices, according to study

Increasingly, synthetic voices are part of our daily lives. From digital assistants like Siri and Alexa to automated teleoperators and answering machines. With the expansion of generative AI, a new type of synthetic voice has been developed: voice clones, capable of recreating an exact replica of a person’s voice from just a few seconds of recording.

LOOK: NASA’s Curiosity vehicle detects chemical compounds compatible with life on Mars

Cloned voices differ from traditional synthetic voices in the number of samples they require. Synthetic voices like Siri require a voice actor to spend hours in a recording booth. In contrast, a cloned voice can be created with as little as 10 seconds of speaking, significantly expanding the number of potential voices as well as the number of possible applications.

Researchers Patti Adank and Han Wang, who specialize in the study of human perception of unclear speech, were fascinated by the idea of ​​the machine-replicated voice. A key question they sought to answer was how easy it is for the average person to understand voice clones.

They suspected that these clones would simply be poor representations of real human voices and that people would have difficulty understanding them. However, what they discovered was completely different.

“At first I thought that the cloned voices would be less intelligible because they were unknown,” reflects Adank. “I found that they were up to 20% more intelligible, which was quite surprising. A small part of our paper is about that experiment, and the majority of it is my collaborator and I desperately trying to figure out what makes those cloned voices more intelligible.”

Initially, the duo presented volunteers with human voices and cloned voices, asking them to rate their intelligibility. After finding that the cloned voices were consistently considered easier to understand, they repeated the experiment with elderly volunteers to determine whether hearing loss altered the effect; with American volunteers (the original group was British) to evaluate whether accent had an influence; and with a filter designed to imitate cochlear implants. In all cases, the cloned voices were superior.

After examining more than 100 acoustic measurements, Adank believes the only way to solve the mystery is to work with collaborators specializing in text-to-speech systems to adapt an existing open source cloning system.

“Now we are going to try to recreate [el efecto] studying how synthesizers work and how they use digital signal processing to generate those voices, simply to better understand this process,” concludes Adank.