The voice cloning company ElevenLabs has announced the launch of its new ‘AI Dubbing’ dubbing tool powered by Artificial Intelligence (AI) that will be able to replace the voice of any original actor with an identical voice in more than 20 languages.
The software startup, which uses generative AI focused on voice-related issues such as voice cloning and text-to-speech transition, pursues goal of “eliminating linguistic barriers to content.”
In this sense, ElevenLabs has launched a new tool for dubbing voices into other languagesallowing users automatically translate any speechmaintaining the actor’s voice and preserving the characteristics and nuances of the original recording.
As defined by the company in a statement on its blog, the tool, which it refers to as ‘AI Dubbing’, is capable of creating a new “efficient and high-quality” way of dubbing audio and video content. In fact, it is designed to facilitate accessibility to content in streaming media, games and movies.
As a novelty, this tool “goes beyond traditional translation and subtitles” since one of its main characteristics is that it is capable of preserving the identity of the speaker, as well as the speech patterns of the original person when translated. That is, it maintains the emphasis, feelings or grimaces of the speaker.
Likewise, the company has detailed that the voice of the same actor can be heard in more than twenty different languages among them Portuguese, Spanish, Japanese, Ukrainian, Polish or Arabic. In general, all languages that the company currently supports in the model are available Eleven Multilingual v2.
To develop this tool, ElevenLabs has explained that combine their multilingual speech synthesis technology and voice cloning technology, as well as text processing. All this with the aim of making the content more accessible to a broader audience, always “maintaining the essence of the original delivery.”
As stated by the director ejecutivo and ElevenLabs, Mati Staniszewski the release of ‘AI Dubbing’ is the company’s biggest step toward eliminating language content barriers. Therefore, “they will help the public enjoy any content they want, regardless of the language they speak.”
On the other hand, Staniszewski has also highlighted the boost it offers to content creators, who will be able to reach a larger audience around the world in a simple way.
In addition to all this, the dubbing tool also includes a self-developed system that allows detecting the voices of different speakers as well as eliminate the background noise of audio. For example, it is able to differentiate music or noise from other conversations from the voice of the main speaker.
This means that the new dubbing tool can effectively handle any audio or video containing multiple characters, ensuring that the dubbing continues to showcase the different voices with their own characteristics.
With ‘AI Dubbing’, ElevenLabs provides one way to automate the dubbing process guaranteeing “high quality” results, as it has been developed, in addition to allowing the creation of multilingual content for creators, educators and media companies.