I Tried Prime Video's New AI Dubbing, and I Have Thoughts

We may earn a commission from links on this page.

When you watch a piece of media produced in a language you don't speak, you have a couple of options. You can, of course, turn on subtitles and follow along by reading. But in some cases, you can switch the audio track to whichever language you speak. This is called the "dub," and it usually requires voice artists to record the lines in this new language—in a way that matches the original tone of the show or movie.

This requires time and money, something studios aren't always willing to part with. As such, not everything has a dub. But what if it could?

On Wednesday, Amazon announced "AI-aided dubbing" for Prime Video. On select titles, viewers can choose to play an AI-generated dub of the script in either English or Latin American Spanish. Amazon says there are 12 titles that support this feature at this time, but only named three: El Cid: La Leyenda; Mi Mamá Lora; and Long Lost. The company also says "localization professionals" work with AI to "ensure quality control." It's not clear what that means, other than there are humans checking these AI dubs to make sure they aren't mistranslating or making other mistakes due to hallucinations.

I have Prime Video, so I was interested to check out these dubs for myself to see how Amazon's AI compared to, you know, human beings. In my search, I couldn't find Mi Mamá Lora, but I could find the other two titles. In order to use the feature, you need to go into the language selector on the compatible title and choose the "AI beta." Then, you can experience what I experienced.

El Cid: La Leyenda

There are a few titles on Prime Video going by the name of El Cid, but the only one that supports AI dubbing is this specific option: a 63 minute-long documentary about Rodrigo Díaz de Vivar, otherwise known as El Cid. I wasn't watching El Cid: La Layenda to learn more about this ruler from medieval Spain—I wanted to hear how well an AI bot would handle the English dubbing of a movie produced in Spanish.

The results are interesting, to say the least. Most of the speaking in the documentary happens via voiceover, which is extremely hit or miss. ("Hit" may be too generous.) When you know the voice is AI-generated, you pick out the bland or odd inflections. It's classic AI-generated speech: items in a list are spoken in inconsistent tones; some words are slurred or artificially slowed down, as if the bot has had a drink or two. But in all honesty, I was expecting worse. This isn't like watching a documentary narrated by the TikTok AI voice. It isn't great (it's not even good), but it does offer a bit more emotion than I expected.

The issues extend to the AI dubs of the documentary's interviewees. In many cases, the voice is quite awkward, stilted, and full of the inconsistencies mentioned above. At times, the voice is completely devoid of any emotion, and does sound more like a voiceover tool from years past, rather than the "high quality" AI voices we've come to expect.

To Amazon's credit, it does sound like the AI uses a different voice for each person it is dubbing. That makes it a bit easier to distinguish between the different speakers, and spares you from having to listen to the same mediocre AI voice for the entire movie.

Long Lost

Long Lost is a little more difficult to judge: The movie, originally produced in English, has an AI dub in Spanish. Since I don't speak Spanish, it's not as easy to pick out the robotic quirks your ear picks up on when you know the language.

Still, Long Lost offered a different experience than El Cid, since this is a movie: You get to hear how the AI handles dubbing over actors. Like the AI voiceover in the documentary, these AI actors aren't as bad as I expected. I was anticipating AI voices delivering their lines totally flat, but there is some emotion here at times. No robot is winning an Oscar here, mind you: In fact, the voices can be quite awkward and lacking, reading the script too loudly, abruptly, or plainly. But, for some lines, I'm not sure I would assume these were AI voices if I didn't already know ahead of time.

Even in more intense scenes, the AI does "try." When the character screams, the AI raises its voice. It's far from perfect, but it's an interesting experiment to see what the AI can do:

In one scene, two characters compete in a game of "Chubby Bunny," in which each has to continue adding marshmallows to their mouths while still being able to say "chubby bunny." At times, the AI actually sounds muffled, and while it isn't "realistic," it's a nice touch. (Perhaps this is where the human intervention comes into play.) At other times, however, the AI sounds totally normal, while the actors' mouths are clearly full of marshmallow.

Another interesting quirk: One of the characters starts speaking in French, so the AI track cuts out and the original audio comes in. It's jarring, because you can hear the difference immediately—not just the different languages, of course, but the difference in how the movie's actual audio sounds when compared to the sterile AI output.

I'd like to try this feature again on a movie produced in Spanish with an AI English dub. I imagine it would be easier for me to pick out the issues, and to know whether more (or all) of the voices would sound totally fake.

Is the future of dubbing AI?

Look, there is no way I would genuinely enjoy watching an entire movie or series with an AI dub. Personally, I'm not a huge fan of human-generated dubs, so I have zero interest in one that, at times, sounds like a robot gunning for the EGOT.

But I have to admit, the tech isn't terrible. At some points, the voices sound realistic enough for me to get the gist, especially when the voiceover would also be monotone and informative—such as during a documentary. And it's interesting that these AI dubs can incorporate elements from the show or movie, like muffled speech during a game of Chubby Bunny.

I don't want to see any voice actors put out of work here. They deserve to get paid for their labor and audiences deserve a high-quality dub on the shows and movies they pay to watch. However, I do think AI dubbing could offer a compromise for the times when no dub is on its way at all: If a studio isn't going to pay for a dub anyway, it would be helpful to have a dub at all—even if it's on the robotic side. That would certainly help visually impaired viewers, who might not be able to take advantage of subtitles.

Of course, it's naive to think studios will only use this tech for those specific uses, rather than eliminate an entire human profession to save some cash. To that end, I'd be happy if the whole AI dubbing practice ends here and now: It's interesting, but it's not good, so please don't replace real voice actors.

View the full article