Deepfake technology is now being used to “visually dub” movies to remove profanity for PG-13 ratings

This is probably the most practical and one of the least creepy ways I’ve seen deepfake technology being used since it was initially created. While it’s had some very impressive showcases, particularly as an alternative to badly CGI’d versions of actors looking years younger than they are or even bringing them back from the dead, this application is a little less drastic and a little more seamless.

In what is being termed “vubbing”, deepfake technology is being used to generate new frames when lines are changed in post or certain things (like profanity) are cut out completely in order to appease the censors (the MPAA, in this case). With deepfake technology falling far below the budget of reshoots, it makes a lot of sense, but that the technology has come this far so quickly is also very impressive.

As No Film School reports, the idea seems to stem from the independent action-thriller Fall. In one particular scene, two friends are climbing an abandoned radio tower when a section of the ladder breaks away and the pair are left stranded on the tower. In what would probably be the natural response for many of us, the occasional profanity slips out regarding the situation but the Motion Picture Association of America (MPAA) didn’t approve it for the PG-13 rating the crew needed in order to maximise the potential audience. With a production budget of what is reported to be only $3 million, reshooting the scenes and angles all over again with new lines just wasn’t an option. So, they turned to deepfake.

It just so happens, though, that Scott Mann, the director and co-writer of Fall founded a company called Flawless in 2021. It was originally established as a company that used deepfake as a way to more realistically dub movies from one language to another – another fantastic use of deepfake technology – and have the mouth movements match up to whatever language the voice was speaking rather than seeing the original mouth movements in the movie’s originally recorded language. Mann realised he could apply this technology to the situation they were facing with Fall and the MPAA.

And it appears to have worked. Actors were even reported to say that they couldn’t tell when things had changed and what was an original recording. “As far as I know, every movement my mouth made in that movie, my mouth made”, said actor Grace Carolina Curry. The movie did get its PG-13 rating in the end, too.

I still think there’s a little way for it to go in order for it to become 100% believable all of the time but in the samples video above, it looks pretty close. It’s only a matter of time before you won’t be able to tell the difference between the AI-generated shots and the real thing. At least, you wouldn’t know if you didn’t realise who the actors were and hadn’t seen the movie already in its native language.

It’s a great tool for the movie industry, no doubt, but it’s definitely scary how good it’s getting. The implications outside of Hollywood are far-reaching.

Now, if only somebody would run Demolition Man through this so that when they say “Pizza Hut” they’re not mouthing the words “Taco Bell”. Or, you know, just put the original dialogue back.

[via No Film School / Lead Image: Lionsgate]