Artificial intelligence has already gone pretty far in face swapping. In fact, there are some deepfake videos that are very difficult to figure out. But Disney Research Studios and ETH Zurich just made it even more precise, and also cheaper than before. And it can be so accurate that it’s kinda creepy.
Disney Research Studio published a paper explaining the research and the results they got with the new algorithm. Of course, you can also see some examples in the video above. The researchers relied on several existing methods: encoder-decoder (autoencoder) methods, GAN-based image-to-image translation, and geometry-based morphable models.
The resulting face swaps gave realistic results at a 1024 x 1024 resolution and even preserved a high level of detail and contrast. For comparison, the researchers only achieved 256 × 256 pixels with DeepFaceLab on an 11GB GPU. or DeepFakes, they were able to produce 128 × 128 images, and the morphable models approach gave a resolution of 500×500 pixels. However, the results of the morphable models approach were typically unrealistic.
I can’t even start to wonder how many bad implications these face swaps could have. However, Disney didn’t have anything bad in mind when developing the algorithm. We all know by now that CGI is used to recreate actors that have aged or deceased. Just remember Carrie Fisher and Peter Cushing in Rogue One. This is the major goal of the new algorithm: to recreate actors accurately and relatively cheap. According to Disney, other applications include stunt scenes that would be dangerous for an actor to perform but still require high-quality face images.
If you’d like to read the full paper, you can find the PDF here.