Neural networks can now turn a 2D photo into a surprisingly accurate 3D render

A detailed and accurate face mapping is a complex task. It requires a series of photos with ideal and consistent lighting from different angles. If you want to capture all the details and imperfections of the face, you need professional lighting and multiple shots. However, a group of researches is on the way of changing this method.

Researchers Hao Li, Shunsuke Saito, Lingyu Wei, Koki Nagano, and Liwen Hu use deep neural networks to create detailed and quite accurate 3D models using a single 2D photo. The researchers used an extensive online face database to create a library of fine details and textures of the face. Neural networks first filter through a network of possible textures, and then scan and blend the suitable facial features and skin tones with the 3D model. And the result is remarkable, even with the photos that aren’t exactly sharp.

The scientists submitted their research to Cornell University, where they state this invention “could widely impact new forms of immersive communication, education, and consumer applications.” Some video games, such as NBA 2K17, already offer the option of scanning your face for the game. However, the results are not always as accurate as you would like them to be. But unlike the NBA 2K17’s face scanner, the use of deep neural networks makes the 3D models very precise.

Since official face databases are easily available online (and tons of photos on social networks on top of that), there are some privacy and ethical questions to be raised. This method can provide a high degree of realism in virtual world, but it can also allow different kind of improper and unethical use. But, let’s hope it will just stay an important discovery and have many useful and ethical applications.

[Photorealistic Facial Texture Inference Using Deep Neural Networks (ArXiv 2016) via Gizmodo]