Neural networks can now turn a 2D photo into a surprisingly accurate 3D render

Dec 8, 2016

Dunja Djudjic

Dunja Djudjic is a multi-talented artist based in Novi Sad, Serbia. With 15 years of experience as a photographer, she specializes in capturing the beauty of nature, travel, and fine art. In addition to her photography, Dunja also expresses her creativity through writing, embroidery, and jewelry making.

Neural networks can now turn a 2D photo into a surprisingly accurate 3D render

Dec 8, 2016

Dunja Djudjic

Dunja Djudjic is a multi-talented artist based in Novi Sad, Serbia. With 15 years of experience as a photographer, she specializes in capturing the beauty of nature, travel, and fine art. In addition to her photography, Dunja also expresses her creativity through writing, embroidery, and jewelry making.

Join the Discussion

Share on:

neural-networkd-3d-face

A detailed and accurate face mapping is a complex task. It requires a series of photos with ideal and consistent lighting from different angles. If you want to capture all the details and imperfections of the face, you need professional lighting and multiple shots. However, a group of researches is on the way of changing this method.

Researchers Hao Li, Shunsuke Saito, Lingyu Wei, Koki Nagano, and Liwen Hu use deep neural networks to create detailed and quite accurate 3D models using a single 2D photo. The researchers used an extensive online face database to create a library of fine details and textures of the face. Neural networks first filter through a network of possible textures, and then scan and blend the suitable facial features and skin tones with the 3D model. And the result is remarkable, even with the photos that aren’t exactly sharp.

The scientists submitted their research to Cornell University, where they state this invention “could widely impact new forms of immersive communication, education, and consumer applications.” Some video games, such as NBA 2K17, already offer the option of scanning your face for the game. However, the results are not always as accurate as you would like them to be. But unlike the NBA 2K17’s face scanner, the use of deep neural networks makes the 3D models very precise.tumblr_ohtqycd8b81qav3uso1_540

Since official face databases are easily available online (and tons of photos on social networks on top of that), there are some privacy and ethical questions to be raised. This method can provide a high degree of realism in virtual world, but it can also allow different kind of improper and unethical use. But, let’s hope it will just stay an important discovery and have many useful and ethical applications.

YouTube video

[Photorealistic Facial Texture Inference Using Deep Neural Networks (ArXiv 2016) via Gizmodo]

Filed Under:

Tagged With:

Find this interesting? Share it with your friends!

Dunja Djudjic

Dunja Djudjic

Dunja Djudjic is a multi-talented artist based in Novi Sad, Serbia. With 15 years of experience as a photographer, she specializes in capturing the beauty of nature, travel, and fine art. In addition to her photography, Dunja also expresses her creativity through writing, embroidery, and jewelry making.

Join the Discussion

DIYP Comment Policy
Be nice, be on-topic, no personal information or flames.

Leave a Reply

Your email address will not be published. Required fields are marked *