Your Next Smart Selection Could Be Powered By A Neural Network

Jun 29, 2016

Udi Tirosh

Udi Tirosh is an entrepreneur, photography inventor, journalist, educator, and writer based in Israel. With over 25 years of experience in the photo-video industry, Udi has built and sold several photography-related brands. Udi has a double degree in mass media communications and computer science.

Your Next Smart Selection Could Be Powered By A Neural Network

Jun 29, 2016

Udi Tirosh

Udi Tirosh is an entrepreneur, photography inventor, journalist, educator, and writer based in Israel. With over 25 years of experience in the photo-video industry, Udi has built and sold several photography-related brands. Udi has a double degree in mass media communications and computer science.

Join the Discussion

Share on:

cnn-masking-03

Admit it! object masking sucks. It’s not that it is impossible, even the hardest selections and masks can be created with some work (and some methods require less work that others). But, in general, masking is a hard and tedious work.

Researchers in the The Chinese University of Hong Kong working with Adobe Research are now showing some work that uses Convolutional Neural Networks to successfully mask portraits. The paper bears the boring name “Automatic Portrait Segmentation for Image Stylization” [PDF link], and it shows how selection and masking can significantly improve if the software knows that it’s making a portrait.

In an essence, the method shows how using several Neural Networks to successfully mask a portrait (and then suspect it to “stylising”). The article focuses on selfie shots (which would explain the stylising motivation), from what I gather, “stylising” is university-speak for instagram filters). In the researcher’s words:

The bulk of these portraits are captured by casual photographers who often lack the necessary skills to consistently take great portraits, or to successfully post-process them. Even with the plethora of easy-to-use automatic image filters that are amenable to novice. This work was done when Xiaoyong was an intern at Adobe Research. photographers, good portrait post-processing requires treating the subject separately from the background in order to make the subject stand out.

But, there is another reason, selfies are not taken by professional photographers (well, not always), so the image may not be well composted, it may not be in goo orientation, face parts may be hidden and other such shenanigans. Once the Network is given a subject though, it can detect those variations and mask out the face.

cnn-masking-02

The following flowchart shows the flow of the algorithm: after a face is detected, it is treated for alignment and normalization and then the final neural network masks it.

cnn-masking-04

Is the algorithm perfect? I guess that considering the fact that some of the researchers are from Adobe, I guess we will find out in one of the next Photoshop versions.

[The Chinese University of Hong Kong (pdf) via SolsticeRetouch]

Filed Under:

Tagged With:

Find this interesting? Share it with your friends!

Udi Tirosh

Udi Tirosh

Udi Tirosh is an entrepreneur, photography inventor, journalist, educator, and writer based in Israel. With over 25 years of experience in the photo-video industry, Udi has built and sold several photography-related brands. Udi has a double degree in mass media communications and computer science.

Join the Discussion

DIYP Comment Policy
Be nice, be on-topic, no personal information or flames.

Leave a Reply

Your email address will not be published. Required fields are marked *

One response to “Your Next Smart Selection Could Be Powered By A Neural Network”

  1. Vertex Avatar
    Vertex

    adobe.. and perfect in the same sentence? :)

    content aware fill is not perfect, shake reduction is not perfect, focus select is not perfect.
    adobe just puts BETA features in it apps and that´s it…..