This AI-based app “paints in” new objects to your photos

Seeing the advancement of photo-related AI is impressive. Just as you can use it to remove stuff from your photos, you can add things, too. A new app named GANPaint Studio can “paint-in” completely new objects to your photos, or “restyle” things that already exist in your shots.

GANPaint Studio is a joint project by researchers from MIT CSAIL, IBM Research, and the MIT-IBM Watson AI Lab. Basically, it lets you use a “brush” to “paint in” different stuff into your photos. But, instead of painting in simple brush strokes, this brush paints in objects that make sense in the context of the photo. You can “draw” trees, domes, brick-textures, doors, windows…

Here’s a short video that demonstrates GANPaint Studio in action:

The researchers explain how the app works:

“The core part of GANPaint Studio is a neural network (GAN) that can produce its own images of a certain category, e.g. kitchen images. In previous work, we analyzed which internal parts of the network are responsible for producing which feature (project GANDissect). This allowed us to modify images that the network produced by “drawing” neurons.”

If you want a full explanation and technical details, you can check them out on GANPaint Studio’s website and even download the paper here.

For now, only the demo version of this app is available, and you can try it out here. We can’t tell when and how it will be implemented, but I sure hope that it won’t be misused like “artificial faces” recently were. And even if that happens, it would be fun to see Adobe’s algorithm determining which images were altered and which weren’t.

[via PetaPixel]