Colorization and restoration of old photos is a painstaking and time-consuming process, especially if you’re working with heavily damaged images. Computer vision team of Mail.ru Group has introduced an AI-powered tool that will make his process simpler and easier. They’ve even launched a website where you can test it out and restore the vintage photos from your old family album. Or any other vintage photos, if you prefer.
The AI-based photo restoration project uses Vision Technology from Mail.ru. It was initially made for restoring old military photos, for different purposes: including them in the “Immortal Regiment” gallery, printing them for 9th of May parades, or putting in family albums.
Fedor Kitashov, a research engineer at the Mail.ru Group computer vision team, explains the process in detail in this article. The method consists of three steps:
- Finding all the image defects: fractures, scuffs, holes
- Inpainting the discovered defects, based on the pixel values around them
- Colorizing the image
Basically, the scientists first trained a neural network to find all damaged areas in images. They worked together with “Immortal Regiment” project, who shared their data with Mail.ru Group. The data consists of photos from The Second World War, and the scientists noticed that people upload “mostly individual or group portraits with a moderate to a large number of defects.”
Credit: Mail.ru
The second step was to inpaint the damaged areas, and this is similar to the PEN-Net AI that we covered earlier today. “To do inpainting we’d upload an original image and a mask where we marked all the clean area with ones, and with zeros — all the pixels we want to inpaint,” Kitashov explains. “For any photo from an open-source image dataset, for example, OpenImagesV4, we add the defects similar to those we see in real life. Then we’d trained the net to restore the missing parts.”
Finally, there’s colorization. While there are several software solutions available, the team decided to come up with their own algorithm for colorization. They say that none of the existing services could color the portraits quickly and efficiently, and they wanted their colorized photos to be more believable.
Credit: Mail.ru
Of course, the tool also works with photos other than military images. I tested it out on different photos I found on Wikimedia Commons and in Europeana gallery.
Lt. Col.Ujvárossy Károly, Credit: Kiss Geza
Serbian writer Miloš Crnjanski:
Frida Kahlo, photographed by her father Guillermo Kahlo:
As you can see, the results are not perfect. I mean, if you’re looking for perfection, there’s still no better solution than a skilled human restoration expert. However, I’d say that this colorization and restoration work is pretty good, considering that it’s something done by artificial intelligence, literally in a matter of seconds.
You can read Fedor Kitashov’s article here. It’s pretty technical, but it could be an interesting read to those of you who have sufficient knowledge in computer engineering. Also, you can take a closer look at the original images and all the processing stages here. And last but not least, feel free to test out the tool yourself on Mail.ru’s website.
FIND THIS INTERESTING? SHARE IT WITH YOUR FRIENDS!