Google’s latest upscaling technology makes CSI TV shows a reality

Google has introduced a new upscaling technology that turns low-resolution images into detailed high-resolution ones It can start with a portrait as tiny as 64×64 upscale it to 1024×1024 while preserving all the detail.

Researchers Jonathan Ho and Chitwan Saharia shared the details of the technology in a post on Google AI Blog. The new upsampling approach relies on diffusion models, which Google first proposed back in 2015. These models learn by gradually corrupting the image until it becomes “pure noise.” Then, the algorithm learns to gradually denoise it until it produces a clean image again.

For the latest approach, the researchers combined Super-Resolution via Repeated Refinements (SR3) and a model for class-conditioned synthesis, called Cascaded Diffusion Models (CDM).

The Image Super Resolution works more or less like the diffusion model described above. After the algorithm is trained, takes as input a low-resolution image, and builds a corresponding high resolution image from pure noise.

CDM is a class-conditional diffusion model trained on ImageNet data to generate high-resolution natural images, and Google built it “as a cascade of multiple diffusion models.”

“This cascade approach involves chaining together multiple generative models over several spatial resolutions: one diffusion model that generates data at a low resolution, followed by a sequence of SR3 super-resolution diffusion models that gradually increase the resolution of the generated image to the highest resolution.”

With the latest upsampling technique, Google also introduced a new data augmentation technique called conditioning augmentation, which should improve the sample quality results of CDM even further. You can read the full blog post on Google AI Blog and learn a bit more about the whole process and its results.

A few uses of this technique come to mind. One of them is upsampling your own photos downloaded from Facebook, Instagram, or WhatsApp that compress them. It can sometimes be faster than searching for originals. On the other hand, it could be tricky because someone can steal a small resolution of your photo, upsample it and use it as they please. Like with many technologies, this one’s not good or bad per se, it all depends on who uses it.

Ultimately, it could be of help to the authorities when trying to identify criminals from low-res security camera shots. Maybe those cringeworthy “Enhance!” moments from the CSI TV series will finally become a reality.

[via DPReview]