DIY Photography

Your one stop shop for everything photo-video

  • News
  • Inspiration
  • Reviews
  • Tutorials
  • DIY
  • Gear
Search

Submit A Story

This video explains Google’s Super Resolution algorithm in just three minutes

May 31, 2019 by John Aldred 6 Comments

  • Share
  • Tweet
  • Flipboard
  • WhatsApp

There are so many new features coming to smartphone cameras these days that it can be difficult to keep up. You’ve got all kinds of HDR, fake bokeh, high resolution and high detail modes that all work in a bunch of different ways. Most of these features are there to overcome limitations in the tiny sensors that smartphones tend to contain.

One feature is Google’s Super Resolution algorithm. A computational photography technique that essentially replaces the demosaicing process used on most digital cameras. It’s quite fascinating, looks much better than most of the alternatives, and it’s pretty clever how it works.

So, to understand why this is such a cool process, one needs to know how Bayer and similar (like Fuji X-Trans) sensors work. Bayer and similar types of sensor are in just about every camera out there except for Sigma Foveon. Each pixel on the sensor is assigned a colour. Each only sees that one colour, either red, green or blue, at a particular level of brightness. This means that the sensor doesn’t see the entire sensor for all colours and results in a lot of missing information. The demosaicing process interpolates this missing data to present you with the final colour image.

Sigma’s Foveon technology actually does see all three colours for every pixel on the sensor, and this is sort of what Google’s attempting to emulate with this technique.

Put simply, Google’s algorithm takes multiple images, using the slight hand movements from the person holding the phone. The slight misalignment of the scene relative to the pixels fills in some that missing data. After aligning all the images, the algorithm is able to pull different pieces of real colour information from real pixel data. Instead of guessing the missing information, it’s able to create three complete (in theory) red, green and blue channels.

We harness natural hand tremor, typical in handheld photography, to acquire a burst of raw frames with small offsets. These frames are then aligned and merged to form a single image with red, green, and blue values at every pixel site. This approach, which includes no explicit demosaicing step, serves to both increase image resolution and boost signal to noise ratio. Our algorithm is robust to challenging scene conditions: local motion, occlusion, or scene changes. It runs at 100 milliseconds per 12-megapixel RAW input burst frame on mass-produced mobile phones.

– Google Research Team

There is no doubt that computational photographer has come on in leaps and bounds over the last few years, and no doubt it will continue to do so in the coming years as smartphone cameras improve and their processors get even faster and more capable.

It’s a very cool technique, and you can read more about Handheld Multi-frame Super-resolution mode here.

[via DPReview]

FIND THIS INTERESTING? SHARE IT WITH YOUR FRIENDS!

  • Share
  • Tweet
  • Flipboard
  • WhatsApp

Related posts:

Does Adobe’s Super Resolution eliminate the need for high resolution cameras? Comparing Topaz Gigapixel AI vs Photoshop Super Resolution – Which is best? Super high resolution “more than 75-megapixel” Canon EOS R on the way next year? AI-powered Super Resolution upscaling is now available in Adobe Camera Raw

Filed Under: news Tagged With: google, smartphone photography, Super Resolution

John Aldred: from diyphotography.net

About John Aldred

John Aldred is a photographer with over 20 years of experience in the portrait and commercial worlds. He is based in Scotland and has been an early adopter - and occasional beta tester - of almost every digital imaging technology in that time. As well as his creative visual work, John uses 3D printing, electronics and programming to create his own photography and filmmaking tools and consults for a number of brands across the industry.

« Judas Priest’s Rob Halford goes full metal, kicks phone out of fan’s hand during concert
LIT xenon smartphone flash cancels Kickstarter campaign despite exceeded funds »

Submit A Story

Get our FREE Lighting Book

DIYP lighting book cover

* download requires newsletter signup
DIYPhotography

Recent Comments

Free Resources

Advanced lighting book

Recent Posts

  • Here’s a bullet time video booth you can build yourself
  • Ricoh has discontinued the HD PENTAX-DA 21mm F3.2AL Limited silver lens
  • This “stellar flower” unravels the twilight’s evolution in 360 degrees
  • Strobes vs Continuous LEDs – Which is right for you?
  • Wave goodbye to Apple’s My Photo Stream next month

Udi Tirosh: from diyphotography.netUdi Tirosh is an entrepreneur, photography inventor, journalist, educator, and writer based in Israel. With over 25 years of experience in the photo-video industry, Udi has built and sold several photography-related brands. Udi has a double degree in mass media communications and computer science.

Alex Baker: from diyphotography.netAlex Baker is a portrait and lifestyle driven photographer based in Valencia, Spain. She works on a range of projects from commercial to fine art and has had work featured in publications such as The Daily Mail, Conde Nast Traveller and El Mundo, and has exhibited work across Europe

David Williams: from diyphotography.netDave Williams is an accomplished travel photographer, writer, and best-selling author from the UK. He is also a photography educator and published Aurora expert. Dave has traveled extensively in recent years, capturing stunning images from around the world in a modified van. His work has been featured in various publications and he has worked with notable brands such as Skoda, EE, Boeing, Huawei, Microsoft, BMW, Conde Nast, Electronic Arts, Discovery, BBC, The Guardian, ESPN, NBC, and many others.

John Aldred: from diyphotography.netJohn Aldred is a photographer with over 20 years of experience in the portrait and commercial worlds. He is based in Scotland and has been an early adopter - and occasional beta tester - of almost every digital imaging technology in that time. As well as his creative visual work, John uses 3D printing, electronics and programming to create his own photography and filmmaking tools and consults for a number of brands across the industry.

Dunja Djudjic: from diyphotography.netDunja Djudjic is a multi-talented artist based in Novi Sad, Serbia. With 15 years of experience as a photographer, she specializes in capturing the beauty of nature, travel, and fine art. In addition to her photography, Dunja also expresses her creativity through writing, embroidery, and jewelry making.

Copyright © DIYPhotography 2006 - 2023 | About | Contact | Advertise | Write for DIYP | Full Disclosure | Privacy Policy