DIY Photography

Hacking Photography - one Picture at a time

  • News
  • Inspiration
  • Reviews
  • Tutorials
  • DIY
  • Gear
Search

Submit A Story

This algorithm creates realistic video from a single photo

Jun 16, 2021 by Dunja Djudjic 1 Comment

  • Share
  • Tweet
  • Flipboard
  • WhatsApp

Researchers at the University of Washington have developed a deep learning method that only needs a single photo to make a believable video. If you have a photo of a waterfall, a river, smoke, or clouds, it predicts the previous and the next frame and creates a pretty cool animation.

The team described their method in a paper and will present it at the Conference on Computer Vision and Pattern Recognition on 22 June. Aleksander Hołyński is a doctoral student in the Paul G. Allen School of Computer Science & Engineering and the lead author of the paper, and he spoke a bit about the project for UW News.

“What’s special about our method is that it doesn’t require any user input or extra information,” Hołyński said. “All you need is a picture. And it produces as output a high-resolution, seamlessly looping video that quite often looks like a real video.” Here’s one of the examples the team ended up with:

Credits: Sarah McQuate/University of Washington

As you probably know, this isn’t the first program that turns a photo into a video or a cinemagraph. You can do it yourself in Photoshop, Premiere Pro, and After Effects. There have also been Photoshop plugins like Artymate, or other predictive algorithms that are still far from perfect. The key is to make the end video believable, and there are a bunch of challenges for doing it right.

Hołyński explains that turning a photo into a video requires the algorithm to predict the future. “And in the real world, there are nearly infinite possibilities of what might happen next,” he adds. So, he and his team trained a neural network with thousands of videos of waterfalls, rivers, oceans, and other material with fluid motion. They would first ask the network to predict the motion of a video only by the first frame. Then, it would compare its prediction with the actual video, which helped it learn to identify clues that tell it what was going to happen next (such as ripples in a stream, for example).

The researchers tried to use “splatting,” a technique that moves each pixel according to its predicted motion. However, it posed another set of challenges. “Think about a flowing waterfall,” Hołyński told UW News. “If you just move the pixels down the waterfall, after a few frames of the video, you’ll have no pixels at the top!” So they had to come up with a solution for this, and they called it “symmetric splatting.” It doesn’t only predict the future, but also “the past” of an image, creating a seamless animation.

For now, the algorithm works best with flowing materials, like rivers, waterfalls, smoke, or clouds. Essentially anything that has predictable and flowing motion. But in the future, the team would like to extend its possibilities and animate other things, for example, someone’s hair blowing in the wind.

[via UW News]

FIND THIS INTERESTING? SHARE IT WITH YOUR FRIENDS!

  • Share
  • Tweet
  • Flipboard
  • WhatsApp

Related posts:

Neural network creates photo-realistic images of people who aren’t real Photographer creates realistic locations for miniature photography on a budget A single photo creates optical illusion that will mess with your mind New Adobe’s algorithm transfers style of one photo onto the other

Filed Under: news Tagged With: algorithm, animation, cinemagraph, cinemagraphs

About Dunja Djudjic

Dunja Djudjic is a multi-talented artist based in Novi Sad, Serbia. With 15 years of experience as a photographer, she specializes in capturing the beauty of nature, travel, and fine art. In addition to her photography, Dunja also expresses her creativity through writing, embroidery, and jewelry making.

« The H&Y 100 filter is one clever magnetic system
This video was made from 37,000 film photos and inspired by early cinema »

Submit A Story

Get our FREE Lighting Book

DIYP lighting book cover

* download requires newsletter signup
DIYPhotography

Recent Comments

Free Resources

Advanced lighting book

Recent Posts

  • TikTok bans AI deep fakes from platform
  • What motivates you to take photographs?
  • DPReview is shutting down in three weeks
  • Panasonic S1H II specs leak – 8K and phase detect AF
  • Adobe launches Firefly AI – no unlicensed photos for training

Alex Baker is a portrait and lifestyle driven photographer based in Valencia, Spain. She works on a range of projects from commercial to fine art and has had work featured in publications such as The Daily Mail, Conde Nast Traveller and El Mundo, and has exhibited work across Europe

Dave Williams is an accomplished travel photographer, writer, and best-selling author from the UK. He is also a photography educator and published Aurora expert. Dave has traveled extensively in recent years, capturing stunning images from around the world in a modified van. His work has been featured in various publications and he has worked with notable brands such as Skoda, EE, Boeing, Huawei, Microsoft, BMW, Conde Nast, Electronic Arts, Discovery, BBC, The Guardian, ESPN, NBC, and many others.

John Aldred is a photographer with over 20 years of experience in the portrait and commercial worlds. He is based in Scotland and has been an early adopter - and occasional beta tester - of almost every digital imaging technology in that time. As well as his creative visual work, John uses 3D printing, electronics and programming to create his own photography and filmmaking tools and consults for a number of brands across the industry.

Dunja Djudjic is a multi-talented artist based in Novi Sad, Serbia. With 15 years of experience as a photographer, she specializes in capturing the beauty of nature, travel, and fine art. In addition to her photography, Dunja also expresses her creativity through writing, embroidery, and jewelry making.

Copyright © DIYPhotography 2006 - 2023 | About | Contact | Advertise | Write for DIYP | Full Disclosure | Privacy Policy