DIY Photography

Your one stop shop for everything photo-video

  • News
  • Inspiration
  • Reviews
  • Tutorials
  • DIY
  • Gear
Search

Submit A Story

You’ve heard of text-to-image AI, now text-to-video is here

Aug 12, 2022 by Alex Baker Add Comment

  • Share
  • Tweet
  • Flipboard
  • WhatsApp

By now, you’ve probably heard of text-to-image prompt-based AI such as DALL-E2, Midjourney, and even Google‘s own version. These programmes pretty much do what they sound like – they take a text prompt or set of instructions, and turn them into an image. It’s usually either photo-realistic or it’s an illustration.

Tests have shown just how powerful this technology can be, and because it’s AI, it means that it’s constantly evolving and getting better and better. Well now, text-to-video AI has arrived, and it could be a complete game changer.

It’s not available yet, of course. We merely have a fleeting glimpse of what might be possible via a Twitter post from Runway. Runway is (according to its Twitter profile) a browser-based “professional video editing powered by artificial intelligence”.

In the example video, we see a man playing tennis on a clay court. A set of written prompts then changes the parameters. First, we see a tennis court on a sandy beach. The video background then changes so that the man appears to be playing tennis on a beach.

#stablediffusion text-to-image checkpoints are now available for research purposes upon request at https://t.co/7SFUVKoUdl

Working on a more permissive release & inpainting checkpoints.

Soon™ coming to @runwayml for text-to-video-editing pic.twitter.com/7XVKydxTeD

— Patrick Esser (@pess_r) August 11, 2022

Next, the prompt says “a tennis court on the moon” and we see the background change accordingly. The prompts go through a number of iterations, all the time the man in the foreground is unaffected.

Commenters have expressed their disbelief at how good this seems to be, with one saying that “the moon would have less gravity so the ball wouldn’t bounce that way!” Realism aside, however, this is a pretty big deal, particularly in the world of special effects and CGI. If you can merely type in a set of prompts to change aspects of a video, well, the mind boggles really.

And not least in terms of deep fakes and potential nefarious uses. But it looks as though this technology is firmly here to stay, and will be continuing to develop at an alarming rate. It will be interesting to see where this goes in the next year or so, no maybe even just the next few months!

FIND THIS INTERESTING? SHARE IT WITH YOUR FRIENDS!

  • Share
  • Tweet
  • Flipboard
  • WhatsApp

Related posts:

‘Do What the F*ck You Want’ is the image licensing option you’ve never heard of Google’s text-to-image AI will create photorealistic images of anything you can possibly imagine Rentmeester v. Nike Jumpman Logo Lawsuit to be Heard in June 2016 Things Clients Say To Photographers – How Many Have You Heard?

Filed Under: news Tagged With: AI, DALL-E 2, midjourney, text to image, text to video

Alex Baker: from diyphotography.net

About Alex Baker

Alex Baker is a portrait and lifestyle driven photographer based in Valencia, Spain. She works on a range of projects from commercial to fine art and has had work featured in publications such as The Daily Mail, Conde Nast Traveller and El Mundo, and has exhibited work across Europe

« Sony adds anti-theft crypto signature tech to in-camera images
PSA: Pixsy had a data breach; photographers, change your passwords now »

Submit A Story

Get our FREE Lighting Book

DIYP lighting book cover

* download requires newsletter signup

Recent Comments

Free Resources

Advanced lighting book

Learn photography

Recent Posts

  • Watch: Amateur astronomer captures a rare giant fireball on Jupiter
  • AI-generated deepfake nude images of school girls shake Spanish town
  • The Nikkor S Plena is a 135mm f/1.8 Z mount portrait lens
  • Meta promises you’ll soon be able to delete Threads without deleting Instagram
  • Getty jumps on the generative AI bandwagon

Udi Tirosh: from diyphotography.netUdi Tirosh is an entrepreneur, photography inventor, journalist, educator, and writer based in Israel. With over 25 years of experience in the photo-video industry, Udi has built and sold several photography-related brands. Udi has a double degree in mass media communications and computer science.

Alex Baker: from diyphotography.netAlex Baker is a portrait and lifestyle driven photographer based in Valencia, Spain. She works on a range of projects from commercial to fine art and has had work featured in publications such as The Daily Mail, Conde Nast Traveller and El Mundo, and has exhibited work across Europe

David Williams: from diyphotography.netDave Williams is an accomplished travel photographer, writer, and best-selling author from the UK. He is also a photography educator and published Aurora expert. Dave has traveled extensively in recent years, capturing stunning images from around the world in a modified van. His work has been featured in various publications and he has worked with notable brands such as Skoda, EE, Boeing, Huawei, Microsoft, BMW, Conde Nast, Electronic Arts, Discovery, BBC, The Guardian, ESPN, NBC, and many others.

John Aldred: from diyphotography.netJohn Aldred is a photographer with over 20 years of experience in the portrait and commercial worlds. He is based in Scotland and has been an early adopter - and occasional beta tester - of almost every digital imaging technology in that time. As well as his creative visual work, John uses 3D printing, electronics and programming to create his own photography and filmmaking tools and consults for a number of brands across the industry.

Dunja Djudjic: from diyphotography.netDunja Djudjic is a multi-talented artist based in Novi Sad, Serbia. With 15 years of experience as a photographer, she specializes in capturing the beauty of nature, travel, and fine art. In addition to her photography, Dunja also expresses her creativity through writing, embroidery, and jewelry making.

Copyright © DIYPhotography 2006 - 2023 | About | Contact | Advertise | Write for DIYP | Full Disclosure | Privacy Policy