Adobe Firefly unveils insane AI tools for text-to-video and audio creation

If you were impressed with Adobe’s Firefly AI text-to-image software, hold onto your hats, you ain’t seen nothin’ yet! Adobe just unveiled today the video version of Firefly. And it’s staggering.

It won’t be available even as a beta version until later this year, but Adobe has wowed everyone at NAB with what it can do so far to enhance your video and audio workflow. And just like the stills, Firefly is still being billed as the ‘ethical AI’ option as it was trained only on stock images and footage already in Adobe’s libraries.

These new features are integrated into the Firefly ecosystem in a way that is designed to help content creators, not eliminate them. I must say that it’s looking very tempting to try out and will greatly increase the speed of getting small projects out into the world. Perfect for small content creators.

The main features are as follows:

Text to color enhancements

It’s easy to create or use a LUT to alter the color of video clips, but it can be time-consuming to create just the right thing. Now you can change color schemes, time of day, or even the seasons in already-recorded videos, instantly altering the mood and setting to evoke a specific tone and feel. A simple prompt like “Make this scene feel warm and inviting” will create what you need in seconds.

Advanced music and sound effects

Don’t have a particular piece of audio or sound effect for your video? Instead of scrolling through those audio libraries, you can just type in what you need. Creators can easily generate royalty-free custom sounds and music to reflect a certain feeling or scene for both temporary and final tracks.

Stunning fonts, text effects, graphics, and logos

This one looks really cool, especially if, like me, you don’t have the patience to learn After Effects. Now, with a few simple words and in a matter of minutes, creators can generate subtitles, logos and title cards and custom contextual animations. The text animations are particularly cool!

Powerful script and B-roll capabilities

I’ve often thought that one positive of these image generators is for creating storyboards and plans. Firefly has jumped on this, making it really easy to make complex animated storyboards. Creators can dramatically accelerate pre-production, production and post-production workflows using AI analysis of script-to-text to automatically create storyboards and previsualizations, as well as recommending b-roll clips for rough or final cuts.

But perhaps the most impressive part of this is the ability to create B-roll, purely from a few lines of text. I mean, how many of us have come away from a shoot and started editing, only to find that we didn’t shoot enough B-roll? It happens. If this is really as good as the video suggests, this feature is absolutely mind-blowing!

Creative assistants and co-pilots

Here’s an example of AI being useful once again. Now you can harness its ability to teach you new skills. With personalized generative AI-powered “how-tos,” users can master new skills and accelerate processes from initial vision to creation and editing.

So there you have it. Adobe Firefly is trailblazing in its AI text-to-video and will be able to be used seamlessly with Premiere Pro and After Effects. It could become an incredibly useful and powerful tool, and I believe, contrary to all the fear-mongering going on currently about AI, could be incredibly helpful for content creators.

Unfortunately, although Firefly is available now for select Beta users, these new AI video tools won’t be available to use until later on in the year.