We all knew it was only a matter of time before Adobe started implementing its Firefly-powered generative fill-type features in its video products. I have to admit, I didn’t think it’d be quite so little time, but it looks like it’s coming soon.
At Adobe MAX, Adobe is showing off a sneak peek of some new features coming to Creative Cloud. Amongst them is “Project Fast Fill”, which brings Firefly generative AI features to Premiere Pro.
Unfortunately, there’s no video of the new features in action yet – although they’re expected to come at some point before the end of Adobe Max. As they become available, you can view them here. Adobe has also released no images to go along with these announcements yet, so I’ve illustrated this article with images created using Adobe’s new Firefly 2 engine.
Adobe Premiere Pro – Project Fast Fill
So, what’s new? Well, the big one is Project Fast Fill. This is essentially the same generative AI that powers Photoshop’s Generative Fill feature, except much more advanced – and hopefully a lot quicker. Naturally, being a feature for Premiere Pro, this one is targeted towards video. Adobe says that it may also be coming to Adobe After Effects.
Project Fast Fill brings the power of Firefly generative AI to video for the first time. Leveraging Generative Fill technology – which Adobe Photoshop already uses to ease the addition, removal, or expansion of content in images with simple Firefly-powered text prompts – Project Fast Fill offers an early look at what human-prompted generative AI could enable inside Adobe video editing tools including Premiere Pro and After Effects.
Adobe Blog
As mentioned, Adobe hasn’t released a video showing exactly how it works yet, but that’s coming in the next day or two. The concept of generative AI for video is intriguing. There’s obviously a lot of opportunity for potential abuse here. Adobe will have to keep a very close eye on that to make sure it doesn’t happen.
Of course, there are plenty of legitimate reasons for using generative AI with video, too. Perhaps you’re shooting a period piece and need to remove a satellite dish off the side of a house in the distance. Or clean up a bit of trash from your footage that you weren’t able to clean up on set.
Exactly what capabilities generative AI in Premiere Pro or After Effects will have remains to be seen. It’s still in the early prototype days and it may be a little while off being implemented just yet. But hopefully, when Adobe releases the Sneaks video for this feature, we’ll learn a lot more.
Other audio & video Sneaks from Adobe MAX
Amongst all of the other things, three other features are being demonstrated for audio and video use. The first is called “Project Dub Dub Dub”, and it’s designed to help automate the dubbing process.
The idea behind it is a concept we’ve seen before. But it’s designed to help you overdub your footage, perhaps with audio from a different take or a dub over a completely different language. It uses the audio to modify the footage to make your actor’s mouth look like they actually spoke the words.
Another is “Project Scene Change”. This lets you take footage from two different cameras pointed at a scene from different viewpoints and then create seamless, smooth motion between the two of them. This is going to be a very interesting one to play with!
Finally, “Project Res Up”, which as the name suggests, is for increasing the resolution of your video footage. It uses a diffusion-based upsampling technology to increase the resolution of the shot. This could be extremely useful for slow-motion shots when limited to 1080p for those high speed in a 4K project.
Plenty of other Sneaks at Adobe MAX
There are plenty more Sneaks besides this one, but I felt this justified its own post, as it’ll be the first video editing application to incorporate generative AI directly into the software itself.
Currently, pretty much all AI tools capable of generating video are dedicated apps. You have to export your clip from your editor, run it through this dedicated tool and then import it back in. Soon (maybe), you’ll be able to do it all directly within Premiere Pro and After Effects.
But other features are coming to Photoshop, including “Project Stardust”, allowing you to more easily select many items from complex scenes, as well as some pretty major AI-powered 3D and design tools.
You can see the list on the Adobe blog, and be sure to keep an eye on the Adobe Max website for the upcoming videos. As to when they’ll actually come to our desktops, though, is anybody’s guess.
Note: All images in this post were generated using the current beta release of Adobe Firefly 2.
FIND THIS INTERESTING? SHARE IT WITH YOUR FRIENDS!