Gen AI Tools, Sora, Firefly Video, Coming in 2024

Adobe’s generative artificial intelligence flagship is Firefly, a text-to-image tool that powers generative fill-in for Photoshop and text-to-vector in Illustrator. However, the 2024 Firefly model will power three AI video tools inside video editing software Premiere Pro: Generative Extend, Object Addition and Removal, and Text to Video.

Premiere already allows video editors to integrate Adobe creative cloud software like After Effects (motion graphics) and Audition (sound mixing). Today’s announcement makes clear that Adobe will take the same approach with AI. The company also announced that it is in “early explorations” to integrate video generation from OpenAI (Sora), Runway, and Pika Labs into Premiere.

Last week, Adobe demonstrated these new gen AI tools for IndieWire and shared examples (see video at the top of the page) of what will be available inside Premiere Pro before the end of the year.

Generative Extend

Every editor has wished for just a few more frames to allow for a smoother edit or transition. The Gen Extend tool would allow editors to use AI to create new frames of up to a “few seconds” that Adobe promises will “seamlessly add frames to make clips longer.”

In the preview video, the tool extends the actress’ stoic performance by a couple of seconds to fill a gap in the editor’s timeline. While extending performance introduces ethical and union issues,…