How AI Will Turn Us All Into Filmmakers

As artificial intelligence sneaks its way into all corners of life—driving, finance, chatbots—everyday video will become ripe for an upgrade.
illustration with a robot and woman looking at a film projector.
ZOHAR LAZAR

In high school, Mackenzie Leake shot a movie about being afraid to get her driver’s license. “A very millennial subject,” she jokes. It gave her a punishing lesson in editing video: Leake spent countless hours, over the course of weeks, “scrubbing” through her footage to find the best shots, then painstakingly assembling them. “It’s a ton of grunt work,” she notes.

Now, seven years later, she’s trying to accelerate the process. A graduate student in computer science at Stanford, Leake is working with a team to make a piece of software that automatically analyzes scripted film footage. The program uses AI to recognize faces and shot types, and even transcribes the spoken words, so it knows where each frame ought to line up with the dialog. Once the AI has done its work, you can specify the style of shot you’d like to use in your edit—full-frame close-ups, wide shots—and voilà: The software splices together a take using those parameters. In essence, it’s a robot that produces a surprisingly decent-looking result in a few seconds. If you don’t like the edit, you can have it generate a new one.

Leake’s work is still in the lab. But pretty soon, I predict, you’ll be using software much like this. Because as AI sneaks its way into all corners of life—driving, finance, chatbots—everyday video will become ripe for an upgrade. An AI-propelled boom is about to transform the way we wield the moving image.

Lord knows we need the help. As platforms like YouTube, Instagram, and Snapchat have proved, video has become a dominant mode of communication and public rhetoric. And carrying a high-quality video camera in our pockets means we’re recording more moving pictures than ever before.

Ah, but to turn those infinite hours of video into something more complex and polished than, say, an Instagram Story? That’s still hard. Editing tools are more useful now, but video is still more time-consuming and trickier to wrestle with than text or photos. In Word or Google Docs, it’s easy to quickly find words and cut and paste and sort them. Video, in contrast, is “inherently big data,” as Leake puts it, combining audio, movement, and imagery.

Which is why AI is so powerful. Consider another tool—in its private beta release—called Reduct.video. A new version under testing lets you upload video, then uses machine learning to transcribe any speech. It tags each word to the corresponding visual frames. This allows for some nifty editing tricks: If you cut and paste a few sentences of transcript together, Reduct composites precisely those chunks of video too (albeit with jump-points that can be a bit jerky).

“You’re editing the video by editing the text,” cofounder Prabhas Pokharel tells me.

One of Reduct’s clients does customer research, recording dozens of hours of interviews, then uses the tool to quickly generate two-minute highlight clips. These AI-­powered capabilities are popping up in major editing tools, too: The firm Digital Anarchy recently launched Transcriptive, which uses autogenerated transcripts to help parse clips in Adobe’s Premiere Pro, a widely used editing suite.

AI-assisted editing won’t make Oscar-­worthy auteurs out of us. But amateur visual storytelling will probably explode in complexity. Even tools for one-to-one video messaging might evolve—AI on our phones could pull together disparate clips into weird, delightful missives. And, of course, AI editing will uncork new forms of digital malfeasance: It’ll be a lot easier to persuasively lie, to make ever-slicker propaganda.

That’s a fearsome subject worthy of the next high school filmmakers.

Write to clive@clivethompson.net.


This article appears in the February issue. Subscribe now.


More Great WIRED Stories