![](https://imageio.forbes.com/specials-images/imageserve/6639504ddc4d4bc3aead9524/0x0.jpg?format=jpg&crop=3154,1773,x136,y0,safe&height=900&width=1600&fit=bounds)
Director Paul Trillo was given early access to Sora, OpenAI’s new text-to-video generator, which he … [+]
Paul Trillo
Director Paul Trilloâs new music video for the band Washed Out, is the first commercial use of OpenAIâs new text-to-video generator, Sora. âThe Hardest Partâ tells the story of a coupleâs life together as an infinite dolly shot, from meet cute in school through adulthood.
âThis was an idea I had almost 10 years ago and then abandoned,â says Trillo. âWith Sora, I was finally able to bring it to life.â
The filmmaker, one of the first to be granted access to OpenAIâs new text-to-video generator, says it took 700 generations to create the 55 selected shots that make up the video.
“I really wanted to do something that was both unique to the hallucinations of Sora while also attempting to create something that felt timeless and works regardless of what technology was being used,â explains Trillo in an interview.
Years ago, he âwanted to do a continuous zoom through a relationship that spans multiple decades beginning in the late â70s,â but working with a dozen actors through several time periods, not to mention âa whole slew of locations,â he says, wasnât realistic for the music video budgets he works with.
But, Trillo explains, Sora presents unique challenges and opportunities to artists. âI think fundamentally, what works as a story, what works on an audience will never change. But our process to get to that finish line is different,â he says.
âWhat is unique about AI is that it’s this more fluid, organic process where you’re ideating. You have your idea, it feeds into the final product, and then the final product gives you a new idea to go back into the writing phase and rewrite,â he adds. âIn this way, pre-production, post-production become really fluid. That’s what’s been really exciting for me: you can actually spend more time trying to craft that story.â
Trilloâs work is characterized by a visionary use of technique and technology in service of story. âThe Hardest Partâ marks Trilloâs 20th Vimeo Staff Pick. His other award-winning short-form work includes âAt the End of the Cul-de-sac,â a 10-minute single- take shot with a drone, the first short film made with Runway AI Gen-2 âThank You For Not Answering,â and a collaboration with French artist Jacques titled âAbsolve,â which was shot on location in the Louvre.
In addition to Trilloâs narrative and video art work, he has also directed music videos for The Shins and ads for Ford, Toyota, IBM, Microsoft, Starbucks, T-Mobile and a Super Bowl commercial for Carvana, which have established him as one of the most eclectic, creative, innovative directors working on the cutting edge of technology today.
Still from “The Hardest Part,” directed by Paul Trillo. It’s all part of the endless dolly through … [+]
Paul Trillo
âSomething I noticed that Sora does really well is making organic transitions between one scene that bleeds into another. And the walls don’t necessarily make sense for how one environment, you know, ends and another one begins,â Trillo says.
This never-ending zoom technique is something he has used in his work for over 10 years, so he immediately thought of how he could use Sora to recreate it. Once he narrowed in on that, even some of the people on the Sora team were surprised.
âWhen I had started to do these really fast, like, hyperlapse things, they’re like, âOh, we didn’t even know it could do that.â The prompt becomes this chemistry formula of words sequenced, timed and paced in a very specific order,â he explains. âYou can trigger events to happen. You can say, like this, then this. So you’re almost just writing a script. Some of these prompts are very detailed.â
You can check out the incredible script-like prompt Trillo gave Sora to produce the infinite dolly shot that drives the video here. âThe right combo of words created that hyperlapse, the infinite dolly shot,â he says. âAnd once I kind of figured that equation out, I would just replace the subjects and the location and the events that are happening.â
At the beginning of April, two weeks before the prestigious TED conference in Vancouver, Jonathan Wells, who commissions video for TED, contacted Trillo about creating something that envisions what the next 40 years of TED would look like. This was a tremendous opportunity for a visionary filmmaker to get his work in front of some of the worldâs most influential people.
In order to convey this never-ending road of ideas and technological progress, Trillo used the hyperlapse technique he developed making âThe Hardest Part,â prompting Sora to fly through future iterations of the TED conference.
âIt’s an indication, I guess, of this new kind of content that is wildly ambitious and weird and has a somewhat acceptable production value that can be turned around in a short amount of time,â Trillo said.