Pinar Yanardag receives NSF CAREER award to democratize video production with artificial intelligence

Today's artificial intelligence (AI) video generators are essentially black boxes. You type a prompt and hope for the best, but you have almost no control over camera angle, motion, or other specific details.

With research supported by a $562,245 National Science Foundation (NSF) Faculty Early Career Development Program (CAREER) award, Pinar Yanardag, assistant professor in the Virginia Tech Department of Computer Science and core faculty at the Sanghani Center for Artificial Intelligence and Data Analytics, wants to open those black boxes to give users "director-like" control over AI video.

The aim of the project is to democratize professional-level video production for all users. "A science teacher should be able to make a compelling visual the same way a Hollywood studio would, minus the budget," she said. "From educators to scientists to indie filmmakers, the goal is the same: turn an idea into a compelling visual story without specialized equipment or expertise."

The research, she said, offers the unique opportunity to mesh her two worlds as Emmy-nominated creative director of “Westworld 2020” and as an academic AI researcher. “Controls people want are already hiding inside these models," Yanardag said. "We don't have to build something new, we just have to unlock what's there to give them the ability to choose the camera angle, the motion, the shot without needing to be a technical expert and without the multimillion-dollar cost of retraining these models from scratch."

"The ability to control AI-generated video the way a professional director would — choosing the angle, the motion, the shot — sounds like it should be complicated and expensive. Pinar's work strives to lower those barriers," said Christine Julien, professor and head of the Department of Computer Science. "Think of a child in rural Appalachia discovering that technology and creativity aren't separate worlds, the same connection that defines Pinar's career. That kind of impact is exactly what the NSF CAREER award is meant to support."

Professional-quality video for everyone

Yanardag has targeted five groups who will benefit from her work, essentially anyone who wants to produce professional-quality video.

Educators can quickly produce custom, engaging instructional videos (animating a scientific process, for example) without a film crew or special equipment.

Scientists in data-scarce fields can generate realistic synthetic video and images to fill gaps where real-world data is hard or expensive to collect, such as medical imaging or agriculture.

Filmmakers and small studios can change camera angles, transfer motion, or repurpose existing footage without costly reshoots, bringing cinematic techniques within reach of people without Hollywood budgets.

Science communicators can turn complex ideas into compelling visual stories.

And students and the general public can provide early, hands-on AI literacy for K–8 students, as well as a clearer public understanding of how AI is reshaping creative work and jobs.

Imagine you film a dancer with a single phone camera locked in one spot, said Yanardag. Normally that one angle is all you'll ever have. With this new technology, you can re-render that same performance as if a camera were smoothly orbiting around the dancer, or looking down from above — viewpoints that were never actually filmed. No second camera, no reshoot, no expensive 3D scanning rig.

“Or,” she said, “suppose you have footage of a car driving down a winding road. You'd like a boat sailing with that exact same sweeping motion, or an airplane gliding low over a beach following the car's path. This new method lifts the motion out of the original clip and applies it to a completely different subject and scene, so a filmmaker can prototype a shot without filming or animating it.”

Yanardag’s CAREER award also funds an integrated education plan that includes:

Yanardag has built her CAREER project on a body of work she has been developing in her Generative Modeling Lab (GEMLAB).

“My group of graduate students has worked with me the last few years learning how to understand and control the internal workings of generative AI models, first for images and increasingly for video,” she said, pointing to preliminary papers co-authored with them that can be found on the GEMLAB website.

Undergraduates are involved, too; several have co-authored papers with Yanardag. Her ongoing education plan deliberately creates more such opportunities.

"The CAREER award lets me pull these threads into one cohesive five-year vision I call ‘generative cinematography,' a universally accessible medium where anyone, of any age or background, can translate their ideas into film," Yanardag said.