On Friday, TriStar Pictures unveiled Here, a $50 million film directed by Robert Zemeckis that utilized real-time generative AI face transformation techniques to depict actors Tom Hanks and Robin Wright over a span of 60 years, representing one of Hollywood’s pioneering full-length films centered around AI-driven visual effects.
The movie is based on a 2014 graphic novel that primarily unfolds in a New Jersey living room across various time periods. Instead of casting different actors to portray the characters at different ages, the production team employed AI to alter Hanks’ and Wright’s appearances throughout the film.
The de-aging technology was developed by Metaphysic, a visual effects firm known for its real-time face swapping and aging effects. During production, the crew monitored two screens: one displaying the actors’ actual appearances and the other showing their appearances at whatever age the scene required.
This information was first shared by Ars Technica, a reliable source for news on technology, tech policy, reviews, and other related topics. Ars is a part of Condé Nast, the parent company of WIRED.
Metaphysic has created a revolutionary facial modification system that utilizes bespoke machine-learning models trained on footage from Hanks’ and Wright’s past films. This involved compiling an extensive dataset that included various facial movements, skin textures, and appearances captured under diverse lighting and camera angles. The outcome is a set of models that can instantaneously perform face transformations, eliminating the extensive manual post-production efforts typically associated with traditional CGI processes.
In contrast to older methods of aging effects that depended on meticulous frame-by-frame adjustments, Metaphysic’s method enables immediate transformations by analyzing facial landmarks and aligning them with trained age variations.
“You couldn’t have made this movie three years ago,” Zemeckis remarked to The New York Times, contributing to an in-depth article about the film. The conventional visual effects needed for such advanced face modifications would typically involve hundreds of artists and a budget significantly closer to that of standard Marvel movies.
This isn’t the first instance where AI techniques have been employed to de-age actors. For the 2023 film Indiana Jones and the Dial of Destiny, ILM utilized a proprietary system named Flux, paired with infrared cameras to gather facial data on set, subsequently using archival footage of Ford for de-aging in post-production. In contrast, Metaphysic’s AI models are capable of processing transformations without any additional hardware, delivering results in real-time during filming.
The film Here emerges as major studios investigate AI implementations beyond mere visual effects. Companies like Runway are crafting tools for text-to-video generation, while others, such as Callaia, develop AI systems for script analysis and pre-production strategies. Nevertheless, recent guild agreements impose stringent restrictions on the application of AI in creative tasks like scriptwriting.
In addition, as highlighted during the SAG-AFTRA union strike last year, Hollywood studios and unions remain engaged in heated discussions regarding AI’s role in filmmaking. While the Screen Actors Guild and Writers Guild managed to secure certain limitations on AI use in recent contracts, many seasoned professionals in the industry perceive the technology as an unavoidable reality. “Everyone’s nervous,” Susan Sprung, CEO of the Producers Guild of America, remarked to The New York Times. “And yet no one’s quite sure what to be nervous about.”
Despite the challenges, The New York Times reports that Metaphysic’s technology has already been utilized in two other films set for release in 2024. Furiosa: A Mad Max Saga used it to revive the character of late actor Richard Carter, whereas Alien: Romulus restored Ian Holm’s android character from the original 1979 film. These applications demanded estate consent due to new California regulations governing AI recreations of performers, commonly referred to as deepfakes.
Not everyone is enthusiastic about the direction AI technology is taking in film. Robert Downey Jr. recently stated in an interview that he would direct his estate to pursue legal action against any attempts to digitally resurrect him for future film appearances. Yet, despite the controversies, Hollywood continues to discover ways to create death-defying (and age-defying) visual achievements on screen—especially when substantial financial incentives are at stake.
This story was first published on Ars Technica.
