Flawless, created by filmmakers and scientists, may have finally crossed the uncanny valley for lip-dubbing content.
Currently, watching a piece of foreign language content dubbed into English is an exercise in discomfort.
Thanks to the proliferation of local-language film and TV — mostly brought to us by streaming platforms — the experience of watching a dubbed Danish crime procedural or a Spanish boarding school soap opera are eerie trips down Uncanny Valley. It's a term widely used for technology that doesn't suspend our disbelief and actually takes on a human likeness, putting it in the mix of dystopian nightmares.
Enter Flawless, an AI-powered filmmaking studio that wants you to stream buzzy shows and still be able to sleep at night (without the mismatched mouth movements and brutal scene cuts). Founded in 2018 by multi-hyphenate directors Scott Mann and Nick Lynes, Flawless' proprietary TrueSync technology maps actors' faces and renders the translations we've seen in the wild west of AI. The company previewed a sizzle reel of its work at this year's Cannes Film Festival, which Variety Can share exclusively here.
Designed to protect artistic copyrights and comply with Hollywood labor guilds, Flawless raises big questions about the value of international content and its ability to thrive in American markets. Variety It can also report that Immaculate is officially in the film distribution game. The company has entered into a joint venture with XYZ Films and T-Shop Productions to acquire foreign film rights. The partnership will translate the English-language projects into what Flawless promises will be “fully lip-synced visuals” and release them in the respective territories. The first title to be released is Victor Danel's award-winning sci-fi romp “UFO Sweden”, which will be released under the domestic title “Watch the Skies”. A full slate of projects is expected in the coming months*.
Maan said along. Variety About fear and loathing of AI in show business, building an ethical code and the mind mechanics of the Uncanny Valley.
How did you get started in this space?
It all started in 2018, my background is film direction, writing and production. I happened to be the first paper to introduce creative AI to the world. It was called “Deep Video Portrait” and came out of the SIGGRAPH conference, which is basically the Oscars for science. It blew my mind.
I had always worked in the VFX realm, and it was clear the impact this technology could have on our industry. I reached out to the authors of the paper, which led to founding the company with them. [my partner] Nick Lines. Christian Thiobault, Pablo Garrido and Hyung Woo Kim founded our science department, and have turned it into the leading computer-driven generative science department in the world. This is our secret sauce.
There is a lot of fear and anxiety surrounding AI in Hollywood right now. The interesting thing about your company is that you are optimistic.
We based this thing from a filmmaking perspective. I love art, and our mission is to see this technology as a tool to create tools. A different view is that it aims to destroy creations and steal art. To Nipster out of it, riding roughshod and disrupting for disruption's sake. But we believe in expressing human stories, and film is the most modern form of that. Tech is enhancing this and adding more layers of creativity. There are benefits. Then you have the economics of this whole business, which relies on certain considerations and an understanding of how it works.
There is a huge lack of education around AI in this town. How does this affect your business?
This is a global problem. It's not just Hollywood. Education and understanding are key. Some companies have — loopholes, let's call it — won't educate because it shows a kind of underhand player. You need to have an honest conversation about how and where it's disruptive, and where the positivity lies. We've prioritized understanding with the studios and labor guilds — including SAG-AFTRA — over the last few years and said, “Look, this is how it's supposed to be used.” Next [AMPTP contract] Negotiation is something that everyone should start talking about now. This change is entering our industry and we must be prepared.
Good point. Is the industry addressing this in a meaningful way?
I can say with confidence that there has been a big change in attitudes. Before last year's strikes, there was a cautious path to greenlighting the use of AI in projects. People were having early conversations and seemed to understand, and it came from a place of trying to improve the process. But there is a fear of crossing over. The industry is in such bad shape right now that many people don't want to rush into the wrong solution.
How did the Immaculate form its moral code?
It emerged from issues that I personally experienced as a filmmaker. I always wanted to fix the dubbing of the film. I did a movie where I had that kind of idea, a movie called “Hist” with Robert De Niro. It was very intimately built and nuanced, and it was a pleasure to present what we did. I watched the foreign language version that was dubbed. The script was completely rewritten and the mouth movements were completely different. I realized, “God, no wonder things don't travel well.”
I found a bug to fix this problem and tried a bunch of VFX like head scanning and other nonsense. It wasn't practical, and it didn't cross the uncanny valley. That's a long way of saying you can't ethically do it if you're removing and destroying all the areas that contribute to the film. We have learned that ethics and legal rights go hand in hand.
Speaking of Uncanny Valley, it's pretty brutal to watch lip-dubbing in its current form. Especially the way streaming has enhanced it.
As humans, we study each other in great detail. If we notice something out of the ordinary, our brains are programmed to go on hyper-alert. Throughout the evolution of filmmaking, our brains have been dumbed down and retrained. Your subconscious is doing a reality check. If something doesn't marry well — especially accuracy and vision — you're immediately putting a huge barrier to any immersion of empathy and belief in what you're seeing. Think how much an actor's face and gestures and performance convey without hearing the voice. There's a lot to it. They are all causes. [present lip dubbing] A problem. But it's good to have a clear problem to solve.
There's also a pretty game-changing factor here for international films. This could open up a new star market and put an end to American language remakes of popular films.
By showing filmmakers our content, especially over the past year, they understand the potential of moving from the local stage to the global stage. This is a great opportunity to get your work done and it's motivating. He is very excited to show his work to a wider audience and especially in the US.
Has a studio or corporation ever tried to acquire you completely?
Yes, when we first fell here in Hollywood. We had a very short promo reel in 2021 and some tried to price out of the box. Nick and I agreed that we wanted to serve everyone, not just the interests of one studio. We want to be Switzerland, because it is otherwise very limited. But it helped us raise money and grow the company. It's a good problem to have.
You also want to get into the distribution game.
We've always seen ourselves as somewhat of a studio. We want to focus on the future. You need to do the whole thing right from pre-production, production to release. All these steps are influenced by technology. My point of view is that we should reach 10 times the audience of films. That, and making amazing movies for less with these tools. If you look at our first one, “UFO Sweden,” it was obviously translated into English but it also gave the filmmakers the freedom to add and change things that they didn't tolerate the first time around. could You can choose to be like an algorithmic component and try to do one thing, but I think the real power comes in making and distributing movies. The whole shebang. And in terms of post-production, we can do more. Look at Pixar, they repeat their movies right up to the moment of release.
* The JV acquisition with Flawless will be led by Ryan Black, XYZ's James Emmanuel Shapiro and Tea Shop's James Harris.