Disney experiments with face projection mapping, animating digital skin on live actor in real time

in Disney, Entertainment, Technology, Theater, Theme Parks

What is the future of live Disney characters? It may lie in the very same technology the company is increasingly using to create dazzling effects in its theme parks.

We all know projection mapping from the many Cinderella Castle shows Disney has rolled out over the last few years, adding fun animation as a projected layer on top of the large structure to enhance nighttime fireworks spectaculars. Now, Disney Research is diving deep into what they call “the first system for live dynamic augmentation of human faces.”

In other words, Disney is developing digital skin that would allow human actors to become just about any type of character on the fly, with virtual makeup morphing and blending with their every movement.


Screen Shot 2017-04-25 at 12.15.42 PM

Screen Shot 2017-04-25 at 12.16.12 PM

Screen Shot 2017-04-25 at 12.16.01 PM

Screen Shot 2017-04-25 at 12.16.17 PM

Screen Shot 2017-04-25 at 12.16.06 PM

The Disney-led research organization notes their biggest challenge with this technology is the inherent latency between a person’s movements and the system’s ability to process and react quickly to update the visuals without any noticeable lag.

Disney’s system “aims at reducing latency during every step of the process, from capture, through processing, to projection” by utilizing “infrared illumination” along with “an optically and computationally aligned high-speed camera.”

This is by no means the first example of facial projection mapping technology. It was most famously used recently to transform Lady Gaga’s face during the 2016 Grammy awards for her David Bowie tribute performance:



Two years ago, the Omote system was shown off online:


And just four weeks ago, this dance performance tech demo appeared online as well:

INORI (prayer) / Making from nobumichi asai on Vimeo.

But Disney claims its system “is the first method which fully supports dynamic facial projection mapping without the requirement of any physical tracking markers and incorporates facial expressions.”

So where will Disney’s new digital magic show up?

It’s not hard to imagine characters in their theme parks or on Broadway stages seeming to transform right before viewers’ eyes. It’s unlikely we’ll see David Bowie, Heath Ledger’s Joker, or V from “V for Vendetta” showing up in a Disney theme park any time soon, but those examples shown in the video above give good pop culture reference points for what is possible using this tech.

Comments Off on Disney experiments with face projection mapping, animating digital skin on live actor in real time