Facing the Future
Can a New Zealand-based lab take virtual assistants from the realm of marketing gimmicks and endow them with real intelligence?
--
One day in the spring of 2010, a computer scientist named Mark Sagar sat down to compose an email to his boss, film director Peter Jackson. Sagar had been the special projects supervisor on Jackson’s 2005 film, King Kong, pioneering the facial motion capture technology that allowed the great ape to finally go beyond chest-beating and really emote.
With two Scientific and Engineering Awards from the Academy of Motion Picture Arts and Sciences already on his shelf, Sagar’s career seemed secure. The technology he’d helped invent — “an expression-based, editable character animation system,” as the academy put it in bestowing one of the Oscars — had played an indispensable role in the creation of the highest grossing movie of all time, James Cameron’s Avatar. He was poised to jump from one tentpole blockbuster to the next.
But the Kenyan-born engineer — whose family moved to New Zealand when he was a child — had been quietly hatching a more ambitious plan. The way he saw it, the motion capture technology he’d helped perfect was little more than puppetry — a way for technicians to map an array of points on an actor’s face to corresponding points on the face of an animated character. When the actor smiled, the character smiled. The results were impressive, but they were only skin-deep. For a character to be truly lifelike, Sagar knew, its expressions would have to come from within. They’d have to be motivated, driven, responding to an array of internal processes, like those of a living creature.
“I got to a point in my career where I’d gone as far as I wanted to,” he recalls, sitting in his office at the headquarters of Soul Machines, the Auckland-based startup he co-founded in 2016. “I thought, ‘Okay, I might actually want to work on different things than making computer-generated apes.’”
Sagar developed a lofty goal: to build an entity that would learn, feel, remember, and interface with people in much the way we interface with each other. He wanted to build a digital human.
In the email, he pitched the idea to Jackson as a tool for immersive storytelling —…
