Microsoft Research Asia Takes Facial Recognition Software A “Nose” Forward

AsianScientist (Aug. 10, 2011) – The human face is a complicated thing, and realistic animation of the human face has been termed the “holy grail” of computer graphics.

Decades of research in computer graphics have developed a number of techniques for capturing 3D moving images of the human face. But all have flaws, capturing insufficient detail or failing to depict accurately a changing expression.

Researchers from Microsoft Research Asia: Haoda Huang, Xin Tong, and Hsiang-Tao Wu, together with Jinxiang Chai from Texas A&M University, have developed a new approach for acquiring high-fidelity 3D facial performances – complete with realistic dynamic wrinkles and fine facial details.

Researchers from Microsoft Research Asia have developed a new approach for acquiring high-fidelity 3D facial performances, complete with realistic dynamic wrinkles and fine facial details!

The team’s research, “Leveraging Motion Capture and 3D Scanning for High-fidelity Facial Performance Acquisition”, will be presented at SIGGRAPH 2011, the 38th International Conference and Exhibition on Computer Graphics and Interactive Techniques, taking place from Aug 7-11 in Vancouver, British Columbia.

Existing methods of capturing faces and expressions include marker-based motion capture and high-resolution scanners. In the marker-based technique, the images were low resolution, although generally accurate. The high-resolution scanners, on the other hand, could capture facial images but typically only for static poses.

The team set out to combine both the accurate motion capture of marker systems with the high resolution of scanners, using the least amount of computing power for an accurate facial reconstruction.

In a two step-process, the team first applied about 100 reflective dots to an actor’s face, who then made a series of facial expressions that was captured by a marker-based motion capture system. Next, the researchers aligned the motion capture with corresponding facial reconstructions on the screen, thereby determining a minimal set of face scans required for accurate facial reconstruction.

This was no easy task, as even a small mis-alignment of fine-grained features such as wrinkles or pores will make the resulting facial reconstruction appear unnatural

“We want to make sure these features align, or you will see some weird artifacts,” Tong said. “A wrinkle may appear, then disappear, then appear—it’s not natural.”

The team’s work may have implications in areas such as computerized filmmaking, to create virtual faces which are more lifelike. Also, the technique could be used to create computer avatars that are more realistic than pre-programmed versions found in devices such as the Xbox360.

However, much more work needs to be done. Currently his team’s scanning technique does not capture synchronized eye and lip movements. In addition, it takes considerable computing power and several hours to successfully register all of the images, which ideally should be done in real time, Tong said.

“There are a lot of challenges,” he concluded, “but it is a very exciting research area.”