Tech Trio Reveals 4D Facial Performance Capture Service

Clear Angle Studios, DI4D, and TexturingXYZ have teamed up on a groundbreaking 4D facial performance capture service, which for the first time produces per-frame map data to deliver a final product almost indistinguishable from recorded human footage of a human. The companies announced the new service, described “as a new standard of performance-driven digital human faces” at SIGGRAPH 2024 last week.

Also, the group shared a video (below) illustrating the building process from the initial mesh to final render, offering a glimpse into the “future of interactive entertainment” and the potential collaborative capability that the three specialist companies could offer vendors in a single service.

Clear Angle Studios provides capture technologies, DI4D provides markerless mesh tracking, and TexturingXYZ enhances the micro-skin details on the maps using advanced Hyperskin technology.

As described the group, 3D scanning produces an accurate digital model of a static, real-life subject. However, even when using a 3D scan, recreating an actor’s movements in a digital human has previously relied on the interpretation of an animator. 4D capture records scans in motion, allowing a photo-realistic digital replication of the actor’s performance that captures all the subtlety and nuances only a real human can produce.

The companies have teamed up to accurately display subtleties of the hyperfine changes that human faces undergo when changing expressions. Those changes include micro skin details such as stretching, contracting, flexing, and flowing in specific ways when differing facial poses are pulled. This results in visible distortions, and changes to the way that light falls across the skin.

According to the companies, details are lost when animating a static frame of skin detail from a single 3D scan across a sequence. The video illustrates how the collaborative 4D process produces a faithful representation of these real-world shifts happening dynamically per frame. In addition to detailing the micro skin texture map accurately across the geometry, the light reflections seen in the renders are also highly accurate and representative of those micro detailing shifts.

“For more than a decade, we’ve provided full body character, head and prop scanning, LiDAR and environment capture along with 3D scan processing for some of the most recognizable studios in the business,” said Clear Angle Studio co-founder and CEO Dominic Ridley. “I’m immensely proud to be able to apply our learnings and expertise from that journey to this advancement in 4D capture, which we and our partners will offer as a single service to the entertainment industry. Once again this is an advancement only made possible through teamwork and collaboration with fantastic people!”

"DI4D pioneered the use of 4D facial capture and have been leaders in the field for over 20 years,” added DI4D co-founder and CEO Colin Urquhart. “We’re very excited to be collaborating with Clear Angle Studios and TexturingXYZ – two world-renowned leaders in the fields of 3D scanning and texture map generation. By combining their expertise with our world-leading 4D tracking capability, we are able to produce groundbreaking 4D data with highly accurate per frame diffuse, normal, and displacement maps, resulting in the most realistic performance-driven facial animation possible."

"At TexturingXYZ, our mission has always been to empower artists with the highest quality texture maps, enabling them to create stunningly realistic digital humans,” concluded TexturingXYZ CEO Jeremy Celeste. “This collaboration is a testament to what can be achieved when industry leaders combine their unique strengths. We're excited to contribute our expertise to this revolutionary 4D capture process, pushing the boundaries of what’s possible in digital entertainment."

Source: Clear Angle Studios

Debbie Diamond Sarto is news editor at Animation World Network.