For animated characters to move us, they first have to move. But bringing virtual characters to life has never been a simple task.
Motion capture technology records and digitally replicates human movement to create 3D animations. Think of Andy Serkis’ work, transformed into Gollum, in the Lord of the Rings film series.
The problem is traditional motion capture relies on lots of expensive, bulky hardware. And it requires skilled professionals operating multi-camera setups, studio environments and special sensor suits worn by actors.
“All of that expense, skill and time is prohibitive,” said Gavan Gravesen, co-founder and CEO of RADiCAL. As its name implies, the New York-based member of NVIDIA’s Inception program for startup companies breaks with tradition. “We’re focused on delivering universal availability, seamless integration and speed — all at low cost,” he said.
The company’s AI-powered solution, driven by NVIDIA GPUs, only requires game developers, 3D graphics artists, filmmakers and AR/VR enthusiasts to have one physical item: a 2D camera, even one on a phone. No other hardware, studios or elaborate sensor gear are needed, which dramatically decreases the cost and effort involved.
Fast Motion Capture
Users anywhere in the world can upload their videos by smartphone or the web directly into RADiCAL’s cloud. The company processes the videos using AI and motion science to detect what’s there, represent human movement in 3D space, and even reconstruct what the camera can’t see.
The algorithms automatically adapt to different body proportions and human body movements, and create a model whose motion mirrors the video subject in 3D. While the model’s smooth motion makes the process look easy, it’s anything but.
“We integrate deep learning, robotics, computer vision and biomechanics, which required a powerful AI development environment,” said Anna-Chiara Bellini, co-founder and chief technology officer of RADiCAL. NVIDIA GPU computing plays a crucial role in making that possible.
Fast Iteration with TensorFlow and CUDA
In developing the AI, the first challenge RADiCAL faced, Bellini says, was the sheer amount of data to process. To develop their algorithms, a single still image could require analysis of up to 6GB of data. But with every second of motion, there are 120 frames, effectively creating a staggering 720GB of data.
Bellini and her team opted for TensorFlow to integrate GPU programming into their research process, giving them a “single execution model” for their entire pipeline. “With the use of ad-hoc kernels written in CUDA to supplement TensorFlow, we’ve come to a point where a simple kernel written in just a couple of hours can save us days in simulation,” she said.
To support its motion models, RADiCAL processes several frames in a staggered way on multi-GPU systems. “Technologies like the most recent generation of NVIDIA NVLink, supporting multiple Tesla V100 GPUs, make that possible,” Bellini said.
Using cloud-based, multi-GPU machines, Bellini and her team decreased the development cycle from ideation through deployment to production by 10x.
“NVIDIA GPUs allow us to work faster, explore more options and use the time of our machine learning engineers more efficiently,” she said. “It’s been a revelation.”
Ready-to-Use Animation Files
After a video is processed, RADiCAL offers live 3D previews presented side by side with the actual video footage through their website and its MOTiON app.
Users can also download animation files of their work in a format that is ready to use for 3D animation, augmented reality, VR, gaming engines and other 3D graphics programs and content channels.
RADiCAL has recently opened a public beta program for early adopters across creative industries. It’s aiming for a full commercial launch within weeks.
The company is one of the nearly 2,800 startups worldwide that have joined our Inception program. Through this program, NVIDIA helps accelerate startups by providing them with access to technology, expertise and marketing support.
