Have you ever been in a VR conference call and thought, well, I know what everyone looks like from the front, but what about the other sides? The Fraunhofer Institute for Telecommunications has your back — and theirs. Researchers there have created a powerful multi-camera setup that captures in 3D every aspect of someone and dumps it straight into a VR experience.
The goal is to create avatars in virtual reality that are not just photorealistic from certain vantage points, but show the entire person, along with all their presence and gestures. Turns out this requires quite a bit of hardware.
“We are currently using more than 20 stereo cameras to map a human,” explained Oliver Schreer, who leads the research group, in a press release. Each depth-sensing camera sends its data to a central processor to be fused into a single representation.
“In developing these algorithms, special care has been taken to ensure they work efficiently and fast, so the movements of dialogue partners can very quickly be converted into a dynamic model,” said Schreer.
It still takes a few seconds, but the result is a virtual model that has every detail intact, from hairstyle to wrinkles in a shirt. Every little movement is also captured, from blinks to high kicks. It can be inserted into a virtual environment and viewers or interlocutors can move all the way around the avatar and inspect it from any angle.
The first application the researchers have worked on is a teleconferencing one, but Schreer sees the system being applied in entertainment, as well. “Film producers could use it to transfer the movement of actors into scenes more easily than ever before,” he said. Or shows could be recorded this way and viewers could navigate the whole scene in VR alongside the characters.
The system is still very lab-bound, of course, and no one is expected to set up 20 cameras in their conference room (well, we might). The team is working on improving the cameras and the software that handles the image streams. Alas, it’ll still be a few years before we get a real Holodeck.
Featured Image: Fraunhofer HHI