Light Field Volume Capture Is the Future of 360
Imagine this: it’s your child’s sixth birthday party. They are sitting at the head of the table, eagerly bouncing up and down in a chair two sizes too large. Your partner comes in from the kitchen, cake in hand, and the children scattered around the room begin a raucous cheer. You move around the table, examining the pattern of balloons on the napkins. In the corner your mother sits in a rocking chair, a blanket across her knees. You walk towards her, and notice that she’s touching her lips, a gesture of quiet joy. You walk the rest of the way around the table, your foot passing harmlessly through a party hat that’s been discarded on the floor. In the doorway you see yourself, laughing as you pass around bright golden plates.
If it seems impossible to imagine, it may be because you haven’t been paying attention to the possibilities of light field volume capture (LFVC). And while it’s true that the technology can’t yet capture a moving picture like the one described above, that’s where it’s headed — faster than most of us realise.
What Exactly is Light Field Volume?
360 video is often equated with virtual reality, but the stark difference between them is a lack of depth of field. With 360, the image is flat. It wraps all around the user, but they can’t move closer to it or further away; their point of view is fixed.
It’s one of the things that holds 360 back from feeling as immersive as it could, and it’s solved by light field volumes. The technology behind LFVs is as complicated as it is simple. Essentially, everything that we see is creating by rays of light. Those light rays bounce off of objects, or are absorbed by them, which creates patterns of light and shadow. A light field is a spherical grid of light rays; by tracking the position, pattern, and colours of that grid (the “volume”), we can “see” whatever objects the light is touching. Each frame of a volumetric video/light field volume capture is a real time rendered 3D model of the scene, creating a geometrically correct 3d representation of what the camera captures. As of right now most of the technology allows 6 DOF (degrees of freedom), up from the 3 DOF of 360 video.
The practical application of that means that any image created with LFVC can be digitally recreated and entered with virtual reality.
What Can It Do Today?
There are currently two methods of light field capture: outside in and inside out. Outside in (8i, Uncorporeal) allows you to capture 360 degrees of volumetric video, ideal for single character captures. Inside out (Lytro, HypeVR, Matterport) will capture everything in the camera array’s field of view. All the companies producing volumetric video systems are equally invested in ways of distributing the experiences they create. One of the technology’s hurdles is the enormous amount of data needed to display the content created.
Who Are the Major Players?
There are more companies working on this every day. Here are just four players, each of whom is using a slightly different version of the technology.
Lytro’s Immerge Camera is a sophisticated machine capable of instant recreations of complex, moving scenes. With Lytro’s technology you can take a photograph — not create a digital recreation of a scene — with a sense of perspective. Check out their gallery for examples. The photographs are impressive, but their real strength comes from the ability to do the same with video. They’re a technology company who generally work with content partners to leverage their tech, but they created Moon as an example to showcase what the technology can do. It’s unfortunately not available for public viewing, but all reports are that it is transportive. Lytro’s camera uses a combination of live action with digitally rendered elements to provide the full 6 degrees of freedom.
Matterport is taking a different tactic. They’re staying away from video and concentrating solely on photographs, specifically targeting the real estate market and gaming fields where photo-realistic 360 representations of a space are ground-breaking. Their camera is smaller and rotates automatically on a stand, capturing light fields through an entire room with only one camera — but the process takes a full minute, meaning it can’t be used to capture moving targets. No video, and probably no portraits (though people used to sit for hours to have a painter render their likeness!).
Otoy, meanwhile, has laser-focused their offering, using their LightStage technology to digitally capture human faces. They also make rendering software, keeping users in their pipeline from capture to render to stream.
HypeVR call their technology by a slightly different name: 360 volumetric video. Their system is based around 14 6k Red Cameras (totalling about $250,000) and LiDAR technology (using lasers to sense depth) and includes a full capture, rendering, compression and streaming suite, along with creative production and post production services.
What’s Next?
The best part of playing with new technology is imagining where it will go from here. The scenario where we began, capturing a 360 video of a family birthday, isn’t possible today; but it’s so close you can almost taste the chocolate cake. The miniaturisation of Lytro’s technology, or the addition of nine or ten lenses in a circle for Matterport, would make that scenario possible. Imagine if every room in the house featured a 360 camera carefully placed in a central location; with the click of an app video could be captured for every special occasion, every important moment. We all love flipping through old photographs (or swiping through them, which is what most of us do these days). How much more fun would it be to walk through them?
Most importantly, this technology is already revolutionising 360 video, which is itself still incredibly new. Can’t see a photograph on your mobile screen? What if you could zoom in simply by bringing the phone closer to your phone, or zoom out by holding it further away? Going on vacation? Take a walk through that Airbnb before you reserve it (and never get stuck in a place with no windows ever again). With or without a headset, LFVC is the future — and the future is now.
—
written by Wren Handman for Hammer & Tusk. Check our our weekly virtual reality newsletter!
