What DeepMind Means for Virtual Reality
Live Capture Virtual Reality is difficult.
360 Video is one way, but it doesn’t afford you the aspect of embodied cognition that gives VR its special powers of hacking identity, shifting perception, and embedding new memories. (Don’t believe me? )
Google might just be changing that.
A bit of (recent) history
Microsoft has a great solution for inward facing volumetric video — but it’s incredibly expensive and can only capture characters, not whole scenes. Also, even if you nail the 106 cameras shooting 30fps with Kinect integrated for depth, the magic definitely comes on the back end here.
Lytro promised outward facing volumetric video from it’s Immerge light-field camera, but its demo Hallelujah reveals the inherent problem… You have only about a foot of agency in any given direction.
After Google aquired Lytro for “about $40 million” and in the mean time, developed a light field 360 photography camera based on their original 360 camera — the Jump Odyssey. You can check out the very beautiful result in Welcome to Light Fields on Steam.
We can’t forget Facebook’s partnership with RED, creating the Surround 360. The image quality on this is sure to be amazing, but I’m not sure how they will exceed the angle options of Lytro’s Hallelujah.
All of these solutions are incredibly expensive, data intensive, and require insane amounts of processing — and none of them offer the viewer a room scale VR experience with true 6DOF.
But where is this all going? For Google at least — it’s a cross department question.
Google’s AI company DeepMind just announced a neural network that can ‘imagine’ a space from different angles. Read the full article here.
In theory, this neural network, paired with a 360 monoscopic video, could create a spatial render of a scene and solve the problem of volumetric video without the expensive hardware.
Pairing this up with open source tools like Seurat — we could see volumetric video on mobile VR .There are surely more developments to be done, but as the fields of AI and VR merge, we are sure to see exponential growth.
