Gaze and Click — Interactive 360 Video
Insider look into creating an immersive documentary
This article is the third, and last installment of my review of working on “Kinshasa Now”, my first VR/360 video project. That was in 2015, and since then I have just been too busy creating new VR/360 video projects to finish this trilogy.
The first article covered capturing the content on location, while the second article talked about the post-production process. Both articles still contain useful information and resources for anyone starting with 360 video production.
“Kinshasa Now” was published on the web and as a Gear VR app and is still quite unique as it was gaze and click 360 video documentary. Interactive 360 video stories are still rare, as are web/interactive documentaries in general.
When was the last time you clicked through one? Right.
Note that I use the past tense, as the project was never publicly available, and has been taken down in the meantime. Now I could start rambling on the importance of digital archiving, but let’s stay on topic.
What follows is a list of tools we tried and tested to program the interactive part of the immersive documentary experience.
KR Pano
KR Pano was used to create the website for the project. At the time webVR and A-frame were too experimental to implement and KR pano was the easiest way to publish 360 video on-line. The website functioned as a click and point adventure, where you use your mouse to move the 360 video sphere and click to make story choices.
KR Pano is basically a player for panoramic images and interactive virtual tours. It is available as a Flash or HTML5 application. Flash is not supported on mobile devices, and is getting less relevant by the year. Adobe plans to end support by 2020.
It is not free, but you get reliable results you can show in a browser (optimized for Chrome) without fearing the experience breaks in the middle of your presentation. Accept for the wifi connection of course.
The game engines
At the time we were creating the interactive content (end of 2015, beginning of 2016), just making 4K 360 video play in Unity 3D and Unreal 4 was a real headache.
By now, Unity 3D (since version 5.6) has better support for 360 video with its new Video Player component.
By the end of 2016, I directed the interactive 360 video component for the historical TV drama “Les Pays d’en Haut”. We used Unity 3D to build the website, Gear VR and cardboard builds, but did buy the asset Easy Movie Texture to smooth out 360 video playback. A more expensive alternative which I used in another project: AVPro Video .
Unreal has still no built in 360 video support, but I found this thread on the unreal forum which seems to offer a solution. The new 4.18 preview version promises major Media Framework updates.
Both engines are free for personal use.
TouchDesigner
TouchDesigner is a node-based visual programming platform, developed by the Toronto based company Derivative. Originally conceived as a real-time render engine, effects generator and prototyping tool for live performances, it has now grown into a multi-tool for interactive work. It has a steep learning curve but you can count on a growing community of artists and technologists to help you when you get stuck.
This is a really powerful tool, and is perfect for installation/performance oriented VR work. You can play ultra-sharp high quality 360 video through a premium headset (using a powerful GPU and a PC to match) using the Hap codec.
It is not free. Their wiki explains the licensing.
This is far from a drag and drop developing environment, so we ended up hiring someone who mastered the application, -who couldn’t finish it due to force majeure circumstances. So then came in…
Wonda VR
Wonda VR is an authoring tool for 360 VR experiences, similar to what DVD Studio Pro was for DVD programming or Director for CD-ROM’s. You can publish to Gear VR and cardboard, using the Wonda VR app for iOS or Android, or go for a (rather expensive) white label export.
It is really an easy creation process you can pick up in a couple of hours. Again, not free, but you can try before you buy (14-day trial) and download the full Wonda VR desktop application after you register.
Finally we had something portable on our Samsung Gear VR, and we could show this prototype while pitching our new project “Without A Roof”. But that’s a story for later.
