Making Augmented Reality a “true” reality

The dream of a true Augmented Reality experience as we walk through the real world, on the street, showing you the menu & availability as you enter the food joint, assisted learning on how to play a piano, without having to take your hands off the piano or as a tool for social change, is here. The remarkable vision, of a look-up interface, that will replace many of our daily gadgets, is the future of technology evolution.

It’s extremely challenging to realize this vision. For starters, we want the system to be light weight, thin form factor, adjust to differences in our human sensory system and deliver a smooth user experience. Dig deeper, and we also want low power and thus low computational complexity, large field of view & resolution, large eye box, good head tracking and gaze tracking for accurate location, low latency rendering and of course lower price. Many of these parameters are not good friends of each other, you get one parameter right at the cost of another, there is no free lunch, at-least not yet. You have to be smart in picking the right parameters you want to optimize. For this ultimate dream to come true, many disciplines have to come together — from optics, computer vision, interaction paradigms, new business models to product design.

Weight & Comfort

By now, I have experimented with Hololens, ODG glasses, DAQRI smart helmet, Meta demo and some other AR devices. Both Hololens and DAQRI smart helmet comes packed with lot of sensors. With all the display optics and sensors, where to place what, which technology to use and how to distribute weight evenly is a challenge. I have also had to adjust the glasses up/down, so I can see the display properly and consistently.

Clear and Smooth User Experience

New interaction paradigms which are natural to humans yet that doesn't make the information cluttered is important. Human evolution has already invested billions of years in training mind to understand how to interpret & interact with the real world. This is not something VR or AR systems can afford to mess with.

Designers & technologists try to balance the trade-offs between visibility and flexibility. We try to experience what it is to have a dynamic extension of user’s identity projected on to the world, have new ways to give feedback that involves more senses and give users creative freedom to manipulate objects.

All of these open up exciting new ways to interact with the world. Meta CEO’s exciting TED talk shows the potential of human centric design & technology https://www.youtube.com/watch?v=H9ZOpQzjukY

Optics/Display/Electronics considerations

AR glasses need to be comfortable and not degrade human sight. Thin form factor, large FOV (Field Of View), good enough resolution, large eye box, calibration to fit different visual parameters are important considerations. Although human eye has 130–135° vertical and 200° horizontal FOV (both eyes combined), its difficult to get higher FOV without compromising on resolution. If it’s just video resolution, we are already there with respect to technology.

The eye box specifies how much the user’s eye can stray away left/right/up/down from the location of the center of the lens without losing the image or suffering significant degradation in image quality. Bigger eye box is better.

Calibration issues is one of the annoying challenges I have seen in every single demo that I have experienced both in Head Mounted Displays as well as Optical See Through Glasses. Inconsistencies in spatial, temporal, visual and social parameters makes the process tough.

When you look at a 3-D scene, the depth at which your eyes are focused varies for objects at different distances. Omitting focus cues will give an artificial experience. Light field displays have become more popular recently (Lytro, Nvidia, Magic Leap), however need more computation and resolution is sacrificed. Compressive sensing techniques have lowered computation requirements but still widely used in research than on commercial products. Pin Light displays offers wide FOV and thin form factor but the image formed is not very sharp, also resolution is not good.

Use-case, Cost & Business model

AR & VR is set to hit $150 Billion, surpassing mobile, by 2020. Gaming and Industrial applications have been the initial commercial targets. Education, health, prosumer and consumer segments will follow soon. Current cost is prohibitive for consumers. Most companies have SDK offering, to get developers to develop applications and content. New business models will evolve like renting or subscription models to make it more affordable and sustainable.

Conclusion

In the next 3–5 years, I am hopeful most of the issues mentioned here will be solved through a combination of technology & design. We now have most parts that can build the future of consumer devices and the winner will be the one who can combine these in unique ways while understanding deeply how human visual system and human interaction system works.

• Raji Kannan