The trend in computing towards more natural user interfaces is unmistakable. Graphical user interfaces have long been dominant, but machines driven by more intuitive inputs, like touch and voice, are now mainstream. Today, audio, motion, and even our thoughts, are the basis for the most innovative computer-user interaction models powered by advanced sensor technology. Each computing paradigm maps to one or more of the five human senses; exploring each sense gives us an indication of the direction in which technology is heading.
Sight – Graphical User Interface
The introduction of the graphical user interface (GUI) drove a step function change in computers as productivity tools, because users could rely heavily on sight, our dominant sense. The GUI was then carried forward and built on with the advent of touchscreen devices. The next frontier for visual user interfaces lies in virtual reality and augmented reality. Innovations within these themes will further carry forward the GUI paradigm. VR and AR rely heavily on sight, but combine it more artfully with other inputs like audio, motion, and touch to create immersive interfaces.
Touch – Touchscreen Devices
PCs leveraged basic touch as a foundational input via the keyboard and the mouse. The iPhone then ushered in a computing era dominated by touch, rejecting the stylus in favor of, as Steve Jobs put it, “the best pointing device in the world” – our fingers. Haptics have pushed touchscreen technology further, making it more sensory, but phones and tablets fall well short of truly immersive computing. Bret Victor summarized the shortcomings of touchscreen devices in his 2011 piece, A Brief Rant on the Future of Interaction Design, which holds up well to this day.
More fully integrating our sense of touch will be critical for the user interfaces of the future. We think that haptic suits are a step we will take on the journey to full immersion, but the best way to trick the user into believing he or she is actually feeling something in VR is to manipulate the neurochemistry of the brain. This early field is known as neurohaptics.
Hearing – Digital Assistants & Hearables
Computers have been capable of understanding a limited human spoken vocabulary since the 1960s. By the 1990s, dictation software was available to the masses. Aside from limited audio feedback and rudimentary speech-to-text transcription, computers did not start widely leveraging sound as an interface until digital assistants began to be integrated into phones.
As digital assistants continue to improve, more and more users are integrating them into their daily routines. In our Robot Fear Index, we found that 43% of Americans had used a digital assistant in the last three months. However, our study of Amazon Echo vs. Google Home showed that Google Home answered just 39.1% of queries correctly vs. the Echo at 34.4%. Clearly we’re early in the transition to audio as a dominant input for computing.
Hearables, like Apple’s AirPods, represent the next step forward for audio as a user interface.
Through Siri, AirPods can handle information requests, dictation, media control, and phone calls; meanwhile, quick glances at the Apple Watch on your wrist will suffice for most notifications. All of this means that your phone stays in your pocket. There’s more from us on AirPods here, but we believe audio as a UI is a key enabler of augmented reality technology and the future of computing. As your “device” becomes a seamless overlay of digital information on the real world, non-visual inputs like audio and motion become critical components to interacting with that device.
Smell & Taste – Advanced VR
Let’s not leave out smell and taste. The pinnacle of VR and AR experiences will include all five senses, not just sight, sound, and sometimes touch like we see today. As noted above, neurohaptics will ultimately enable lifelike touch experiences created by direct influence on the brain. The same neurochemical manipulation will be capable of enabling smell and taste. This form of advanced VR carries the promise of truly multi-sensory computing, making our digital experiences much more lifelike than previously imagined, perhaps indistinguishable from the real thing.
As we track the arc of computing technology over the past few decades and peer into the future, it clearly bends towards more advanced user interfaces. Bringing all five senses together into a fully immersive future of computing. A future in which we interface with machines and each other in completely new ways.
Disclaimer: We actively write about the themes in which we invest: artificial intelligence, robotics, virtual reality, and augmented reality. From time to time, we will write about companies that are in our portfolio. Content on this site including opinions on specific themes in technology, market estimates, and estimates and commentary regarding publicly traded or private companies is not intended for use in making investment decisions. We hold no obligation to update any of our projections. We express no warranties about any estimates or opinions we make.