Natural User Interfaces or NUI
Trend watching has become a profession in recent years and there are sites dedicated to monitoring trends of all kinds. I spend a good deal of my time writing about trends in technology and observing a number of key trends that Microsoft is focused on. Individual trends such as Big Data, the Internet of Things and cloud computing are fascinating on their own—but over the last year it’s become clear to me that a number of trends are maturing and colliding at the same time and helping to fuel one big trend we’re focused on—Natural User Interfaces or NUI. Allow me to explain what NUI is and some of the trends behind it.
In simple terms, NUI is concerned with making technology much more natural to interact with. That may be through voice control of a TV (rather than fumbling with remotes and on screen menus), it may be through playing a video game just using your body with no need to master a controller of any kind, or it may be logging in to a system by simply having the system recognize you. It’s no coincidence that all of these examples are features of Kinect for Xbox 360, which has taken NUI in to 18 million homes in a little over a year. Kinect is a great example of NUI in action—taking advantage of advanced computing capabilities to remove the barriers for interaction. Kinect is just one example though—the touch screen interfaces that we have become familiar with on our smartphones are examples of NUI. Behind Kinect and smartphones there are a set of underlying trends that are enabling natural interaction. Let’s take a brief look at two—the proliferation of sensors and big data.
Kinect contains a number of “sensor”—a series of cameras, including a depth camera that enable Kinect to recognize you–as well as a sophisticated audio array of microphones that can pinpoint your voice, even in a very noisy room. When you stop and think about the number of sensors (or computers of some kind) in your life you begin to notice that there are far more than the traditional devices you often associate with computing. Increasingly our homes are filled with electronics in the form of household appliances, lighting systems, heating systems, alarm systems, even electronic vacuum cleaners. Each of these systems has a degree of intelligence and over time more and more of them will become connected to the Internet and connected together. Our cars already have hundreds of computers and sensors and they’re getting smarter each year. Soon they’ll be connecting to and “talking with” smart streets and cities to help us find a parking space more quickly and pay for it automatically.
Our smartphones are already filled with sensors—GPS, accelerometers, compasses, light meters and more. Some phones already increase the ringer volume when the light sensors detect it’s dark (and assume your phone is in your pocket or bag). As we allow them to, our smartphones are set to get smarter. The GPS knows where we are and the calendar knows where we’re going so perhaps it’ll connect to our car and pre-program the route? All of these scenarios, big or small are examples of our interaction with technology becoming more natural. And all of them generate data—every click, every non-click, every movement, every gesture. Each one is data and the more there is, the more natural technology can become, as we can teach it by example using techniques known as machine learning.
Devices generate data and data enables smarter devices. All of this will lead to more natural ways to interact with technology—and there is another trend we’re very focused on at Microsoft that we believe will help fuel this shift – displays everywhere. We foresee a world where almost any surface can become an interactive display and we’re busy pushing the boundaries with transparently displays, displays that can see as well as project and technology that turns the palm of your hand in to a display.
Bookmark this page for “Natural user interface” and check back regularly as these articles update on a very frequent basis. The view is set to “news”. Try clicking on “video” and “2” for more articles.