Last week, a friend of mine wanted to play a song for me on his home stereo system. He took out his phone, started an app, switched on the amplifier, selected the audio source type, then the device, search mode, the music folder (I’m fetching coffee in the meantime), then the artist and the album, before finally hitting “play.” No faster than taking a CD out of its case and putting it into the CD player like in the good old days.
As I create speech and language technology for a living, I was happy to point out how speech recognition can simplify our interactions with complicated gadgets: a brief voice command like “play the Joshua Tree album” is a natural and efficient shortcut through tedious menus. You press the speech button on your phone or watch, speak the command, and the task is carried out.
We’re seeing this simplicity of voice being applied to cars and a variety of apps and devices – and with the arrival of the Internet of Things where devices have very little or no screen at all, there is simply no easy way to operate your device other than natural human conversation.
When it comes to voice in the car, however; convenience is great, but safety is critical – notably when there are a myriad of apps and services now available as part of the connected infotainment system. A simple “Hello car, I want to go to 4657 Mission Boulevard in Pacific Beach” enters the destination quickly and safely. The same goes for finding music, places of interest (POI) and now even the ability to find reservations and book a table.
Breaking the device boundaries: Bringing IoT and connected cars together
In a world where virtually everything is now connected with a layer of intelligence, there is a risk that devices and in-car systems become too complicated to use. But, where voice, natural language and smart design are integrated, engaging these systems becomes, well, more human.
However, there’s still a disconnect when it comes to the simplicity of our interactions with these smart cars and devices…not everything that is connected is connected to each other.
That’s about to change too, with in-car systems extending their reach beyond the car and traditional infotainment applications, and connecting to appliances and systems that are available in a smart home.
When cars and things “talk” to each other – we experience simplicity and instant gratification like never before. Consider:
- Sitting in your living room, you ask your car via your home system for the charge status, to lock the doors, turn on the heater, or activate other functions of your car.
- In your car, 15 minutes away from home, you can already turn on the heating, and while you pull into the driveway, you say “Hello Dragon, turn on the lights, deactivate the alarm, and open the garage.”
Gone are the days of having to use three different phone or remote control apps separately to operate lights, alarm, and the garage door – simply speaking these commands to the car saves a lot of time and effort.
Further, additional cross-device experiences through mobile companion apps will come to market that connect a phone and a car, so you can simply ask “where did I park my car?” and the phone guides you.
The ultimate personal assistant
If I were fortunate enough to have a personal assistant like the brilliant Pepper Potts, I could greet her with a short “Hello Pepper, I’m home,” and she’d kindly arrange that lights are turned on, the alarm off, and the garage opens. She knows the functions that my various appliances and services have, and can control all of them in a smart way. Sadly, I don’t have Pepper Potts as my aide – but I may never need her with speech-driven virtual personal assistants proving a helpful alternative.
These assistants should be able to control all of our appliances. Imagine in the future, every time you buy a new device or service, like a music player or a new home security system, your assistant will automatically learn to operate its features. From that music player, the assistant software learns the functions to play songs or Internet radio, and it generates speech commands for those on its own. Thus, over time, your personal assistant will know how to operate all the functions from all your devices – and will let you control them by voice.
This will also allow the assistant to combine actions across devices in a smart way. Consider the example above, when you get home, your assistant will understand it should turn on lights and open the garage door. You won’t need to request all of these actions separately. Or, the assistant knows that just before 7:30 a.m., it should tee up your favorite playlist along with the traffic report and emails in your car, and turn off your TV before you leave.
In the same fashion that the assistant can learn the functions from all your connected devices, it will also know the devices from which you can talk to it. This way, no matter whether you are addressing your digital aide via your living room, phone, or car, you always reach your own assistant. And things start to get really amazing when you can also bring your assistant into devices that are not your own, like a hotel room TV or your rental car while you travel.
From human–machine interfaces to human–function interfaces
While I was in school, human–machine interaction (HMI) started to really take off, the study of good design that allows operating devices intuitively and efficiently. But now, I find the term HMI to be a bit of a misnomer. It’s really not the complicated ‘machine’ with which I want to interact, but rather directly with its functions – I want an experience that is intuitive. I don’t want to study the user’s manual to find out which buttons I need to press on my new amplifier before it plays music. I just want it to play that album, as quickly as possible.
This at its heart is the promise of speech and language technology: to make our lives easier, more productive, and more enjoyable. And that is what drives innovation for cross-device virtual assistants. No matter whether you ask your car to warm up your house while you’re on your way home, or ask your home hub system to prepare navigation info for your weekend road trip – your ubiquitous personal assistant is awaiting your command.