What’s next:
In the Labs

×

Buffalo buffalo Buffalo buffalo buffalo buffalo Buffalo buffalo: Part 1 of What’s left to tackle in voice technology

We have a long way to go before voice technology has reached the human level of interactions. Ken Arakelian starts this four-part blog series talking about where we are today in voice, where we’re going and where Nuance is leading the way.

By

Me: “Alexa – what’s the temperature going to be today?”

Alexa: “Right now the temperature is 56 degrees today in with cloudy skies.  Today you can expect clouds and showers with a high of 60 degrees and a low of 44 degrees. ”

Me: “What about tomorrow?”

Alexa: [blank stare]

Me:  “Ugh – Alexa – what will the temperature be tomorrow?”

Voice as a computer interface has come a long way, but it’s still clunky and nothing like talking to another person. Our amazement with how far the technology has come since voice recognition in IVRs came on the scene in the 1980s can make us forget the remaining problems we have to tackle to get to human-level interactions. In this blog series, I’m going to take each remaining hurdle and talk about where we are today, where we’re going and how Nuance is leading the way.

 

Part 1: Automatically generating dialog for conversations is a complex problem to solve.

“Buffalo buffalo Buffalo buffalo buffalo buffalo Buffalo buffalo.” Believe it or not, this is a grammatically correct sentence and illustrates why automating natural language processing and conversation is hard. If you’re wondering what the Buffalo sentence means you can click the link and read about it (helpful tip – take an Advil). The tl;dr (too long; didn’t read) version is that the word “buffalo” can be a proper noun, noun, or a verb, so the sentence translates to something about how buffalo from Buffalo bully (aka buffalo) buffalo, etc…

This is obviously an extreme example, but it just goes to show that there is plenty of meaning and “nuance” hidden in the words people choose that computers haven’t been “taught” to understand yet.

Here’s an example that may resonate more with English speakers:

SHE never told him that she loved him. (but someone else did)

She NEVER told him that she loved him. (zero times in their entire relationship)

She never TOLD him that she loved him. (she showed it but never said it out loud)

She never told HIM that she loved him. (but told everybody else)

She never told him that SHE loved him. (but that someone else did)

She never told him that she LOVED him. (only that he liked him and thought was funny)

She never told him that she loved HIM. (she said she loved someone else)

As a live, English-speaking human, you would catch the subtle changes in meaning just by placing inflection on different words. However, artificial intelligence would have to be taught that kind of nuance.

Another great illustration of the complexity of language can be seen in a video of physicist Richard Feynman, apparently being condescending to his interviewer: Richard Feynman Magnets – YouTube. The interviewer is simply asking Dr. Feynman to explain magnetism to him, and Dr. Feynman refuses and dismisses the question, saying that the interviewer won’t understand. The net of the video is that Dr. Feynman can’t explain magnetism in a meaningful way without a shared frame of reference – and he and the interviewer don’t share one. The interviewer doesn’t have the degrees that Dr. Feynman has, so he equates it to explaining to an alien why his wife is in the hospital with a broken leg. Well, she slipped and fell. Why did she slip and fall? Well, she was walking on ice. Why is ice slippery? …etc., on down into deeper and deeper levels of complexity – for seven minutes – and never answers the magnetism question. (One viewer posted, “This is why no one talks to you at parties.”)

This complexity is at the core of the problem we need to solve for computers to “learn” how to converse with humans. Nuance is making great advances in automating conversation.  Currently the state of the art in this area is still Simple Question Answering (essentially Enterprise Search front-ended with Natural Language Understanding). See Paul Tepper’s Post on advances in automating conversation. Nuance is working internally and with research partners on encoding the general knowledge that computers need in order to decipher the buffalo sentence and to have a frame of reference to converse with humans.

So, just in case you didn’t have a frame of reference when reading this blog post, go back and read the Wikipedia entry on the buffalo sentence and watch the Dr. Feynman video. Then you’ll understand the monstrous task we have in bringing voice technology up to human-level interactions.

Next time: Part 2: Sentiment and Emotion in Voice – “Your customer seems angry – umm – now what?”

Read full article

More from the editor

Beyond the algorithms: Shaping the future of the Automotive Assistant for autonomous cars
New study reveals how drivers want to get notifications in the cars of the future
Dragon celebrates its 20th anniversary
Two decades later, we’re still talking to our computers – and much more
Nuance Research Conference 2017: Reflections on Deep Learning and AI innovation
Keynotes from John Searle & Barbara Grosz inspire Nuance’s global voice, AI research teams
1,000 years of emoji history and what Machine Learning means for its future
A look at emoji: how they’ve changed over time and where they are going
Dragon, do you speak my dialect?
Once divisive, now unifying, dialects play an important role in defining who we are
Nuance’s inaugural engineering conference, Ncode, rocks it in Montreal
Q&A with Nuance Mobile R&D from the event
How the technology transcribing your meetings actually works
Simple isn’t always as simple as it seems
Why we’re using Deep Learning for our Dragon speech recognition engine
Unique application of Neural Nets results in greater productivity
Winograd Schema Challenge: Can computers reason like humans?
Results from the inaugural Winograd Schema Challenge unveiled at the IJCAI-16 in New York
Hearing is like seeing – for our brains and for machines
How CNNs developed for image recognition help with ASR and NLU, too
Part 1: How to avoid 5 common automotive HMI usability pitfalls
Audio and touch input are at the core of a powerful automotive HMI system
Part 3 – AI for customer care: Using Machine Learning to solve customer requests
Turning big data into big knowledge for better customer service
Part 2 – AI for customer care: Turning ‘bags of words’ into meaning with machine learning
Machine learning and AI turn big data into big knowledge for a better customer experience
Nuance and DFKI help students create interactive appliances of the future with speech tools
Providing easy to use speech tech helps usher forth tomorrow’s interactive appliances
Part 1 – AI for customer care: Human assisted virtual agents get smart with big knowledge
Machine learning and AI turn big data into big knowledge for a better customer experience
Mercedes-Benz’s Margarete Wies discusses the future of the connected car
Extending digital living with infotainment systems, autonomous vehicles, and more
In a galaxy (not so) far, far away
Star Wars and the relationship between man and machine
Then and NAO: Bringing conversational robots to homes, hotels, and hospitals
Aldebaran's NAO and Pepper show the power of specialized voice experiences for robotics
How many Neural Nets does it take to catch the big fish in Machine Learning?
NLU and AI innovation goes deeper so machines can understand human language
KITT – Please open the garage
How talking cars that talk to "things" make life simpler, smarter
Meet Lisa, a world championship robot with a lot to say
How this student-built social robot can lend a helpful hand to our aging society
Just be yourself: More on variation, voice biometrics, and the science of voice technology
Using Deep Neural Networks to add variation and improve accuracy
Taking a pause to discuss speaker variation… and Machine Learning
New research observes variation in communication to abstract meaning
Innovating dialog: How machines use and make sense of ellipses
Building speech systems that naturally use ellipses in human-machine interaction
Innovating dialogue: How machines make sense of anaphora
Building speech systems that naturally use anaphora in human-machine interaction
#iLookLikeAnEngineer: Breaking down gender stereotypes in tech
An inside look at Nuance and how diversity fuels innovation
Lost in translation: A solo trip abroad and discovering the art of language
Sunrises in Spain and machines you can talk to
Innovating dialogue: How machines make sense of paraphrasing and adult language
Building speech systems that naturally use paraphrases in human-machine interaction
Innovating machine dialog: Brush up on your Greek and read Aristotle
Building systems that can make sense of Rhetoric and nuanced speech
Deep learning, coming to a car near you
Computing systems inspired by the human brain change the way we interact with cars
Getting “deep” about “deep learning”
A detailed exploration of deep machine learning, a concept rooted in metaphors
The personality of Science: The traits that help define an industry
Nuance researchers answer, “What qualities does a good researcher possess?”
Technology inspired by humans – A look back at NRC 2015
Reflections from Nuance Research Conference 2015
The intersection of Science Fiction, super-pi, and technology innovation
An ode to Mr. Spock and to chasing the impossible
Star Trek, Mr. Spock, and a highly sought-after future
How a vision for the future helped allay the anxieties of the time.
It’s time to take off your tinfoil hats: AI is safe for human consumption
Exploring the effects of artificial intelligence on our daily lives
Nuance senior research scientist David Martin receives AAAI Senior Member status
Leading Artificial Intelligence industry group recognizes Martin for career achievements
Can machines think?
Nuance to host annual Winograd Schema Challenge, an alternative to the Turing Test
Humanizing technology through Cognitive Computing and Artificial Intelligence
Nuance furthers AI investment with DFKI research center
Why “innovation” doesn’t always have to be new (or at least on first sight)
Decades old concepts give light to revolutionary innovations
Ethics and design: Doing the right thing
The importance of design stretches far beyond basic appeal
Will machine language bring about the demise of voice actors?
The science behind creating next-generation synthetic voices
Can we build ‘Her’?: What Samantha tells us about the future of AI
The journey to making virtual assistants more humanlike
Innovation and design: The coolness (and unusability) of our rich friends’ houses
The challenges of designing for fanfare vs. usability
The never-ending evolution of natural language understanding
Rapid development in natural language understanding creates new possibilities
Nuance’s Ron Kaplan awarded honorary doctorate from University of Copenhagen
Award recognizes significant contributions to linguistics and natural language
Video: Innovating a relationship between people and technology
Making technology that works *for* us - not against us
Nuance’s Peter Patel-Schneider receives prestigious SWSA Ten-Year Award
SWSA award honors most impactful research
Beyond the GUI: It’s time for a conversational user interface
Conversational user interface promoting new interactions between people and devices
Voice recognition and the dawn of intelligent systems
Examining the rapid progress of voice recognition and natural language understanding
Nuance opens new Mobile Innovation Center in Cambridge’s Central Square
The new mobile innovation center is home to the expanding segment of Nuance’s R&D.
Nuance Chief Technology Officer Vlad Sejnoha named 2013 CTO of the year
Nuance CTO presented with the CTO of the Year Award at Mass TLC Awards
Smart watches need intelligent systems
Here's how you deliver intelligent systems for the evolving wearables ecosystem
Nuance and Intel keep their heads in the cloud
Collaborating on cloud computing to advance intelligent NLU systems
Got GPUs? Nuance puts groundbreaking NVIDIA GPUs to work to accelerate voice innovation
Ushering a new era in Machine Learning
Beyond the algorithms: Shaping the future of the Automotive Assistant for autonomous cars
New study reveals how drivers want to get notifications in the cars of the future
The history of emoji and its future with Machine Learning
1,000 years of emoji history and what Machine Learning means for its future
A look at emoji: how they’ve changed over time and where they are going
New technology can transcribe meetings between colleagues
How the technology transcribing your meetings actually works
Simple isn’t always as simple as it seems
Seeing is like hearing for machines and human brains
Hearing is like seeing – for our brains and for machines
How CNNs developed for image recognition help with ASR and NLU, too
Machine learning turns bags of words from big data into big knowledge for customer care
Part 2 – AI for customer care: Turning ‘bags of words’ into meaning with machine learning
Machine learning and AI turn big data into big knowledge for a better customer experience
The Future Mobility vehicle becomes a contextual and highly personalized digital living space.
Mercedes-Benz’s Margarete Wies discusses the future of the connected car
Extending digital living with infotainment systems, autonomous vehicles, and more
How many Neural Nets does it take to catch the big fish in Machine Learning?
How many Neural Nets does it take to catch the big fish in Machine Learning?
NLU and AI innovation goes deeper so machines can understand human language
Variation can improve accuracy of speaker verification for voice biometrics
Just be yourself: More on variation, voice biometrics, and the science of voice technology
Using Deep Neural Networks to add variation and improve accuracy
in communication, speech systems are built to interpret and use rhetorical devices like anaphora
Innovating dialogue: How machines make sense of anaphora
Building speech systems that naturally use anaphora in human-machine interaction
in communication, speech systems are built to make sense of and use rhetorical devices like paraphrase
Innovating dialogue: How machines make sense of paraphrasing and adult language
Building speech systems that naturally use paraphrases in human-machine interaction
Getting “deep” about “deep learning”
A detailed exploration of deep machine learning, a concept rooted in metaphors
On Super Pi Day, we celebrate those who dare to chase the impossible and innovate a futuristic world full of things even Mr. Spock couldn't imagine
The intersection of Science Fiction, super-pi, and technology innovation
An ode to Mr. Spock and to chasing the impossible
aaai-senior-member-david-martin
Nuance senior research scientist David Martin receives AAAI Senior Member status
Leading Artificial Intelligence industry group recognizes Martin for career achievements
Why “innovation” doesn’t always have to be new (or at least on first sight)
Decades old concepts give light to revolutionary innovations
Can we build ‘Her’?: What Samantha tells us about the future of AI
The journey to making virtual assistants more humanlike
Nuance’s Ron Kaplan awarded honorary doctorate from University of Copenhagen
Award recognizes significant contributions to linguistics and natural language
Beyond the GUI: It’s time for a conversational user interface
Conversational user interface promoting new interactions between people and devices
Nuance Chief Technology Officer Vlad Sejnoha named 2013 CTO of the year
Nuance CTO presented with the CTO of the Year Award at Mass TLC Awards
Got GPUs? Nuance puts groundbreaking NVIDIA GPUs to work to accelerate voice innovation
Ushering a new era in Machine Learning
Dragon speech recognition software celebrates its 20 year anniversary
Dragon celebrates its 20th anniversary
Two decades later, we’re still talking to our computers – and much more
Nuance speech technology can understand over 80 languages and their dialects
Dragon, do you speak my dialect?
Once divisive, now unifying, dialects play an important role in defining who we are
Dragon uses deep learning for more accurate speech recognition.
Why we’re using Deep Learning for our Dragon speech recognition engine
Unique application of Neural Nets results in greater productivity
How to get the key foundation right for automotive HMI with audio and touch input
Part 1: How to avoid 5 common automotive HMI usability pitfalls
Audio and touch input are at the core of a powerful automotive HMI system
DFKI students use nuance speech tools to create interactive IoT applications
Nuance and DFKI help students create interactive appliances of the future with speech tools
Providing easy to use speech tech helps usher forth tomorrow’s interactive appliances
Star Wars’ portrayal of relationships between robots and humans is becoming today’s reality with new technological advancements
In a galaxy (not so) far, far away
Star Wars and the relationship between man and machine
Connected cars are becoming more connected to the IoT and more useful, employing ubiquitous personal assistants that exist across devices and experiences
KITT – Please open the garage
How talking cars that talk to "things" make life simpler, smarter
Speech systems need to observe and deal with pauses and other variation to elicit more natural communication between man and machine
Taking a pause to discuss speaker variation… and Machine Learning
New research observes variation in communication to abstract meaning
Female Nuance engineers share stories about combatting gender stereotypes in the tech industry
#iLookLikeAnEngineer: Breaking down gender stereotypes in tech
An inside look at Nuance and how diversity fuels innovation
The ancient Greeks discovered rhetorical devices which are now common in everyday language - something we need to specially design speech systems to accommodate
Innovating machine dialog: Brush up on your Greek and read Aristotle
Building systems that can make sense of Rhetoric and nuanced speech
Childlike curiosity, being comfortable with a blank page... Nuance researchers share what qualities they think a good researcher possesses
The personality of Science: The traits that help define an industry
Nuance researchers answer, “What qualities does a good researcher possess?”
Star Trek, Mr. Spock, and the future of patient care
Star Trek, Mr. Spock, and a highly sought-after future
How a vision for the future helped allay the anxieties of the time.
winograd-schema-challenge
Can machines think?
Nuance to host annual Winograd Schema Challenge, an alternative to the Turing Test
Ethics and design: Doing the right thing
The importance of design stretches far beyond basic appeal
Innovation and design: The coolness (and unusability) of our rich friends’ houses
The challenges of designing for fanfare vs. usability
Video: Innovating a relationship between people and technology
Making technology that works *for* us - not against us
Voice recognition and the dawn of intelligent systems
Examining the rapid progress of voice recognition and natural language understanding
Wearables like smart watches need intelligent systems to enable a meaningful human-device interaction
Smart watches need intelligent systems
Here's how you deliver intelligent systems for the evolving wearables ecosystem
bringing together leading minds to tackle advancements in AI
Nuance Research Conference 2017: Reflections on Deep Learning and AI innovation
Keynotes from John Searle & Barbara Grosz inspire Nuance’s global voice, AI research teams
Nuance engineers gather at Ncode
Nuance’s inaugural engineering conference, Ncode, rocks it in Montreal
Q&A with Nuance Mobile R&D from the event
Contestants for the Winograd Schema Challenge build intelligent systems to test natural language and reasoning capabilities.
Winograd Schema Challenge: Can computers reason like humans?
Results from the inaugural Winograd Schema Challenge unveiled at the IJCAI-16 in New York
How you can use machine learning and natural language methods to accurately answer customer service questions
Part 3 – AI for customer care: Using Machine Learning to solve customer requests
Turning big data into big knowledge for better customer service
An agent in a call center supports virtual agents
Part 1 – AI for customer care: Human assisted virtual agents get smart with big knowledge
Machine learning and AI turn big data into big knowledge for a better customer experience
Machine Learning, Neural Nets, and advanced voice technology are making the robots for homes, banks, hotels, and more, even smarter
Then and NAO: Bringing conversational robots to homes, hotels, and hospitals
Aldebaran's NAO and Pepper show the power of specialized voice experiences for robotics
Students from the University of Koblenz-Landau built Lisa, a helpful social robot who can communicate with humans and perform daily tasks.
Meet Lisa, a world championship robot with a lot to say
How this student-built social robot can lend a helpful hand to our aging society
How machine speech systems use and make sense of ellipses rhetorical devices
Innovating dialog: How machines use and make sense of ellipses
Building speech systems that naturally use ellipses in human-machine interaction
Traveling alone in Spain, I formed a new appreciation for language as an art and the technology we’ve built to interpret and interact with people
Lost in translation: A solo trip abroad and discovering the art of language
Sunrises in Spain and machines you can talk to
deep learning connected car echnology
Deep learning, coming to a car near you
Computing systems inspired by the human brain change the way we interact with cars
Nuance Research Conference 2015 explored R&D topics like Deep Neural Nets, Artificial Intelligence, Natural Language Understanding, Anaphora, and more
Technology inspired by humans – A look back at NRC 2015
Reflections from Nuance Research Conference 2015
Two things so different can live in harmony - these are the positive effects of artificial intelligence on humanity
It’s time to take off your tinfoil hats: AI is safe for human consumption
Exploring the effects of artificial intelligence on our daily lives
artificial-intelligence
Humanizing technology through Cognitive Computing and Artificial Intelligence
Nuance furthers AI investment with DFKI research center
Will machine language bring about the demise of voice actors?
The science behind creating next-generation synthetic voices
The never-ending evolution of natural language understanding
Rapid development in natural language understanding creates new possibilities
Nuance’s Peter Patel-Schneider receives prestigious SWSA Ten-Year Award
SWSA award honors most impactful research
Nuance opens new Mobile Innovation Center in Cambridge’s Central Square
The new mobile innovation center is home to the expanding segment of Nuance’s R&D.
Nuance and Intel keep their heads in the cloud
Collaborating on cloud computing to advance intelligent NLU systems
Show more articles