What’s next:
In the Labs

×

“Customized voices are a great way to enhance and distinguish a brand”

New advancements in deep-learning enable Nuance to offer a large variety of mobility assistant voices that sound more human-like, can be customized according to customer demands and enable a great user experience across a large variety of day-to-day use cases. Johan Wouters, Director of Product Management gives some insights into our latest advancements in deep-learning based text-to-speech, voice creation and customer benefits of these solutions.

By
Customized voices are a great way to enhance and distinguish a brand

Personalization is one of the key features that distinguishes modern in-car smart assistants, mobility assistants or personal assistants from traditional infotainment systems as well as from one another. Building on white-labeled development platforms like Nuance Dragon Drive, car manufacturers have a chance to develop a personalized experience that is tailored to users’ specific needs and enhances their experience to one that is – unlike solutions provided by global tech giants – uniquely theirs.

Sophisticated smart assistant solutions not only enable smart command and control functionalities but also dialogue-based collaboration. When users are having a real conversation with their systems, these systems not only need to understand the queries but need to be able to deliver an appropriate response –with respect to content and context but also voice, intonation, and style.

New advancements in deep-learning enable Nuance to offer a large variety of system voices that sound more human-like, can be customized according to customer demands and enable a great user experience across a large variety of day-to-day use cases. I sat down with Johan Wouters, Director of Product Management to talk about our latest advancements in deep-learning based text-to-speech, voice creation and customer benefits of these solutions.

 

Johan, what is the key rationale behind customizing the voice used in a mobility assistant?

Personalized custom voices are important features of a unique user experience. The first direct impression of a speech system is the quality and personality of the voice output. This is why we are offering customized voices to our customers. In addition to a unique set of capabilities included in the assistant, these voices are a great way to enhance and distinguish a brand. We are able to adjust voice characteristics like gender, age, timbre, accent, and speaking style. In addition, our natural language generation (NLG) can adjust the wording to reflect the message and driver context.

 

At CES we showed how our mobility assistant reacts to different user emotions detected by voice and facial expressions. Can you expand a bit on that principle?

We are convinced that this principle strengthens the emotional connection between the user and the mobility assistant. Users feel understood by the system and are more likely to establish a kind of emotional relationship to it. On the other hand, adjusting the text-to-speech output – like the messaging style and the sentence lengths, for example – can support or counteract different moods and improve the user experience. If the user is chatty, the system offers longer, more colloquial prompts. If the user is not, the information is delivered in a rather neutral and factual style. This system variation is based on real-time detection of the user’s mood. But another possibility is adjusting the output based on learned preferences and context.

 

Let’s start with user preferences: How can the user benefit from personalization?

Nuance’s multi-style voices make it possible to render information in different ways. For example, if the system is providing a weather forecast, it can take into account individual weather preferences. Some people prefer cooler temperatures, others want to know about brightness, others are interested in wind speed, etc.  There are many ways to derive personalized information, such as by learning it from interactions, asking specific questions, or having the user create profile settings. With our advanced embedded capabilities, the information can stay in the car and does not need to be uploaded to the cloud. Alternatively, the personalized data can be uploaded in a secure way so that it can also be used by a rental or shared car.

 

How can contextual awareness influence the voice output of the system?  

When thinking about semi-autonomous driving and different transfer-of-control scenarios, the system could adapt the voice output depending on context. Again, we are talking about a scenario where the assistant is monitoring in real-time, but here, it is about fusing in-cabin and external sensor information. For example, if the system tracks that the driver’s eyes are off the road and a transfer of control is required, the voice output could be more forceful and assertive.

 

Which additional features affect the user experience in everyday scenarios?

Nuance offers multi-lingual modalities which is important for a positive user experience when crossing borders or entering regions with foreign street names, etc. Multi-lingual capabilities ensure that landmarks, street names, etc. are pronounced according to the local phonetic rules and not according to the language set for the system. In addition, our latest research prototypes of human-like voices are almost indistinguishable from actual human speech. I brought some examples and bet you won’t be able to get all of them right.

 

Read full article

More from the editor

Not a trust fall, but a trust rise to the occasion
Drive Lab Study investigates how voice-system transparency fosters trust
Q&A: How our latest internal hackathon brought innovation to non-profits
The SS11 Innovation Challenge for Social Good
Sorry, Team “Yanny” – AI says it’s “Laurel”
An audio clip is at the center of the latest viral debate
A tribute to Stephen Hawking and his iconic voice
Celebrating the life – and voice – of the renowned scientist
How the machines will adjust to us: A short story about “conversational AI” growing up
The evolution of conversational artificial intelligence
Multimodal interaction – How machines learn to understand pointing
Smart assistants combine speech, gesture and hand writing for human-like user experience
Buffalo buffalo Buffalo buffalo buffalo buffalo Buffalo buffalo: Part 1 of What’s left to tackle in voice technology
Automatically generating dialog for conversations is a complex problem to solve.
Beyond the algorithms: Shaping the future of the Automotive Assistant for autonomous cars
New study reveals how drivers want to get notifications in the cars of the future
Dragon celebrates its 20th anniversary
Two decades later, we’re still talking to our computers – and much more
Nuance Research Conference 2017: Reflections on Deep Learning and AI innovation
Keynotes from John Searle & Barbara Grosz inspire Nuance’s global voice, AI research teams
1,000 years of emoji history and what Machine Learning means for its future
A look at emoji: how they’ve changed over time and where they are going
Dragon, do you speak my dialect?
Once divisive, now unifying, dialects play an important role in defining who we are
Nuance’s inaugural engineering conference, Ncode, rocks it in Montreal
Q&A with Nuance Mobile R&D from the event
How the technology transcribing your meetings actually works
Simple isn’t always as simple as it seems
Why we’re using Deep Learning for our Dragon speech recognition engine
Unique application of Neural Nets results in greater productivity
Winograd Schema Challenge: Can computers reason like humans?
Results from the inaugural Winograd Schema Challenge unveiled at the IJCAI-16 in New York
Hearing is like seeing – for our brains and for machines
How CNNs developed for image recognition help with ASR and NLU, too
Part 1: How to avoid 5 common automotive HMI usability pitfalls
Audio and touch input are at the core of a powerful automotive HMI system
Part 3 – AI for customer care: Using Machine Learning to solve customer requests
Turning big data into big knowledge for better customer service
Part 2 – AI for customer care: Turning ‘bags of words’ into meaning with machine learning
Machine learning and AI turn big data into big knowledge for a better customer experience
Nuance and DFKI help students create interactive appliances of the future with speech tools
Providing easy to use speech tech helps usher forth tomorrow’s interactive appliances
Part 1 – AI for customer care: Human assisted virtual agents get smart with big knowledge
Machine learning and AI turn big data into big knowledge for a better customer experience
Mercedes-Benz’s Margarete Wies discusses the future of the connected car
Extending digital living with infotainment systems, autonomous vehicles, and more
In a galaxy (not so) far, far away
Star Wars and the relationship between man and machine
Then and NAO: Bringing conversational robots to homes, hotels, and hospitals
Aldebaran's NAO and Pepper show the power of specialized voice experiences for robotics
How many Neural Nets does it take to catch the big fish in Machine Learning?
NLU and AI innovation goes deeper so machines can understand human language
KITT – Please open the garage
How talking cars that talk to "things" make life simpler, smarter
Meet Lisa, a world championship robot with a lot to say
How this student-built social robot can lend a helpful hand to our aging society
Just be yourself: More on variation, voice biometrics, and the science of voice technology
Using Deep Neural Networks to add variation and improve accuracy
Taking a pause to discuss speaker variation… and Machine Learning
New research observes variation in communication to abstract meaning
Innovating dialog: How machines use and make sense of ellipses
Building speech systems that naturally use ellipses in human-machine interaction
Innovating dialogue: How machines make sense of anaphora
Building speech systems that naturally use anaphora in human-machine interaction
#iLookLikeAnEngineer: Breaking down gender stereotypes in tech
An inside look at Nuance and how diversity fuels innovation
Lost in translation: A solo trip abroad and discovering the art of language
Sunrises in Spain and machines you can talk to
Innovating dialogue: How machines make sense of paraphrasing and adult language
Building speech systems that naturally use paraphrases in human-machine interaction
Innovating machine dialog: Brush up on your Greek and read Aristotle
Building systems that can make sense of Rhetoric and nuanced speech
Deep learning, coming to a car near you
Computing systems inspired by the human brain change the way we interact with cars
Getting “deep” about “deep learning”
A detailed exploration of deep machine learning, a concept rooted in metaphors
The personality of Science: The traits that help define an industry
Nuance researchers answer, “What qualities does a good researcher possess?”
Technology inspired by humans – A look back at NRC 2015
Reflections from Nuance Research Conference 2015
The intersection of Science Fiction, super-pi, and technology innovation
An ode to Mr. Spock and to chasing the impossible
Star Trek, Mr. Spock, and a highly sought-after future
How a vision for the future helped allay the anxieties of the time.
It’s time to take off your tinfoil hats: AI is safe for human consumption
Exploring the effects of artificial intelligence on our daily lives
Nuance senior research scientist David Martin receives AAAI Senior Member status
Leading Artificial Intelligence industry group recognizes Martin for career achievements
Can machines think?
Nuance to host annual Winograd Schema Challenge, an alternative to the Turing Test
Humanizing technology through Cognitive Computing and Artificial Intelligence
Nuance furthers AI investment with DFKI research center
Why “innovation” doesn’t always have to be new (or at least on first sight)
Decades old concepts give light to revolutionary innovations
Ethics and design: Doing the right thing
The importance of design stretches far beyond basic appeal
Will machine language bring about the demise of voice actors?
The science behind creating next-generation synthetic voices
Can we build ‘Her’?: What Samantha tells us about the future of AI
The journey to making virtual assistants more humanlike
Innovation and design: The coolness (and unusability) of our rich friends’ houses
The challenges of designing for fanfare vs. usability
The never-ending evolution of natural language understanding
Rapid development in natural language understanding creates new possibilities
Nuance’s Ron Kaplan awarded honorary doctorate from University of Copenhagen
Award recognizes significant contributions to linguistics and natural language
Video: Innovating a relationship between people and technology
Making technology that works *for* us - not against us
Nuance’s Peter Patel-Schneider receives prestigious SWSA Ten-Year Award
SWSA award honors most impactful research
Beyond the GUI: It’s time for a conversational user interface
Conversational user interface promoting new interactions between people and devices
Voice recognition and the dawn of intelligent systems
Examining the rapid progress of voice recognition and natural language understanding
Nuance opens new Mobile Innovation Center in Cambridge’s Central Square
The new mobile innovation center is home to the expanding segment of Nuance’s R&D.
Nuance Chief Technology Officer Vlad Sejnoha named 2013 CTO of the year
Nuance CTO presented with the CTO of the Year Award at Mass TLC Awards
Smart watches need intelligent systems
Here's how you deliver intelligent systems for the evolving wearables ecosystem
Nuance and Intel keep their heads in the cloud
Collaborating on cloud computing to advance intelligent NLU systems
Got GPUs? Nuance puts groundbreaking NVIDIA GPUs to work to accelerate voice innovation
Ushering a new era in Machine Learning
When it comes to autonomous vehicles it is not enough to simply strive for innovative, but instead achieve ideal interactions
Not a trust fall, but a trust rise to the occasion
Drive Lab Study investigates how voice-system transparency fosters trust
Stephen Hawking speaks famous words with his computer-generated text-to-speech voice
A tribute to Stephen Hawking and his iconic voice
Celebrating the life – and voice – of the renowned scientist
Buffalo buffalo Buffalo buffalo buffalo buffalo Buffalo buffalo: Part 1 of What’s left to tackle in voice technology
Automatically generating dialog for conversations is a complex problem to solve.
bringing together leading minds to tackle advancements in AI
Nuance Research Conference 2017: Reflections on Deep Learning and AI innovation
Keynotes from John Searle & Barbara Grosz inspire Nuance’s global voice, AI research teams
Nuance engineers gather at Ncode
Nuance’s inaugural engineering conference, Ncode, rocks it in Montreal
Q&A with Nuance Mobile R&D from the event
Contestants for the Winograd Schema Challenge build intelligent systems to test natural language and reasoning capabilities.
Winograd Schema Challenge: Can computers reason like humans?
Results from the inaugural Winograd Schema Challenge unveiled at the IJCAI-16 in New York
How you can use machine learning and natural language methods to accurately answer customer service questions
Part 3 – AI for customer care: Using Machine Learning to solve customer requests
Turning big data into big knowledge for better customer service
An agent in a call center supports virtual agents
Part 1 – AI for customer care: Human assisted virtual agents get smart with big knowledge
Machine learning and AI turn big data into big knowledge for a better customer experience
Machine Learning, Neural Nets, and advanced voice technology are making the robots for homes, banks, hotels, and more, even smarter
Then and NAO: Bringing conversational robots to homes, hotels, and hospitals
Aldebaran's NAO and Pepper show the power of specialized voice experiences for robotics
Students from the University of Koblenz-Landau built Lisa, a helpful social robot who can communicate with humans and perform daily tasks.
Meet Lisa, a world championship robot with a lot to say
How this student-built social robot can lend a helpful hand to our aging society
How machine speech systems use and make sense of ellipses rhetorical devices
Innovating dialog: How machines use and make sense of ellipses
Building speech systems that naturally use ellipses in human-machine interaction
Traveling alone in Spain, I formed a new appreciation for language as an art and the technology we’ve built to interpret and interact with people
Lost in translation: A solo trip abroad and discovering the art of language
Sunrises in Spain and machines you can talk to
deep learning connected car echnology
Deep learning, coming to a car near you
Computing systems inspired by the human brain change the way we interact with cars
Nuance Research Conference 2015 explored R&D topics like Deep Neural Nets, Artificial Intelligence, Natural Language Understanding, Anaphora, and more
Technology inspired by humans – A look back at NRC 2015
Reflections from Nuance Research Conference 2015
Two things so different can live in harmony - these are the positive effects of artificial intelligence on humanity
It’s time to take off your tinfoil hats: AI is safe for human consumption
Exploring the effects of artificial intelligence on our daily lives
artificial-intelligence
Humanizing technology through Cognitive Computing and Artificial Intelligence
Nuance furthers AI investment with DFKI research center
Will machine language bring about the demise of voice actors?
The science behind creating next-generation synthetic voices
The never-ending evolution of natural language understanding
Rapid development in natural language understanding creates new possibilities
Nuance’s Peter Patel-Schneider receives prestigious SWSA Ten-Year Award
SWSA award honors most impactful research
Nuance opens new Mobile Innovation Center in Cambridge’s Central Square
The new mobile innovation center is home to the expanding segment of Nuance’s R&D.
Nuance and Intel keep their heads in the cloud
Collaborating on cloud computing to advance intelligent NLU systems
Nuance SS11 Innovation Challenge
Q&A: How our latest internal hackathon brought innovation to non-profits
The SS11 Innovation Challenge for Social Good
How the machines will adjust to us: A short story about “conversational AI” growing up
The evolution of conversational artificial intelligence
Beyond the algorithms: Shaping the future of the Automotive Assistant for autonomous cars
New study reveals how drivers want to get notifications in the cars of the future
The history of emoji and its future with Machine Learning
1,000 years of emoji history and what Machine Learning means for its future
A look at emoji: how they’ve changed over time and where they are going
New technology can transcribe meetings between colleagues
How the technology transcribing your meetings actually works
Simple isn’t always as simple as it seems
Seeing is like hearing for machines and human brains
Hearing is like seeing – for our brains and for machines
How CNNs developed for image recognition help with ASR and NLU, too
Machine learning turns bags of words from big data into big knowledge for customer care
Part 2 – AI for customer care: Turning ‘bags of words’ into meaning with machine learning
Machine learning and AI turn big data into big knowledge for a better customer experience
The Future Mobility vehicle becomes a contextual and highly personalized digital living space.
Mercedes-Benz’s Margarete Wies discusses the future of the connected car
Extending digital living with infotainment systems, autonomous vehicles, and more
How many Neural Nets does it take to catch the big fish in Machine Learning?
How many Neural Nets does it take to catch the big fish in Machine Learning?
NLU and AI innovation goes deeper so machines can understand human language
Variation can improve accuracy of speaker verification for voice biometrics
Just be yourself: More on variation, voice biometrics, and the science of voice technology
Using Deep Neural Networks to add variation and improve accuracy
in communication, speech systems are built to interpret and use rhetorical devices like anaphora
Innovating dialogue: How machines make sense of anaphora
Building speech systems that naturally use anaphora in human-machine interaction
in communication, speech systems are built to make sense of and use rhetorical devices like paraphrase
Innovating dialogue: How machines make sense of paraphrasing and adult language
Building speech systems that naturally use paraphrases in human-machine interaction
Getting “deep” about “deep learning”
A detailed exploration of deep machine learning, a concept rooted in metaphors
On Super Pi Day, we celebrate those who dare to chase the impossible and innovate a futuristic world full of things even Mr. Spock couldn't imagine
The intersection of Science Fiction, super-pi, and technology innovation
An ode to Mr. Spock and to chasing the impossible
aaai-senior-member-david-martin
Nuance senior research scientist David Martin receives AAAI Senior Member status
Leading Artificial Intelligence industry group recognizes Martin for career achievements
Why “innovation” doesn’t always have to be new (or at least on first sight)
Decades old concepts give light to revolutionary innovations
Can we build ‘Her’?: What Samantha tells us about the future of AI
The journey to making virtual assistants more humanlike
Nuance’s Ron Kaplan awarded honorary doctorate from University of Copenhagen
Award recognizes significant contributions to linguistics and natural language
Beyond the GUI: It’s time for a conversational user interface
Conversational user interface promoting new interactions between people and devices
Nuance Chief Technology Officer Vlad Sejnoha named 2013 CTO of the year
Nuance CTO presented with the CTO of the Year Award at Mass TLC Awards
Got GPUs? Nuance puts groundbreaking NVIDIA GPUs to work to accelerate voice innovation
Ushering a new era in Machine Learning
yanny-laurel-audio-clip-headphones
Sorry, Team “Yanny” – AI says it’s “Laurel”
An audio clip is at the center of the latest viral debate
Smart multimodal assistants, such as Nuance Dragon Drive, now also include gaze detection based on eye-tracking
Multimodal interaction – How machines learn to understand pointing
Smart assistants combine speech, gesture and hand writing for human-like user experience
Dragon speech recognition software celebrates its 20 year anniversary
Dragon celebrates its 20th anniversary
Two decades later, we’re still talking to our computers – and much more
Nuance speech technology can understand over 80 languages and their dialects
Dragon, do you speak my dialect?
Once divisive, now unifying, dialects play an important role in defining who we are
Dragon uses deep learning for more accurate speech recognition.
Why we’re using Deep Learning for our Dragon speech recognition engine
Unique application of Neural Nets results in greater productivity
How to get the key foundation right for automotive HMI with audio and touch input
Part 1: How to avoid 5 common automotive HMI usability pitfalls
Audio and touch input are at the core of a powerful automotive HMI system
DFKI students use nuance speech tools to create interactive IoT applications
Nuance and DFKI help students create interactive appliances of the future with speech tools
Providing easy to use speech tech helps usher forth tomorrow’s interactive appliances
Star Wars’ portrayal of relationships between robots and humans is becoming today’s reality with new technological advancements
In a galaxy (not so) far, far away
Star Wars and the relationship between man and machine
Connected cars are becoming more connected to the IoT and more useful, employing ubiquitous personal assistants that exist across devices and experiences
KITT – Please open the garage
How talking cars that talk to "things" make life simpler, smarter
Speech systems need to observe and deal with pauses and other variation to elicit more natural communication between man and machine
Taking a pause to discuss speaker variation… and Machine Learning
New research observes variation in communication to abstract meaning
Female Nuance engineers share stories about combatting gender stereotypes in the tech industry
#iLookLikeAnEngineer: Breaking down gender stereotypes in tech
An inside look at Nuance and how diversity fuels innovation
The ancient Greeks discovered rhetorical devices which are now common in everyday language - something we need to specially design speech systems to accommodate
Innovating machine dialog: Brush up on your Greek and read Aristotle
Building systems that can make sense of Rhetoric and nuanced speech
Childlike curiosity, being comfortable with a blank page... Nuance researchers share what qualities they think a good researcher possesses
The personality of Science: The traits that help define an industry
Nuance researchers answer, “What qualities does a good researcher possess?”
Star Trek, Mr. Spock, and the future of patient care
Star Trek, Mr. Spock, and a highly sought-after future
How a vision for the future helped allay the anxieties of the time.
winograd-schema-challenge
Can machines think?
Nuance to host annual Winograd Schema Challenge, an alternative to the Turing Test
Ethics and design: Doing the right thing
The importance of design stretches far beyond basic appeal
Innovation and design: The coolness (and unusability) of our rich friends’ houses
The challenges of designing for fanfare vs. usability
Video: Innovating a relationship between people and technology
Making technology that works *for* us - not against us
Voice recognition and the dawn of intelligent systems
Examining the rapid progress of voice recognition and natural language understanding
Wearables like smart watches need intelligent systems to enable a meaningful human-device interaction
Smart watches need intelligent systems
Here's how you deliver intelligent systems for the evolving wearables ecosystem
Show more articles