New insights in voice perception
It growls, it grumbles, it scratches
Voices are full of information. Not just with the information in words themselves, but also with emotions, education, age and maybe even gender. But not when you are deaf and wear a cochlear implant. Voices, music; everything sounds like screeching and scratching.
How bad is that? That’s what professor in audiology Deniz Başkent wants to know. What do you miss? UKrant listened in.
Thanks to Etienne Gaudrain, Leanne Nagels, Emile de Kleine and Allison Loiselle for making and editing the used sound fragments in this article.
Eerste tegel
1What if…
You’re in the park with your boyfriend. He goes to get an ice cream while you sit in the grass, people-watching. Then you hear a voice behind you.
You may not realize it, but a lot is happening in those few seconds.
First, your boyfriend uses his vocal chords to create vibrations in the air. Then he uses the cavity of his mouth, along with his tongue and lips, to shape those vibrations and push them out in your direction.
These sound waves travel to your outer ear, which guides them through the ear canal. There, they make your eardrums vibrate. Three tiny bones amplify the sound; the fluid inside the cochlea, a snail-shaped structure behind the bones, ripples. Tiny hair cells sitting on top of the membrane ride the wave. Those in the center of the membrane will detect the low tenor of your boyfriend’s voice. Those at the outer ends will pick up the high-pitched scream of a nearby child.
The top of the hair cells bump into the overlying structure within the ear, and bend. That causes a tiny channel to open up, and chemicals to rush in. Those chemicals create an electric signal, which is carried to the brain by the auditory nerve.
Finally, you are able to make sense of what your boyfriend just told you.
‘There you are, gorgeous.’
You lean back to accept a kiss and your ice cream.
That’s all great, of course. The fact that you were able to process this information is a truly impressive feat of evolution. But what if those loving sounds come across differently?
Fragment
Tweede tegel
Afbeelding oor
2‘This is what it sounds like when you’re deaf and wear a cochlear implant’, explains RUG professor of auditory perception, Deniz Başkent.
Başkent has been researching sound and the perception of voices for many years now. She is fascinated by the simple fact that listening to voices tells us so much more than the information transmitted by the words.
‘Most importantly, by listening to voices, we get information about the identity of the person who is speaking’, Başkent says. ‘Sound perception tells you who is talking, whether that person is an international, whether that person is educated, whether that person is male or female. And that’s all very helpful, because it can tell you how to interact with this person.’
In addition to generating a mental image of the person, we can also guess how that person feels. ‘The words “What have you done?” may be angry or playful, joking or sarcastic. Intonations make a big difference’, Başkent explains.
You don’t even need words to be able to hear those emotions.
Emotion 1
Emotion 2
Emotion 3
So when your boyfriend – whom you have identified through his voice – says ‘there you are, gorgeous’, you instantly relax, because you have all the information you need to be at ease.
But when you can only hear with help from a cochlear implant (a small device placed directly under the skin above your ear connected to electrodes in the cochlea) your auditory nerve is not stimulated in the usual acoustic way, but electrically. The result is something that the brain registers as a low, and rough, ‘sound’.
Imagine if that’s the world you heard all the time.
‘Of course, implants have come a long way’, says Başkent. ‘But most attention so far has focused on transferring information.’ So the voices lack all these extra layers of personal information.
The question is: what do you miss when you’re unable to distinguish different voices? What does voice perception really mean in our communication?
That’s what Başkent wants to know and that’s what she’s been researching, together with her group and collaborators – psychologists, engineers, linguists and people from artificial intelligence -, using the VICI grant of 1,5 million euros she received in 2017. She started out by establishing a baseline for people who aren’t hearing impaired – what do we hear and at what age do we hear it? She studied children up to twelve years old.
The initial results reveal that we know way less about the importance of voice perception than we even realized.
Derde tegel
Afbeelding oor
3How often has your mother moaned, ‘Look what you’ve done’? Like that time you stole a cookie and hoped she wouldn’t notice that the stack had gone from ten to nine?
How often did your father ask ‘Did you forget what time you had to be home?’
They may have said it playfully or with anger, disappointment, frustration. And they will have expected you to identify that emotion. The problem is that children often don’t, says Başkent.
But since she is only in the first, unpublished stage of her research, she hesitates to make any strong statements. Nonetheless all the data she has collected points in roughly the same direction. Abilities we take for granted – recognizing emotions in a voice, making a mental picture of the person speaking, being able to home in on one voice when many people talk – take way longer to develop then we thought.
‘The results are really surprising’, says Başkent, pointing to a set of data. ‘It’s not until kids are around twelve years old that they can really do it.’
So when you talk to children, you should be really explicit in what you mean. ‘You really can’t expect them to figure out what you mean themselves.’
Why we didn’t realize this before? The fact is, it isn’t that easy to research. Highly technical researchers had to develop software that allowed her to change every aspect of a voice, like the pitch, or the effects that the mouth cavity has on a voice.
It took Başkent and her team three years to develop the right tools to catch children’s attention. The test she now uses is almost like a video game and encourages the children to try their best, yielding the best results.
But it’s all been worth it, says Başkent. Because children don’t only struggle to distinguish emotions. They also struggle to tell voices apart, and even to distinguish male from female voices.
That has consequences. What about children in crowded classrooms that get the label ADHD, because they do not seem to listen? What if they just can’t? When they hear things like this?
Focus
‘Children from around 5, 6, 7 years old only understand half of the words they hear in a noisy place’, Başkent found. ‘But the teacher thinks everything is okay.’
Acoustics in schools are often bad. Buildings are old and there are rarely carpets or curtains to dampen the ambient sounds. We may be able to help children hear and understand better simply by putting foam under their tables.
Başkent hopes that her results will really make change. ‘I want to help people. These children are very vulnerable.’
Vierde tegel
Afbeelding oor
4And now?
Başkent has only just begun. Because she now has some idea how the voice perception of normal people develops, she can start to research what happens when you’re dependent on a cochlear implant.
Again, she has to rely on children. Grown-ups may have learned the essential skills in different ways, or may distort results by drawing on previous experience.
Are these children less able to distinguish emotions? Will they ever learn? Or are they able to perform the same trick in a different way? ‘Maybe they are able to learn from watching faces’, Başkent says. ‘Or maybe they adjust to the implant and learn it in a different way, through the neuroplasticity of the brain.’
So to them these voices that sound so scratchy to our ears, contain the same emotional and mental layers.
Fragment’
Baskent and her team are currently looking for normal-hearing monolingual Dutch children between 4 and 12 years old that would like to participate in the eye-tracking part of the study.
The study will be carried out at the Harmony building and will take around 45-60 minutes. Children will receive a gift card of 5 euro, a small gift and a travel reimbursement. Want to know more? Check www.picka-onderzoek.nl or email [email protected].