New smartwatch app alerts deaf and hard-of-hearing users to common home-related sounds
Prototype used machine learning to classify sounds in real time
November 10, 2020
Smartwatches offer people a private method for getting notifications about their surroundings -- such as a phone call, health alerts or an upcoming package delivery.
Now University of Washington researchers have developed SoundWatch, a smartwatch app for deaf and hard-of-hearing people who want to be aware of nearby sounds. When the smartwatch picks up a sound the user is interested in -- examples include a siren, a microwave beeping or a bird chirping -- SoundWatch will identify it and send the user a friendly buzz along with information about the sound.
"This technology provides people with a way to experience sounds that require an action -- such as getting food from the microwave when it beeps," said lead author Dhruv Jain, a UW researcher in the Paul G. Allen School of Computer Science & Engineering.
Jain was born hard of hearing. "I use the watch prototype to notice birds chirping and waterfall sounds when I am hiking. It makes me feel present in nature. My hope is that other deaf and hard-of-hearing people who are interested in sounds will also find SoundWatch helpful."
The team started this project by designing a system called HomeSound for deaf and hard-of-hearing people who wanted to know what was going on around their homes.
"I used to sleep through the fire alarm," said Jain.
The researchers tested HomeSound in the Seattle-area homes of six deaf or hard-of-hearing participants for three weeks. Participants were instructed to go about their lives as normal and complete weekly surveys.
Based on feedback, a prototype used machine learning to classify sounds in real time. The researchers created a dataset of over 31 hours of 19 common home-related sounds, such as a dog bark or a cat meow, a baby crying or a knock on the door.-- NSF Public Affairs, firstname.lastname@example.org