SpeakEasy
FEB 2025
FEB 2025
Partners
Second Place • UCSC SVC Designathon
Partners
Second Place • UCSC SVC Designathon
What if practicing speech felt more like a game, and less like a test?
What if practicing speech felt more like a game, and less like a test?
Context
Context
Over 70 million people stutter globally, yet most speech technologies fail to serve them, because they're trained only on fluent speech.
Current speech technologies, such as automated phone menus or computer-generated captions, often fail to work well for people who stutter due to training on fluent speech. To make these speech technologies more responsive and accessible for people who stutter, it is important to collect more diverse stuttered speech data to improve these technologies.
Over 70 million people stutter globally, yet most speech technologies fail to serve them, because they're trained only on fluent speech.
Current speech technologies, such as automated phone menus or computer-generated captions, often fail to work well for people who stutter due to training on fluent speech. To make these speech technologies more responsive and accessible for people who stutter, it is important to collect more diverse stuttered speech data to improve these technologies.
Problem
Problem
Design a mobile app that empowers individuals who stutter to practice speaking confidently in a supportive and low-pressure environment. The app should also encourage voluntary data contribution for more inclusive AI speech technologies, with a focus on user consent, transparency, and clarity around dataset use. You may incorporate elements like gamification, progress tracking, or personalized goals to make the experience engaging and user-driven.
Design a mobile app that empowers individuals who stutter to practice speaking confidently in a supportive and low-pressure environment. The app should also encourage voluntary data contribution for more inclusive AI speech technologies, with a focus on user consent, transparency, and clarity around dataset use. You may incorporate elements like gamification, progress tracking, or personalized goals to make the experience engaging and user-driven.
How might we empower individuals who stutter to practice speaking confidently in a supportive environment while contributing to more inclusive speech recognition technologies?
How might we empower individuals who stutter to practice speaking confidently in a supportive environment while contributing to more inclusive speech recognition technologies?
User Research
User Research
To set the stage for our work, we interviewed several students and individuals who stutter, refined our objectives based on their feedback, and distilled our findings into these 3 key insights
People felt kind of lost in their practice without personalized goals or progress checkpoints, so it was touch to stay on track.
Standard ASR kept choking on disfluent or stuttered speech, effectively leaving those voices out of the training models that AI use to understand speech.
Users struggle to find speech practice that's both engaging and motivating.
To set the stage for our work, we interviewed several students and individuals who stutter, refined our objectives based on their feedback, and distilled our findings into these 3 key insights
People felt kind of lost in their practice without personalized goals or progress checkpoints, so it was touch to stay on track.
Standard ASR kept choking on disfluent or stuttered speech, effectively leaving those voices out of the training models that AI use to understand speech.
Users struggle to find speech practice that's both engaging and motivating.
Solution
Solution
In a 48-hour sprint, insights gave us just what we needed to master this multiplayer gamified mobile app. low-pressure speech trivia and practice for stutterers, with friend-based leaderboards and online 'play with friends' matching to build community confidence: the recipe for success.
Hello, SpeakEasy!
In a 48-hour sprint, insights gave us just what we needed to master this multiplayer gamified mobile app. low-pressure speech trivia and practice for stutterers, with friend-based leaderboards and online 'play with friends' matching to build community confidence: the recipe for success.
Hello, SpeakEasy!
User Persona
User Persona
Based on our research, we defined a primary user persona that reflect our target users’ pain points, goals, and requirements.
Based on our research, we defined a primary user persona that reflect our target users’ pain points, goals, and requirements.
Design Iterations
Design Iterations
We went through a lot of design iterations throughout the design process. Debates would spark about almost anything, from font and color to usability and design hierarchy. The more debates the better, it all helped us shape the best possible app at the end. Below is an example of an iteration of the home screen.
We went through a lot of design iterations throughout the design process. Debates would spark about almost anything, from font and color to usability and design hierarchy. The more debates the better, it all helped us shape the best possible app at the end. Below is an example of an iteration of the home screen.
Data Collection
Data Collection
When it comes to the data collection, the back end will consist of waveforms from users' responses paired with the expected or inferred words. These data pairs will be sent to companies under contract.
These key-value pairs will help speech technology companies train their models on more diverse data, making speech recognition more inclusive for individuals who may not speak as fluidly as others.
When it comes to the data collection, the back end will consist of waveforms from users' responses paired with the expected or inferred words. These data pairs will be sent to companies under contract.
These key-value pairs will help speech technology companies train their models on more diverse data, making speech recognition more inclusive for individuals who may not speak as fluidly as others.
Trust & Transparency
Trust &
Transparency
To ensure transparency, we have implemented a consent notice before users sign in, clearly outlining how their information will be used and how the games will function.
To ensure transparency, we have implemented a consent notice before users sign in, clearly outlining how their information will be used and how the games will function.
Intentionality
Intentionality
We aimed to create an engaging and enjoyable app while also leveraging speech data to help individuals who stutter. This would contribute to improving data collection and speech technologies.
We aimed to create an engaging and enjoyable app while also leveraging speech data to help individuals who stutter. This would contribute to improving data collection and speech technologies.
Accessibility
Accessibility
We sought to implement as many inclusive features as possible to make the app enjoyable for everyone. In particular, we designed functions to support individuals who stutter, ensuring they feel included and considered throughout the design process.
We sought to implement as many inclusive features as possible to make the app enjoyable for everyone. In particular, we designed functions to support individuals who stutter, ensuring they feel included and considered throughout the design process.
Readability
Readability
Our goal was to maintain a clean and readable interface while also incorporating vibrant and inviting design elements to appeal to users of all ages and demographics.
Our goal was to maintain a clean and readable interface while also incorporating vibrant and inviting design elements to appeal to users of all ages and demographics.
Inclusivity
Inclusivity
We wanted to ensure that people of all ages and backgrounds could have a great experience. To achieve this, we incorporated a variety of options to accommodate different preferences and needs.
We wanted to ensure that people of all ages and backgrounds could have a great experience. To achieve this, we incorporated a variety of options to accommodate different preferences and needs.
Challenges
Challenges
We did realize some issues after the competition — most people who do stutter won’t stutter with a single word. Usually stuttering occurs in long sentences or phrases.
To fix this, we would implement games that take in a longer answer, maybe having the users ask brain teasers or trivia to their friends through the speech to help capture more of the data.
Another Issue we ran into was the fact that we are in fact using the technology that we are trying to fix, by transcribing the stuttered speech as answer inputs. What if our app fails to understand the stuttered speech?
To fix this, we later added a text input option after 3 failed speech attempts. Additionally, the more the platform gets used and the more data we collect, the better the speech models would get.
We did realize some issues after the competition — most people who do stutter won’t stutter with a single word. Usually stuttering occurs in long sentences or phrases.
To fix this, we would implement games that take in a longer answer, maybe having the users ask brain teasers or trivia to their friends through the speech to help capture more of the data.
Another Issue we ran into was the fact that we are in fact using the technology that we are trying to fix, by transcribing the stuttered speech as answer inputs. What if our app fails to understand the stuttered speech?
To fix this, we later added a text input option after 3 failed speech attempts. Additionally, the more the platform gets used and the more data we collect, the better the speech models would get.
Final Thoughts
Final
Thoughts
Winning 2nd place was definitely a plus, but what really stuck with me from this experience was just how quickly our team clicked. It was the first time I’d ever worked with this group, but from the jump, ideas were flying around like popcorn, every single person brought something brilliant to the table. I remember thinking, "Yep, these are my people. I want to do every designathon with them from now on." Amazing designers, inspiring collaborators, and just a dream team all around.
Winning 2nd place was definitely a plus, but what really stuck with me from this experience was just how quickly our team clicked. It was the first time I’d ever worked with this group, but from the jump, ideas were flying around like popcorn, every single person brought something brilliant to the table. I remember thinking, "Yep, these are my people. I want to do every designathon with them from now on." Amazing designers, inspiring collaborators, and just a dream team all around.