DOVER

Language learning meets binging watching

Challenge

" One major difficulty language learners encounter along the way is speaking improvement. Mostly, they won’t gain the fluency unless they spend time staying abroad. "

The language learning market seems saturated, but when peeled away, there are so few products that can help intermediate learners to develop their fluency. Many learners can read and listen, but struggle with speaking. Their progress often stalls because of limited access to practice with native speakers and lack of motivation. We want to address this pitfall in exist solutions by designing an effective and engaging way to help learners improve their speaking.

Solution

Dover is a language learning platform that pairs speaking practice with video streaming services like Netflix and Hulu. It provides a fun and effective approach for learners to practice speaking through dubbing over their favorite TV shows. By listening and mimicking dialogues of native speakers(a.k.a echoing practice), learners gain correct mouth muscle memory to improve their fluency and pronunciation.

Context
MHCID Capstone, 
University of Washington

Team members
Ann Lin
Liyi Chu
Collin Walker
Amy Roberts

Responsibility
Product Design
Interaction model
Experience flow
UI design
Video editing
User Research

1. Sync account + watch shows
2. Select dialogue to practice
3. Echoing practice and record
4. Get visual and audio feedback 
5. Improve until pass 
6. Enjoy dubbing over

DESIGN PROCESS

___   
   ___ 
01
Discover

How can we help language learners improve
their speaking effectively?

First Round Research

Subject Matter Expert Interview 

We identified target user and  paint points by interviewing  five experts in language teaching and technology. Experts reveled it's really challenging for learners' to improve speaking when they can’t spend time staying in a country where the language is spoken. By only learning from a course setting, it’s hard for students to engage in organic conversations and stay motivated to keep improving. This is especially detrimental to intermediate learners who are at the cusp of developing fluency.

Competitive Analysis

After investigating current products, we decided to develop a solution in the independent - entertaining area. Our target users can freely practice by their own and better stay motivated in fun learning activities.

What we found

Target User

  • Intermediate level learners who live in a non target - language environment

Pain Points

  • Feel uncomfortable when practicing speaking in front of people
  • Inconvenient and inadequate access to practice with native speakers
  • Be bogged down by robotic conversation practice in typical course setting
___   
   ___ 
02
UNDERSTAND
To acquire any language, you need to repeat words and phrases often, so repeat things you like. When we do something pleasurable, dopamine is released in the brain and that makes us want to do it again. 
                                                                     - Dr. Michael Erard, linguistics author                                                                                           - Dr. Michael Erard, linguistics author

First Round Research

Research question

How can some learners acquired strong speaking ability, even if they had never been staying abroad?

Semi-Structured Interview

We conducted semi-structured interviews to gain insights from 8 successful learners’ personal experience . We asked learners to draw a timeline and talk about when and what they did to learn. Telling stories along with a timeline can better remind people of details in a long journey. At beginning, they had trouble telling us how they “learned” the language. But when we asked what they did “involving” the target language, the learners began telling vivid stories. As it turns out, they never thought of it as “learning. Also, after exploring different language learning methods with 5 teaching experts, we learned that echoing practice is a powerful way for learners to develop fluency.

Linda, Taiwanese English Learner
‘‘ I watched through the entire series of Friends three times. It helped me learn natural conversation as well as about American culture .
American Spanish learner
‘‘ I like La Liga and I wanted to see what Spanish news sources were saying about  those players.
Dr. Alexander Arguelles, Linguist who can speak 50languages
‘‘ When [echoing] properly, you put the correct resonance of a new speech  form directly into your auditory system and simultaneously seek to match that resonance with your vocal output . . . you get into the physical habit of reciting entire dialogues aloud and correctly. How can this fail to improve your fluency?

Analysis and Synthesis

We clustered research findings into topics to help surface the patterns. Top five design principles came out which can best address our target users' pain points.

Dr. Alexander Arguelles, Linguist who can speak 50 languages
When [echoing] properly, you put the correct resonance of a new speech form directly into your auditory system and simultaneously seek to match that resonance with your vocal output . . . . If you get into the physical habit of reciting entire dialogues aloud and correctly, how can this fail to improve your fluency?
Adam McGarity, Program Manager of
World Languages at City University of Seattle
I ask my students write a monologue, record a native  speaker saying it, and then echo the native speaker.
Taiwanese English Learners
I watched through the entire series of Friends three times. It helped me learn natural conversation as well as about American culture.
Taiwanese English Learners
I had a great passion about old 80's English songs… If there was any new vocabulary in the lyrics, I would look it up in the dictionary in order to sing the song.
American Spanish learner
I like La Liga and I wanted to see what Spanish news sources were saying about those players
___   
   ___  
03
DESIGN

Create an experience to engage learners in practicing speaking via echoing methods

So Our Design Must...

  • Incorporate the echoing method into practices 
  • Provide a sense of achievement
  • Design fun activities as the primary motivation beyond language learning
  • Include context in learning material  
  • Generate feedback immediately

Ideation

We start with unstructured ideation first (faucet thinking) and then narrowed down by adding design constraints one at a time (funnel thinking). This helped us foster a more natural flow during the brainstorming session that created promising concepts that were unconventional and grounded in our research findings.

Videos clips from Youtube
Videos from streaming service
Learners enjoy more fun of dubbing over a scene like a voice actor and a binge watching experience.
Mobile device as a to-go solution
Mobile as remote paired with bigger screen devices
Learners feel more conformable to speak in a private space and more often watch shows on TV instead of mobile.
Getting feedback for your speech through voice analysis system
Learners gain a sense of achievement when developing fluency through echoing practice.

Concept Refinement

User interviews helped us evaluate and finalize the product concept. We selected ‘’Dubbing over shows''  as  the final concept. Our final concept involves assisted conversation mimicking that helps the learner pronounce in a correct manner. Once they successfully mimic the conversations with assistance, then they can enjoy the fun of freeform dubbing over the scenes. Building from what we learned in the interviews, we also fleshed out the final concept while focusing on our design goal. This ensured that our design solution provides an engaging user experience that a user would love to use, instead of just a product that the we “think” a user will love to use.

Videos clips from Youtube
Videos from streaming service
Learners enjoy more fun of dubbing over a scene like a voice actor and a binge watching experience.
Mobile device as a to-go solution
Mobile as remote paired with bigger screen devices
Learners feel more conformable to speak in a private space and more often watch shows on TV instead of mobile.
Getting feedback for your speech through voice analysis system
Learners gain a sense of achievement when developing fluency through echoing practice.

How our concept evolved

Videos clips from Youtube
Videos from streaming service
Learners enjoy more fun of dubbing over a scene like a voice actor and a binge watching experience.
Mobile device as a to-go solution
Mobile as remote paired with bigger screen devices
Learners feel more conformable to speak in a private space and more often watch shows on TV instead of mobile.
Getting feedback for your speech through voice analysis system
Learners gain a sense of achievement when developing fluency through echoing practice.

II. Concept Refinement

User interviews helped us evaluate and finalize the product concept. Building from what we learned in the interviews, we also fleshed out the final concept while focusing on our design goal. This ensured that our design solution provides an engaging user experience that a user would love to use, instead of just a product that the we “think” a user will love to use.

We chose ‘’Dubbing over shows''  as  the final concept. This involves assisted conversation mimicking that helps the learner pronounce in a correct manner. Once they successfully mimic the conversations with assistance, then they can enjoy the fun of freeform dubbing over the scenes.
BASED ON FINDINGS
Learners enjoy more fun of dubbing over a scene like a voice actor and a binge watching experience.
Learners feel more conformable to speak in a private space and more often watch shows on TV instead of mobile.
Learners can gain a sense of achievement when seeing their progress in developing fluency.
HOW OUR CONCEPT EVOLVED
Videos from TV streaming service
Video clips from Youtube
Mobile as a remote paired with bigger screen devices
Mobile device as a to-go solution
Generating immediate feedback through voice analysis system for users to improve in each practice.
___   
   ___  
INTERACTION

How to intervene video watching experience in a streamline flow to integrate echoing practice?

Interaction Design

In this stage, there were three main challenges we tackled on :

Through countless iterations of combing through every detail, we finally distill a simplified and intuitive interaction model and flows for all complex tasks. The interaction design led to a series of wireframe to guide the development of the UI and prototype.

 We broke each major task down and thought through detailed interaction for each activity.
This interaction model led to a series of UI wireframe and the development of  interactive prototype.
This interaction model led to a series of UI wireframe and the development of  interactive prototype.
Interaction Model
___   
   ___  
04
Test & learn

How do we quickly build an interactive prototype to answer critical design questions?

Prototype

Our functioning prototype was built on Adobe Premiere and Unified Remote (a custom mobile remote platform). Participants could record their voice directly on top of the video and have it played back to them. These functions gave us the ability to validate the main test goals.

  • Do learners find engagement in echoing practice and dubbing over?
  • Does the remote UI allow user to practice easily while enjoying video watching?
  • How do user feel about the interaction model and the flow?

Usability test

We ran test with 5 language learners. Each participant was asked to complete tasks out-loud, followed with an interview. Through identifying main takeaways in the usability test, we further improved the ease of use of our final design.

Improving ease of use

Autoplay the feedback in practice mode right after a learner record

Since learners will repeat the practice loop frequently ( listen to, echo, get feedback ) to improve. It takes too many clicks to complete a practice loop. Dover will automatically playback the most recent recording to reduce one manual click. Learners only need to pay attention on switching between two buttons ( repeat and record ) and enjoy a smooth practice experience.

Navigate the video and chose dialogues for practice

While watching a show, by double tapping the subtitle, learners can mark the dialogue for practice afterward. Furthermore, learners are able to navigate the video to the next or previous lines effortlessly by just swiping left or right on the subtitle area. They won’t need to split their attention on rewinding the video to an accurate point, so watching experience won’t be disrupted.

Engagement of Dover

I definitely think that recording myself is a better way of learning and more interactive...I'm very entertained and I would do this for other shows!

— Canadian Spanish learner
I was not feeling very confident as I hadn’t spoken in five years, but playing with Dover reminded me I hadn’t forgotten.

— American Spanish learner
I could see Dover as a fun social activity where everyone was recording various lines of dialogue throughout a show or movies.

— American Spanish learner
One reason why I would use this is because the shows are made for real native speakers to watch.
  
— Chinese Spanish Learner

UI Design

Final UI Design
___   
   ___  
05
Product

Introducing Dover

Watching Mode

  • Watch videos
  • Navigate the dialogue         
  • Select a line to echo

Practice Mode ( echoing )

  • Listen to the dialogue carefully
  • Copy the dialogue and record your speech
  • Receive feedback and improve
  • Unlock dubbing over feature

Performance Mode ( dubbing over )

  • Dub over scenes by recording your voice
  • Save the dubbed over video to collection
  • Share with friends