This blog is an extension of one of my Inclusive-Design project last semester.

Problem Identified

Speech is a fundamental tool in our everyday communication. Yet, speech impairment is quite common in adults as well as children. It has a prevalence of 7% in India and is listed as the fifth-highest occurring disability in the country.
I wanted to explore how might a design intervention directly assist children with speech disorders to talk, encourage and improve their communication in the long run.

The Primary Research

The primary research part of this project was done along with my friend, Ashima Mahajan.
To better understand the speech/ communication challenges faced by children, we visited a Special School with students diagnosed with Cerebral Palsy (CP). We interacted with the teachers, a lot of students (made some new friends) and collected an in-depth primary data/statistic of about 10-12 students mainly in the age-group of 11-15yrs.
Since the majority of the students were diagnosed with Quadriplegic CP, their limb movements were very much restricted and were wheelchair-bound. But most of them could speak in Keywords or very short sentences. Another important finding was that a lot of the students had a well-developed brain, so they faced no problem in understanding what was being said to them.

Initial Concepts

Picture Exchange Communication (PEC) was found to be the most effective method of communication in their school. Therefore, my initial few design concepts involved tablet applications with picture cards. Individuals could select the picture of what they want to say with a tap or by using an adaptive switch. (For example, a child selects an image of the toilet seat if he/she needs to use the restroom)
PEC cards did not encourage the children in attempting to speak. Also one of the main drawbacks of this concept was that it was relatively mundane for the children and failed to provide an intuitive, engaging and a fun user-experience.

Proposed Solution

While ideating & exploring a more playful & gamified product interaction, I came across the ‘Talking-Tom Cat’ App. It is a famous game in which the objective is to take care of a virtual pet-cat named Tom. The element of surprise in the game is that Tom can repeat the words you speak into your microphone using a synthesized voice.

Using this application as a platform, I conceptualized an ‘Inclusive Mode’ designed for specially-abled children with speech impairment.

How it will work

A parent or a guardian needs to fill in some basic details like their child’s name, age, etc before getting started. Unlike the normal Talking-Tom who repeats after you, here, Tom will help the child to complete his/her sentences using Language Processing and Predictive Analytics.

The general idea is that the child can speak to Tom in his/her keywords or broken sentences. The predictive algorithm will then kick in to refine, complete the sentence and Tom will speak back the full, corrected sentence back to the child.
The child can tap (or use an adaptive switch) to select from either ‘YES’ or ‘NO’ to validate the sentence repeated by Tom.

This will encourage the child to try & form complete sentences in communication. The corrected sentence spoken back by Tom can also be sent to the parent or teacher’s phone or spoken out aloud by Tom so that the nearby helper can attend to their child immediately (This can be particularly useful for the wheelchair-bound children). Example: “ I WANT TO USE THE TOILET” or “I AM HUNGRY”.
This will also empower the children with a playful tool of communication with their guardians or people nearby.

Along with the child improving his/her speech and sentence formation, The Machine learning algorithm will make Tom smarter, wiser after each use. It will eventually improve voice recognition and analyze usage patterns to become ‘Daily routine smart’ to better assist the child every day. Example: At around 12-1 PM, Tom will know it is the child’s lunchtime or which subject(s) is the child facing difficulty in school. Or how to call out friend’s names, etc.

The Way ahead

Using ML, the possibilities are endless. In this way, The Talking-Tom Cat can evolve to become a speech-impaired child’s very own virtual assistant!
I still have to further refine the prototype and test this concept with the users.

Thank you for reading until the very end. I would love to hear your thoughts on it!

Read more blogs/ articles: https://manasmano.com/blog/