Experience Design | Generative AI | Assistive Tech
Reducing time taken by non-verbal communicators in creating new content faster for communication.
Imagine using a communication method where you must select words and pictures. You can group these pictures and customize them according to your needs.
Avaz AAC(Augmentative and Alternative Communication) is a picture-based Communication & Learning App for people with special needs such as Autism, Cerebral Palset etc. Avaz empowers people to communicate their thoughts, emotions, ideas, needs and wants and also assists them in language understanding and learning.
AAC (Augmentative and Alternative Communication) - Communication devices, systems, strategies and tools that replace or support spoken language. These tools support a person who has difficulties communicating using speech.
Website: https://avazapp.com/
![](https://framerusercontent.com/images/elCAHdQ8u2mhbHuPbugVu3zuwSc.png)
Introduction to the problem
Avaz realized users had difficulty setting up new communication folders and words due to the complicated long process, so they aimed to simplify it to save users time.
User Research
Ethnography
Observations were done with the AAC users - SLPs, caregivers, and teachers. They were asked to build a folder and vocabulary adding.
90%
users took long time to add words for a required context
70%
of the users added less than 10 words in one go for a particular context.
![](https://framerusercontent.com/images/DN6zcSzvyPFMZVfesPQHVJBi7g.png)
Deeper Insights
Users were asked about the tasks and their choices while performing certain tasks.
![](https://framerusercontent.com/images/9nv1Qmn3CAjF6hlf6Ww1Sb5oybo.png)
Difficulty in modeling new language needs
![](https://framerusercontent.com/images/eeg4tGRTFlfoO9OOG6Wi5jSh2bk.png)
Difficult Navigation
Usability Testing + Task Analysis
Observations were done with the AAC users - SLPs and teachers. They were asked to build a folder and vocabulary adding for a new context of 'cooking in the kitchen'
![](https://framerusercontent.com/images/MoV4Ow2XqRlvQuN08vey4xtvUw.png?scale-down-to=2048)
4-6 steps
To add content to a single cell - folder/ word
20-30 secs
Time spent on adding word/ folder to each word
25 minutes
Time spent to add around 50 words
The task was-
Tedious and time-taking
Users experience confusion in creating folders with appropriate vocabulary.
Language modeling is one of the core requirements for the process.
Goal
To make the vocabulary + language expansion fast and easier to assist users in folder creation and language modeling.
Solution Overview
We worked on a three-day-long design sprint to come up with ideas to improve the overall experience. I explored design intervention on various touch points and in various ways using generative AI and language expansion methods.
Improving + increasing access to the words
Assisting users in consolidating context-related words - Making folders for users for quick communication needs
Assist users in modeling language - Helping users with not just vocabulary but also support phrases
Our Solution
.
#Option 1 - Quickly adding the words
Home Grid
Settings
Edit Cell
Add New
Add New
Choose what to add
Choose what to add
Repeat the process for n number of times
Cell Detail
Cell Detail
Old flow of adding words - long, time taking and repetitive
.
New flow of adding word - QUICK | EFFICIENT
Home Grid
Settings
Edit Cell
Add New
Select ‘Add Multiple Words’
Add Words
.
#Option 2 - Helping the user in creating for the impromptu context.
I revamped the existing grid section to provide a smart 'QUICK' folder with all context-based vocabulary groups. A user can sift through all or create a new one based on the category to find exactly what they need and save time getting to the vocabulary they will need.
Old flow of adding words - long, time taking and repetitive
Home Grid
Settings
Edit Cell
Add New
Add New
Choose what to add
Choose what to add
Repeat the process for n number of times
Cell Detail
Cell Detail
.
New flow of adding words using AI
Home Grid
Settings
Edit Cell
Add New
Prompt for generation
_
![](https://framerusercontent.com/images/6vJ12BcZitZCUZfL2XZdnzB8DQQ.png)
![](https://framerusercontent.com/images/cRijaXNCT4pH9Eo1jcCIIl8Zzk.png)
Assistance in language modelling - Picture Mode
Assisted vocabulary expansion is done for the picture mode as well.
![](https://framerusercontent.com/images/OYsuhqZ5DXFfTnUGFwAu1IA060.png)
![](https://framerusercontent.com/images/NQFhE2GssA6TUrcDgaOtRpB6O0.png)
Assistance in language modelling
Often complete training for language modeling is not possible, which can create gap in communication. Assisted language modelling is done to help users with custom communication option.
![](https://framerusercontent.com/images/w1zbs1dUkmjiJUtRelM4U0lgo.png)
![](https://framerusercontent.com/images/qkSQr2KSsKA0lBNHAGcmFSA.png)
Hello Doctor
These are then filled with vocabulary associated with the ‘dentist visit’.
Testing the impact
To understand the impact on our users I conducted a usability test with 3 speech language pathologists and their students.
Task 01
Create a new QUICK FOLDER
Task 02
Add words for a given context
Task 03
Edit/ Add and Delete cells if needed
Task 04
Request for the service using new feature
Observation
We observed great satisfaction and positive feedback from the users. They liked the new quicker way of adding the words and language modelling for communication.
01
Time taken to create a new folder with 50 words
25 minutes
< 1 minutes
02
Time taken to communicate in new context
120 seconds
15 seconds
97%
of the time saved by users in creating new vocabulary
8x
faster communication than previous method
Retrospective
What did I learn?
This was my first attempt at integrating Gen AI into any of my design practice.
The collaborative sprint gave me a fun opportunity to think about my feat and navigate the challenging and stressful attempt.
This was also a novel attempt at figuring out what can be catered with a simple AAC device with text and images which can be life-changing for someone just with small additions and changes.
Challenges and Changes
The main challenge was tackling functional complexity and depth within assistive technology software.
Some of the language learning and expansion concepts were new, and finding ways to integrate them into the product was challenging.
For changes, I would like to explore more UI variations and test my solutions with wider user groups to receive feedback and integrate with our solutions.
Other Projects
![](https://framerusercontent.com/images/pofvkr1Zvc6pbYN9lia0t0o9zgA.png)
#1 Using generative AI for image making
![](https://framerusercontent.com/images/uSTvEyXl9lxBn7NhCSiotKdU9y4.png)
#1 Using generative AI for image making
![](https://framerusercontent.com/images/LdrZt6ZBgWSVjmBRXv39gEfI.png)
.
Curated and designed with love, caffeine and berries.
Monika, 2024 🌼