Experience Design | Generative AI | Assistive Tech

Reducing time taken by non-verbal communicators in creating new content faster for communication.

Imagine using a communication method where you must select words and pictures. You can group these pictures and customize them according to your needs.

Avaz AAC(Augmentative and Alternative Communication) is a picture-based Communication & Learning App for people with special needs such as Autism, Cerebral Palset etc. Avaz empowers people to communicate their thoughts, emotions, ideas, needs and wants and also assists them in language understanding and learning.

AAC (Augmentative and Alternative Communication) - Communication devices, systems, strategies and tools that replace or support spoken language. These tools support a person who has difficulties communicating using speech.

Website: https://avazapp.com/

Introduction to the problem

Avaz realized users had difficulty setting up new communication folders and words due to the complicated long process, so they aimed to simplify it to save users time.

User Research

Ethnography

Observations were done with the AAC users - SLPs, caregivers, and teachers. They were asked to build a folder and vocabulary adding.

90%

users took long time to add words for a required context

70%

of the users added less than 10 words in one go for a particular context.

Deeper Insights

Users were asked about the tasks and their choices while performing certain tasks.

Difficulty in modeling new language needs

Difficult Navigation

Usability Testing + Task Analysis

Observations were done with the AAC users - SLPs and teachers. They were asked to build a folder and vocabulary adding for a new context of 'cooking in the kitchen'

4-6 steps

To add content to a single cell - folder/ word

20-30 secs

Time spent on adding word/ folder to each word

25 minutes

Time spent to add around 50 words

The task was-

  • Tedious and time-taking

  • Users experience confusion in creating folders with appropriate vocabulary.

  • Language modeling is one of the core requirements for the process.

Goal

To make the vocabulary + language expansion fast and easier to assist users in folder creation and language modeling.

Solution Overview

We worked on a three-day-long design sprint to come up with ideas to improve the overall experience. I explored design intervention on various touch points and in various ways using generative AI and language expansion methods.

  1. Improving + increasing access to the words

  2. Assisting users in consolidating context-related words - Making folders for users for quick communication needs

  3. Assist users in modeling language - Helping users with not just vocabulary but also support phrases

Our Solution

.

#Option 1 - Quickly adding the words

Home Grid

Settings

Edit Cell

Add New

Add New

Choose what to add

Choose what to add

Repeat the process for n number of times

Cell Detail

Cell Detail

Old flow of adding words - long, time taking and repetitive

.

New flow of adding word - QUICK | EFFICIENT

Home Grid

Settings

Edit Cell

Add New

Select ‘Add Multiple Words’

Add Words

.

#Option 2 - Helping the user in creating for the impromptu context.

I revamped the existing grid section to provide a smart 'QUICK' folder with all context-based vocabulary groups. A user can sift through all or create a new one based on the category to find exactly what they need and save time getting to the vocabulary they will need.

Old flow of adding words - long, time taking and repetitive

Home Grid

Settings

Edit Cell

Add New

Add New

Choose what to add

Choose what to add

Repeat the process for n number of times

Cell Detail

Cell Detail

.

New flow of adding words using AI

Home Grid

Settings

Edit Cell

Add New

Prompt for generation

_

Assistance in language modelling - Picture Mode

Assisted vocabulary expansion is done for the picture mode as well.

Assistance in language modelling

Often complete training for language modeling is not possible, which can create gap in communication. Assisted language modelling is done to help users with custom communication option.

Hello Doctor

These are then filled with vocabulary associated with the ‘dentist visit’.

Testing the impact

To understand the impact on our users I conducted a usability test with 3 speech language pathologists and their students.

Task 01

Create a new QUICK FOLDER

Task 02

Add words for a given context

Task 03

Edit/ Add and Delete cells if needed

Task 04

Request for the service using new feature

Observation

We observed great satisfaction and positive feedback from the users. They liked the new quicker way of adding the words and language modelling for communication.

01

Time taken to create a new folder with 50 words

25 minutes

< 1 minutes

02

Time taken to communicate in new context

120 seconds

15 seconds

97%

of the time saved by users in creating new vocabulary

8x

faster communication than previous method

Retrospective

What did I learn?

  • This was my first attempt at integrating Gen AI into any of my design practice.


  • The collaborative sprint gave me a fun opportunity to think about my feat and navigate the challenging and stressful attempt.


  • This was also a novel attempt at figuring out what can be catered with a simple AAC device with text and images which can be life-changing for someone just with small additions and changes.

Challenges and Changes

  • The main challenge was tackling functional complexity and depth within assistive technology software.


  • Some of the language learning and expansion concepts were new, and finding ways to integrate them into the product was challenging.


  • For changes, I would like to explore more UI variations and test my solutions with wider user groups to receive feedback and integrate with our solutions.

Other Projects

#1 Using generative AI for image making

#1 Using generative AI for image making

.

Curated and designed with love, caffeine and berries.

Monika, 2024 🌼