Inclusivity through technology

Posted on September 24, 2020

Emotions can be difficult to read for anyone. But imagine the challenge for those who have disabilities that impact their ability to communicate their feelings. With funding from the new Queen’s Wicked Ideas Competition, Queen’s researchers Claire Davies (Mechanical and Materials Engineering) and Nora Fayed (Rehabilitation Therapy) are working to develop assistive technology that can make it easier for children with disabilities to tell others how they are feeling.

“Children with complex medical conditions often have difficulty communicating with others,” says Davies. “Their parents may understand how they are feeling, but others, such as caregivers, may not. So, imagine if you were upset or sad, but couldn’t communicate that to people who are caring for you.”

Davies and Fayed are exploring the use of algorithms that use physiological signals, such as heart rate or sweat, along with feedback from parents, who are most likely to understand how the child is feeling, to determine which signals align with particular emotions. The idea is to then develop an app with an interface, such as a screen that would attach to a wheelchair, that could tell others how the child is feeling.

“This could have a substantial impact on the way a child with a disability interacts with caregivers, and with peers,” says Davies. The team is currently working with families in the community to identify which physiological signals are important and how the presence of parents or other family members can affect emotion.

This is not the first time that Davies has collaborated with the Faculty of Rehabilitation Therapy. Her Mech 393 course – Biomechanical Product Development – is part of an interdisciplinary initiative that brings biomechanical engineering and occupational therapy students together to make custom assistive devices for real users in the community.

“A multidisciplinary approach allows us to look at these complex problems from many perspectives,” says Davies. “Our hope is that this kind of research will give people with disabilities a more inclusive and equitable role in society.”

Here is Sally, a participant in our study. Physiological signals like galvanic skin response and heart rate variability provide insights into what she is feeling. Of course, mum told us that this is a family experience of pure joy. By co-constructing meaning from caregivers and integrating physiological signals, we can interpret Sally’s feelings and animate her expression on an iPad attached to her wheelchair. Now, kids her own age know how she’s feeling even if they don’t understand her facial expressions.