March 23, 2018 by Arvind Sanjeev
The e-tshirt is about predicting challenging behavior in autistic children to help prevent meltdowns.
Meltdowns among children classified as autistic can be a dreadful experience. Such challenging behavior leads to self-inflicting actions that injure themselves and sometimes others around them. It is usually observed when the child is trying to learn something new or when in a new environment. Teachers and parents try their best to avoid meltdowns but such behavior has no plan and can be difficult to predict. Here this project presents a novel solution that has been developed for predicting challenging behavior in autistic children, which will help parents and teachers prevent meltdowns and avoid stressing them too much. The developed solution is a new kind of visual representation of the body vitals of the child through the physical medium of clothing. An electrophoretic display has been seamlessly embedded on a t-shirt that presents data which on observing can allow parents and teachers to predict and prevent meltdowns before they occur. The E-tshirt project video:
Important reads for this project:
E-tshirt: Technical story of Development
E-tshirt: DIY tutorial
“When you have a meltdown, it’s as if the world is ending. Everything is too much and you feel like an overwhelming darkness has engulfed your very being. Irrepressible anger that may seem completely irrational to an outsider can be inwardly devastating us internally.” - Person diagnosed with Autism Spectrum Disorder
Many of the people classified as autistic have difficulty in conveying their thoughts to others through speech and facial expressions. In moments of excruciating pain they are not able to communicate their predicament to others around them and results in strenuous motor activity which thus leads to self inflicting behavior. Their incapability to be understood leads to escalating levels of stress and frustration that also adds to instigating a meltdown. A teacher or parent cannot understand their condition and predict before such meltdowns occur because they appear calm and relaxed on the outside before suddenly jolting into such challenging behavior. This project presents a technology that allows people to convey their internal feelings based on Autonomic Nervous Activity (ANS) to a caregiver and help them in preventing meltdowns [1].
Heart rate, electrodermal activity, and skin temperature are important parameters that can help understand the emotional condition of a person. It has been observed that the resting heart rate of an autistic person is around twice (120 bpm) that of others (60-80 bpm) and also experiences high levels of electrodermal activity [1]. In order to calm down their sympathetic nervous system they usually engage in self-stimulation activities which done repeatedly helps them to relax [2].
Meltdowns usually occur when the person is trying to learn something new or when in a new environment. Most of the time, teachers observe such challenging behavior when they try to teach a new lesson to an autistic child. The inability to learn this new lesson despite repeated attempts can lead to mounting frustration that eventually results in a meltdown. In such situations, observing the escalating heart rates, electrodermal activity and skin conductance can allow the teacher to avert a meltdown by letting the child calm down. These vital body parameters are therefore paramount in understanding them.
‘Dress is the way in which individuals learn to live in their bodies and feel at home in them. Dress is . . . an intimate experience of the body’ [3].
Clothing supports non-verbal and subtle communication, it is a tool that is a creative manifestation of the wearer and possesses powerful potential for social interaction and creative expression. Moreover, garments when infused with technology can be an anti-depressive form of embodiment which improves quality of life [4].
E-tshirt is a t-shirt which I have prototyped with an embedded electrophoretic based display that that can reveal ANS data such as: heart rate, skin conductance response and body temperature. Eink displays are lightweight and flexible. They ensure best outdoor visibility by providing high contrast and reflectivity. Furthermore, these displays require power only when refreshing or changing the contents of the display and does not require power to stay on. Due to these characteristics, Eink displays offer the best solution for serving as a wearable display in this project.
Through this research project I have thus conceived and prototyped a body worn display through which I am learning to implement meaningful manifestations of an autistic person’s autonomic data that can assist a caregiver. Moreover, I also work to understand the fundamental issues in having such body worn displays before it could be applied to the body or integrated into people’s lives in a way that is truly meaningful. The decision of communicating one’s autonomic data to another is truly left to the wearer and the technology can only be used to its potential if the person is willing to share it. Here, I am thus trying to unlock this potential through technology by following a user centered approach in the hope of creating profound artifacts that would eventually lead to better technology development in this area.
Consider the following scenario which is the current story at most of the schools providing education to children with special needs:
Michael enjoys going to his school in his own way. He has a very caring teacher that works with him and teaches him new lessons every day. One day during the middle of a lesson, she was waiting for an answer from Michael based on a question she asked about one of his favorite stories; she asked him a couple of times as she knew that this was something he couldn’t forget. Without warning Michael started screaming and waving his hands around abruptly, she had unanticipatedly triggered a meltdown that lead to self-injurious behavior. It was days like these that the teacher wanted to avoid by truly understanding how the children were feeling inside and till what stage they can nudge them to teach new lessons.
Now refer to an alternative scenario where technology is used to help Michael and his teacher to understand each other better without having to experience challenging behavior:
It’s been one week after the incident and Michael is happy than ever to go to school. His teacher greets him and says that she has something special for him today. She takes out a colorful t-shirt from behind her and gives it to Michael. She then tells him that this would allow them to understand each other better by displaying his biometric data to her. She asks him whether he wants to try it and on and have a go, Michael is excited and changes his t-shirt. Now the class commences and it’s Michael’s turn in the group to respond to the game that they are playing. His teacher is around him and notices that he is ignoring the group and is turning away, before she asks him to face the group she finds that his t-shirt is showing elevated levels of heart rate and decides to let Michael leave the group and plays one of his favorite songs on his ipod. In time, Michael relaxes and shows decreased heart rate and joins the group to listen to a story from their teacher. Together, Michael and his teacher worked to avoid a dreadful meltdown with the assistance of his new t-shirt that allowed his teacher to recognise his inner feelings.
As this scenario explains, the technology presented here would be able to assist Michael’s teacher in understanding his emotional status. The data can be used to help the teacher adapt her style of teaching to accommodate for Michael’s emotional well being.
Clothing is our second skin, it is the primary interface to our environment and becomes a medium which exchanges information like: emotions and experiences. It is one of the foremost things used by people for expressing themselves and plays a prominent role in our lives.
Attention has been given here to make sure that the design is as simple as possible relying on the concept that ‘Simple is the Best’ [5]. Simple forms of subtle communication is employed since too many possibilities for interaction will complicate its use and lead to confusion. In this section, I shall discuss about the consequent hardware peripherals used for prototyping this project.
Hardware design
In this project I am using parameters such as heart rate, skin temperature and galvanic skin response to determine the user’s emotional state. Various sensors are used for achieving this. The entire system is hosted on a custom four layer PCB that was designed and fabricated for the E-tshirt. An STM32L152 microcontroller was selected for its low power consumption to control the entire system. This ARM Cortex - M3 controller offers 256 KB flash memory and 32 KB RAM along with a 12-bit ADC.
As shown in the block diagram, three sensors are used for getting the autonomic data. Mainly, the pulse sensor that uses reflective pulse oximetry technique to determine the heart rate. Here, since the hemoglobin in the blood offers different optical properties with respect to the oxygen content in blood, we are able to distinguish between the different heart beats.
Next is the galvanic skin response sensor which consists of two electrodes placed inside the t-shirt in contact with the person’s body. Based on the stress levels of the person, they exhibit greater skin conductivity due to the activation of the sweat glands. Finally we have a temperature sensor: LM35 that is embedded inside the t-shirt near the user’s shoulder that can assist in reading the skin temperature.
The data obtained from these sensors are then processed by the microcontroller to generate accurate values of the autonomic data. This is then displayed through the e-ink display. The 6-inch e-ink display used here is a thin and flexible panel on a plastic substrate manufactured by LG, the LB060X02. The display was sourced from China. The driver for the display was created for STM32L152 using uGFX platform.
Operation
E-tshirt consists of a 6 inch electrophoretic display embedded onto the t-shirt which is positioned on the chest area. This positioning allows the caregiver to interact face to face with the child as well as monitor his autonomic data. The inner surface of the t-shirt having the connectors has been lined with a layer of soft cloth to ensure maximum comfort for the child. The strategic positioning of the display, battery and the controller board allows maximum flexibility for the child to move around without feeling any weights that are drooping from their t-shirt.
The E-ink screen displays the autonomic data such as: heart rate, skin temperature and galvanic skin response that is updated every 500 milliseconds owing to the slow refresh rate of the electrophoretic display. The caregiver can then directly obtain the child’s autonomic information by viewing this high contrast displaying any daylight conditions and offers subtle interaction as well as discretion, unlike flashy and bright LED screens.
Through the interaction with Dr.Neena, it was understood that E-tshirt apart from being really helpful in assisting parents of children with ASD can also be used in a lot of other fields. It can readily be used to assist people having Asperger’s syndrome, which also lies in the same spectrum as ASD. People classified with Asperger’s also face frequent meltdowns that get triggered from excessive stress. Apart from this, this technology can also serve people with chronic diseases such as stress induced asthmas and cardiac arrhythmia by sharing their autonomic data with caregivers who interact with them daily. This technology allows them to understand what situations create greater stress and triggers anxiety in them; thus measures can be taken to subsequently avoid those incidents. The same use case and application can be extended to other chronic illnesses as well like PTSD (Post Traumatic Stress Disorder) and Anxiety disorders.
This open source project would thus allow people to take advantage of the E-tshirt technology to create solutions for people in distress. This will also allow the identification of several other use cases where the technology can be leveraged.
I would like to start by thanking Dr.Neena Shilen (Developmental Pediatrics, Sunrise Hospital) for giving me valuable guidance in designing this project. And also wish to express my gratitude to Mr.Anirudh Sharma (MIT Media Lab, Graviky Labs) and Mr.Sijo Kuruvilla George for their valuable insight and counsel. I would also like thank Mrs.Ancy Paulose, Mr.Antony Aby and Mr.Loui Paulose for assisting with altering the fabric of the T-shirt. The work done by Mr.Petteri Aimonen for reverse engineering an eink display served as a guide for creating the driver for interfacing the display in this project and I wish to thank him. I also thank Dr.Sanjeev S, Dr.Usha Rani G, Mr.Jerry Mannel and Mrs.Bhavna Jerry for their invaluable guidance.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
An interaction designer and engineer. Yahoo-Accenture had also awarded him as the "Most Promising Innovator".
