It is surprising if we haven’t met a child with Autism. For a matter of fact, 85% of humans show signs of Autism. But most of us neither attend therapies nor have difficulty in communicating. Whilst there are so many kinds of therapies that take place, Art Therapy has been proved to be one of the effective and promising methods. Children with Autism Spectrum Disorder are believed to be more creative than an average human and is one way they express their thoughts and emotions.
Now taking this into context with 3 highlight factors of:
framing the problem statement of
How can Art Therapy be made easily accessible and help in connecting therapists and children with Autism during these tough times?
We developed Express.
Now Express is a paint application which is made specifically for children with Autism under the ambit of Art Therapy. This paint application includes 2 login conditions including one for the therapist/doctor and another being the parent login.
As we are familiar with submitting art and drawing for competitions this follows the same format, where you open a new project and allow the child to experiment with all their creative skills through a set of colors and a brush. With a user friendly interface it could be very easy and knowledgeable for the child to get used to this application. Once the drawing/art is completed the parent just needs to submit it.
On the other hand, we are even giving an option to capture a picture of existing or already made drawings on papers. They just need to select and submit it. And this puts an end to the task of the parent.
The doctor logs in to the app as well, through a unique generated user code and finds all the submissions in the welcome page by children that come under his provision. That is not it. Express has a unique feature where the doctor also receives a full fledged report on the drawing.
This includes a complete image processed analysis of the percentage of every color used showing the statistics of the color proportions. In addition to this it provides a complete timeline of the entire artwork. With the physical Art Therapies the doctor usually sees the end version of the drawing but here, with it's continuous screen capturing method, the doctor could just move a slider and see the artwork right from the first point of the color to the end version.
Once the artwork has been submitted following the steps mentioned in Parental Perspectives, the image shall be processed using technologies such as Computer Vision and Color extraction. In which, the colors in the artwork are extracted and are put in a statistical manner in the form of pie charts and histograms. With most of the things happening in the back-end, our efforts are on reducing the time span for the processing to take place and provide accurate and reliable results.
These statistics will be put in the form of a report with again a user friendly interface. So that it is very easy to interpret the data provided and for easier understanding.
While talking about a concept of communication through colors, every color signifies either mood, situation etc. And taking advantage of children with ASD are more creative than the average human, and usually show amazing artistic skills, this will allow us to interpret the emotional state of the child.
For example Red signifies anger, harsh feelings while subtle colors like yellow or white signify calmness or peace. In this way, the statistics that we will be providing will hel the doctor to understand the emotional status of the child, due to which rather than sticking to the conventional method of Art Therapy the doctor can focus on specific problems that the child is going through. With this we try to provide easy interpretable data, for providing specific treatments accordingly.
In the future aspects we wish to include ML considering we have enough data of artworks, colors and some other factors, which would help us to minimize the time span for the therapists to decide on the course of actions.
Since we are currently dealing with a very small segment of Autism because not a lot of children go through Art Therapy, we wish to act small and think big. Which also means if non-verbal communication could be dealt with, which enables us to even deal with mute people or anyone who does actually express themselves either through writing or drawing.
In the future we also wish to include an aspect of education, by providing Express as a platform to learn through Machine Learning Algorithms and Convolution Neural Networks.
Autism as a neural disorder is not fully understood by doctors or therapists and thus the treatment/therapy of the same happens in a very deconstructed and experimental way. Due to which it either the child is not completely understood and they follow a curriculum that every child with autism follows which is not at all feasible considering them to be the future of the nation. To overcome this generalisation, we developed Express, to deal with specific problems for a specific child, to attain maximum results and effectiveness.
Drawings are a distilled form of a person’s consciousness, and if properly interpreted can provide a clear insight into the person. To streamline this process, we created an app which acts as a bridge between the patient and their therapist. With the main intention of how can we make Art Therapy accessible to everyone the application was made to be user friendly on the basis of subtle colors used for the UI, easy controls and a lot of fun.
The app is developed using Flutter as a base for fast prototyping which uses Dart as a language to develop this application.
On the back-end Firebase has been used to take care of the database may it be about the login credentials or the artwork submissions and reports as well.
We have used Visual studio code/ Android studio as code editor which have flexible integrations for GitHub which is being used for collaborating with other teammates during the tough times virtually.
OpenCV/pallete recognition API of Flutter are used to distinguish and extract colors from the artworks and then present it in the form of a statistical report.
The prototype mentioned above only scratches the surface, given enough time and resources, machine learning algorithms can be utilized to fully analyse the illustration and provide an accurate assessment to the users.
With a limited time frame, it was pretty impossible for us to train a dataset which would help us in effectively extract each colors from the artwork. Due to which one of the major problem we faced was to actually find a way to extract these colors in a very short time.
We have been successful in building the prototyped version of the drawing app on the first hand.
Including to which we have enabled the app to extract primary color recognition without any Machine Learning Algorithms.
In the course of this project we have got in contact with many institutions where children with Autism are treated and also Art Therapy being conducted, which has somewhat given us a step to take forward and initiate this project.
Since, this project comes under the ambit of Invent Your own Challenge, and the topic doesn't include the sector of Space and related technology, we didn't find the right way to actually utilize the data that has been collected since the course of time.
But which doesn't restrict us from using other resources. In the future we look forward to integrating cloud services such as AWS for better computations and have definitely used Miro to its extent who are the Global Collaborators.
https://drive.google.com/drive/folders/1EfFL2xo2yRZCinP_IKwrZWfodsHExXB4?usp=sharing
The following link can be referred for the Pitch Deck Presentation and the Proof of Concept(PoC) video of express, which we believe could lend you enough understandability about the project.
The above link helped in understanding the color palettes that could be used to design the UI of the application.
The official documents of Flutter helped us in the overall building of the application
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6124538/
The official artile referred to understand the aspects of Art Therapy
https://www.researchgate.net/publication/304996838_Art_Therapy_-_A_Review_of_Methodology
One more research paper to add to the set of papers which was used to understand the concept of Art Therapy